[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:22:01.067502  8743 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.137.254:37327
I20260709 12:22:01.069061  8743 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:22:01.070120  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:01.087724  8749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:01.087561  8750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:01.088743  8743 server_base.cc:1061] running on GCE node
W20260709 12:22:01.087543  8752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:02.279340  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:02.279603  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:02.279901  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599722279856 us; error 0 us; skew 500 ppm
I20260709 12:22:02.280738  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:02.285899  8743 webserver.cc:533] Webserver started at http://127.8.137.254:41609/ using document root <none> and password file <none>
I20260709 12:22:02.286760  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:02.286999  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:02.287569  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:02.293447  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/master-0-root/instance:
uuid: "492193a9dae74af0ae2404da27362f2a"
format_stamp: "Formatted at 2026-07-09 12:22:02 on dist-test-slave-kpwv"
I20260709 12:22:02.301004  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.004s
I20260709 12:22:02.305948  8758 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:02.307170  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:02.307459  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/master-0-root
uuid: "492193a9dae74af0ae2404da27362f2a"
format_stamp: "Formatted at 2026-07-09 12:22:02 on dist-test-slave-kpwv"
I20260709 12:22:02.307766  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:02.366569  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:02.368103  8743 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:22:02.368525  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:02.440304  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.254:37327
I20260709 12:22:02.440411  8809 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.254:37327 every 8 connection(s)
I20260709 12:22:02.448319  8810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:02.469748  8810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a: Bootstrap starting.
I20260709 12:22:02.475747  8810 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:02.477612  8810 log.cc:826] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a: Log is configured to *not* fsync() on all Append() calls
I20260709 12:22:02.482141  8810 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a: No bootstrap required, opened a new log
I20260709 12:22:02.502442  8810 raft_consensus.cc:359] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492193a9dae74af0ae2404da27362f2a" member_type: VOTER }
I20260709 12:22:02.503002  8810 raft_consensus.cc:385] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:02.503314  8810 raft_consensus.cc:740] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492193a9dae74af0ae2404da27362f2a, State: Initialized, Role: FOLLOWER
I20260709 12:22:02.504360  8810 consensus_queue.cc:260] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492193a9dae74af0ae2404da27362f2a" member_type: VOTER }
I20260709 12:22:02.504870  8810 raft_consensus.cc:399] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:02.505170  8810 raft_consensus.cc:493] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:02.505510  8810 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:02.513664  8810 raft_consensus.cc:515] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492193a9dae74af0ae2404da27362f2a" member_type: VOTER }
I20260709 12:22:02.514565  8810 leader_election.cc:304] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 492193a9dae74af0ae2404da27362f2a; no voters: 
I20260709 12:22:02.517067  8810 leader_election.cc:290] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:02.517416  8813 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:02.520581  8813 raft_consensus.cc:697] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 1 LEADER]: Becoming Leader. State: Replica: 492193a9dae74af0ae2404da27362f2a, State: Running, Role: LEADER
I20260709 12:22:02.521579  8813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492193a9dae74af0ae2404da27362f2a" member_type: VOTER }
I20260709 12:22:02.523378  8810 sys_catalog.cc:565] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:02.535408  8815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 492193a9dae74af0ae2404da27362f2a. Latest consensus state: current_term: 1 leader_uuid: "492193a9dae74af0ae2404da27362f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492193a9dae74af0ae2404da27362f2a" member_type: VOTER } }
I20260709 12:22:02.536033  8815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:02.537498  8814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "492193a9dae74af0ae2404da27362f2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "492193a9dae74af0ae2404da27362f2a" member_type: VOTER } }
I20260709 12:22:02.538175  8814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:02.541033  8822 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:02.557947  8822 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:02.563272  8743 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:02.578263  8822 catalog_manager.cc:1383] Generated new cluster ID: e28fdd64a8384e2b985a3e1e70ef161d
I20260709 12:22:02.578660  8822 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:02.604440  8822 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:02.605975  8822 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:02.620595  8822 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a: Generated new TSK 0
I20260709 12:22:02.621562  8822 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:02.635944  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:02.644596  8833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:02.646483  8834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:02.649355  8836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:02.650401  8743 server_base.cc:1061] running on GCE node
I20260709 12:22:02.651221  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:02.651410  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:02.651621  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599722651585 us; error 0 us; skew 500 ppm
I20260709 12:22:02.652266  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:02.656916  8743 webserver.cc:533] Webserver started at http://127.8.137.193:45437/ using document root <none> and password file <none>
I20260709 12:22:02.657505  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:02.657712  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:02.658018  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:02.659214  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-0-root/instance:
uuid: "929101e20d6944f3abf2b312b22616c4"
format_stamp: "Formatted at 2026-07-09 12:22:02 on dist-test-slave-kpwv"
I20260709 12:22:02.664945  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:22:02.669438  8841 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:02.670356  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:22:02.670679  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "929101e20d6944f3abf2b312b22616c4"
format_stamp: "Formatted at 2026-07-09 12:22:02 on dist-test-slave-kpwv"
I20260709 12:22:02.670964  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:02.691476  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:02.692826  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:02.695729  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:02.700314  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:02.700525  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:02.700846  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:02.701022  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:02.808370  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:34237
I20260709 12:22:02.808499  8904 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:34237 every 8 connection(s)
I20260709 12:22:02.815608  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:02.824848  8909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:02.829100  8910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:02.838750  8743 server_base.cc:1061] running on GCE node
I20260709 12:22:02.840065  8905 heartbeater.cc:344] Connected to a master server at 127.8.137.254:37327
I20260709 12:22:02.840570  8905 heartbeater.cc:461] Registering TS with master...
W20260709 12:22:02.840612  8912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:02.841718  8905 heartbeater.cc:507] Master 127.8.137.254:37327 requested a full tablet report, sending...
I20260709 12:22:02.841813  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:02.842229  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:02.842453  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599722842434 us; error 0 us; skew 500 ppm
I20260709 12:22:02.843304  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:02.845047  8775 ts_manager.cc:194] Registered new tserver with Master: 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237)
I20260709 12:22:02.846518  8743 webserver.cc:533] Webserver started at http://127.8.137.194:43743/ using document root <none> and password file <none>
I20260709 12:22:02.847213  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:02.847476  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:02.847540  8775 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35882
I20260709 12:22:02.847817  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:02.849309  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-1-root/instance:
uuid: "34e1340365c14e6c8d1954a320d9d013"
format_stamp: "Formatted at 2026-07-09 12:22:02 on dist-test-slave-kpwv"
I20260709 12:22:02.856170  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.000s	sys 0.008s
I20260709 12:22:02.860957  8917 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:02.861829  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:22:02.862151  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-1-root
uuid: "34e1340365c14e6c8d1954a320d9d013"
format_stamp: "Formatted at 2026-07-09 12:22:02 on dist-test-slave-kpwv"
I20260709 12:22:02.862493  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:02.880687  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:02.882377  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:02.884966  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:02.888708  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:02.888968  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:02.889266  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:02.889492  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:02.977494  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.194:33849
I20260709 12:22:02.977574  8979 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.194:33849 every 8 connection(s)
I20260709 12:22:02.983819  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:02.995401  8984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:02.996742  8983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:03.000988  8980 heartbeater.cc:344] Connected to a master server at 127.8.137.254:37327
I20260709 12:22:03.001335  8980 heartbeater.cc:461] Registering TS with master...
W20260709 12:22:03.001267  8986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:03.002247  8980 heartbeater.cc:507] Master 127.8.137.254:37327 requested a full tablet report, sending...
I20260709 12:22:03.003018  8743 server_base.cc:1061] running on GCE node
I20260709 12:22:03.004130  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:03.004374  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:03.004572  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599723004541 us; error 0 us; skew 500 ppm
I20260709 12:22:03.004498  8775 ts_manager.cc:194] Registered new tserver with Master: 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:03.005234  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:03.006352  8775 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35884
I20260709 12:22:03.009824  8743 webserver.cc:533] Webserver started at http://127.8.137.195:41023/ using document root <none> and password file <none>
I20260709 12:22:03.010313  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:03.010521  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:03.010848  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:03.012032  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-2-root/instance:
uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
format_stamp: "Formatted at 2026-07-09 12:22:03 on dist-test-slave-kpwv"
I20260709 12:22:03.016656  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:22:03.020116  8991 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:03.020926  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:22:03.021201  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-2-root
uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
format_stamp: "Formatted at 2026-07-09 12:22:03 on dist-test-slave-kpwv"
I20260709 12:22:03.021471  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783599720900375-8743-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:03.051860  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:03.053226  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:03.054777  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:03.057094  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:03.057272  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:03.057575  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:03.057736  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:03.155879  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.195:36209
I20260709 12:22:03.155972  9054 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.195:36209 every 8 connection(s)
I20260709 12:22:03.171993  9055 heartbeater.cc:344] Connected to a master server at 127.8.137.254:37327
I20260709 12:22:03.172367  9055 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:03.173121  9055 heartbeater.cc:507] Master 127.8.137.254:37327 requested a full tablet report, sending...
I20260709 12:22:03.175078  8775 ts_manager.cc:194] Registered new tserver with Master: 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:03.176182  8743 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015561792s
I20260709 12:22:03.176337  8775 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35886
I20260709 12:22:03.228080  8775 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35900:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:03.231959  8775 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:03.332823  8870 tablet_service.cc:1511] Processing CreateTablet for tablet 8f886b747d5f4f29b5ccacd465aa542f (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:03.334709  8868 tablet_service.cc:1511] Processing CreateTablet for tablet b8a990e66edc4710afc8a0bf0625b910 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:03.335845  8866 tablet_service.cc:1511] Processing CreateTablet for tablet 8de765fd04334493963ed7f7c894b468 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:03.332926  8869 tablet_service.cc:1511] Processing CreateTablet for tablet 596dacc9c2934bb59018b1428443fcde (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:03.337039  8867 tablet_service.cc:1511] Processing CreateTablet for tablet 0128bab39b3d40b9acd0d2f8aef1ea57 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:03.334481  8870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f886b747d5f4f29b5ccacd465aa542f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.338353  8867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0128bab39b3d40b9acd0d2f8aef1ea57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.341732  8866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de765fd04334493963ed7f7c894b468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.339478  8944 tablet_service.cc:1511] Processing CreateTablet for tablet 596dacc9c2934bb59018b1428443fcde (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:03.346318  8869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596dacc9c2934bb59018b1428443fcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.348881  8865 tablet_service.cc:1511] Processing CreateTablet for tablet ed5a17df6ece4515bf9e210a3b96fd94 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:03.349045  8943 tablet_service.cc:1511] Processing CreateTablet for tablet b8a990e66edc4710afc8a0bf0625b910 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:03.348129  8941 tablet_service.cc:1511] Processing CreateTablet for tablet 8de765fd04334493963ed7f7c894b468 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:03.343446  8942 tablet_service.cc:1511] Processing CreateTablet for tablet 0128bab39b3d40b9acd0d2f8aef1ea57 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:03.347141  8945 tablet_service.cc:1511] Processing CreateTablet for tablet 8f886b747d5f4f29b5ccacd465aa542f (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:03.353006  8868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8a990e66edc4710afc8a0bf0625b910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.354835  8944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596dacc9c2934bb59018b1428443fcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.371959  8940 tablet_service.cc:1511] Processing CreateTablet for tablet ed5a17df6ece4515bf9e210a3b96fd94 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:03.373095  8865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed5a17df6ece4515bf9e210a3b96fd94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.375803  8942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0128bab39b3d40b9acd0d2f8aef1ea57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.377266  8941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de765fd04334493963ed7f7c894b468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.382444  8945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f886b747d5f4f29b5ccacd465aa542f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.384173  8940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed5a17df6ece4515bf9e210a3b96fd94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.388988  8943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8a990e66edc4710afc8a0bf0625b910. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.508352  9019 tablet_service.cc:1511] Processing CreateTablet for tablet 8f886b747d5f4f29b5ccacd465aa542f (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:22:03.514200  9017 tablet_service.cc:1511] Processing CreateTablet for tablet b8a990e66edc4710afc8a0bf0625b910 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:22:03.506218  8864 tablet_service.cc:1511] Processing CreateTablet for tablet f6bb8b14ec80438c8982e8d19952ae13 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:03.530534  9019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f886b747d5f4f29b5ccacd465aa542f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.532681  8864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6bb8b14ec80438c8982e8d19952ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.548106  9018 tablet_service.cc:1511] Processing CreateTablet for tablet 596dacc9c2934bb59018b1428443fcde (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:22:03.516440  9016 tablet_service.cc:1511] Processing CreateTablet for tablet 0128bab39b3d40b9acd0d2f8aef1ea57 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:22:03.554390  9017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b8a990e66edc4710afc8a0bf0625b910. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:22:03.557078  8824 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:03.556981  9015 tablet_service.cc:1511] Processing CreateTablet for tablet 8de765fd04334493963ed7f7c894b468 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:22:03.563666  9014 tablet_service.cc:1511] Processing CreateTablet for tablet ed5a17df6ece4515bf9e210a3b96fd94 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:22:03.563843  8939 tablet_service.cc:1511] Processing CreateTablet for tablet f6bb8b14ec80438c8982e8d19952ae13 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:03.567565  8939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6bb8b14ec80438c8982e8d19952ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.569840  9018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 596dacc9c2934bb59018b1428443fcde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.573371  9014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed5a17df6ece4515bf9e210a3b96fd94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.575251  9013 tablet_service.cc:1511] Processing CreateTablet for tablet f6bb8b14ec80438c8982e8d19952ae13 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:22:03.580029  9016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0128bab39b3d40b9acd0d2f8aef1ea57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.581480  9013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6bb8b14ec80438c8982e8d19952ae13. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.584755  9015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de765fd04334493963ed7f7c894b468. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.660691  9012 tablet_service.cc:1511] Processing CreateTablet for tablet 7270f78f86d84264b8792b80a5c14f45 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:03.663729  9012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7270f78f86d84264b8792b80a5c14f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.658252  8938 tablet_service.cc:1511] Processing CreateTablet for tablet 7270f78f86d84264b8792b80a5c14f45 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:03.669308  8938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7270f78f86d84264b8792b80a5c14f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.691354  8864 tablet_service.cc:1511] Processing CreateTablet for tablet eca4b422654d4f0cbe4fb6cce4abb820 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:03.692605  8864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca4b422654d4f0cbe4fb6cce4abb820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.696550  8870 tablet_service.cc:1511] Processing CreateTablet for tablet 7270f78f86d84264b8792b80a5c14f45 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:22:03.699676  9074 tablet_bootstrap.cc:492] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:03.700222  9075 tablet_bootstrap.cc:492] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
I20260709 12:22:03.697765  8870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7270f78f86d84264b8792b80a5c14f45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.702471  8941 tablet_service.cc:1511] Processing CreateTablet for tablet eca4b422654d4f0cbe4fb6cce4abb820 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:03.703999  8941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca4b422654d4f0cbe4fb6cce4abb820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.731940  9076 tablet_bootstrap.cc:492] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:03.744257  9074 tablet_bootstrap.cc:654] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.747633  9075 tablet_bootstrap.cc:654] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.749308  9018 tablet_service.cc:1511] Processing CreateTablet for tablet eca4b422654d4f0cbe4fb6cce4abb820 (DEFAULT_TABLE table=test-workload [id=3971d42f5f4a4ddab1f9f86b14f92d31]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:22:03.751204  9018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eca4b422654d4f0cbe4fb6cce4abb820. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:03.759706  9076 tablet_bootstrap.cc:654] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.774562  9075 tablet_bootstrap.cc:492] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:03.775066  9074 tablet_bootstrap.cc:492] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:03.775605  9074 ts_tablet_manager.cc:1403] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.122s	user 0.017s	sys 0.006s
I20260709 12:22:03.775075  9075 ts_tablet_manager.cc:1403] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.098s	user 0.018s	sys 0.002s
I20260709 12:22:03.780428  9076 tablet_bootstrap.cc:492] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:03.780871  9076 ts_tablet_manager.cc:1403] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.050s	user 0.013s	sys 0.000s
I20260709 12:22:03.781919  9075 raft_consensus.cc:359] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.782770  9075 raft_consensus.cc:385] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.783174  9075 raft_consensus.cc:740] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.784045  9075 consensus_queue.cc:260] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.784323  9074 raft_consensus.cc:359] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.785038  9074 raft_consensus.cc:385] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.785348  9074 raft_consensus.cc:740] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.788146  9076 raft_consensus.cc:359] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.788867  9076 raft_consensus.cc:385] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.789158  9076 raft_consensus.cc:740] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.789844  9076 consensus_queue.cc:260] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.790663  9075 ts_tablet_manager.cc:1434] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.005s
I20260709 12:22:03.791538  8980 heartbeater.cc:499] Master 127.8.137.254:37327 was elected leader, sending a full tablet report...
I20260709 12:22:03.791816  9075 tablet_bootstrap.cc:492] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
I20260709 12:22:03.794533  9055 heartbeater.cc:499] Master 127.8.137.254:37327 was elected leader, sending a full tablet report...
I20260709 12:22:03.798353  9075 tablet_bootstrap.cc:654] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.799809  9076 ts_tablet_manager.cc:1434] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:22:03.800755  9076 tablet_bootstrap.cc:492] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:03.804003  9075 tablet_bootstrap.cc:492] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:03.804463  9075 ts_tablet_manager.cc:1403] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.013s	user 0.005s	sys 0.004s
I20260709 12:22:03.791046  9074 consensus_queue.cc:260] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.807394  9075 raft_consensus.cc:359] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.808084  9075 raft_consensus.cc:385] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.808380  9075 raft_consensus.cc:740] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.809089  9075 consensus_queue.cc:260] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.811352  9075 ts_tablet_manager.cc:1434] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:03.812274  9075 tablet_bootstrap.cc:492] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
I20260709 12:22:03.814018  9076 tablet_bootstrap.cc:654] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.819253  8905 heartbeater.cc:499] Master 127.8.137.254:37327 was elected leader, sending a full tablet report...
I20260709 12:22:03.820703  9074 ts_tablet_manager.cc:1434] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.045s	user 0.011s	sys 0.005s
I20260709 12:22:03.821652  9074 tablet_bootstrap.cc:492] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:03.822136  9075 tablet_bootstrap.cc:654] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.838259  9075 tablet_bootstrap.cc:492] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:03.838766  9075 ts_tablet_manager.cc:1403] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.027s	user 0.012s	sys 0.004s
I20260709 12:22:03.840945  9074 tablet_bootstrap.cc:654] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.841589  9075 raft_consensus.cc:359] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.842321  9075 raft_consensus.cc:385] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.842621  9075 raft_consensus.cc:740] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.843482  9075 consensus_queue.cc:260] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.845657  9075 ts_tablet_manager.cc:1434] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:22:03.847260  9075 tablet_bootstrap.cc:492] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
I20260709 12:22:03.847352  9076 tablet_bootstrap.cc:492] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:03.847832  9076 ts_tablet_manager.cc:1403] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.047s	user 0.004s	sys 0.011s
I20260709 12:22:03.850762  9076 raft_consensus.cc:359] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.854369  9075 tablet_bootstrap.cc:654] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.854399  9076 raft_consensus.cc:385] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.858989  9076 raft_consensus.cc:740] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.859980  9076 consensus_queue.cc:260] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.863267  9076 ts_tablet_manager.cc:1434] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:22:03.865514  9076 tablet_bootstrap.cc:492] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:03.878237  9074 tablet_bootstrap.cc:492] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:03.878753  9074 ts_tablet_manager.cc:1403] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.057s	user 0.017s	sys 0.000s
I20260709 12:22:03.879531  9076 tablet_bootstrap.cc:654] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.881748  9083 raft_consensus.cc:493] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.881719  9074 raft_consensus.cc:359] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.882531  9074 raft_consensus.cc:385] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.882959  9074 raft_consensus.cc:740] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.882424  9083 raft_consensus.cc:515] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.883663  9074 consensus_queue.cc:260] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.886683  9083 leader_election.cc:290] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:03.890383  9076 tablet_bootstrap.cc:492] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:03.890864  9076 ts_tablet_manager.cc:1403] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.009s
I20260709 12:22:03.894441  9076 raft_consensus.cc:359] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.895145  9076 raft_consensus.cc:385] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.895427  9076 raft_consensus.cc:740] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.896117  9076 consensus_queue.cc:260] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.898161  9076 ts_tablet_manager.cc:1434] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:22:03.898759  9075 tablet_bootstrap.cc:492] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:03.899719  9075 ts_tablet_manager.cc:1403] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.053s	user 0.014s	sys 0.019s
I20260709 12:22:03.901018  9076 tablet_bootstrap.cc:492] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:03.904160  9075 raft_consensus.cc:359] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.904994  9075 raft_consensus.cc:385] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.909308  9074 ts_tablet_manager.cc:1434] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.004s
I20260709 12:22:03.910521  9074 tablet_bootstrap.cc:492] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:03.905328  9075 raft_consensus.cc:740] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
W20260709 12:22:03.913683  9056 tablet.cc:2406] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:03.915719  9075 consensus_queue.cc:260] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.922254  9080 raft_consensus.cc:493] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.922767  9080 raft_consensus.cc:515] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.924362  9076 tablet_bootstrap.cc:654] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.925336  9080 leader_election.cc:290] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:03.922224  9075 ts_tablet_manager.cc:1434] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.002s
I20260709 12:22:03.931056  9075 tablet_bootstrap.cc:492] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
I20260709 12:22:03.951949  9074 tablet_bootstrap.cc:654] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.966315  9076 tablet_bootstrap.cc:492] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:03.967283  9076 ts_tablet_manager.cc:1403] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.066s	user 0.011s	sys 0.005s
I20260709 12:22:03.969157  9075 tablet_bootstrap.cc:654] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:03.975015  9089 raft_consensus.cc:493] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:03.975540  9089 raft_consensus.cc:515] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.974694  9076 raft_consensus.cc:359] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.980801  9076 raft_consensus.cc:385] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:03.981468  9076 raft_consensus.cc:740] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:03.982200  9029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:03.982507  9089 leader_election.cc:290] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:03.985630  9076 consensus_queue.cc:260] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:03.987406  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de765fd04334493963ed7f7c894b468" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:03.987550  8955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013" is_pre_election: true
W20260709 12:22:03.995410  8842 leader_election.cc:343] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:03.996953  8843 leader_election.cc:343] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:03.992996  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de765fd04334493963ed7f7c894b468" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013" is_pre_election: true
I20260709 12:22:03.999675  8954 raft_consensus.cc:2468] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 0.
I20260709 12:22:04.000526  9076 ts_tablet_manager.cc:1434] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.003s
I20260709 12:22:04.001509  9076 tablet_bootstrap.cc:492] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:04.003567  9074 tablet_bootstrap.cc:492] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:03.993040  9028 raft_consensus.cc:2468] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 0.
I20260709 12:22:04.005699  8842 leader_election.cc:304] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:04.006699  9089 raft_consensus.cc:2804] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.007092  9089 raft_consensus.cc:493] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.007606  9074 ts_tablet_manager.cc:1403] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.097s	user 0.025s	sys 0.008s
I20260709 12:22:04.007607  9089 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.008755  9075 tablet_bootstrap.cc:492] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:04.010512  9075 ts_tablet_manager.cc:1403] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.081s	user 0.009s	sys 0.014s
I20260709 12:22:04.010478  8843 leader_election.cc:304] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 929101e20d6944f3abf2b312b22616c4; no voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e
I20260709 12:22:04.011638  9083 raft_consensus.cc:2749] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:04.013384  9081 raft_consensus.cc:493] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.014655  9081 raft_consensus.cc:515] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.016103  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.017262  9028 raft_consensus.cc:2468] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.019579  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
I20260709 12:22:04.020064  9081 leader_election.cc:290] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:04.022570  8918 leader_election.cc:304] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
I20260709 12:22:04.025653  9080 raft_consensus.cc:2804] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.026113  9080 raft_consensus.cc:493] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.026515  9080 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.026186  9075 raft_consensus.cc:359] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.022372  9076 tablet_bootstrap.cc:654] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
W20260709 12:22:04.029770  8921 leader_election.cc:343] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.033753  9089 raft_consensus.cc:515] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.026983  9075 raft_consensus.cc:385] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.038208  9080 raft_consensus.cc:515] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.044940  9075 raft_consensus.cc:740] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.043360  9074 raft_consensus.cc:359] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.048571  9074 raft_consensus.cc:385] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.048897  9074 raft_consensus.cc:740] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.049230  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:04.049638  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:04.049576  9074 consensus_queue.cc:260] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.050264  9089 leader_election.cc:290] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 election: Requested vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.048585  9080 leader_election.cc:290] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
W20260709 12:22:04.050906  8921 leader_election.cc:343] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.051496  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de765fd04334493963ed7f7c894b468" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:04.052008  8954 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.047288  9075 consensus_queue.cc:260] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.052548  9029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de765fd04334493963ed7f7c894b468" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:04.053171  9029 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.053844  9075 ts_tablet_manager.cc:1434] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.043s	user 0.000s	sys 0.006s
I20260709 12:22:04.054792  9075 tablet_bootstrap.cc:492] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
I20260709 12:22:04.057214  9074 ts_tablet_manager.cc:1434] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.049s	user 0.000s	sys 0.007s
I20260709 12:22:04.050231  9028 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.058143  9074 tablet_bootstrap.cc:492] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:04.061622  9075 tablet_bootstrap.cc:654] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.069649  8954 raft_consensus.cc:2468] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 1.
I20260709 12:22:04.074465  9074 tablet_bootstrap.cc:654] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.075461  9029 raft_consensus.cc:2468] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 1.
I20260709 12:22:04.075570  9080 raft_consensus.cc:493] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.075799  8843 leader_election.cc:304] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:04.072932  9028 raft_consensus.cc:2468] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.076561  9083 raft_consensus.cc:2804] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.077610  8918 leader_election.cc:304] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:04.076042  9080 raft_consensus.cc:515] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.081212  9080 leader_election.cc:290] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.081714  9080 raft_consensus.cc:2804] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.081802  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
I20260709 12:22:04.082183  9080 raft_consensus.cc:697] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:04.082430  8880 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.083607  8921 leader_election.cc:304] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:04.083632  9080 consensus_queue.cc:237] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.095404  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013" is_pre_election: true
I20260709 12:22:04.099502  9076 tablet_bootstrap.cc:492] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:04.099916  9075 tablet_bootstrap.cc:492] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:04.099961  9106 raft_consensus.cc:2804] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.104015  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.110296  9075 ts_tablet_manager.cc:1403] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.056s	user 0.016s	sys 0.007s
I20260709 12:22:04.108142  8772 catalog_manager.cc:5697] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:04.110285  9106 raft_consensus.cc:493] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.106655  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
I20260709 12:22:04.110291  9076 ts_tablet_manager.cc:1403] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.109s	user 0.015s	sys 0.006s
I20260709 12:22:04.110467  9028 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.125028  9075 raft_consensus.cc:359] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.127235  9105 raft_consensus.cc:493] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.127705  9075 raft_consensus.cc:385] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.127789  9105 raft_consensus.cc:515] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.128140  9075 raft_consensus.cc:740] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.126852  9083 raft_consensus.cc:697] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 1 LEADER]: Becoming Leader. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
W20260709 12:22:04.127118  8995 leader_election.cc:343] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.125636  9106 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.130839  8995 leader_election.cc:304] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:04.129387  9076 raft_consensus.cc:359] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.133220  9076 raft_consensus.cc:385] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.133520  9076 raft_consensus.cc:740] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.134083  9081 raft_consensus.cc:2749] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:04.134240  9076 consensus_queue.cc:260] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.136443  9076 ts_tablet_manager.cc:1434] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:22:04.131142  9075 consensus_queue.cc:260] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.140710  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.141304  9028 raft_consensus.cc:2468] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.142107  9074 tablet_bootstrap.cc:492] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:04.142555  9074 ts_tablet_manager.cc:1403] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.085s	user 0.008s	sys 0.003s
I20260709 12:22:04.142450  8918 leader_election.cc:304] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
I20260709 12:22:04.139653  9105 leader_election.cc:290] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.143534  9080 raft_consensus.cc:2804] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.144791  9080 raft_consensus.cc:493] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.143752  9075 ts_tablet_manager.cc:1434] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 12:22:04.144234  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
I20260709 12:22:04.145366  9080 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.146489  9075 tablet_bootstrap.cc:492] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
W20260709 12:22:04.146783  8921 leader_election.cc:343] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.145380  9076 tablet_bootstrap.cc:492] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:04.148149  9083 consensus_queue.cc:237] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.149152  9074 raft_consensus.cc:359] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.150068  9074 raft_consensus.cc:385] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.150374  9074 raft_consensus.cc:740] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.151108  9074 consensus_queue.cc:260] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.153419  9074 ts_tablet_manager.cc:1434] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:22:04.154322  9074 tablet_bootstrap.cc:492] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:04.156674  9080 raft_consensus.cc:515] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.157933  9106 raft_consensus.cc:515] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.169924  9076 tablet_bootstrap.cc:654] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.169711  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:04.171449  8880 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.174686  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:04.175745  9028 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.174335  9080 leader_election.cc:290] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.181408  8879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:04.183557  9029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
W20260709 12:22:04.184070  8921 leader_election.cc:343] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.184396  9029 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.187801  9028 raft_consensus.cc:2468] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.189291  8918 leader_election.cc:304] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:04.190263  9080 raft_consensus.cc:2804] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.181826  9106 leader_election.cc:290] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.190570  9029 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.190740  9080 raft_consensus.cc:697] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:04.192216  8918 leader_election.cc:304] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
I20260709 12:22:04.192111  9080 consensus_queue.cc:237] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.192996  9106 raft_consensus.cc:2804] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.193817  9106 raft_consensus.cc:697] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:04.194795  9106 consensus_queue.cc:237] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.199690  9075 tablet_bootstrap.cc:654] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.208632  8774 catalog_manager.cc:5697] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 929101e20d6944f3abf2b312b22616c4 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "929101e20d6944f3abf2b312b22616c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:04.212862  9074 tablet_bootstrap.cc:654] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.214645  8880 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.252146  9074 tablet_bootstrap.cc:492] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:04.252673  9074 ts_tablet_manager.cc:1403] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.099s	user 0.008s	sys 0.004s
I20260709 12:22:04.253455  8772 catalog_manager.cc:5697] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:04.255666  9074 raft_consensus.cc:359] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.255458  8772 catalog_manager.cc:5697] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:04.256385  9074 raft_consensus.cc:385] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.256728  9074 raft_consensus.cc:740] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.257439  9074 consensus_queue.cc:260] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.263832  9075 tablet_bootstrap.cc:492] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:04.264442  9075 ts_tablet_manager.cc:1403] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.118s	user 0.012s	sys 0.010s
I20260709 12:22:04.267589  9076 tablet_bootstrap.cc:492] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:04.268025  9076 ts_tablet_manager.cc:1403] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.123s	user 0.017s	sys 0.031s
I20260709 12:22:04.269615  9075 raft_consensus.cc:359] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.270335  9075 raft_consensus.cc:385] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.270694  9075 raft_consensus.cc:740] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.271474  9075 consensus_queue.cc:260] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.270501  9076 raft_consensus.cc:359] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.272105  9074 ts_tablet_manager.cc:1434] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:22:04.272820  9076 raft_consensus.cc:385] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.273229  9076 raft_consensus.cc:740] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.273679  9074 tablet_bootstrap.cc:492] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:04.274040  9076 consensus_queue.cc:260] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.275274  9075 ts_tablet_manager.cc:1434] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 12:22:04.276266  9075 tablet_bootstrap.cc:492] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
I20260709 12:22:04.276588  9076 ts_tablet_manager.cc:1434] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:04.277518  9076 tablet_bootstrap.cc:492] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:04.279844  9080 raft_consensus.cc:493] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.280298  9080 raft_consensus.cc:515] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.282318  9080 leader_election.cc:290] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.282980  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6bb8b14ec80438c8982e8d19952ae13" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
I20260709 12:22:04.283679  8880 raft_consensus.cc:2468] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.283701  9029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6bb8b14ec80438c8982e8d19952ae13" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.284675  9029 raft_consensus.cc:2468] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.284912  8921 leader_election.cc:304] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:04.285656  9080 raft_consensus.cc:2804] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.286007  9080 raft_consensus.cc:493] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.286355  9080 raft_consensus.cc:3060] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.292819  9080 raft_consensus.cc:515] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.294808  9080 leader_election.cc:290] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.296875  9029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6bb8b14ec80438c8982e8d19952ae13" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:04.297392  9029 raft_consensus.cc:3060] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.300076  9076 tablet_bootstrap.cc:654] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.302996  9074 tablet_bootstrap.cc:654] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.303719  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6bb8b14ec80438c8982e8d19952ae13" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:04.304308  8880 raft_consensus.cc:3060] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.313025  8880 raft_consensus.cc:2468] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.316399  9076 tablet_bootstrap.cc:492] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:04.316977  9029 raft_consensus.cc:2468] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.317243  9076 ts_tablet_manager.cc:1403] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.040s	user 0.002s	sys 0.008s
I20260709 12:22:04.318161  8918 leader_election.cc:304] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
I20260709 12:22:04.321221  9080 raft_consensus.cc:2804] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.321684  9080 raft_consensus.cc:697] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:04.322432  9080 consensus_queue.cc:237] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.322353  9076 raft_consensus.cc:359] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.323773  9076 raft_consensus.cc:385] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.324223  9076 raft_consensus.cc:740] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.325286  9076 consensus_queue.cc:260] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.327891  9076 ts_tablet_manager.cc:1434] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:22:04.328958  9076 tablet_bootstrap.cc:492] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:04.316730  9075 tablet_bootstrap.cc:654] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.337455  9076 tablet_bootstrap.cc:654] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.347208  8773 catalog_manager.cc:5697] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:04.353410  9075 tablet_bootstrap.cc:492] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:04.353847  9075 ts_tablet_manager.cc:1403] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.078s	user 0.007s	sys 0.013s
I20260709 12:22:04.357259  9075 raft_consensus.cc:359] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.358112  9075 raft_consensus.cc:385] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.360575  9074 tablet_bootstrap.cc:492] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:04.361166  9074 ts_tablet_manager.cc:1403] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.014s
I20260709 12:22:04.358455  9075 raft_consensus.cc:740] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.364744  9074 raft_consensus.cc:359] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.364929  9075 consensus_queue.cc:260] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.365585  9074 raft_consensus.cc:385] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.365918  9074 raft_consensus.cc:740] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.365271  9076 tablet_bootstrap.cc:492] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:04.366951  9076 ts_tablet_manager.cc:1403] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.013s
I20260709 12:22:04.366663  9074 consensus_queue.cc:260] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.369081  9074 ts_tablet_manager.cc:1434] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:22:04.369938  9074 tablet_bootstrap.cc:492] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:04.372969  9075 ts_tablet_manager.cc:1434] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:22:04.376241  9075 tablet_bootstrap.cc:492] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013: Bootstrap starting.
I20260709 12:22:04.376137  9080 raft_consensus.cc:493] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.376569  9074 tablet_bootstrap.cc:654] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.376720  9080 raft_consensus.cc:515] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.381153  9029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.382102  9029 raft_consensus.cc:2468] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.382161  9076 raft_consensus.cc:359] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.382465  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
I20260709 12:22:04.382889  9076 raft_consensus.cc:385] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.383177  9076 raft_consensus.cc:740] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.380530  9080 leader_election.cc:290] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.383869  9076 consensus_queue.cc:260] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.383354  8918 leader_election.cc:304] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
W20260709 12:22:04.385185  8921 leader_election.cc:343] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.386025  9076 ts_tablet_manager.cc:1434] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 12:22:04.386765  9076 tablet_bootstrap.cc:492] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e: Bootstrap starting.
I20260709 12:22:04.387627  9080 raft_consensus.cc:2804] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.388963  9075 tablet_bootstrap.cc:654] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.389379  9080 raft_consensus.cc:493] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.391744  9080 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.400436  9075 tablet_bootstrap.cc:492] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013: No bootstrap required, opened a new log
I20260709 12:22:04.404074  9076 tablet_bootstrap.cc:654] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.404703  9075 ts_tablet_manager.cc:1403] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013: Time spent bootstrapping tablet: real 0.029s	user 0.006s	sys 0.008s
I20260709 12:22:04.406517  9080 raft_consensus.cc:515] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.409757  9080 leader_election.cc:290] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.411547  9075 raft_consensus.cc:359] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.412202  9075 raft_consensus.cc:385] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.412541  9075 raft_consensus.cc:740] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.412148  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:04.413183  9029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:04.413861  9029 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.413385  9075 consensus_queue.cc:260] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
W20260709 12:22:04.414182  8921 leader_election.cc:343] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.416273  9075 ts_tablet_manager.cc:1434] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:22:04.419881  9029 raft_consensus.cc:2468] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.421001  8918 leader_election.cc:304] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:04.421650  9080 raft_consensus.cc:2804] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.422115  9080 raft_consensus.cc:697] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:04.410471  9074 tablet_bootstrap.cc:492] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:04.422806  9074 ts_tablet_manager.cc:1403] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.007s
I20260709 12:22:04.423192  9080 consensus_queue.cc:237] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.423997  9076 tablet_bootstrap.cc:492] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e: No bootstrap required, opened a new log
I20260709 12:22:04.424401  9076 ts_tablet_manager.cc:1403] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent bootstrapping tablet: real 0.038s	user 0.016s	sys 0.000s
I20260709 12:22:04.426003  9074 raft_consensus.cc:359] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.426723  9074 raft_consensus.cc:385] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.427018  9074 raft_consensus.cc:740] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.427526  9076 raft_consensus.cc:359] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.428290  9076 raft_consensus.cc:385] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.428599  9076 raft_consensus.cc:740] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.429243  9076 consensus_queue.cc:260] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.431694  9076 ts_tablet_manager.cc:1434] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:22:04.432827  8773 catalog_manager.cc:5697] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:04.434868  9074 consensus_queue.cc:260] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.442449  9074 ts_tablet_manager.cc:1434] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 12:22:04.443581  9074 tablet_bootstrap.cc:492] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:04.451572  9074 tablet_bootstrap.cc:654] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.452235  9081 raft_consensus.cc:493] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.452625  9081 raft_consensus.cc:515] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.454289  9081 leader_election.cc:290] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:04.454995  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
W20260709 12:22:04.456497  8995 leader_election.cc:343] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.456698  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013" is_pre_election: true
I20260709 12:22:04.457232  8954 raft_consensus.cc:2468] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 0.
I20260709 12:22:04.458245  8993 leader_election.cc:304] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:04.459055  9081 raft_consensus.cc:2804] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.459393  9081 raft_consensus.cc:493] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.459733  9081 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.459957  9074 tablet_bootstrap.cc:492] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:04.460361  9074 ts_tablet_manager.cc:1403] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.017s	user 0.007s	sys 0.004s
I20260709 12:22:04.462956  9074 raft_consensus.cc:359] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.463693  9074 raft_consensus.cc:385] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.464021  9074 raft_consensus.cc:740] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.464658  9074 consensus_queue.cc:260] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.466615  9074 ts_tablet_manager.cc:1434] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:22:04.467419  9074 tablet_bootstrap.cc:492] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4: Bootstrap starting.
I20260709 12:22:04.467860  9081 raft_consensus.cc:515] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.470356  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4"
W20260709 12:22:04.472146  8995 leader_election.cc:343] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:04.473165  9074 tablet_bootstrap.cc:654] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:04.474046  9081 leader_election.cc:290] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:04.474249  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:04.474854  8954 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.479724  8954 raft_consensus.cc:2468] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 1.
I20260709 12:22:04.480964  8993 leader_election.cc:304] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:04.481734  9081 raft_consensus.cc:2804] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.482085  9080 raft_consensus.cc:493] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.483016  9083 raft_consensus.cc:493] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.483541  9074 tablet_bootstrap.cc:492] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4: No bootstrap required, opened a new log
I20260709 12:22:04.483426  9083 raft_consensus.cc:515] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.484055  9074 ts_tablet_manager.cc:1403] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 12:22:04.485363  9083 leader_election.cc:290] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.487462  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013" is_pre_election: true
I20260709 12:22:04.487300  9074 raft_consensus.cc:359] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.488157  9074 raft_consensus.cc:385] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:04.488560  9074 raft_consensus.cc:740] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Initialized, Role: FOLLOWER
I20260709 12:22:04.489375  9074 consensus_queue.cc:260] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.490731  9104 raft_consensus.cc:493] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.491307  9104 raft_consensus.cc:515] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.482502  9080 raft_consensus.cc:515] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.493609  9104 leader_election.cc:290] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.494329  9029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.495096  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
I20260709 12:22:04.495517  9029 raft_consensus.cc:2393] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 929101e20d6944f3abf2b312b22616c4 in current term 1: Already voted for candidate 34e1340365c14e6c8d1954a320d9d013 in this term.
I20260709 12:22:04.495678  8880 raft_consensus.cc:2468] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.497298  9074 ts_tablet_manager.cc:1434] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:22:04.496620  9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.497664  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013" is_pre_election: true
I20260709 12:22:04.498092  9027 raft_consensus.cc:2468] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 0.
I20260709 12:22:04.498369  8921 leader_election.cc:304] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:04.496912  8842 leader_election.cc:304] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 929101e20d6944f3abf2b312b22616c4; no voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e
I20260709 12:22:04.499187  9106 raft_consensus.cc:2804] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.499534  9106 raft_consensus.cc:493] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.499758  9104 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.499975  9106 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.502005  9081 raft_consensus.cc:697] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 LEADER]: Becoming Leader. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:04.496623  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.502740  9080 leader_election.cc:290] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.503717  9028 raft_consensus.cc:2393] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 929101e20d6944f3abf2b312b22616c4 in current term 1: Already voted for candidate 34e1340365c14e6c8d1954a320d9d013 in this term.
I20260709 12:22:04.503356  9081 consensus_queue.cc:237] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.506839  9104 raft_consensus.cc:2749] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:04.511318  8842 leader_election.cc:304] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 929101e20d6944f3abf2b312b22616c4; no voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e
I20260709 12:22:04.514971  9129 raft_consensus.cc:493] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.515676  9104 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.515411  9129 raft_consensus.cc:515] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.518111  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca4b422654d4f0cbe4fb6cce4abb820" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4" is_pre_election: true
I20260709 12:22:04.518733  8880 raft_consensus.cc:2468] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 0.
I20260709 12:22:04.519167  9106 raft_consensus.cc:515] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.519959  8995 leader_election.cc:304] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:04.521234  9129 leader_election.cc:290] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:04.521701  9129 raft_consensus.cc:2804] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:04.522022  9129 raft_consensus.cc:493] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:04.522343  9129 raft_consensus.cc:3060] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.522274  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca4b422654d4f0cbe4fb6cce4abb820" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013" is_pre_election: true
I20260709 12:22:04.522626  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:04.522991  8954 raft_consensus.cc:2468] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 0.
I20260709 12:22:04.523160  8880 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.529858  9080 consensus_queue.cc:1048] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:04.531214  9106 leader_election.cc:290] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.533874  8880 raft_consensus.cc:2468] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.534528  9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:04.535111  9027 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.536316  8921 leader_election.cc:304] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:04.537961  9106 raft_consensus.cc:2804] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.538806  9106 raft_consensus.cc:697] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:04.539880  9106 consensus_queue.cc:237] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.544435  9027 raft_consensus.cc:2468] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 1.
I20260709 12:22:04.549962  8772 catalog_manager.cc:5697] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195). New cstate: current_term: 1 leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:04.556728  9129 raft_consensus.cc:515] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.559405  8880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca4b422654d4f0cbe4fb6cce4abb820" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:04.559993  8880 raft_consensus.cc:3060] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:22:04.560542  8824 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:04.565833  9104 raft_consensus.cc:2749] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:04.569334  8774 catalog_manager.cc:5697] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 0 to 1, leader changed from <none> to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:04.570746  8880 raft_consensus.cc:2468] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 1.
I20260709 12:22:04.571929  8995 leader_election.cc:304] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:04.572746  9081 raft_consensus.cc:2804] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:04.573228  9081 raft_consensus.cc:697] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 LEADER]: Becoming Leader. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:04.574019  9081 consensus_queue.cc:237] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.576787  9129 leader_election.cc:290] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 1 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:04.580152  9105 consensus_queue.cc:1048] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:04.580513  8879 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.582798  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eca4b422654d4f0cbe4fb6cce4abb820" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:04.583352  8954 raft_consensus.cc:3060] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.589665  8954 raft_consensus.cc:2468] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 1.
I20260709 12:22:04.591037  9104 raft_consensus.cc:493] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:04.591485  9104 raft_consensus.cc:515] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:04.593479  9104 leader_election.cc:290] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:04.593817  9080 consensus_queue.cc:1048] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:04.594995  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" is_pre_election: true
I20260709 12:22:04.597187  8954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34e1340365c14e6c8d1954a320d9d013" is_pre_election: true
I20260709 12:22:04.597992  8954 raft_consensus.cc:2393] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 929101e20d6944f3abf2b312b22616c4 in current term 1: Already voted for candidate 79e3c530a0414a60b7d1c0d914a86a7e in this term.
I20260709 12:22:04.599095  8843 leader_election.cc:304] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 929101e20d6944f3abf2b312b22616c4; no voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e
I20260709 12:22:04.600068  9104 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:04.615978  9104 raft_consensus.cc:2749] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:04.622215  9080 consensus_queue.cc:1048] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:04.627948  9106 consensus_queue.cc:1048] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:04.624092  8772 catalog_manager.cc:5697] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195). New cstate: current_term: 1 leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:04.642239  9080 consensus_queue.cc:1048] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:04.652158  9083 consensus_queue.cc:1048] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:04.688580  9104 consensus_queue.cc:1048] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:04.730351  9106 consensus_queue.cc:1048] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:04.742523  9080 consensus_queue.cc:1048] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:04.827585  8954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:42234
I20260709 12:22:04.828171  8954 raft_consensus.cc:606] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Received request to transfer leadership to TS 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:04.833611  8954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:42234
I20260709 12:22:04.834170  8954 raft_consensus.cc:606] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Received request to transfer leadership to TS 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:05.133322  9081 consensus_queue.cc:1048] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:05.133461  9080 consensus_queue.cc:1048] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:05.144940  9081 consensus_queue.cc:1048] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:05.145582  9106 consensus_queue.cc:1048] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:05.151913  9081 consensus_queue.cc:1048] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:05.153400  9080 consensus_queue.cc:1048] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:05.197973  9155 consensus_queue.cc:1048] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:05.225109  9155 raft_consensus.cc:993] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:05.225591  9155 raft_consensus.cc:1081] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Signalling peer 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:05.231128  8879 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:38696
I20260709 12:22:05.231765  9155 raft_consensus.cc:993] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:05.232191  9155 raft_consensus.cc:1081] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Signalling peer 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:05.231741  8879 raft_consensus.cc:493] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.233459  8879 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.237854  8955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:42234
I20260709 12:22:05.238430  8955 raft_consensus.cc:606] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Received request to transfer leadership to TS 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:05.255342  9167 consensus_queue.cc:1048] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:05.248150  8879 raft_consensus.cc:515] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.287154  8877 raft_consensus.cc:1240] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Rejecting Update request from peer 34e1340365c14e6c8d1954a320d9d013 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:05.239516  8878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:38696
I20260709 12:22:05.287969  8878 raft_consensus.cc:493] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.290086  8878 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.291124  8953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:05.289656  8879 leader_election.cc:290] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 2 election: Requested vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:05.291775  8953 raft_consensus.cc:3055] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.292104  8953 raft_consensus.cc:740] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:05.292894  8953 consensus_queue.cc:260] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.293880  8953 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.295562  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:05.296100  9028 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.302492  9028 raft_consensus.cc:2468] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 2.
I20260709 12:22:05.303755  8842 leader_election.cc:304] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:05.305244  9104 raft_consensus.cc:2804] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.305585  8955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:42234
I20260709 12:22:05.305747  9104 raft_consensus.cc:697] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Becoming Leader. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:05.306296  8955 raft_consensus.cc:606] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Received request to transfer leadership to TS 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:05.306787  9104 consensus_queue.cc:237] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.324132  8878 raft_consensus.cc:515] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.326772  8878 leader_election.cc:290] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 2 election: Requested vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:05.327503  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:05.328169  9028 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.329568  8955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:05.330129  8955 raft_consensus.cc:3055] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.330458  8955 raft_consensus.cc:740] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:05.334321  8955 consensus_queue.cc:260] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.335258  9028 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 2.
I20260709 12:22:05.335601  8955 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.336336  8842 leader_election.cc:304] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:05.337123  9104 raft_consensus.cc:2804] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.337697  9104 raft_consensus.cc:697] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Becoming Leader. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:05.337893  8878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8de765fd04334493963ed7f7c894b468"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
mode: GRACEFUL
new_leader_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:38716
I20260709 12:22:05.338474  8878 raft_consensus.cc:606] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 1 LEADER]: Received request to transfer leadership to TS 34e1340365c14e6c8d1954a320d9d013
I20260709 12:22:05.338614  9104 consensus_queue.cc:237] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.340308  8772 catalog_manager.cc:5697] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 reported cstate change: term changed from 1 to 2, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 929101e20d6944f3abf2b312b22616c4 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "929101e20d6944f3abf2b312b22616c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.350632  8953 raft_consensus.cc:2468] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 2.
I20260709 12:22:05.355034  8955 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 2.
I20260709 12:22:05.368875  8772 catalog_manager.cc:5697] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 reported cstate change: term changed from 1 to 2, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 929101e20d6944f3abf2b312b22616c4 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "929101e20d6944f3abf2b312b22616c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.382508  9028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45"
dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
mode: GRACEFUL
new_leader_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:34368
I20260709 12:22:05.383134  9028 raft_consensus.cc:606] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 LEADER]: Received request to transfer leadership to TS 34e1340365c14e6c8d1954a320d9d013
I20260709 12:22:05.688887  9028 raft_consensus.cc:1275] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.690450  9104 consensus_queue.cc:1048] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:05.702116  8955 raft_consensus.cc:1275] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.704977  9083 consensus_queue.cc:1048] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:05.711702  9163 raft_consensus.cc:993] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:05.712308  9162 raft_consensus.cc:1081] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Signalling peer 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:05.714802  8878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:38696
I20260709 12:22:05.715574  8878 raft_consensus.cc:493] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.716044  8878 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.724381  8878 raft_consensus.cc:515] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.726969  8955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:05.727506  8955 raft_consensus.cc:3055] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.727826  8955 raft_consensus.cc:740] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:05.728705  8955 consensus_queue.cc:260] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.729803  8955 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.731165  8878 leader_election.cc:290] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 2 election: Requested vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:05.731419  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:05.732424  9028 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.736166  8955 raft_consensus.cc:2468] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 2.
I20260709 12:22:05.737228  8843 leader_election.cc:304] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:05.738010  9104 raft_consensus.cc:2804] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.738722  9028 raft_consensus.cc:2468] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 2.
I20260709 12:22:05.738456  9104 raft_consensus.cc:697] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Becoming Leader. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:05.740233  9104 consensus_queue.cc:237] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.746865  8955 raft_consensus.cc:1275] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.748111  9104 consensus_queue.cc:1048] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:05.748325  8772 catalog_manager.cc:5697] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 reported cstate change: term changed from 1 to 2, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 929101e20d6944f3abf2b312b22616c4 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "929101e20d6944f3abf2b312b22616c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.761209  9028 raft_consensus.cc:1275] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:05.762692  9180 consensus_queue.cc:1048] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:05.770068  9104 raft_consensus.cc:993] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4: : Instructing follower 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:05.770803  9083 raft_consensus.cc:1081] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 1 LEADER]: Signalling peer 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:05.772342  8953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8de765fd04334493963ed7f7c894b468"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:42206
I20260709 12:22:05.772887  8953 raft_consensus.cc:493] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.773160  8953 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.779407  8953 raft_consensus.cc:515] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.781530  8953 leader_election.cc:290] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 2 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:05.782719  9027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de765fd04334493963ed7f7c894b468" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:05.783272  9027 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.785851  8878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de765fd04334493963ed7f7c894b468" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:05.786396  8878 raft_consensus.cc:3055] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.786742  8878 raft_consensus.cc:740] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:05.787467  8878 consensus_queue.cc:260] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.788416  8878 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.788980  9027 raft_consensus.cc:2468] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 2.
I20260709 12:22:05.789909  9028 raft_consensus.cc:1240] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Rejecting Update request from peer 929101e20d6944f3abf2b312b22616c4 for earlier term 1. Current term is 2. Ops: []
I20260709 12:22:05.790134  8918 leader_election.cc:304] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
I20260709 12:22:05.790982  9162 raft_consensus.cc:2804] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.791427  9162 raft_consensus.cc:697] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:05.792105  9162 consensus_queue.cc:237] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.799942  8878 raft_consensus.cc:2468] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 2.
I20260709 12:22:05.801383  9162 raft_consensus.cc:993] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:05.801823  9163 raft_consensus.cc:1081] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Signalling peer 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:05.801966  8772 catalog_manager.cc:5697] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 1 to 2, leader changed from 929101e20d6944f3abf2b312b22616c4 (127.8.137.193) to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.803179  8878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:38696
I20260709 12:22:05.803609  8878 raft_consensus.cc:493] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.803889  8878 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.808977  8878 raft_consensus.cc:515] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.810546  8878 leader_election.cc:290] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 2 election: Requested vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:05.811226  8953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:05.811595  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:05.811745  8953 raft_consensus.cc:3055] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.812052  8953 raft_consensus.cc:740] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:05.812083  9028 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.812741  8953 consensus_queue.cc:260] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.813577  8953 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.816566  9028 raft_consensus.cc:2468] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 2.
I20260709 12:22:05.817422  8842 leader_election.cc:304] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:05.818071  9104 raft_consensus.cc:2804] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.818066  8953 raft_consensus.cc:2468] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 2.
I20260709 12:22:05.818722  9104 raft_consensus.cc:697] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Becoming Leader. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:05.819456  9104 consensus_queue.cc:237] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.824890  8772 catalog_manager.cc:5697] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 reported cstate change: term changed from 1 to 2, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 929101e20d6944f3abf2b312b22616c4 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "929101e20d6944f3abf2b312b22616c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:05.890319  9167 raft_consensus.cc:993] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e: : Instructing follower 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:05.890815  9170 raft_consensus.cc:1081] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 LEADER]: Signalling peer 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:05.892566  8953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:42222
I20260709 12:22:05.893263  8953 raft_consensus.cc:493] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:05.893571  8953 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.898171  8953 raft_consensus.cc:515] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.899813  8953 leader_election.cc:290] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 2 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:05.900601  8878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:05.900770  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:05.901180  8878 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.901268  9028 raft_consensus.cc:3055] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:05.901608  9028 raft_consensus.cc:740] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:05.902386  9028 consensus_queue.cc:260] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.903554  9028 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:05.907704  8878 raft_consensus.cc:2468] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 2.
I20260709 12:22:05.908974  8921 leader_election.cc:304] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:05.909638  9028 raft_consensus.cc:2468] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 2.
I20260709 12:22:05.909651  9163 raft_consensus.cc:2804] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:05.910193  9163 raft_consensus.cc:697] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:05.910948  9163 consensus_queue.cc:237] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:05.916481  8772 catalog_manager.cc:5697] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 1 to 2, leader changed from 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195) to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:06.217026  9028 raft_consensus.cc:1275] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.218165  9163 consensus_queue.cc:1048] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:06.226696  8878 raft_consensus.cc:1275] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.228379  9162 consensus_queue.cc:1048] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:22:06.241714  8976 debug-util.cc:398] Leaking SignalData structure 0x7b080012f060 after lost signal to thread 8746
W20260709 12:22:06.242389  8976 debug-util.cc:398] Leaking SignalData structure 0x7b080012f740 after lost signal to thread 8809
W20260709 12:22:06.243979  8976 debug-util.cc:398] Leaking SignalData structure 0x7b08001f7d20 after lost signal to thread 8979
I20260709 12:22:06.259985  8953 raft_consensus.cc:1275] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.261371  9104 consensus_queue.cc:1048] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:06.273680  9028 raft_consensus.cc:1275] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.279770  9104 consensus_queue.cc:1048] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:06.370476  8953 raft_consensus.cc:1275] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.371753  9187 consensus_queue.cc:1048] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:06.386153  9028 raft_consensus.cc:1275] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.387565  9187 consensus_queue.cc:1048] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:06.433832  8878 raft_consensus.cc:1275] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.435086  9162 consensus_queue.cc:1048] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:06.445823  9028 raft_consensus.cc:1275] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:06.448417  9162 consensus_queue.cc:1048] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:08.417119  8743 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:22:08.417896  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:08.437206  8878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
mode: GRACEFUL
new_leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
 from {username='slave'} at 127.0.0.1:44666
I20260709 12:22:08.437714  8878 raft_consensus.cc:606] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Received request to transfer leadership to TS 79e3c530a0414a60b7d1c0d914a86a7e
I20260709 12:22:08.449348  8953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
 from {username='slave'} at 127.0.0.1:45460
I20260709 12:22:08.450022  8953 raft_consensus.cc:606] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Received request to transfer leadership to TS 79e3c530a0414a60b7d1c0d914a86a7e
I20260709 12:22:08.451495  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:08.452142  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:08.659260  9196 raft_consensus.cc:993] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4: : Instructing follower 79e3c530a0414a60b7d1c0d914a86a7e to start an election
I20260709 12:22:08.659720  9187 raft_consensus.cc:1081] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Signalling peer 79e3c530a0414a60b7d1c0d914a86a7e to start an election
I20260709 12:22:08.661342  9027 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57"
dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
 from {username='slave'} at 127.0.0.1:34342
I20260709 12:22:08.661821  9027 raft_consensus.cc:493] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.662137  9027 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.666695  9027 raft_consensus.cc:515] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:08.668843  9027 leader_election.cc:290] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 3 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:08.669353  8953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:08.669857  8878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:08.669977  8953 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.670497  8878 raft_consensus.cc:3055] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:08.670858  8878 raft_consensus.cc:740] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:08.671543  8878 consensus_queue.cc:260] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:08.672740  8878 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:08.677069  8878 raft_consensus.cc:2468] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 3.
I20260709 12:22:08.677438  8953 raft_consensus.cc:2468] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 3.
I20260709 12:22:08.678109  8995 leader_election.cc:304] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:08.678896  9170 raft_consensus.cc:2804] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:08.680197  9170 raft_consensus.cc:697] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 LEADER]: Becoming Leader. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:08.681110  9170 consensus_queue.cc:237] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:08.689312  8775 catalog_manager.cc:5697] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e reported cstate change: term changed from 2 to 3, leader changed from 929101e20d6944f3abf2b312b22616c4 (127.8.137.193) to 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195). New cstate: current_term: 3 leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:08.819365  9204 raft_consensus.cc:993] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 79e3c530a0414a60b7d1c0d914a86a7e to start an election
I20260709 12:22:08.819995  9203 raft_consensus.cc:1081] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Signalling peer 79e3c530a0414a60b7d1c0d914a86a7e to start an election
I20260709 12:22:08.821491  9028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94"
dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
 from {username='slave'} at 127.0.0.1:34356
I20260709 12:22:08.822096  9028 raft_consensus.cc:493] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:08.822464  9028 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.827380  9028 raft_consensus.cc:515] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:08.829113  9028 leader_election.cc:290] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 2 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:08.829866  8878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:08.830256  8953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:08.830453  8878 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.830861  8953 raft_consensus.cc:3055] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:08.831190  8953 raft_consensus.cc:740] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:08.831961  8953 consensus_queue.cc:260] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:08.833114  8953 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:08.836937  8878 raft_consensus.cc:2468] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 2.
I20260709 12:22:08.838161  8995 leader_election.cc:304] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:08.839075  9170 raft_consensus.cc:2804] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:08.839542  9170 raft_consensus.cc:697] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 LEADER]: Becoming Leader. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:08.839779  8953 raft_consensus.cc:2468] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 2.
I20260709 12:22:08.840415  9170 consensus_queue.cc:237] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:08.847903  8775 catalog_manager.cc:5697] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e reported cstate change: term changed from 1 to 2, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195). New cstate: current_term: 2 leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:09.248603  8878 raft_consensus.cc:1275] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Refusing update from remote peer 79e3c530a0414a60b7d1c0d914a86a7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:09.249958  9170 consensus_queue.cc:1048] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:09.260962  8955 raft_consensus.cc:1275] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Refusing update from remote peer 79e3c530a0414a60b7d1c0d914a86a7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:09.267473  9197 consensus_queue.cc:1048] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:09.368820  8955 raft_consensus.cc:1275] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Refusing update from remote peer 79e3c530a0414a60b7d1c0d914a86a7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.370481  9208 consensus_queue.cc:1048] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:09.374106  8878 raft_consensus.cc:1275] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Refusing update from remote peer 79e3c530a0414a60b7d1c0d914a86a7e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:09.376065  9208 consensus_queue.cc:1048] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:22:09.453121  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:09.458696  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:09.459086  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:10.460270  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:10.465693  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:10.466205  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:11.557684  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:11.563179  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:11.563577  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:22:12.343811  8806 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 8746
W20260709 12:22:12.344594  8806 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 8809
W20260709 12:22:12.345345  8806 debug-util.cc:398] Leaking SignalData structure 0x7b080010bf20 after lost signal to thread 8904
W20260709 12:22:12.346037  8806 debug-util.cc:398] Leaking SignalData structure 0x7b0800118220 after lost signal to thread 8979
W20260709 12:22:12.346777  8806 debug-util.cc:398] Leaking SignalData structure 0x7b080019a3e0 after lost signal to thread 9054
I20260709 12:22:12.564587  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:12.569530  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:12.569932  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:13.570807  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:13.575672  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:13.576059  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:14.576980  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:14.583132  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:14.583499  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:15.584384  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:15.589277  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:15.589684  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:16.590574  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:16.595259  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:16.595669  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:17.596625  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:17.601823  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:17.602402  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:18.628835  8743 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:22:18.633381  8879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
mode: GRACEFUL
new_leader_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:38716
I20260709 12:22:18.633988  8879 raft_consensus.cc:606] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Received request to transfer leadership to TS 34e1340365c14e6c8d1954a320d9d013
I20260709 12:22:18.638441  8879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
mode: GRACEFUL
new_leader_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:38716
I20260709 12:22:18.638918  8879 raft_consensus.cc:606] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Received request to transfer leadership to TS 34e1340365c14e6c8d1954a320d9d013
I20260709 12:22:18.643410  8879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
mode: GRACEFUL
new_leader_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:38716
I20260709 12:22:18.644003  8879 raft_consensus.cc:606] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Received request to transfer leadership to TS 34e1340365c14e6c8d1954a320d9d013
I20260709 12:22:18.648608  9028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57"
dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
mode: GRACEFUL
new_leader_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:34368
I20260709 12:22:18.648993  9028 raft_consensus.cc:606] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 LEADER]: Received request to transfer leadership to TS 34e1340365c14e6c8d1954a320d9d013
I20260709 12:22:18.655099  9221 raft_consensus.cc:993] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4: : Instructing follower 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:18.655666  9218 raft_consensus.cc:1081] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Signalling peer 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:18.656401  9028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94"
dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
mode: GRACEFUL
new_leader_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:34368
I20260709 12:22:18.656952  9028 raft_consensus.cc:606] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 LEADER]: Received request to transfer leadership to TS 34e1340365c14e6c8d1954a320d9d013
I20260709 12:22:18.657410  8953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:42206
I20260709 12:22:18.657917  8953 raft_consensus.cc:493] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.658346  8953 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.665400  8953 raft_consensus.cc:515] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.667735  8953 leader_election.cc:290] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 3 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:18.668324  8879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:18.668627  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:18.669039  8879 raft_consensus.cc:3055] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.669232  9028 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.669381  8879 raft_consensus.cc:740] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:18.670276  8879 consensus_queue.cc:260] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.671595  8879 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.675819  9028 raft_consensus.cc:2468] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 3.
I20260709 12:22:18.677126  8918 leader_election.cc:304] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
I20260709 12:22:18.678009  9220 raft_consensus.cc:2804] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:18.678440  8879 raft_consensus.cc:2468] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 3.
I20260709 12:22:18.679122  9220 raft_consensus.cc:697] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:18.680154  9220 consensus_queue.cc:237] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.688975  8773 catalog_manager.cc:5697] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 2 to 3, leader changed from 929101e20d6944f3abf2b312b22616c4 (127.8.137.193) to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.785372  9219 raft_consensus.cc:993] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e: : Instructing follower 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:18.785912  9228 raft_consensus.cc:1081] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 LEADER]: Signalling peer 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:18.787240  8953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:42222
I20260709 12:22:18.787690  8953 raft_consensus.cc:493] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.787964  8953 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.792480  8953 raft_consensus.cc:515] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.794504  8953 leader_election.cc:290] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 4 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:18.794927  8879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:18.795120  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0128bab39b3d40b9acd0d2f8aef1ea57" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:18.795521  8879 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.795640  9028 raft_consensus.cc:3055] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:18.795998  9028 raft_consensus.cc:740] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:18.796638  9028 consensus_queue.cc:260] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.797472  9028 raft_consensus.cc:3060] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:18.800074  8879 raft_consensus.cc:2468] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 4.
I20260709 12:22:18.801198  8921 leader_election.cc:304] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:18.801605  9028 raft_consensus.cc:2468] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 4.
I20260709 12:22:18.801935  9220 raft_consensus.cc:2804] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:18.802392  9220 raft_consensus.cc:697] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 4 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:18.803292  9220 consensus_queue.cc:237] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.809960  8773 catalog_manager.cc:5697] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 3 to 4, leader changed from 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195) to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:18.812884  9221 raft_consensus.cc:993] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4: : Instructing follower 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:18.813297  9218 raft_consensus.cc:1081] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Signalling peer 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:18.814930  8953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:42206
I20260709 12:22:18.815448  8953 raft_consensus.cc:493] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:18.815752  8953 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.820510  8953 raft_consensus.cc:515] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.821852  8953 leader_election.cc:290] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 3 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:18.822568  8879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:18.822786  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f886b747d5f4f29b5ccacd465aa542f" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:18.823176  8879 raft_consensus.cc:3055] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:18.823264  9028 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.823526  8879 raft_consensus.cc:740] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:18.824326  8879 consensus_queue.cc:260] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.825338  8879 raft_consensus.cc:3060] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:18.827868  9028 raft_consensus.cc:2468] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 3.
I20260709 12:22:18.828716  8918 leader_election.cc:304] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
I20260709 12:22:18.829445  9220 raft_consensus.cc:2804] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:18.829839  8879 raft_consensus.cc:2468] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 3.
I20260709 12:22:18.829859  9220 raft_consensus.cc:697] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:18.830919  9220 consensus_queue.cc:237] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:18.837385  8773 catalog_manager.cc:5697] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 2 to 3, leader changed from 929101e20d6944f3abf2b312b22616c4 (127.8.137.193) to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:19.103564  9219 raft_consensus.cc:993] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e: : Instructing follower 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:19.104004  9228 raft_consensus.cc:1081] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 LEADER]: Signalling peer 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:19.105358  8953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:42222
I20260709 12:22:19.105782  8953 raft_consensus.cc:493] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.106062  8953 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.109884  8953 raft_consensus.cc:515] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:19.111756  8953 leader_election.cc:290] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 3 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:19.112512  8879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:19.112709  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:19.113026  8879 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.113153  9028 raft_consensus.cc:3055] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.113476  9028 raft_consensus.cc:740] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:19.114192  9028 consensus_queue.cc:260] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:19.115272  9028 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.118796  8879 raft_consensus.cc:2468] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 3.
I20260709 12:22:19.119855  8921 leader_election.cc:304] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:19.120486  9220 raft_consensus.cc:2804] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:19.120801  9028 raft_consensus.cc:2468] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 3.
I20260709 12:22:19.120954  9220 raft_consensus.cc:697] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:19.121784  9220 consensus_queue.cc:237] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:19.128057  8773 catalog_manager.cc:5697] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 2 to 3, leader changed from 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195) to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:19.167704  9221 raft_consensus.cc:993] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4: : Instructing follower 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:19.168110  9221 raft_consensus.cc:1081] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Signalling peer 34e1340365c14e6c8d1954a320d9d013 to start an election
I20260709 12:22:19.169523  8953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
 from {username='slave'} at 127.0.0.1:42206
I20260709 12:22:19.169966  8953 raft_consensus.cc:493] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:19.170293  8953 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.170315  9028 raft_consensus.cc:1275] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.171600  9220 consensus_queue.cc:1048] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.175031  8953 raft_consensus.cc:515] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:19.177462  8879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:19.177917  8953 leader_election.cc:290] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 3 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:19.177966  8879 raft_consensus.cc:3055] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:19.178359  8879 raft_consensus.cc:740] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:19.179210  8879 consensus_queue.cc:260] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:19.181413  8879 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.181723  8878 raft_consensus.cc:1275] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.183749  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "34e1340365c14e6c8d1954a320d9d013" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:19.184239  9028 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:19.188213  8879 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 3.
I20260709 12:22:19.188284  9234 consensus_queue.cc:1048] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:19.189213  8921 leader_election.cc:304] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:19.190146  9234 raft_consensus.cc:2804] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:19.190781  9234 raft_consensus.cc:697] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Becoming Leader. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:19.192291  9234 consensus_queue.cc:237] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:19.194800  9028 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34e1340365c14e6c8d1954a320d9d013 in term 3.
I20260709 12:22:19.200945  8773 catalog_manager.cc:5697] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 reported cstate change: term changed from 2 to 3, leader changed from 929101e20d6944f3abf2b312b22616c4 (127.8.137.193) to 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "34e1340365c14e6c8d1954a320d9d013" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:19.212626  9028 raft_consensus.cc:1275] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:19.213534  9234 consensus_queue.cc:1048] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:19.221235  8879 raft_consensus.cc:1275] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:19.223057  9222 consensus_queue.cc:1048] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:19.377200  8879 raft_consensus.cc:1275] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.378310  9222 consensus_queue.cc:1048] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.386269  9028 raft_consensus.cc:1275] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.388206  9220 consensus_queue.cc:1048] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.520481  8879 raft_consensus.cc:1275] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.521513  9220 consensus_queue.cc:1048] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.529755  9028 raft_consensus.cc:1275] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.531103  9234 consensus_queue.cc:1048] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.666290  9028 raft_consensus.cc:1275] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.667502  9234 consensus_queue.cc:1048] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:19.676538  8879 raft_consensus.cc:1275] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Refusing update from remote peer 34e1340365c14e6c8d1954a320d9d013: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:19.678637  9220 consensus_queue.cc:1048] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:21.698000  8743 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:22:21.699007  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:21.707029  8953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:45460
I20260709 12:22:21.707677  8953 raft_consensus.cc:606] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Received request to transfer leadership to TS 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:21.709955  8953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:45460
I20260709 12:22:21.710353  8953 raft_consensus.cc:606] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Received request to transfer leadership to TS 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:21.712283  8953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8de765fd04334493963ed7f7c894b468"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
 from {username='slave'} at 127.0.0.1:45460
I20260709 12:22:21.712675  8953 raft_consensus.cc:606] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Received request to transfer leadership to TS 79e3c530a0414a60b7d1c0d914a86a7e
I20260709 12:22:21.714566  8953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:45460
I20260709 12:22:21.714983  8953 raft_consensus.cc:606] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Received request to transfer leadership to TS 929101e20d6944f3abf2b312b22616c4
I20260709 12:22:21.716825  8953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94"
dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
mode: GRACEFUL
new_leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
 from {username='slave'} at 127.0.0.1:45460
I20260709 12:22:21.717218  8953 raft_consensus.cc:606] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Received request to transfer leadership to TS 79e3c530a0414a60b7d1c0d914a86a7e
I20260709 12:22:21.718128  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:22:21.718540  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:21.801751  9248 raft_consensus.cc:993] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:21.802412  9252 raft_consensus.cc:1081] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Signalling peer 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:21.804250  8878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:38696
I20260709 12:22:21.804956  8878 raft_consensus.cc:493] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:21.805450  8878 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.811546  8878 raft_consensus.cc:515] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:21.813571  8878 leader_election.cc:290] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 3 election: Requested vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:21.814343  8953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:21.814875  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7270f78f86d84264b8792b80a5c14f45" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:21.815192  8953 raft_consensus.cc:3055] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:21.815414  9028 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.815507  8953 raft_consensus.cc:740] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:21.816401  8953 consensus_queue.cc:260] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:21.817781  8953 raft_consensus.cc:3060] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:21.822636  9028 raft_consensus.cc:2468] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 3.
I20260709 12:22:21.824122  8842 leader_election.cc:304] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:21.824952  8953 raft_consensus.cc:2468] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 3.
I20260709 12:22:21.826330  9253 raft_consensus.cc:2804] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:21.827514  9253 raft_consensus.cc:697] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 3 LEADER]: Becoming Leader. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:21.828306  9253 consensus_queue.cc:237] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:21.835950  8775 catalog_manager.cc:5697] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 reported cstate change: term changed from 2 to 3, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 929101e20d6944f3abf2b312b22616c4 (127.8.137.193). New cstate: current_term: 3 leader_uuid: "929101e20d6944f3abf2b312b22616c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:22.081319  9251 raft_consensus.cc:993] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 79e3c530a0414a60b7d1c0d914a86a7e to start an election
I20260709 12:22:22.081879  9248 raft_consensus.cc:1081] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Signalling peer 79e3c530a0414a60b7d1c0d914a86a7e to start an election
I20260709 12:22:22.083685  9027 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94"
dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
 from {username='slave'} at 127.0.0.1:34356
I20260709 12:22:22.084259  9027 raft_consensus.cc:493] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:22.084681  9252 raft_consensus.cc:993] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 79e3c530a0414a60b7d1c0d914a86a7e to start an election
I20260709 12:22:22.084628  9027 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.085225  9248 raft_consensus.cc:1081] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Signalling peer 79e3c530a0414a60b7d1c0d914a86a7e to start an election
I20260709 12:22:22.086702  9028 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8de765fd04334493963ed7f7c894b468"
dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
 from {username='slave'} at 127.0.0.1:34356
I20260709 12:22:22.087314  9028 raft_consensus.cc:493] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:22.087664  9028 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.092303  9027 raft_consensus.cc:515] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.093940  9028 raft_consensus.cc:515] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.095167  9027 leader_election.cc:290] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 4 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:22.095852  9028 leader_election.cc:290] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 3 election: Requested vote from peers 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237), 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849)
I20260709 12:22:22.097159  8953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:22.097234  8878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed5a17df6ece4515bf9e210a3b96fd94" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:22.097682  8877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de765fd04334493963ed7f7c894b468" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "929101e20d6944f3abf2b312b22616c4"
I20260709 12:22:22.097841  8878 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.097687  8953 raft_consensus.cc:3055] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:22.098264  8877 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.098377  8953 raft_consensus.cc:740] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:22.099123  8953 consensus_queue.cc:260] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.100138  8953 raft_consensus.cc:3060] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.101557  8955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de765fd04334493963ed7f7c894b468" candidate_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:22.102103  8955 raft_consensus.cc:3055] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:22.102399  8955 raft_consensus.cc:740] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:22.103173  8955 consensus_queue.cc:260] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.104110  8878 raft_consensus.cc:2468] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 4.
I20260709 12:22:22.104331  8955 raft_consensus.cc:3060] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:22.105443  8995 leader_election.cc:304] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:22.106316  8953 raft_consensus.cc:2468] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 4.
I20260709 12:22:22.106551  9250 raft_consensus.cc:2804] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:22.107192  9250 raft_consensus.cc:697] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 LEADER]: Becoming Leader. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:22.108368  9250 consensus_queue.cc:237] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.109191  8955 raft_consensus.cc:2468] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 3.
I20260709 12:22:22.110316  8993 leader_election.cc:304] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34e1340365c14e6c8d1954a320d9d013, 79e3c530a0414a60b7d1c0d914a86a7e; no voters: 
I20260709 12:22:22.104962  8877 raft_consensus.cc:2468] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e3c530a0414a60b7d1c0d914a86a7e in term 3.
I20260709 12:22:22.111992  9260 raft_consensus.cc:2804] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:22.114291  9260 raft_consensus.cc:697] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 LEADER]: Becoming Leader. State: Replica: 79e3c530a0414a60b7d1c0d914a86a7e, State: Running, Role: LEADER
I20260709 12:22:22.115403  9260 consensus_queue.cc:237] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.121531  8775 catalog_manager.cc:5697] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e reported cstate change: term changed from 3 to 4, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195). New cstate: current_term: 4 leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:22.131834  8775 catalog_manager.cc:5697] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e reported cstate change: term changed from 2 to 3, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195). New cstate: current_term: 3 leader_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:22.164655  9251 raft_consensus.cc:993] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:22.165182  9248 raft_consensus.cc:1081] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Signalling peer 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:22.166577  8878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:38696
I20260709 12:22:22.167059  8878 raft_consensus.cc:493] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:22.167340  8878 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.171275  8878 raft_consensus.cc:515] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.172581  8878 leader_election.cc:290] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 4 election: Requested vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:22.173365  8955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:22.173516  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b8a990e66edc4710afc8a0bf0625b910" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:22.173885  8955 raft_consensus.cc:3055] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:22.173975  9028 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.174219  8955 raft_consensus.cc:740] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:22.175005  8955 consensus_queue.cc:260] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.176083  8955 raft_consensus.cc:3060] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.179838  9028 raft_consensus.cc:2468] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 4.
I20260709 12:22:22.180918  8842 leader_election.cc:304] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:22.181656  9253 raft_consensus.cc:2804] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:22.181648  8955 raft_consensus.cc:2468] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 4.
I20260709 12:22:22.182271  9253 raft_consensus.cc:697] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 4 LEADER]: Becoming Leader. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:22.183194  9253 consensus_queue.cc:237] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.189899  8775 catalog_manager.cc:5697] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 reported cstate change: term changed from 3 to 4, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 929101e20d6944f3abf2b312b22616c4 (127.8.137.193). New cstate: current_term: 4 leader_uuid: "929101e20d6944f3abf2b312b22616c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:22.278005  9251 raft_consensus.cc:993] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013: : Instructing follower 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:22.278434  9248 raft_consensus.cc:1081] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Signalling peer 929101e20d6944f3abf2b312b22616c4 to start an election
I20260709 12:22:22.279757  8878 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde"
dest_uuid: "929101e20d6944f3abf2b312b22616c4"
 from {username='slave'} at 127.0.0.1:38696
I20260709 12:22:22.280192  8878 raft_consensus.cc:493] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:22.280443  8878 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.284327  8878 raft_consensus.cc:515] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.285790  8878 leader_election.cc:290] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 4 election: Requested vote from peers 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194:33849), 79e3c530a0414a60b7d1c0d914a86a7e (127.8.137.195:36209)
I20260709 12:22:22.286556  8955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "34e1340365c14e6c8d1954a320d9d013"
I20260709 12:22:22.286764  9028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "596dacc9c2934bb59018b1428443fcde" candidate_uuid: "929101e20d6944f3abf2b312b22616c4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "79e3c530a0414a60b7d1c0d914a86a7e"
I20260709 12:22:22.287048  8955 raft_consensus.cc:3055] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:22.287225  9028 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.287361  8955 raft_consensus.cc:740] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 34e1340365c14e6c8d1954a320d9d013, State: Running, Role: LEADER
I20260709 12:22:22.288190  8955 consensus_queue.cc:260] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.288980  8955 raft_consensus.cc:3060] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:22.291260  9028 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 4.
I20260709 12:22:22.292101  8842 leader_election.cc:304] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e3c530a0414a60b7d1c0d914a86a7e, 929101e20d6944f3abf2b312b22616c4; no voters: 
I20260709 12:22:22.292757  9253 raft_consensus.cc:2804] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:22.293162  8955 raft_consensus.cc:2468] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 929101e20d6944f3abf2b312b22616c4 in term 4.
I20260709 12:22:22.293231  9253 raft_consensus.cc:697] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 4 LEADER]: Becoming Leader. State: Replica: 929101e20d6944f3abf2b312b22616c4, State: Running, Role: LEADER
I20260709 12:22:22.294137  9253 consensus_queue.cc:237] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } }
I20260709 12:22:22.300603  8775 catalog_manager.cc:5697] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 reported cstate change: term changed from 3 to 4, leader changed from 34e1340365c14e6c8d1954a320d9d013 (127.8.137.194) to 929101e20d6944f3abf2b312b22616c4 (127.8.137.193). New cstate: current_term: 4 leader_uuid: "929101e20d6944f3abf2b312b22616c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:22.427378  8955 raft_consensus.cc:1275] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:22.428558  9253 consensus_queue.cc:1048] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:22.431559  9028 raft_consensus.cc:1275] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:22.433058  9253 consensus_queue.cc:1048] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:22.518011  8955 raft_consensus.cc:1275] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Refusing update from remote peer 79e3c530a0414a60b7d1c0d914a86a7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:22.519268  9260 consensus_queue.cc:1048] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:22.527393  8878 raft_consensus.cc:1275] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Refusing update from remote peer 79e3c530a0414a60b7d1c0d914a86a7e: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:22.528961  9261 consensus_queue.cc:1048] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:22.584872  8955 raft_consensus.cc:1275] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:22.586325  9254 consensus_queue.cc:1048] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:22.591430  8877 raft_consensus.cc:1275] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Refusing update from remote peer 79e3c530a0414a60b7d1c0d914a86a7e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:22.592851  9261 consensus_queue.cc:1048] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "929101e20d6944f3abf2b312b22616c4" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:22.597353  9028 raft_consensus.cc:1275] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:22.601373  9271 consensus_queue.cc:1048] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:22.606613  8955 raft_consensus.cc:1275] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Refusing update from remote peer 79e3c530a0414a60b7d1c0d914a86a7e: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:22.611426  9261 consensus_queue.cc:1048] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:22.680413  8955 raft_consensus.cc:1275] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:22.681748  9271 consensus_queue.cc:1048] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34e1340365c14e6c8d1954a320d9d013" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 33849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:22.689954  9028 raft_consensus.cc:1275] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Refusing update from remote peer 929101e20d6944f3abf2b312b22616c4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:22.691408  9253 consensus_queue.cc:1048] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e3c530a0414a60b7d1c0d914a86a7e" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 36209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:22.719650  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:22.724388  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:22.724821  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:23.726231  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:23.731473  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:23.731858  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:24.732753  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:24.737361  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:24.737668  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:25.738723  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:25.743646  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:25.743965  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:26.744930  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:26.749760  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:26.750082  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:27.751087  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:27.755714  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:27.756011  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:28.756969  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:28.762017  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:28.762454  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:29.763442  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:29.768143  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:29.768530  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:30.769429  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:30.774130  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:30.774565  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:31.801803  8743 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:22:31.802488  8743 tablet_server.cc:179] TabletServer@127.8.137.193:0 shutting down...
I20260709 12:22:31.833789  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:31.834563  8743 tablet_replica.cc:333] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.835530  8743 raft_consensus.cc:2243] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:31.836189  8743 raft_consensus.cc:2272] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:31.839095  8743 tablet_replica.cc:333] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.839854  8743 raft_consensus.cc:2243] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:31.840394  8743 raft_consensus.cc:2272] T eca4b422654d4f0cbe4fb6cce4abb820 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:31.841864  8921 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:22:31.843343  8743 tablet_replica.cc:333] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.844214  8743 raft_consensus.cc:2243] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:31.844808  8743 raft_consensus.cc:2272] T ed5a17df6ece4515bf9e210a3b96fd94 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:31.845402  8921 consensus_peers.cc:597] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 -> Peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Couldn't send request to peer 929101e20d6944f3abf2b312b22616c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:31.846732  8743 tablet_replica.cc:333] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.847405  8743 raft_consensus.cc:2243] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:31.848340  8743 raft_consensus.cc:2272] T 7270f78f86d84264b8792b80a5c14f45 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:31.850355  8743 tablet_replica.cc:333] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.850925  8743 raft_consensus.cc:2243] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 4 LEADER]: Raft consensus shutting down.
W20260709 12:22:31.851658  8921 consensus_peers.cc:597] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 -> Peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Couldn't send request to peer 929101e20d6944f3abf2b312b22616c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:31.851748  8743 raft_consensus.cc:2272] T b8a990e66edc4710afc8a0bf0625b910 P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:31.853412  8743 tablet_replica.cc:333] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.853886  8743 raft_consensus.cc:2243] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:31.854249  8743 raft_consensus.cc:2272] T f6bb8b14ec80438c8982e8d19952ae13 P 929101e20d6944f3abf2b312b22616c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:31.855923  8743 tablet_replica.cc:333] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.856374  8743 raft_consensus.cc:2243] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:31.857100  8743 raft_consensus.cc:2272] T 596dacc9c2934bb59018b1428443fcde P 929101e20d6944f3abf2b312b22616c4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:31.858747  8743 tablet_replica.cc:333] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.859185  8743 raft_consensus.cc:2243] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:31.859615  8743 raft_consensus.cc:2272] T 8de765fd04334493963ed7f7c894b468 P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:31.861254  8743 tablet_replica.cc:333] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4: stopping tablet replica
I20260709 12:22:31.861685  8743 raft_consensus.cc:2243] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:31.862109  8743 raft_consensus.cc:2272] T 8f886b747d5f4f29b5ccacd465aa542f P 929101e20d6944f3abf2b312b22616c4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:31.950677  8743 tablet_server.cc:196] TabletServer@127.8.137.193:0 shutdown complete.
W20260709 12:22:31.951037  8921 consensus_peers.cc:597] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 -> Peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Couldn't send request to peer 929101e20d6944f3abf2b312b22616c4. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34237: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:31.969532  8743 tablet_server.cc:179] TabletServer@127.8.137.194:0 shutting down...
W20260709 12:22:31.975381  8995 consensus_peers.cc:597] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e -> Peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Couldn't send request to peer 929101e20d6944f3abf2b312b22616c4. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34237: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:31.999634  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:32.000407  8743 tablet_replica.cc:333] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.001253  8743 raft_consensus.cc:2243] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.001873  8743 raft_consensus.cc:2272] T eca4b422654d4f0cbe4fb6cce4abb820 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:22:32.004215  8995 consensus_peers.cc:597] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e -> Peer 929101e20d6944f3abf2b312b22616c4 (127.8.137.193:34237): Couldn't send request to peer 929101e20d6944f3abf2b312b22616c4. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34237: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:22:32.004331  8743 tablet_replica.cc:333] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.005065  8743 raft_consensus.cc:2243] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:32.005867  8743 raft_consensus.cc:2272] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.007603  8743 tablet_replica.cc:333] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.008105  8743 raft_consensus.cc:2243] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:32.008854  8743 raft_consensus.cc:2272] T 8f886b747d5f4f29b5ccacd465aa542f P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.010475  8743 tablet_replica.cc:333] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.011026  8743 raft_consensus.cc:2243] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.011420  8743 raft_consensus.cc:2272] T 596dacc9c2934bb59018b1428443fcde P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.012959  8743 tablet_replica.cc:333] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.013419  8743 raft_consensus.cc:2243] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.013787  8743 raft_consensus.cc:2272] T 7270f78f86d84264b8792b80a5c14f45 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.015375  8743 tablet_replica.cc:333] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.015848  8743 raft_consensus.cc:2243] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.016255  8743 raft_consensus.cc:2272] T b8a990e66edc4710afc8a0bf0625b910 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.017915  8743 tablet_replica.cc:333] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.018364  8743 raft_consensus.cc:2243] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:32.019071  8743 raft_consensus.cc:2272] T f6bb8b14ec80438c8982e8d19952ae13 P 34e1340365c14e6c8d1954a320d9d013 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.020601  8743 tablet_replica.cc:333] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.021021  8743 raft_consensus.cc:2243] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.021391  8743 raft_consensus.cc:2272] T 8de765fd04334493963ed7f7c894b468 P 34e1340365c14e6c8d1954a320d9d013 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.023015  8743 tablet_replica.cc:333] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013: stopping tablet replica
I20260709 12:22:32.023449  8743 raft_consensus.cc:2243] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.023833  8743 raft_consensus.cc:2272] T ed5a17df6ece4515bf9e210a3b96fd94 P 34e1340365c14e6c8d1954a320d9d013 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.075930  8743 tablet_server.cc:196] TabletServer@127.8.137.194:0 shutdown complete.
I20260709 12:22:32.094139  8743 tablet_server.cc:179] TabletServer@127.8.137.195:0 shutting down...
I20260709 12:22:32.120697  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:22:32.121321  8743 tablet_replica.cc:333] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.121958  8743 raft_consensus.cc:2243] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:32.122828  8743 raft_consensus.cc:2272] T eca4b422654d4f0cbe4fb6cce4abb820 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.124601  8743 tablet_replica.cc:333] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.125136  8743 raft_consensus.cc:2243] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.125576  8743 raft_consensus.cc:2272] T 8f886b747d5f4f29b5ccacd465aa542f P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.127269  8743 tablet_replica.cc:333] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.127748  8743 raft_consensus.cc:2243] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.128149  8743 raft_consensus.cc:2272] T 0128bab39b3d40b9acd0d2f8aef1ea57 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.129732  8743 tablet_replica.cc:333] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.130199  8743 raft_consensus.cc:2243] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:22:32.131011  8743 raft_consensus.cc:2272] T 8de765fd04334493963ed7f7c894b468 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.132637  8743 tablet_replica.cc:333] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.133100  8743 raft_consensus.cc:2243] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:22:32.133932  8743 raft_consensus.cc:2272] T ed5a17df6ece4515bf9e210a3b96fd94 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.135849  8743 tablet_replica.cc:333] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.136307  8743 raft_consensus.cc:2243] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.136734  8743 raft_consensus.cc:2272] T 7270f78f86d84264b8792b80a5c14f45 P 79e3c530a0414a60b7d1c0d914a86a7e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.138212  8743 tablet_replica.cc:333] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.138749  8743 raft_consensus.cc:2243] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.139219  8743 raft_consensus.cc:2272] T b8a990e66edc4710afc8a0bf0625b910 P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.140815  8743 tablet_replica.cc:333] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.141259  8743 raft_consensus.cc:2243] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.141640  8743 raft_consensus.cc:2272] T f6bb8b14ec80438c8982e8d19952ae13 P 79e3c530a0414a60b7d1c0d914a86a7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.143237  8743 tablet_replica.cc:333] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e: stopping tablet replica
I20260709 12:22:32.143625  8743 raft_consensus.cc:2243] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:22:32.144068  8743 raft_consensus.cc:2272] T 596dacc9c2934bb59018b1428443fcde P 79e3c530a0414a60b7d1c0d914a86a7e [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.171695  8743 tablet_server.cc:196] TabletServer@127.8.137.195:0 shutdown complete.
I20260709 12:22:32.189105  8743 master.cc:562] Master@127.8.137.254:37327 shutting down...
I20260709 12:22:32.401400  8743 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:22:32.402050  8743 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:22:32.402704  8743 tablet_replica.cc:333] T 00000000000000000000000000000000 P 492193a9dae74af0ae2404da27362f2a: stopping tablet replica
I20260709 12:22:32.439383  8743 master.cc:584] Master@127.8.137.254:37327 shutdown complete.
34e1340365c14e6c8d1954a320d9d013  4
79e3c530a0414a60b7d1c0d914a86a7e  1
929101e20d6944f3abf2b312b22616c4  4
34e1340365c14e6c8d1954a320d9d013  3
79e3c530a0414a60b7d1c0d914a86a7e  3
929101e20d6944f3abf2b312b22616c4  3
34e1340365c14e6c8d1954a320d9d013  8
79e3c530a0414a60b7d1c0d914a86a7e  1
929101e20d6944f3abf2b312b22616c4  0
34e1340365c14e6c8d1954a320d9d013  3
79e3c530a0414a60b7d1c0d914a86a7e  3
929101e20d6944f3abf2b312b22616c4  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (31417 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:22:32.481237  8743 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.137.254:36741
I20260709 12:22:32.482414  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:32.488204  9294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:32.488255  9293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:32.489205  9296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:32.489470  8743 server_base.cc:1061] running on GCE node
I20260709 12:22:32.490437  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:32.490687  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:32.490839  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599752490812 us; error 0 us; skew 500 ppm
I20260709 12:22:32.491379  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:32.493794  8743 webserver.cc:533] Webserver started at http://127.8.137.254:42355/ using document root <none> and password file <none>
I20260709 12:22:32.494267  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:32.494451  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:32.494982  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:32.496109  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/master-0-root/instance:
uuid: "ae4afe2b778048959d9bf0888e13198b"
format_stamp: "Formatted at 2026-07-09 12:22:32 on dist-test-slave-kpwv"
I20260709 12:22:32.500341  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:22:32.503652  9301 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.504329  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 12:22:32.504631  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/master-0-root
uuid: "ae4afe2b778048959d9bf0888e13198b"
format_stamp: "Formatted at 2026-07-09 12:22:32 on dist-test-slave-kpwv"
I20260709 12:22:32.504900  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:32.531257  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:32.532513  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:32.569885  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.254:36741
I20260709 12:22:32.569955  9352 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.254:36741 every 8 connection(s)
I20260709 12:22:32.573773  9353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:32.583119  9353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b: Bootstrap starting.
I20260709 12:22:32.587837  9353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:32.591842  9353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b: No bootstrap required, opened a new log
I20260709 12:22:32.593863  9353 raft_consensus.cc:359] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4afe2b778048959d9bf0888e13198b" member_type: VOTER }
I20260709 12:22:32.594265  9353 raft_consensus.cc:385] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:32.594462  9353 raft_consensus.cc:740] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae4afe2b778048959d9bf0888e13198b, State: Initialized, Role: FOLLOWER
I20260709 12:22:32.594982  9353 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4afe2b778048959d9bf0888e13198b" member_type: VOTER }
I20260709 12:22:32.595455  9353 raft_consensus.cc:399] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:22:32.595655  9353 raft_consensus.cc:493] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:22:32.595937  9353 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:32.599603  9353 raft_consensus.cc:515] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4afe2b778048959d9bf0888e13198b" member_type: VOTER }
I20260709 12:22:32.600087  9353 leader_election.cc:304] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae4afe2b778048959d9bf0888e13198b; no voters: 
I20260709 12:22:32.601135  9353 leader_election.cc:290] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:22:32.601516  9356 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:32.602778  9356 raft_consensus.cc:697] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 1 LEADER]: Becoming Leader. State: Replica: ae4afe2b778048959d9bf0888e13198b, State: Running, Role: LEADER
I20260709 12:22:32.603434  9356 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4afe2b778048959d9bf0888e13198b" member_type: VOTER }
I20260709 12:22:32.603869  9353 sys_catalog.cc:565] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:22:32.607918  9357 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae4afe2b778048959d9bf0888e13198b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4afe2b778048959d9bf0888e13198b" member_type: VOTER } }
I20260709 12:22:32.608551  9357 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:32.608402  9358 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae4afe2b778048959d9bf0888e13198b. Latest consensus state: current_term: 1 leader_uuid: "ae4afe2b778048959d9bf0888e13198b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae4afe2b778048959d9bf0888e13198b" member_type: VOTER } }
I20260709 12:22:32.608959  9358 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [sys.catalog]: This master's current role is: LEADER
I20260709 12:22:32.611584  9362 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:22:32.618342  9362 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:22:32.619264  8743 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:22:32.626998  9362 catalog_manager.cc:1383] Generated new cluster ID: 208d48f16d0c4b379c78a595b2865ccf
I20260709 12:22:32.627249  9362 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:22:32.646719  9362 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:22:32.647855  9362 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:22:32.660568  9362 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b: Generated new TSK 0
I20260709 12:22:32.661125  9362 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:22:32.685243  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:32.692284  9375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:32.692512  9374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:32.694923  8743 server_base.cc:1061] running on GCE node
W20260709 12:22:32.694964  9377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:32.696091  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:32.696265  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:32.696393  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599752696383 us; error 0 us; skew 500 ppm
I20260709 12:22:32.696841  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:32.699028  8743 webserver.cc:533] Webserver started at http://127.8.137.193:43899/ using document root <none> and password file <none>
I20260709 12:22:32.699467  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:32.699618  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:32.699815  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:32.700779  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-0-root/instance:
uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
format_stamp: "Formatted at 2026-07-09 12:22:32 on dist-test-slave-kpwv"
I20260709 12:22:32.707552  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.000s	sys 0.008s
I20260709 12:22:32.715574  9382 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.716493  8743 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 12:22:32.716832  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
format_stamp: "Formatted at 2026-07-09 12:22:32 on dist-test-slave-kpwv"
I20260709 12:22:32.717204  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:32.739410  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:32.741096  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:32.742799  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:32.746008  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:32.746279  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.746551  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:32.746786  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:22:32.784749  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:43325
I20260709 12:22:32.784854  9445 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:43325 every 8 connection(s)
I20260709 12:22:32.790087  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:32.796506  9450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:32.797042  9451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:32.800215  9453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:32.800585  8743 server_base.cc:1061] running on GCE node
I20260709 12:22:32.802088  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:22:32.802297  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:32.802517  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599752802494 us; error 0 us; skew 500 ppm
I20260709 12:22:32.803164  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:32.803610  9446 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36741
I20260709 12:22:32.803913  9446 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:32.804761  9446 heartbeater.cc:507] Master 127.8.137.254:36741 requested a full tablet report, sending...
I20260709 12:22:32.805581  8743 webserver.cc:533] Webserver started at http://127.8.137.194:40557/ using document root <none> and password file <none>
I20260709 12:22:32.806201  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:32.806439  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:32.806805  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:32.807185  9318 ts_manager.cc:194] Registered new tserver with Master: eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325)
I20260709 12:22:32.807969  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-1-root/instance:
uuid: "c391b1d7ed4a499c94f868ca14e87074"
format_stamp: "Formatted at 2026-07-09 12:22:32 on dist-test-slave-kpwv"
I20260709 12:22:32.808881  9318 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33970
I20260709 12:22:32.812292  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.004s
I20260709 12:22:32.815587  9458 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.816299  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:22:32.816596  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-1-root
uuid: "c391b1d7ed4a499c94f868ca14e87074"
format_stamp: "Formatted at 2026-07-09 12:22:32 on dist-test-slave-kpwv"
I20260709 12:22:32.816874  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:32.837865  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:32.839128  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:32.840523  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:32.842605  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:32.842828  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.843052  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:32.843217  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.882640  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.194:38799
I20260709 12:22:32.882764  9520 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.194:38799 every 8 connection(s)
I20260709 12:22:32.887205  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:22:32.894203  9524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:32.896410  9525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:22:32.896945  9527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:22:32.899088  8743 server_base.cc:1061] running on GCE node
I20260709 12:22:32.900063  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:22:32.900177  9521 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36741
W20260709 12:22:32.900331  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:22:32.900563  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599752900546 us; error 0 us; skew 500 ppm
I20260709 12:22:32.900590  9521 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:32.901181  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:22:32.901372  9521 heartbeater.cc:507] Master 127.8.137.254:36741 requested a full tablet report, sending...
I20260709 12:22:32.903448  9318 ts_manager.cc:194] Registered new tserver with Master: c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:32.903704  8743 webserver.cc:533] Webserver started at http://127.8.137.195:45479/ using document root <none> and password file <none>
I20260709 12:22:32.904234  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:22:32.904429  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:22:32.904698  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:22:32.904989  9318 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33986
I20260709 12:22:32.905972  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-2-root/instance:
uuid: "073792a321ca4850993a30440770651b"
format_stamp: "Formatted at 2026-07-09 12:22:32 on dist-test-slave-kpwv"
I20260709 12:22:32.910444  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:22:32.913491  9532 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.914240  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:22:32.914529  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-2-root
uuid: "073792a321ca4850993a30440770651b"
format_stamp: "Formatted at 2026-07-09 12:22:32 on dist-test-slave-kpwv"
I20260709 12:22:32.914834  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783599720900375-8743-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:22:32.929056  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:22:32.930143  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:22:32.931509  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:22:32.933787  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:22:32.933980  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.934231  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:22:32.934386  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:22:32.973564  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.195:37873
I20260709 12:22:32.973670  9595 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.195:37873 every 8 connection(s)
I20260709 12:22:32.987279  9596 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36741
I20260709 12:22:32.987653  9596 heartbeater.cc:461] Registering TS with master...
I20260709 12:22:32.988379  9596 heartbeater.cc:507] Master 127.8.137.254:36741 requested a full tablet report, sending...
I20260709 12:22:32.990156  9318 ts_manager.cc:194] Registered new tserver with Master: 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:32.990247  8743 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013351486s
I20260709 12:22:32.991525  9318 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33996
I20260709 12:22:33.013314  9318 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34006:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:22:33.016162  9318 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:22:33.094544  9410 tablet_service.cc:1511] Processing CreateTablet for tablet bcb007f7d565438ba215d4ccad1c2e62 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:33.096014  9410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb007f7d565438ba215d4ccad1c2e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.099783  9409 tablet_service.cc:1511] Processing CreateTablet for tablet f6ae3e6fbb274773ab45fee44b483907 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:33.101141  9409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ae3e6fbb274773ab45fee44b483907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.102949  9408 tablet_service.cc:1511] Processing CreateTablet for tablet 082b54dc8c5b40b5a30cdac841befab9 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:33.103688  9555 tablet_service.cc:1511] Processing CreateTablet for tablet 87182730c6014f38bb9b60e4ed2486b1 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:33.104295  9408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 082b54dc8c5b40b5a30cdac841befab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.105026  9555 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87182730c6014f38bb9b60e4ed2486b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.106511  9407 tablet_service.cc:1511] Processing CreateTablet for tablet 88f20b0eaf3a4f82a482e301b778fdfe (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:33.107803  9407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f20b0eaf3a4f82a482e301b778fdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.108201  9554 tablet_service.cc:1511] Processing CreateTablet for tablet b87d95fb9055412597d6ea198986cf8e (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:33.109475  9554 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87d95fb9055412597d6ea198986cf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.109517  9553 tablet_service.cc:1511] Processing CreateTablet for tablet bb471b9e458540c488a45423b29c9882 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:33.111023  9553 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb471b9e458540c488a45423b29c9882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.111637  9406 tablet_service.cc:1511] Processing CreateTablet for tablet 283fdca0e28146e99bab344a8a595fd3 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:33.112931  9406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283fdca0e28146e99bab344a8a595fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.114907  9552 tablet_service.cc:1511] Processing CreateTablet for tablet f4b231a1993d47de80e4fb53cca4c3de (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:33.116190  9552 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4b231a1993d47de80e4fb53cca4c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.115895  9560 tablet_service.cc:1511] Processing CreateTablet for tablet bcb007f7d565438ba215d4ccad1c2e62 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:33.117141  9560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb007f7d565438ba215d4ccad1c2e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.119848  9559 tablet_service.cc:1511] Processing CreateTablet for tablet f6ae3e6fbb274773ab45fee44b483907 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:33.121057  9559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ae3e6fbb274773ab45fee44b483907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.122725  9551 tablet_service.cc:1511] Processing CreateTablet for tablet 59b9ed1fc7864936bebdafc1a048f703 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:33.123998  9551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b9ed1fc7864936bebdafc1a048f703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.123760  9558 tablet_service.cc:1511] Processing CreateTablet for tablet 082b54dc8c5b40b5a30cdac841befab9 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:33.124985  9558 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 082b54dc8c5b40b5a30cdac841befab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.127827  9556 tablet_service.cc:1511] Processing CreateTablet for tablet 283fdca0e28146e99bab344a8a595fd3 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:33.129070  9556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283fdca0e28146e99bab344a8a595fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.131830  9557 tablet_service.cc:1511] Processing CreateTablet for tablet 88f20b0eaf3a4f82a482e301b778fdfe (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:33.133066  9557 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f20b0eaf3a4f82a482e301b778fdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.140362  9405 tablet_service.cc:1511] Processing CreateTablet for tablet 87182730c6014f38bb9b60e4ed2486b1 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:33.141634  9405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87182730c6014f38bb9b60e4ed2486b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.143622  9404 tablet_service.cc:1511] Processing CreateTablet for tablet b87d95fb9055412597d6ea198986cf8e (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:33.144925  9404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87d95fb9055412597d6ea198986cf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.147799  9403 tablet_service.cc:1511] Processing CreateTablet for tablet bb471b9e458540c488a45423b29c9882 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:33.150238  9401 tablet_service.cc:1511] Processing CreateTablet for tablet 59b9ed1fc7864936bebdafc1a048f703 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:33.154062  9402 tablet_service.cc:1511] Processing CreateTablet for tablet f4b231a1993d47de80e4fb53cca4c3de (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:33.156293  9403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb471b9e458540c488a45423b29c9882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.162504  9401 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b9ed1fc7864936bebdafc1a048f703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.170924  9402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4b231a1993d47de80e4fb53cca4c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.189436  9486 tablet_service.cc:1511] Processing CreateTablet for tablet bcb007f7d565438ba215d4ccad1c2e62 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:22:33.192215  9485 tablet_service.cc:1511] Processing CreateTablet for tablet f6ae3e6fbb274773ab45fee44b483907 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:22:33.195863  9484 tablet_service.cc:1511] Processing CreateTablet for tablet 082b54dc8c5b40b5a30cdac841befab9 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:22:33.198393  9483 tablet_service.cc:1511] Processing CreateTablet for tablet 88f20b0eaf3a4f82a482e301b778fdfe (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:22:33.201406  9482 tablet_service.cc:1511] Processing CreateTablet for tablet 283fdca0e28146e99bab344a8a595fd3 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:22:33.208873  9477 tablet_service.cc:1511] Processing CreateTablet for tablet 59b9ed1fc7864936bebdafc1a048f703 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:22:33.211669  9478 tablet_service.cc:1511] Processing CreateTablet for tablet f4b231a1993d47de80e4fb53cca4c3de (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:22:33.216449  9479 tablet_service.cc:1511] Processing CreateTablet for tablet bb471b9e458540c488a45423b29c9882 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:22:33.210363  9481 tablet_service.cc:1511] Processing CreateTablet for tablet 87182730c6014f38bb9b60e4ed2486b1 (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:22:33.218618  9480 tablet_service.cc:1511] Processing CreateTablet for tablet b87d95fb9055412597d6ea198986cf8e (DEFAULT_TABLE table=test-workload [id=ebb112d394ea4da2a18f0d86b82dd47c]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:22:33.221421  9486 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb007f7d565438ba215d4ccad1c2e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.222131  9481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 87182730c6014f38bb9b60e4ed2486b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.226634  9612 tablet_bootstrap.cc:492] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.226994  9484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 082b54dc8c5b40b5a30cdac841befab9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.231302  9478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4b231a1993d47de80e4fb53cca4c3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.235088  9480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b87d95fb9055412597d6ea198986cf8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.235834  9479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb471b9e458540c488a45423b29c9882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.238883  9485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6ae3e6fbb274773ab45fee44b483907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.239491  9482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283fdca0e28146e99bab344a8a595fd3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.242905  9477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59b9ed1fc7864936bebdafc1a048f703. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.247881  9483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88f20b0eaf3a4f82a482e301b778fdfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:22:33.248625  9611 tablet_bootstrap.cc:492] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.286896  9613 tablet_bootstrap.cc:492] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.288820  9612 tablet_bootstrap.cc:654] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.291412  9611 tablet_bootstrap.cc:654] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.293943  9613 tablet_bootstrap.cc:654] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.302649  9612 tablet_bootstrap.cc:492] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.303630  9612 ts_tablet_manager.cc:1403] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.077s	user 0.011s	sys 0.007s
I20260709 12:22:33.305366  9611 tablet_bootstrap.cc:492] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.305886  9611 ts_tablet_manager.cc:1403] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.010s
I20260709 12:22:33.306576  9612 raft_consensus.cc:359] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.307394  9612 raft_consensus.cc:385] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.307807  9613 tablet_bootstrap.cc:492] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.307698  9612 raft_consensus.cc:740] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.308244  9613 ts_tablet_manager.cc:1403] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.004s
I20260709 12:22:33.308521  9612 consensus_queue.cc:260] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.308892  9611 raft_consensus.cc:359] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.309525  9611 raft_consensus.cc:385] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.310020  9611 raft_consensus.cc:740] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.310684  9446 heartbeater.cc:499] Master 127.8.137.254:36741 was elected leader, sending a full tablet report...
I20260709 12:22:33.311128  9612 ts_tablet_manager.cc:1434] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:22:33.310833  9613 raft_consensus.cc:359] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.310794  9611 consensus_queue.cc:260] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.311714  9613 raft_consensus.cc:385] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.312088  9612 tablet_bootstrap.cc:492] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.312184  9613 raft_consensus.cc:740] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.313077  9613 consensus_queue.cc:260] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.321192  9612 tablet_bootstrap.cc:654] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.333930  9612 tablet_bootstrap.cc:492] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.334632  9612 ts_tablet_manager.cc:1403] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.009s
I20260709 12:22:33.338294  9612 raft_consensus.cc:359] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.339440  9612 raft_consensus.cc:385] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.339910  9612 raft_consensus.cc:740] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.341035  9612 consensus_queue.cc:260] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.344240  9612 ts_tablet_manager.cc:1434] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:22:33.345489  9612 tablet_bootstrap.cc:492] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.353888  9612 tablet_bootstrap.cc:654] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.357059  9611 ts_tablet_manager.cc:1434] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.051s	user 0.023s	sys 0.027s
I20260709 12:22:33.357283  9521 heartbeater.cc:499] Master 127.8.137.254:36741 was elected leader, sending a full tablet report...
I20260709 12:22:33.357856  9613 ts_tablet_manager.cc:1434] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.049s	user 0.020s	sys 0.027s
I20260709 12:22:33.358889  9596 heartbeater.cc:499] Master 127.8.137.254:36741 was elected leader, sending a full tablet report...
I20260709 12:22:33.359452  9611 tablet_bootstrap.cc:492] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.360644  9612 tablet_bootstrap.cc:492] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.361092  9612 ts_tablet_manager.cc:1403] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.016s	user 0.014s	sys 0.000s
I20260709 12:22:33.363420  9613 tablet_bootstrap.cc:492] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.364019  9612 raft_consensus.cc:359] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.364723  9612 raft_consensus.cc:385] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.364990  9612 raft_consensus.cc:740] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.365854  9611 tablet_bootstrap.cc:654] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.365762  9612 consensus_queue.cc:260] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.368221  9612 ts_tablet_manager.cc:1434] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:22:33.369227  9612 tablet_bootstrap.cc:492] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.369997  9613 tablet_bootstrap.cc:654] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.376523  9612 tablet_bootstrap.cc:654] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.377596  9611 tablet_bootstrap.cc:492] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.378139  9611 ts_tablet_manager.cc:1403] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.009s
I20260709 12:22:33.381052  9611 raft_consensus.cc:359] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.381858  9611 raft_consensus.cc:385] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.382272  9611 raft_consensus.cc:740] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.383190  9611 consensus_queue.cc:260] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.385495  9611 ts_tablet_manager.cc:1434] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 12:22:33.386417  9611 tablet_bootstrap.cc:492] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.392886  9611 tablet_bootstrap.cc:654] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.413790  9617 raft_consensus.cc:493] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.414848  9617 raft_consensus.cc:515] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.431839  9617 leader_election.cc:290] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.436650  9613 tablet_bootstrap.cc:492] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.437283  9613 ts_tablet_manager.cc:1403] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.079s	user 0.034s	sys 0.036s
I20260709 12:22:33.441004  9613 raft_consensus.cc:359] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.441917  9613 raft_consensus.cc:385] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.446007  9612 tablet_bootstrap.cc:492] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.442291  9613 raft_consensus.cc:740] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.446547  9612 ts_tablet_manager.cc:1403] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.078s	user 0.034s	sys 0.011s
I20260709 12:22:33.448259  9611 tablet_bootstrap.cc:492] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.448765  9611 ts_tablet_manager.cc:1403] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.063s	user 0.023s	sys 0.036s
I20260709 12:22:33.449070  9612 raft_consensus.cc:359] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.449651  9612 raft_consensus.cc:385] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.449926  9612 raft_consensus.cc:740] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.450692  9612 consensus_queue.cc:260] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.450709  9613 consensus_queue.cc:260] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.451536  9611 raft_consensus.cc:359] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.452224  9611 raft_consensus.cc:385] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.452508  9611 raft_consensus.cc:740] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.453462  9613 ts_tablet_manager.cc:1434] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.003s
I20260709 12:22:33.453238  9611 consensus_queue.cc:260] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.454457  9613 tablet_bootstrap.cc:492] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.459992  9611 ts_tablet_manager.cc:1434] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:22:33.460963  9611 tablet_bootstrap.cc:492] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.465348  9613 tablet_bootstrap.cc:654] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.466578  9619 raft_consensus.cc:493] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.467435  9611 tablet_bootstrap.cc:654] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.467065  9619 raft_consensus.cc:515] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.468912  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.470084  9612 ts_tablet_manager.cc:1434] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
W20260709 12:22:33.470903  9385 leader_election.cc:343] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.471118  9612 tablet_bootstrap.cc:492] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.474525  9619 leader_election.cc:290] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.478790  9619 raft_consensus.cc:493] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.479215  9619 raft_consensus.cc:515] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.479918  9612 tablet_bootstrap.cc:654] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.481317  9619 leader_election.cc:290] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.488659  9611 tablet_bootstrap.cc:492] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.489295  9611 ts_tablet_manager.cc:1403] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.029s	user 0.003s	sys 0.016s
I20260709 12:22:33.493130  9611 raft_consensus.cc:359] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.494051  9611 raft_consensus.cc:385] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.494369  9611 raft_consensus.cc:740] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.494928  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.495347  9611 consensus_queue.cc:260] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.497223  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.497907  9611 ts_tablet_manager.cc:1434] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:22:33.499621  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
W20260709 12:22:33.501899  9533 leader_election.cc:343] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:33.503063  9533 leader_election.cc:343] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:22:33.504192  9386 leader_election.cc:343] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 073792a321ca4850993a30440770651b (127.8.137.195:37873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.504796  9386 leader_election.cc:304] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb02c76cdbdd4cf190ca05caf642fe88; no voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:33.508119  9611 tablet_bootstrap.cc:492] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.510531  9617 raft_consensus.cc:2749] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:33.512169  9612 tablet_bootstrap.cc:492] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.512701  9612 ts_tablet_manager.cc:1403] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.042s	user 0.023s	sys 0.011s
I20260709 12:22:33.514263  9611 tablet_bootstrap.cc:654] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.514158  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.515530  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.516654  9613 tablet_bootstrap.cc:492] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.517093  9613 ts_tablet_manager.cc:1403] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.009s
I20260709 12:22:33.519876  9612 raft_consensus.cc:359] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
W20260709 12:22:33.520653  9535 leader_election.cc:343] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.520078  9613 raft_consensus.cc:359] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.521201  9613 raft_consensus.cc:385] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.521130  9535 leader_election.cc:304] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 073792a321ca4850993a30440770651b; no voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:33.520948  9612 raft_consensus.cc:385] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.522085  9612 raft_consensus.cc:740] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.522135  9619 raft_consensus.cc:2749] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:22:33.522483  9535 leader_election.cc:343] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.521587  9613 raft_consensus.cc:740] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.523307  9611 tablet_bootstrap.cc:492] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.523087  9535 leader_election.cc:304] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 073792a321ca4850993a30440770651b; no voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:33.523121  9612 consensus_queue.cc:260] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.523866  9611 ts_tablet_manager.cc:1403] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 12:22:33.523993  9619 raft_consensus.cc:2749] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:33.523679  9613 consensus_queue.cc:260] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.525657  9612 ts_tablet_manager.cc:1434] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:22:33.526634  9612 tablet_bootstrap.cc:492] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.526932  9611 raft_consensus.cc:359] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.527627  9611 raft_consensus.cc:385] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.527915  9611 raft_consensus.cc:740] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.528626  9611 consensus_queue.cc:260] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.530819  9611 ts_tablet_manager.cc:1434] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:22:33.531720  9611 tablet_bootstrap.cc:492] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.534410  9612 tablet_bootstrap.cc:654] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.537860  9611 tablet_bootstrap.cc:654] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.538398  9613 ts_tablet_manager.cc:1434] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.021s	user 0.001s	sys 0.005s
I20260709 12:22:33.546787  9613 tablet_bootstrap.cc:492] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.547295  9612 tablet_bootstrap.cc:492] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.547752  9612 ts_tablet_manager.cc:1403] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.006s
I20260709 12:22:33.550580  9612 raft_consensus.cc:359] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.551350  9612 raft_consensus.cc:385] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.551636  9612 raft_consensus.cc:740] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.552420  9611 tablet_bootstrap.cc:492] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.552873  9613 tablet_bootstrap.cc:654] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.552942  9611 ts_tablet_manager.cc:1403] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.021s	user 0.004s	sys 0.008s
I20260709 12:22:33.552361  9612 consensus_queue.cc:260] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.554899  9612 ts_tablet_manager.cc:1434] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:22:33.555832  9612 tablet_bootstrap.cc:492] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.555876  9611 raft_consensus.cc:359] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.556602  9611 raft_consensus.cc:385] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.556906  9611 raft_consensus.cc:740] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.557709  9611 consensus_queue.cc:260] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.562702  9612 tablet_bootstrap.cc:654] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.563246  9611 ts_tablet_manager.cc:1434] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:22:33.564242  9611 tablet_bootstrap.cc:492] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.564462  9613 tablet_bootstrap.cc:492] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.564905  9613 ts_tablet_manager.cc:1403] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.005s
I20260709 12:22:33.567732  9613 raft_consensus.cc:359] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.568365  9613 raft_consensus.cc:385] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.568815  9613 raft_consensus.cc:740] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.569579  9613 consensus_queue.cc:260] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.570254  9619 raft_consensus.cc:493] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.570700  9619 raft_consensus.cc:515] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.572310  9611 tablet_bootstrap.cc:654] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.572641  9619 leader_election.cc:290] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.573834  9613 ts_tablet_manager.cc:1434] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:33.574128  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ed1fc7864936bebdafc1a048f703" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.574752  9613 tablet_bootstrap.cc:492] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
W20260709 12:22:33.575543  9533 leader_election.cc:343] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.577086  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ed1fc7864936bebdafc1a048f703" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.577695  9495 raft_consensus.cc:2468] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 0.
I20260709 12:22:33.581321  9613 tablet_bootstrap.cc:654] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.585263  9611 tablet_bootstrap.cc:492] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.585846  9611 ts_tablet_manager.cc:1403] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 12:22:33.588023  9620 raft_consensus.cc:493] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.588478  9620 raft_consensus.cc:515] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.585906  9535 leader_election.cc:304] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:33.589506  9611 raft_consensus.cc:359] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.590700  9611 raft_consensus.cc:385] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.590999  9620 leader_election.cc:290] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.591259  9611 raft_consensus.cc:740] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.592142  9611 consensus_queue.cc:260] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.590761  9619 raft_consensus.cc:2804] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.594025  9619 raft_consensus.cc:493] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.594635  9619 raft_consensus.cc:3060] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.603256  9617 raft_consensus.cc:493] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.603232  9619 raft_consensus.cc:515] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.603999  9617 raft_consensus.cc:515] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.606313  9617 leader_election.cc:290] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.607650  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ed1fc7864936bebdafc1a048f703" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:33.609172  9612 tablet_bootstrap.cc:492] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.609754  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082b54dc8c5b40b5a30cdac841befab9" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.611258  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082b54dc8c5b40b5a30cdac841befab9" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:33.609761  9612 ts_tablet_manager.cc:1403] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.027s
W20260709 12:22:33.613162  9385 leader_election.cc:343] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.615228  9612 raft_consensus.cc:359] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
W20260709 12:22:33.615774  9366 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:22:33.615907  9612 raft_consensus.cc:385] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.616323  9612 raft_consensus.cc:740] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
W20260709 12:22:33.617405  9533 leader_election.cc:343] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.611994  9570 raft_consensus.cc:2468] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 0.
I20260709 12:22:33.618551  9611 ts_tablet_manager.cc:1434] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.015s
I20260709 12:22:33.619669  9611 tablet_bootstrap.cc:492] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.621299  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.621920  9420 raft_consensus.cc:2468] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 0.
I20260709 12:22:33.623302  9459 leader_election.cc:304] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:33.624161  9620 raft_consensus.cc:2804] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.623901  9386 leader_election.cc:304] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:33.624560  9620 raft_consensus.cc:493] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.624930  9620 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.625516  9617 raft_consensus.cc:2804] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.626099  9611 tablet_bootstrap.cc:654] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.626010  9617 raft_consensus.cc:493] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.626536  9617 raft_consensus.cc:3060] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.630929  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:33.631789  9620 raft_consensus.cc:515] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
W20260709 12:22:33.632413  9462 leader_election.cc:343] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 073792a321ca4850993a30440770651b (127.8.137.195:37873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.634464  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:33.635022  9420 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.642967  9612 consensus_queue.cc:260] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.645510  9641 raft_consensus.cc:493] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.645617  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ed1fc7864936bebdafc1a048f703" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:33.646238  9495 raft_consensus.cc:3060] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.646948  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b"
W20260709 12:22:33.648294  9462 leader_election.cc:343] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 073792a321ca4850993a30440770651b (127.8.137.195:37873): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.646054  9641 raft_consensus.cc:515] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.649641  9420 raft_consensus.cc:2468] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 1.
I20260709 12:22:33.650399  9641 leader_election.cc:290] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.650789  9459 leader_election.cc:304] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 073792a321ca4850993a30440770651b
I20260709 12:22:33.651880  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.652716  9419 raft_consensus.cc:2468] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 0.
I20260709 12:22:33.653910  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.654803  9620 leader_election.cc:290] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.655576  9620 raft_consensus.cc:2804] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.656234  9620 raft_consensus.cc:697] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Becoming Leader. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
W20260709 12:22:33.655674  9535 leader_election.cc:343] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.657636  9533 leader_election.cc:304] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:33.657357  9620 consensus_queue.cc:237] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.658772  9641 raft_consensus.cc:2804] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.659250  9641 raft_consensus.cc:493] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.659674  9641 raft_consensus.cc:3060] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.663220  9612 ts_tablet_manager.cc:1434] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.001s
I20260709 12:22:33.666034  9495 raft_consensus.cc:2468] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 1.
I20260709 12:22:33.671110  9535 leader_election.cc:304] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:33.666719  9619 leader_election.cc:290] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.667261  9641 raft_consensus.cc:515] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.674278  9619 raft_consensus.cc:2804] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.674845  9619 raft_consensus.cc:697] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Becoming Leader. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:33.675580  9641 leader_election.cc:290] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.675666  9619 consensus_queue.cc:237] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.676247  9613 tablet_bootstrap.cc:492] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.676931  9613 ts_tablet_manager.cc:1403] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.102s	user 0.028s	sys 0.036s
I20260709 12:22:33.677152  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:33.668872  9617 raft_consensus.cc:515] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.669438  9612 tablet_bootstrap.cc:492] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.679751  9420 raft_consensus.cc:3060] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.688051  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
W20260709 12:22:33.689460  9535 leader_election.cc:343] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.690497  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082b54dc8c5b40b5a30cdac841befab9" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:33.691709  9611 tablet_bootstrap.cc:492] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
W20260709 12:22:33.692044  9385 leader_election.cc:343] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.690446  9613 raft_consensus.cc:359] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.692544  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082b54dc8c5b40b5a30cdac841befab9" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:33.693423  9613 raft_consensus.cc:385] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.693586  9570 raft_consensus.cc:3060] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.693784  9612 tablet_bootstrap.cc:654] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.693835  9613 raft_consensus.cc:740] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.694088  9420 raft_consensus.cc:2468] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 1.
I20260709 12:22:33.692158  9611 ts_tablet_manager.cc:1403] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.073s	user 0.024s	sys 0.013s
I20260709 12:22:33.694958  9613 consensus_queue.cc:260] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.693315  9316 catalog_manager.cc:5697] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 reported cstate change: term changed from 0 to 1, leader changed from <none> to c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "c391b1d7ed4a499c94f868ca14e87074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.692895  9617 leader_election.cc:290] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.697194  9611 raft_consensus.cc:359] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.698417  9533 leader_election.cc:304] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:33.698678  9611 raft_consensus.cc:385] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.699173  9611 raft_consensus.cc:740] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.699903  9611 consensus_queue.cc:260] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.701231  9613 ts_tablet_manager.cc:1434] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:22:33.701934  9611 ts_tablet_manager.cc:1434] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:22:33.702203  9613 tablet_bootstrap.cc:492] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.702907  9611 tablet_bootstrap.cc:492] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.699893  9646 raft_consensus.cc:2804] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.704813  9646 raft_consensus.cc:697] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 1 LEADER]: Becoming Leader. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:33.707588  9646 consensus_queue.cc:237] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.707286  9315 catalog_manager.cc:5697] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b reported cstate change: term changed from 0 to 1, leader changed from <none> to 073792a321ca4850993a30440770651b (127.8.137.195). New cstate: current_term: 1 leader_uuid: "073792a321ca4850993a30440770651b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.708384  9611 tablet_bootstrap.cc:654] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.711328  9570 raft_consensus.cc:2468] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 1.
I20260709 12:22:33.711627  9613 tablet_bootstrap.cc:654] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.712746  9386 leader_election.cc:304] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:33.713912  9617 raft_consensus.cc:2804] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.714380  9617 raft_consensus.cc:697] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:33.715193  9617 consensus_queue.cc:237] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.719291  9644 raft_consensus.cc:493] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.719915  9644 raft_consensus.cc:515] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.725998  9612 tablet_bootstrap.cc:492] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.726442  9612 ts_tablet_manager.cc:1403] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.008s
I20260709 12:22:33.727424  9611 tablet_bootstrap.cc:492] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.727857  9611 ts_tablet_manager.cc:1403] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.008s
I20260709 12:22:33.728914  9644 leader_election.cc:290] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.730736  9611 raft_consensus.cc:359] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.731621  9611 raft_consensus.cc:385] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.731985  9611 raft_consensus.cc:740] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.735023  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:33.738296  9570 raft_consensus.cc:2468] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 0.
I20260709 12:22:33.735960  9619 raft_consensus.cc:493] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.730561  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.736310  9611 consensus_queue.cc:260] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.730933  9646 raft_consensus.cc:493] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.744457  9386 leader_election.cc:304] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
W20260709 12:22:33.745072  9385 leader_election.cc:343] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.744421  9646 raft_consensus.cc:515] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.746794  9612 raft_consensus.cc:359] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.747393  9612 raft_consensus.cc:385] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.747660  9612 raft_consensus.cc:740] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.748090  9619 raft_consensus.cc:515] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.748965  9611 ts_tablet_manager.cc:1434] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 12:22:33.749863  9611 tablet_bootstrap.cc:492] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b: Bootstrap starting.
I20260709 12:22:33.750684  9619 leader_election.cc:290] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.751343  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.751929  9420 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 0.
I20260709 12:22:33.752158  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.752801  9495 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 0.
I20260709 12:22:33.753504  9533 leader_election.cc:304] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:33.748457  9617 raft_consensus.cc:2804] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.754221  9617 raft_consensus.cc:493] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.754349  9619 raft_consensus.cc:2804] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.754575  9617 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.754752  9619 raft_consensus.cc:493] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.755179  9619 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.756292  9611 tablet_bootstrap.cc:654] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.751863  9318 catalog_manager.cc:5697] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.758939  9612 consensus_queue.cc:260] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.762444  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.763088  9420 raft_consensus.cc:2468] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 0.
I20260709 12:22:33.763789  9617 raft_consensus.cc:515] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.764186  9613 tablet_bootstrap.cc:492] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.764428  9533 leader_election.cc:304] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:33.764719  9613 ts_tablet_manager.cc:1403] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.063s	user 0.011s	sys 0.025s
I20260709 12:22:33.765193  9641 raft_consensus.cc:2804] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.765703  9641 raft_consensus.cc:493] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.766064  9641 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.767107  9495 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.767894  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:33.768569  9646 leader_election.cc:290] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
W20260709 12:22:33.769749  9385 leader_election.cc:343] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.770346  9612 ts_tablet_manager.cc:1434] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 12:22:33.771178  9617 leader_election.cc:290] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.772358  9612 tablet_bootstrap.cc:492] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.772619  9641 raft_consensus.cc:515] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.773718  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:33.774257  9570 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.775241  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:33.775775  9420 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.781245  9613 raft_consensus.cc:359] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.781949  9613 raft_consensus.cc:385] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.782239  9613 raft_consensus.cc:740] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.786412  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:33.785192  9619 raft_consensus.cc:515] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.786718  9612 tablet_bootstrap.cc:654] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.782209  9420 raft_consensus.cc:2468] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 1.
I20260709 12:22:33.787355  9613 consensus_queue.cc:260] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.792270  9570 raft_consensus.cc:2468] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 1.
I20260709 12:22:33.792532  9533 leader_election.cc:304] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:33.793418  9646 raft_consensus.cc:2804] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.793413  9386 leader_election.cc:304] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:33.788151  9641 leader_election.cc:290] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.795552  9617 raft_consensus.cc:2804] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.799902  9495 raft_consensus.cc:2468] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 0.
I20260709 12:22:33.800763  9617 raft_consensus.cc:697] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:33.801644  9617 consensus_queue.cc:237] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.797897  9619 leader_election.cc:290] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.805855  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:33.806452  9496 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.812798  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:33.814885  9611 tablet_bootstrap.cc:492] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b: No bootstrap required, opened a new log
I20260709 12:22:33.818209  9613 ts_tablet_manager.cc:1434] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.051s	user 0.002s	sys 0.004s
I20260709 12:22:33.820581  9611 ts_tablet_manager.cc:1403] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b: Time spent bootstrapping tablet: real 0.071s	user 0.018s	sys 0.025s
I20260709 12:22:33.821231  9613 tablet_bootstrap.cc:492] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.810722  9646 raft_consensus.cc:697] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Becoming Leader. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:33.824227  9611 raft_consensus.cc:359] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.825101  9611 raft_consensus.cc:385] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.825465  9611 raft_consensus.cc:740] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.826352  9646 consensus_queue.cc:237] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.826539  9611 consensus_queue.cc:260] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.828445  9420 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.829882  9611 ts_tablet_manager.cc:1434] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:33.831125  9612 tablet_bootstrap.cc:492] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.831485  9612 ts_tablet_manager.cc:1403] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.060s	user 0.009s	sys 0.012s
I20260709 12:22:33.832429  9496 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 1.
I20260709 12:22:33.833608  9535 leader_election.cc:304] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: 
I20260709 12:22:33.834700  9658 raft_consensus.cc:2804] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.834980  9612 raft_consensus.cc:359] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.835332  9658 raft_consensus.cc:697] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Becoming Leader. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:33.835651  9612 raft_consensus.cc:385] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.835969  9612 raft_consensus.cc:740] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.836609  9612 consensus_queue.cc:260] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.836440  9658 consensus_queue.cc:237] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.840351  9612 ts_tablet_manager.cc:1434] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:22:33.841213  9612 tablet_bootstrap.cc:492] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88: Bootstrap starting.
I20260709 12:22:33.846961  9612 tablet_bootstrap.cc:654] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.849198  9613 tablet_bootstrap.cc:654] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.850950  9620 raft_consensus.cc:493] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.851430  9620 raft_consensus.cc:515] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.849749  9316 catalog_manager.cc:5697] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.849123  9420 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 1.
I20260709 12:22:33.853528  9620 leader_election.cc:290] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.854992  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d95fb9055412597d6ea198986cf8e" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
W20260709 12:22:33.856340  9459 leader_election.cc:343] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:22:33.859395  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d95fb9055412597d6ea198986cf8e" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:33.860196  9570 raft_consensus.cc:2468] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 0.
I20260709 12:22:33.861733  9462 leader_election.cc:304] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:33.863369  9620 raft_consensus.cc:2804] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.863933  9620 raft_consensus.cc:493] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.864343  9612 tablet_bootstrap.cc:492] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88: No bootstrap required, opened a new log
I20260709 12:22:33.864537  9620 raft_consensus.cc:3060] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.864849  9612 ts_tablet_manager.cc:1403] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.004s
I20260709 12:22:33.866287  9613 tablet_bootstrap.cc:492] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.866775  9613 ts_tablet_manager.cc:1403] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.046s	user 0.012s	sys 0.006s
I20260709 12:22:33.867565  9612 raft_consensus.cc:359] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.868196  9612 raft_consensus.cc:385] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.868455  9612 raft_consensus.cc:740] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.869139  9612 consensus_queue.cc:260] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.870077  9613 raft_consensus.cc:359] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.869652  9318 catalog_manager.cc:5697] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b reported cstate change: term changed from 0 to 1, leader changed from <none> to 073792a321ca4850993a30440770651b (127.8.137.195). New cstate: current_term: 1 leader_uuid: "073792a321ca4850993a30440770651b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.871114  9612 ts_tablet_manager.cc:1434] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:22:33.871500  9613 raft_consensus.cc:385] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.871778  9613 raft_consensus.cc:740] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.871655  9318 catalog_manager.cc:5697] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b reported cstate change: term changed from 0 to 1, leader changed from <none> to 073792a321ca4850993a30440770651b (127.8.137.195). New cstate: current_term: 1 leader_uuid: "073792a321ca4850993a30440770651b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.872519  9613 consensus_queue.cc:260] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.873787  9620 raft_consensus.cc:515] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.875072  9613 ts_tablet_manager.cc:1434] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:22:33.876012  9613 tablet_bootstrap.cc:492] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.876456  9620 leader_election.cc:290] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.877023  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d95fb9055412597d6ea198986cf8e" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:33.877557  9570 raft_consensus.cc:3060] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.880156  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d95fb9055412597d6ea198986cf8e" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:33.880748  9420 raft_consensus.cc:3060] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.882871  9613 tablet_bootstrap.cc:654] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.885039  9570 raft_consensus.cc:2468] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 1.
I20260709 12:22:33.886525  9462 leader_election.cc:304] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: 
I20260709 12:22:33.887750  9620 raft_consensus.cc:2804] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.888283  9620 raft_consensus.cc:697] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Becoming Leader. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:33.889057  9620 consensus_queue.cc:237] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.887748  9420 raft_consensus.cc:2468] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 1.
I20260709 12:22:33.898193  9613 tablet_bootstrap.cc:492] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.898851  9620 raft_consensus.cc:493] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.899355  9620 raft_consensus.cc:515] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.899124  9315 catalog_manager.cc:5697] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 reported cstate change: term changed from 0 to 1, leader changed from <none> to c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "c391b1d7ed4a499c94f868ca14e87074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.899374  9613 ts_tablet_manager.cc:1403] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.024s	user 0.015s	sys 0.001s
I20260709 12:22:33.902041  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.902696  9420 raft_consensus.cc:2468] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 0.
I20260709 12:22:33.903738  9459 leader_election.cc:304] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:33.903712  9613 raft_consensus.cc:359] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.904448  9613 raft_consensus.cc:385] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.904690  9613 raft_consensus.cc:740] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.904857  9645 raft_consensus.cc:2804] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:33.905246  9645 raft_consensus.cc:493] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:33.905589  9645 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.905378  9613 consensus_queue.cc:260] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.909111  9316 catalog_manager.cc:5697] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b reported cstate change: term changed from 0 to 1, leader changed from <none> to 073792a321ca4850993a30440770651b (127.8.137.195). New cstate: current_term: 1 leader_uuid: "073792a321ca4850993a30440770651b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:33.910483  9645 raft_consensus.cc:515] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.913218  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:33.913780  9419 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.915371  9620 leader_election.cc:290] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.917683  9645 leader_election.cc:290] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.917845  9613 ts_tablet_manager.cc:1434] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.008s
I20260709 12:22:33.918893  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:33.919095  9613 tablet_bootstrap.cc:492] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.919026  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:33.919756  9569 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.920897  9419 raft_consensus.cc:2468] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 1.
I20260709 12:22:33.921949  9459 leader_election.cc:304] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:33.922838  9645 raft_consensus.cc:2804] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:33.923368  9645 raft_consensus.cc:697] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Becoming Leader. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:33.924150  9645 consensus_queue.cc:237] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.926067  9613 tablet_bootstrap.cc:654] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.927155  9668 raft_consensus.cc:493] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.927618  9668 raft_consensus.cc:515] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.929057  9569 raft_consensus.cc:2468] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 1.
I20260709 12:22:33.931097  9668 leader_election.cc:290] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:33.932087  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.932602  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:33.933135  9419 raft_consensus.cc:2393] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c391b1d7ed4a499c94f868ca14e87074 in current term 1: Already voted for candidate 073792a321ca4850993a30440770651b in this term.
I20260709 12:22:33.934520  9459 leader_election.cc:304] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074; no voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:33.936009  9668 raft_consensus.cc:3060] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.938174  9613 tablet_bootstrap.cc:492] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.938700  9613 ts_tablet_manager.cc:1403] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
I20260709 12:22:33.941381  9613 raft_consensus.cc:359] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.941991  9613 raft_consensus.cc:385] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.942250  9613 raft_consensus.cc:740] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.942905  9613 consensus_queue.cc:260] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.943245  9668 raft_consensus.cc:2749] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:33.944701  9613 ts_tablet_manager.cc:1434] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:22:33.945341  9613 tablet_bootstrap.cc:492] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074: Bootstrap starting.
I20260709 12:22:33.949357  9316 catalog_manager.cc:5697] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 reported cstate change: term changed from 0 to 1, leader changed from <none> to c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "c391b1d7ed4a499c94f868ca14e87074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:33.951056  9613 tablet_bootstrap.cc:654] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074: Neither blocks nor log segments found. Creating new log.
I20260709 12:22:33.955317  9613 tablet_bootstrap.cc:492] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074: No bootstrap required, opened a new log
I20260709 12:22:33.955705  9613 ts_tablet_manager.cc:1403] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074: Time spent bootstrapping tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:22:33.957871  9613 raft_consensus.cc:359] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.958328  9613 raft_consensus.cc:385] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:22:33.958577  9613 raft_consensus.cc:740] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Initialized, Role: FOLLOWER
I20260709 12:22:33.959183  9613 consensus_queue.cc:260] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.960822  9613 ts_tablet_manager.cc:1434] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.002s
I20260709 12:22:33.976454  9658 raft_consensus.cc:493] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:33.976948  9658 raft_consensus.cc:515] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:33.978639  9658 leader_election.cc:290] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:33.979322  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:33.979651  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:33.980002  9419 raft_consensus.cc:2393] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 073792a321ca4850993a30440770651b in current term 1: Already voted for candidate c391b1d7ed4a499c94f868ca14e87074 in this term.
I20260709 12:22:33.980964  9533 leader_election.cc:304] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 073792a321ca4850993a30440770651b; no voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:33.981562  9658 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:33.985955  9658 raft_consensus.cc:2749] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:22:33.988891  9597 tablet.cc:2406] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:22:34.059612  9645 consensus_queue.cc:1048] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.069232  9617 raft_consensus.cc:493] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.069751  9617 raft_consensus.cc:515] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.070168  9645 consensus_queue.cc:1048] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.071813  9617 leader_election.cc:290] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.072955  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ed1fc7864936bebdafc1a048f703" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:34.074203  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59b9ed1fc7864936bebdafc1a048f703" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c391b1d7ed4a499c94f868ca14e87074" is_pre_election: true
I20260709 12:22:34.075047  9496 raft_consensus.cc:2393] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eb02c76cdbdd4cf190ca05caf642fe88 in current term 1: Already voted for candidate 073792a321ca4850993a30440770651b in this term.
I20260709 12:22:34.076680  9385 leader_election.cc:304] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eb02c76cdbdd4cf190ca05caf642fe88; no voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:34.077441  9617 raft_consensus.cc:3060] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.084334  9617 raft_consensus.cc:2749] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:34.088443  9645 raft_consensus.cc:493] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.088840  9645 raft_consensus.cc:515] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.090423  9645 leader_election.cc:290] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.091105  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:34.091439  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:34.091619  9419 raft_consensus.cc:2468] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 0.
I20260709 12:22:34.092021  9569 raft_consensus.cc:2468] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 0.
I20260709 12:22:34.092563  9459 leader_election.cc:304] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:34.093307  9645 raft_consensus.cc:2804] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:22:34.093586  9645 raft_consensus.cc:493] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:22:34.093976  9645 raft_consensus.cc:3060] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.098300  9645 raft_consensus.cc:515] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.099745  9645 leader_election.cc:290] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.100373  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:34.100806  9419 raft_consensus.cc:3060] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.100689  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:34.101248  9569 raft_consensus.cc:3060] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.105198  9419 raft_consensus.cc:2468] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 1.
I20260709 12:22:34.105754  9569 raft_consensus.cc:2468] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 1.
I20260709 12:22:34.106160  9459 leader_election.cc:304] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:34.106897  9645 raft_consensus.cc:2804] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:22:34.107334  9645 raft_consensus.cc:697] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Becoming Leader. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:34.108064  9645 consensus_queue.cc:237] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.114205  9315 catalog_manager.cc:5697] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 reported cstate change: term changed from 0 to 1, leader changed from <none> to c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "c391b1d7ed4a499c94f868ca14e87074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.162987  9658 consensus_queue.cc:1048] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.172947  9619 consensus_queue.cc:1048] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.179332  9619 consensus_queue.cc:1048] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.193013  9641 consensus_queue.cc:1048] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.212378  9645 raft_consensus.cc:493] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:22:34.212859  9645 raft_consensus.cc:515] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.214382  9645 leader_election.cc:290] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.215211  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082b54dc8c5b40b5a30cdac841befab9" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" is_pre_election: true
I20260709 12:22:34.215510  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "082b54dc8c5b40b5a30cdac841befab9" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073792a321ca4850993a30440770651b" is_pre_election: true
I20260709 12:22:34.216296  9569 raft_consensus.cc:2393] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c391b1d7ed4a499c94f868ca14e87074 in current term 1: Already voted for candidate eb02c76cdbdd4cf190ca05caf642fe88 in this term.
I20260709 12:22:34.217262  9462 leader_election.cc:304] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074; no voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:34.218063  9645 raft_consensus.cc:3060] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.219856  9496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
mode: GRACEFUL
new_leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:35686
I20260709 12:22:34.220407  9496 raft_consensus.cc:606] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Received request to transfer leadership to TS eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:34.224332  9645 raft_consensus.cc:2749] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:22:34.228132  9496 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
mode: GRACEFUL
new_leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:35686
I20260709 12:22:34.228586  9496 raft_consensus.cc:606] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Received request to transfer leadership to TS eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:34.240665  9569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3"
dest_uuid: "073792a321ca4850993a30440770651b"
mode: GRACEFUL
new_leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:55362
I20260709 12:22:34.241086  9569 raft_consensus.cc:606] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Received request to transfer leadership to TS eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:34.244973  9641 consensus_queue.cc:1048] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.252859  9569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb471b9e458540c488a45423b29c9882"
dest_uuid: "073792a321ca4850993a30440770651b"
mode: GRACEFUL
new_leader_uuid: "c391b1d7ed4a499c94f868ca14e87074"
 from {username='slave'} at 127.0.0.1:55362
I20260709 12:22:34.253486  9569 raft_consensus.cc:606] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Received request to transfer leadership to TS c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:34.258440  9641 consensus_queue.cc:1048] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.259809  9617 consensus_queue.cc:1048] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.262669  9495 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.264631  9569 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de"
dest_uuid: "073792a321ca4850993a30440770651b"
mode: GRACEFUL
new_leader_uuid: "c391b1d7ed4a499c94f868ca14e87074"
 from {username='slave'} at 127.0.0.1:55362
I20260709 12:22:34.265587  9658 raft_consensus.cc:993] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b: : Instructing follower c391b1d7ed4a499c94f868ca14e87074 to start an election
I20260709 12:22:34.265816  9569 raft_consensus.cc:606] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 1 LEADER]: Received request to transfer leadership to TS c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:34.266024  9658 raft_consensus.cc:1081] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Signalling peer c391b1d7ed4a499c94f868ca14e87074 to start an election
I20260709 12:22:34.268013  9494 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb471b9e458540c488a45423b29c9882"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
 from {username='slave'} at 127.0.0.1:35682
I20260709 12:22:34.268611  9494 raft_consensus.cc:493] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.268983  9494 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.273612  9641 consensus_queue.cc:1048] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.278703  9494 raft_consensus.cc:515] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.290110  9617 consensus_queue.cc:1048] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.294436  9619 consensus_queue.cc:1048] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.296447  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:34.297045  9419 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.298148  9493 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:22:34.304704  9494 leader_election.cc:290] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 2 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.308202  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:34.308917  9569 raft_consensus.cc:3055] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:34.309314  9569 raft_consensus.cc:740] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:34.309933  9419 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 2.
I20260709 12:22:34.310276  9569 consensus_queue.cc:260] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.311878  9569 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.313151  9459 leader_election.cc:304] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:34.314023  9645 raft_consensus.cc:2804] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:34.314515  9645 raft_consensus.cc:697] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 2 LEADER]: Becoming Leader. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:34.315609  9619 raft_consensus.cc:993] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b: : Instructing follower eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:34.315373  9645 consensus_queue.cc:237] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.316032  9646 raft_consensus.cc:1081] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Signalling peer eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:34.317960  9617 consensus_queue.cc:1048] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.322518  9569 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 2.
I20260709 12:22:34.320348  9420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:50912
I20260709 12:22:34.324146  9617 consensus_queue.cc:1048] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.324617  9315 catalog_manager.cc:5697] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 reported cstate change: term changed from 1 to 2, leader changed from 073792a321ca4850993a30440770651b (127.8.137.195) to c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "c391b1d7ed4a499c94f868ca14e87074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.324433  9420 raft_consensus.cc:493] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.326337  9420 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.335968  9420 raft_consensus.cc:515] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.339270  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:34.339946  9570 raft_consensus.cc:3055] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:34.340282  9570 raft_consensus.cc:740] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:34.341184  9570 consensus_queue.cc:260] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.343459  9494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:34.344023  9494 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.342370  9570 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.349960  9494 raft_consensus.cc:2468] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:34.350668  9570 raft_consensus.cc:2468] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:34.350975  9385 leader_election.cc:304] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:34.352339  9420 leader_election.cc:290] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.353523  9617 raft_consensus.cc:2804] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:34.354104  9617 raft_consensus.cc:697] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:34.355197  9617 consensus_queue.cc:237] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.359403  9645 consensus_queue.cc:1048] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:34.378739  9620 consensus_queue.cc:1048] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:22:34.378407  9318 catalog_manager.cc:5697] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 1 to 2, leader changed from 073792a321ca4850993a30440770651b (127.8.137.195) to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.387872  9620 raft_consensus.cc:993] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074: : Instructing follower eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:34.388335  9620 raft_consensus.cc:1081] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Signalling peer eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:34.390283  9420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:50916
I20260709 12:22:34.390890  9420 raft_consensus.cc:493] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.391266  9420 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.398590  9420 raft_consensus.cc:515] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.401108  9420 leader_election.cc:290] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.401365  9494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:34.401917  9494 raft_consensus.cc:3055] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:34.402242  9494 raft_consensus.cc:740] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:34.402670  9568 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:34.403199  9568 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.403036  9494 consensus_queue.cc:260] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.404292  9494 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.410434  9568 raft_consensus.cc:2468] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:34.411207  9494 raft_consensus.cc:2468] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:34.411548  9386 leader_election.cc:304] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:34.412333  9617 raft_consensus.cc:2804] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:34.412361  9620 consensus_queue.cc:1048] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.412899  9617 raft_consensus.cc:697] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:34.413640  9617 consensus_queue.cc:237] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.423579  9318 catalog_manager.cc:5697] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 1 to 2, leader changed from c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194) to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.424432  9668 consensus_queue.cc:1048] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.495604  9668 consensus_queue.cc:1048] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.505965  9620 consensus_queue.cc:1048] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:22:34.687758  9620 raft_consensus.cc:993] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074: : Instructing follower eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:34.688192  9645 raft_consensus.cc:1081] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Signalling peer eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:34.689635  9420 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:50916
I20260709 12:22:34.690104  9420 raft_consensus.cc:493] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.690402  9420 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.695415  9420 raft_consensus.cc:515] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.696926  9420 leader_election.cc:290] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.697702  9494 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:34.698017  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:34.698199  9494 raft_consensus.cc:3055] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:34.698539  9494 raft_consensus.cc:740] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:34.698531  9570 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.699337  9494 consensus_queue.cc:260] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.700207  9494 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.703321  9570 raft_consensus.cc:2468] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:34.704341  9386 leader_election.cc:304] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:34.704988  9617 raft_consensus.cc:2804] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:34.705201  9494 raft_consensus.cc:2468] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:34.705476  9617 raft_consensus.cc:697] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:34.706307  9617 consensus_queue.cc:237] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.711728  9318 catalog_manager.cc:5697] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 1 to 2, leader changed from c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194) to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.782191  9494 raft_consensus.cc:1275] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:34.783300  9617 consensus_queue.cc:1048] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.790092  9570 raft_consensus.cc:1275] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:34.791898  9693 consensus_queue.cc:1048] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.799561  9658 raft_consensus.cc:993] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b: : Instructing follower c391b1d7ed4a499c94f868ca14e87074 to start an election
I20260709 12:22:34.799983  9619 raft_consensus.cc:1081] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 1 LEADER]: Signalling peer c391b1d7ed4a499c94f868ca14e87074 to start an election
I20260709 12:22:34.801378  9494 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
 from {username='slave'} at 127.0.0.1:35682
I20260709 12:22:34.801837  9494 raft_consensus.cc:493] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:34.802206  9494 raft_consensus.cc:3060] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.807457  9494 raft_consensus.cc:515] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.809474  9494 leader_election.cc:290] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 2 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:34.810122  9420 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:34.810495  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4b231a1993d47de80e4fb53cca4c3de" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:34.810683  9420 raft_consensus.cc:3060] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.811021  9570 raft_consensus.cc:3055] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:34.811286  9570 raft_consensus.cc:740] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:34.812081  9570 consensus_queue.cc:260] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.813251  9570 raft_consensus.cc:3060] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:34.816130  9420 raft_consensus.cc:2468] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 2.
I20260709 12:22:34.817214  9459 leader_election.cc:304] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:34.817953  9645 raft_consensus.cc:2804] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:34.818380  9645 raft_consensus.cc:697] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 2 LEADER]: Becoming Leader. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:34.818967  9570 raft_consensus.cc:2468] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 2.
I20260709 12:22:34.819207  9645 consensus_queue.cc:237] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:34.825253  9318 catalog_manager.cc:5697] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 reported cstate change: term changed from 1 to 2, leader changed from 073792a321ca4850993a30440770651b (127.8.137.195) to c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "c391b1d7ed4a499c94f868ca14e87074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:34.845885  9494 raft_consensus.cc:1275] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:34.847187  9570 raft_consensus.cc:1275] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Refusing update from remote peer c391b1d7ed4a499c94f868ca14e87074: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:34.847121  9693 consensus_queue.cc:1048] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.848393  9620 consensus_queue.cc:1048] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.854924  9568 raft_consensus.cc:1275] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:34.856431  9693 consensus_queue.cc:1048] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:34.861789  9420 raft_consensus.cc:1275] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Refusing update from remote peer c391b1d7ed4a499c94f868ca14e87074: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:34.864035  9620 consensus_queue.cc:1048] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:35.133364  9568 raft_consensus.cc:1275] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:35.134580  9693 consensus_queue.cc:1048] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:35.143321  9494 raft_consensus.cc:1275] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:35.144582  9694 consensus_queue.cc:1048] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:35.252555  9568 raft_consensus.cc:1275] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Refusing update from remote peer c391b1d7ed4a499c94f868ca14e87074: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:35.253665  9620 consensus_queue.cc:1048] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:35.261695  9420 raft_consensus.cc:1275] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Refusing update from remote peer c391b1d7ed4a499c94f868ca14e87074: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:35.263221  9620 consensus_queue.cc:1048] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:37.397625  8743 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:22:37.398348  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:37.421458  9417 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
mode: GRACEFUL
new_leader_uuid: "073792a321ca4850993a30440770651b"
 from {username='slave'} at 127.0.0.1:45814
I20260709 12:22:37.421998  9417 raft_consensus.cc:606] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Received request to transfer leadership to TS 073792a321ca4850993a30440770651b
I20260709 12:22:37.424232  9417 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
mode: GRACEFUL
new_leader_uuid: "073792a321ca4850993a30440770651b"
 from {username='slave'} at 127.0.0.1:45814
I20260709 12:22:37.424736  9417 raft_consensus.cc:606] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 LEADER]: Received request to transfer leadership to TS 073792a321ca4850993a30440770651b
I20260709 12:22:37.425676  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:22:37.426121  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:37.873383  9713 raft_consensus.cc:993] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88: : Instructing follower 073792a321ca4850993a30440770651b to start an election
I20260709 12:22:37.873864  9720 raft_consensus.cc:1081] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Signalling peer 073792a321ca4850993a30440770651b to start an election
I20260709 12:22:37.875494  9570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe"
dest_uuid: "073792a321ca4850993a30440770651b"
 from {username='slave'} at 127.0.0.1:55356
I20260709 12:22:37.875998  9570 raft_consensus.cc:493] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:37.876309  9570 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.881728  9570 raft_consensus.cc:515] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:37.883471  9570 leader_election.cc:290] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 3 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:37.884207  9417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:37.884476  9493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:37.884838  9417 raft_consensus.cc:3055] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:37.884989  9493 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.885186  9417 raft_consensus.cc:740] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:37.885962  9417 consensus_queue.cc:260] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:37.886920  9417 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:37.889907  9493 raft_consensus.cc:2468] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 3.
I20260709 12:22:37.891011  9535 leader_election.cc:304] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: 
I20260709 12:22:37.891649  9417 raft_consensus.cc:2468] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 3.
I20260709 12:22:37.891709  9715 raft_consensus.cc:2804] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:37.892751  9715 raft_consensus.cc:697] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 3 LEADER]: Becoming Leader. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:37.893471  9715 consensus_queue.cc:237] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:37.902159  9315 catalog_manager.cc:5697] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b reported cstate change: term changed from 2 to 3, leader changed from eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193) to 073792a321ca4850993a30440770651b (127.8.137.195). New cstate: current_term: 3 leader_uuid: "073792a321ca4850993a30440770651b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:37.971716  9713 raft_consensus.cc:993] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88: : Instructing follower 073792a321ca4850993a30440770651b to start an election
I20260709 12:22:37.972163  9720 raft_consensus.cc:1081] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 LEADER]: Signalling peer 073792a321ca4850993a30440770651b to start an election
I20260709 12:22:37.973637  9570 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62"
dest_uuid: "073792a321ca4850993a30440770651b"
 from {username='slave'} at 127.0.0.1:55356
I20260709 12:22:37.974128  9570 raft_consensus.cc:493] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:37.974445  9570 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:37.979116  9570 raft_consensus.cc:515] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:37.981103  9570 leader_election.cc:290] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 2 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:37.981323  9417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:37.981670  9493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:37.981958  9417 raft_consensus.cc:3055] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:37.982220  9493 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:37.982295  9417 raft_consensus.cc:740] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:37.983136  9417 consensus_queue.cc:260] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:37.984275  9417 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:37.988862  9493 raft_consensus.cc:2468] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 2.
I20260709 12:22:37.990064  9535 leader_election.cc:304] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: 
I20260709 12:22:37.990653  9417 raft_consensus.cc:2468] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 2.
I20260709 12:22:37.990763  9715 raft_consensus.cc:2804] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:37.991333  9715 raft_consensus.cc:697] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 2 LEADER]: Becoming Leader. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:37.992096  9715 consensus_queue.cc:237] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:38.001485  9315 catalog_manager.cc:5697] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b reported cstate change: term changed from 1 to 2, leader changed from eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193) to 073792a321ca4850993a30440770651b (127.8.137.195). New cstate: current_term: 2 leader_uuid: "073792a321ca4850993a30440770651b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: HEALTHY } } }
W20260709 12:22:38.010876  9591 debug-util.cc:398] Leaking SignalData structure 0x7b08000a5f20 after lost signal to thread 8746
W20260709 12:22:38.011722  9591 debug-util.cc:398] Leaking SignalData structure 0x7b080014f7c0 after lost signal to thread 9352
W20260709 12:22:38.013399  9591 debug-util.cc:398] Leaking SignalData structure 0x7b08001cf660 after lost signal to thread 9520
W20260709 12:22:38.014287  9591 debug-util.cc:398] Leaking SignalData structure 0x7b08001d0a80 after lost signal to thread 9595
I20260709 12:22:38.427264  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:38.433511  9417 raft_consensus.cc:1275] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Refusing update from remote peer 073792a321ca4850993a30440770651b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:38.435101  9721 consensus_queue.cc:1048] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:38.435745  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:38.436234  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:38.454210  9493 raft_consensus.cc:1275] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Refusing update from remote peer 073792a321ca4850993a30440770651b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:38.455631  9715 consensus_queue.cc:1048] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:22:38.540459  9493 raft_consensus.cc:1275] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Refusing update from remote peer 073792a321ca4850993a30440770651b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:38.541761  9715 consensus_queue.cc:1048] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:38.550801  9417 raft_consensus.cc:1275] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Refusing update from remote peer 073792a321ca4850993a30440770651b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:38.553588  9715 consensus_queue.cc:1048] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:39.437325  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:39.443673  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:39.444152  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:40.445078  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:40.451647  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:40.452064  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:41.453253  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:41.459929  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:41.460471  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:42.461503  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:42.466761  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:42.467240  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:43.468195  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:43.474558  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:43.475049  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:44.476014  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:44.482237  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:44.482734  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:45.483637  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:45.489020  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:45.489398  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:46.490294  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:46.495988  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:46.496479  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:47.528362  8743 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:22:47.532871  9570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62"
dest_uuid: "073792a321ca4850993a30440770651b"
mode: GRACEFUL
new_leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:55362
I20260709 12:22:47.533483  9570 raft_consensus.cc:606] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 2 LEADER]: Received request to transfer leadership to TS eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:47.543452  9570 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe"
dest_uuid: "073792a321ca4850993a30440770651b"
mode: GRACEFUL
new_leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:55362
I20260709 12:22:47.543884  9570 raft_consensus.cc:606] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 3 LEADER]: Received request to transfer leadership to TS eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:47.551268  9493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
mode: GRACEFUL
new_leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:35686
I20260709 12:22:47.551736  9493 raft_consensus.cc:606] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Received request to transfer leadership to TS eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:47.556602  9493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b87d95fb9055412597d6ea198986cf8e"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
mode: GRACEFUL
new_leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:35686
I20260709 12:22:47.557054  9493 raft_consensus.cc:606] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Received request to transfer leadership to TS eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:47.561631  9493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb471b9e458540c488a45423b29c9882"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
mode: GRACEFUL
new_leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:35686
I20260709 12:22:47.562109  9493 raft_consensus.cc:606] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 2 LEADER]: Received request to transfer leadership to TS eb02c76cdbdd4cf190ca05caf642fe88
I20260709 12:22:47.684576  9734 raft_consensus.cc:993] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b: : Instructing follower eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.685014  9734 raft_consensus.cc:1081] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 3 LEADER]: Signalling peer eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.686575  9419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:50912
I20260709 12:22:47.687139  9419 raft_consensus.cc:493] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.687484  9419 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:47.693324  9419 raft_consensus.cc:515] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.695181  9419 leader_election.cc:290] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 4 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:47.696245  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:47.696228  9493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "88f20b0eaf3a4f82a482e301b778fdfe" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:47.697103  9570 raft_consensus.cc:3055] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:47.697221  9493 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:47.697373  9570 raft_consensus.cc:740] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:47.698120  9570 consensus_queue.cc:260] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.699049  9570 raft_consensus.cc:3060] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:47.703517  9493 raft_consensus.cc:2468] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 4.
I20260709 12:22:47.704973  9385 leader_election.cc:304] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:47.705294  9570 raft_consensus.cc:2468] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 4.
I20260709 12:22:47.705996  9745 raft_consensus.cc:2804] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:47.706967  9745 raft_consensus.cc:697] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:47.708505  9745 consensus_queue.cc:237] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.715219  9315 catalog_manager.cc:5697] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 3 to 4, leader changed from 073792a321ca4850993a30440770651b (127.8.137.195) to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 4 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.793952  9744 raft_consensus.cc:993] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074: : Instructing follower eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.794584  9753 raft_consensus.cc:1081] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Signalling peer eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.796156  9419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b87d95fb9055412597d6ea198986cf8e"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:50916
I20260709 12:22:47.796666  9419 raft_consensus.cc:493] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.797052  9419 raft_consensus.cc:3060] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.801604  9419 raft_consensus.cc:515] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.803261  9419 leader_election.cc:290] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:47.804121  9493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d95fb9055412597d6ea198986cf8e" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:47.804276  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b87d95fb9055412597d6ea198986cf8e" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:47.804692  9493 raft_consensus.cc:3055] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:47.804757  9570 raft_consensus.cc:3060] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.805034  9493 raft_consensus.cc:740] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:47.805848  9493 consensus_queue.cc:260] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.806852  9493 raft_consensus.cc:3060] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.809865  9570 raft_consensus.cc:2468] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:47.811333  9386 leader_election.cc:304] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:47.811844  9493 raft_consensus.cc:2468] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:47.812345  9747 raft_consensus.cc:2804] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:47.812827  9747 raft_consensus.cc:697] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:47.813624  9747 consensus_queue.cc:237] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.815980  9734 raft_consensus.cc:993] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b: : Instructing follower eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.816447  9734 raft_consensus.cc:1081] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 2 LEADER]: Signalling peer eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.818782  9417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:50912
I20260709 12:22:47.819496  9417 raft_consensus.cc:493] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.819911  9417 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:47.820706  9744 raft_consensus.cc:993] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074: : Instructing follower eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.821449  9753 raft_consensus.cc:1081] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 2 LEADER]: Signalling peer eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.823653  9419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb471b9e458540c488a45423b29c9882"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:50916
I20260709 12:22:47.824189  9419 raft_consensus.cc:493] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.824549  9419 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:47.825138  9315 catalog_manager.cc:5697] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 1 to 2, leader changed from c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194) to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.826896  9417 raft_consensus.cc:515] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.829458  9493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:47.829983  9493 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:47.830569  9570 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:47.830780  9417 leader_election.cc:290] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 3 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:47.830670  9419 raft_consensus.cc:515] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.831231  9570 raft_consensus.cc:3055] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:47.831540  9570 raft_consensus.cc:740] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:47.832691  9419 leader_election.cc:290] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 3 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:47.832325  9570 consensus_queue.cc:260] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.833504  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:47.834062  9496 raft_consensus.cc:3055] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:47.833514  9570 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:47.834079  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:47.834542  9496 raft_consensus.cc:740] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:47.834856  9569 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:47.835417  9496 consensus_queue.cc:260] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.836217  9493 raft_consensus.cc:2468] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 3.
I20260709 12:22:47.836580  9496 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:47.839321  9385 leader_election.cc:304] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:47.840308  9747 raft_consensus.cc:2804] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:47.840778  9747 raft_consensus.cc:697] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:47.841799  9570 raft_consensus.cc:2468] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 3.
I20260709 12:22:47.841638  9747 consensus_queue.cc:237] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.844137  9569 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 3.
I20260709 12:22:47.845273  9386 leader_election.cc:304] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:47.846097  9745 raft_consensus.cc:2804] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:47.849323  9745 raft_consensus.cc:697] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:47.849336  9496 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 3.
I20260709 12:22:47.850293  9745 consensus_queue.cc:237] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.851272  9315 catalog_manager.cc:5697] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 2 to 3, leader changed from 073792a321ca4850993a30440770651b (127.8.137.195) to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 3 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.863201  9315 catalog_manager.cc:5697] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 2 to 3, leader changed from c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194) to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 3 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:47.865324  9744 raft_consensus.cc:993] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074: : Instructing follower eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.865813  9744 raft_consensus.cc:1081] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Signalling peer eb02c76cdbdd4cf190ca05caf642fe88 to start an election
I20260709 12:22:47.867410  9419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
 from {username='slave'} at 127.0.0.1:50916
I20260709 12:22:47.868065  9419 raft_consensus.cc:493] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:47.868427  9419 raft_consensus.cc:3060] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.874145  9419 raft_consensus.cc:515] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.875681  9419 leader_election.cc:290] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Requested vote from peers c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:47.876456  9496 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:47.876753  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "87182730c6014f38bb9b60e4ed2486b1" candidate_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:47.876999  9496 raft_consensus.cc:3055] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:22:47.877264  9569 raft_consensus.cc:3060] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.877354  9496 raft_consensus.cc:740] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:47.878289  9496 consensus_queue.cc:260] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.879349  9496 raft_consensus.cc:3060] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:22:47.883576  9569 raft_consensus.cc:2468] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:47.884816  9386 leader_election.cc:304] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:47.885529  9745 raft_consensus.cc:2804] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:22:47.885960  9745 raft_consensus.cc:697] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Becoming Leader. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:47.886062  9496 raft_consensus.cc:2468] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb02c76cdbdd4cf190ca05caf642fe88 in term 2.
I20260709 12:22:47.886894  9745 consensus_queue.cc:237] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:47.893287  9315 catalog_manager.cc:5697] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 reported cstate change: term changed from 1 to 2, leader changed from c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194) to eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:48.106391  9496 raft_consensus.cc:1275] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:48.107761  9745 consensus_queue.cc:1048] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:48.119117  9569 raft_consensus.cc:1275] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:48.120879  9747 consensus_queue.cc:1048] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:48.221139  9493 raft_consensus.cc:1275] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:48.222369  9747 consensus_queue.cc:1048] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:48.231534  9569 raft_consensus.cc:1275] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:48.233120  9747 consensus_queue.cc:1048] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:48.333026  9493 raft_consensus.cc:1275] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:48.334231  9747 consensus_queue.cc:1048] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:48.336278  9569 raft_consensus.cc:1275] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:48.337711  9747 consensus_queue.cc:1048] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:48.343874  9570 raft_consensus.cc:1275] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:48.345155  9747 consensus_queue.cc:1048] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:48.361090  9493 raft_consensus.cc:1275] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:48.365263  9745 consensus_queue.cc:1048] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:48.479215  9569 raft_consensus.cc:1275] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:48.480705  9757 consensus_queue.cc:1048] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:48.486214  9493 raft_consensus.cc:1275] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Refusing update from remote peer eb02c76cdbdd4cf190ca05caf642fe88: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:22:48.487676  9772 consensus_queue.cc:1048] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:22:50.599138  8743 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:22:50.599804  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:50.607667  9419 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
mode: GRACEFUL
new_leader_uuid: "073792a321ca4850993a30440770651b"
 from {username='slave'} at 127.0.0.1:45814
I20260709 12:22:50.608227  9419 raft_consensus.cc:606] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Received request to transfer leadership to TS 073792a321ca4850993a30440770651b
I20260709 12:22:50.610513  9419 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb471b9e458540c488a45423b29c9882"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
mode: GRACEFUL
new_leader_uuid: "073792a321ca4850993a30440770651b"
 from {username='slave'} at 127.0.0.1:45814
I20260709 12:22:50.611027  9419 raft_consensus.cc:606] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Received request to transfer leadership to TS 073792a321ca4850993a30440770651b
I20260709 12:22:50.613131  9419 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
mode: GRACEFUL
new_leader_uuid: "c391b1d7ed4a499c94f868ca14e87074"
 from {username='slave'} at 127.0.0.1:45814
I20260709 12:22:50.613574  9419 raft_consensus.cc:606] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Received request to transfer leadership to TS c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:50.615726  9419 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907"
dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
mode: GRACEFUL
new_leader_uuid: "c391b1d7ed4a499c94f868ca14e87074"
 from {username='slave'} at 127.0.0.1:45814
I20260709 12:22:50.616189  9419 raft_consensus.cc:606] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Received request to transfer leadership to TS c391b1d7ed4a499c94f868ca14e87074
I20260709 12:22:50.617275  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:22:50.617765  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:50.631183  9771 raft_consensus.cc:993] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88: : Instructing follower 073792a321ca4850993a30440770651b to start an election
I20260709 12:22:50.631986  9776 raft_consensus.cc:1081] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Signalling peer 073792a321ca4850993a30440770651b to start an election
I20260709 12:22:50.633383  9569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb471b9e458540c488a45423b29c9882"
dest_uuid: "073792a321ca4850993a30440770651b"
 from {username='slave'} at 127.0.0.1:55356
I20260709 12:22:50.633916  9569 raft_consensus.cc:493] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:50.634268  9569 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:50.639143  9569 raft_consensus.cc:515] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.641162  9569 leader_election.cc:290] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 4 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:50.641988  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:50.642113  9493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb471b9e458540c488a45423b29c9882" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:50.642640  9419 raft_consensus.cc:3055] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:50.642748  9493 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:50.642928  9419 raft_consensus.cc:740] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:50.643664  9419 consensus_queue.cc:260] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.644512  9419 raft_consensus.cc:3060] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:50.648717  9493 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 4.
I20260709 12:22:50.650017  9535 leader_election.cc:304] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: 
I20260709 12:22:50.650385  9419 raft_consensus.cc:2468] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 4.
I20260709 12:22:50.651054  9746 raft_consensus.cc:2804] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:50.652108  9746 raft_consensus.cc:697] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 4 LEADER]: Becoming Leader. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:50.652834  9746 consensus_queue.cc:237] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.659685  9315 catalog_manager.cc:5697] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b reported cstate change: term changed from 3 to 4, leader changed from eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193) to 073792a321ca4850993a30440770651b (127.8.137.195). New cstate: current_term: 4 leader_uuid: "073792a321ca4850993a30440770651b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:50.819648  9778 raft_consensus.cc:993] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88: : Instructing follower c391b1d7ed4a499c94f868ca14e87074 to start an election
I20260709 12:22:50.820137  9776 raft_consensus.cc:1081] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Signalling peer c391b1d7ed4a499c94f868ca14e87074 to start an election
I20260709 12:22:50.821683  9493 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
 from {username='slave'} at 127.0.0.1:35674
I20260709 12:22:50.822101  9493 raft_consensus.cc:493] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:50.822392  9493 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.826567  9493 raft_consensus.cc:515] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.828753  9493 leader_election.cc:290] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 3 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:50.829579  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:50.829851  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6ae3e6fbb274773ab45fee44b483907" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:50.830179  9419 raft_consensus.cc:3055] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:50.830384  9569 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.830529  9419 raft_consensus.cc:740] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:50.831351  9419 consensus_queue.cc:260] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.832468  9419 raft_consensus.cc:3060] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:50.836803  9778 raft_consensus.cc:993] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88: : Instructing follower c391b1d7ed4a499c94f868ca14e87074 to start an election
I20260709 12:22:50.837247  9569 raft_consensus.cc:2468] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 3.
I20260709 12:22:50.837344  9776 raft_consensus.cc:1081] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Signalling peer c391b1d7ed4a499c94f868ca14e87074 to start an election
I20260709 12:22:50.838532  9462 leader_election.cc:304] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: 
I20260709 12:22:50.839150  9493 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62"
dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
 from {username='slave'} at 127.0.0.1:35674
I20260709 12:22:50.839370  9419 raft_consensus.cc:2468] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 3.
I20260709 12:22:50.839824  9493 raft_consensus.cc:493] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:50.839833  9777 raft_consensus.cc:2804] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:50.840363  9493 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:50.841475  9777 raft_consensus.cc:697] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 3 LEADER]: Becoming Leader. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:50.842350  9777 consensus_queue.cc:237] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.845615  9493 raft_consensus.cc:515] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.849200  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:50.849460  9493 leader_election.cc:290] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 4 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), 073792a321ca4850993a30440770651b (127.8.137.195:37873)
I20260709 12:22:50.849781  9419 raft_consensus.cc:3055] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:22:50.849591  9569 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb007f7d565438ba215d4ccad1c2e62" candidate_uuid: "c391b1d7ed4a499c94f868ca14e87074" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "073792a321ca4850993a30440770651b"
I20260709 12:22:50.850176  9419 raft_consensus.cc:740] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:50.850436  9569 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:50.850991  9419 consensus_queue.cc:260] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.851795  9419 raft_consensus.cc:3060] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:22:50.852072  9315 catalog_manager.cc:5697] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 reported cstate change: term changed from 2 to 3, leader changed from eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193) to c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "c391b1d7ed4a499c94f868ca14e87074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:50.856307  9419 raft_consensus.cc:2468] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 4.
I20260709 12:22:50.856756  9569 raft_consensus.cc:2468] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c391b1d7ed4a499c94f868ca14e87074 in term 4.
I20260709 12:22:50.857591  9459 leader_election.cc:304] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c391b1d7ed4a499c94f868ca14e87074, eb02c76cdbdd4cf190ca05caf642fe88; no voters: 
I20260709 12:22:50.858403  9777 raft_consensus.cc:2804] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:22:50.858886  9777 raft_consensus.cc:697] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 4 LEADER]: Becoming Leader. State: Replica: c391b1d7ed4a499c94f868ca14e87074, State: Running, Role: LEADER
I20260709 12:22:50.859725  9777 consensus_queue.cc:237] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:50.866385  9315 catalog_manager.cc:5697] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 reported cstate change: term changed from 3 to 4, leader changed from eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193) to c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "c391b1d7ed4a499c94f868ca14e87074" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: UNKNOWN } } }
I20260709 12:22:51.152316  9778 raft_consensus.cc:993] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88: : Instructing follower 073792a321ca4850993a30440770651b to start an election
I20260709 12:22:51.152942  9778 raft_consensus.cc:1081] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Signalling peer 073792a321ca4850993a30440770651b to start an election
I20260709 12:22:51.154531  9569 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3"
dest_uuid: "073792a321ca4850993a30440770651b"
 from {username='slave'} at 127.0.0.1:55356
I20260709 12:22:51.155046  9569 raft_consensus.cc:493] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:22:51.155330  9569 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:51.159546  9569 raft_consensus.cc:515] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:51.161049  9569 leader_election.cc:290] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 3 election: Requested vote from peers eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325), c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799)
I20260709 12:22:51.161988  9419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eb02c76cdbdd4cf190ca05caf642fe88"
I20260709 12:22:51.162082  9493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283fdca0e28146e99bab344a8a595fd3" candidate_uuid: "073792a321ca4850993a30440770651b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "c391b1d7ed4a499c94f868ca14e87074"
I20260709 12:22:51.162789  9419 raft_consensus.cc:3055] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:22:51.162878  9493 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:51.163125  9419 raft_consensus.cc:740] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: eb02c76cdbdd4cf190ca05caf642fe88, State: Running, Role: LEADER
I20260709 12:22:51.164048  9419 consensus_queue.cc:260] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:51.164942  9419 raft_consensus.cc:3060] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:22:51.168236  9493 raft_consensus.cc:2468] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 3.
I20260709 12:22:51.170087  9417 raft_consensus.cc:1275] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Refusing update from remote peer 073792a321ca4850993a30440770651b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:51.169920  9535 leader_election.cc:304] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073792a321ca4850993a30440770651b, c391b1d7ed4a499c94f868ca14e87074; no voters: 
I20260709 12:22:51.170949  9746 raft_consensus.cc:2804] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:22:51.171097  9419 raft_consensus.cc:2468] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073792a321ca4850993a30440770651b in term 3.
I20260709 12:22:51.172199  9746 raft_consensus.cc:697] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 3 LEADER]: Becoming Leader. State: Replica: 073792a321ca4850993a30440770651b, State: Running, Role: LEADER
I20260709 12:22:51.172595  9788 consensus_queue.cc:1048] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:51.173580  9746 consensus_queue.cc:237] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } }
I20260709 12:22:51.183450  9318 catalog_manager.cc:5697] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b reported cstate change: term changed from 2 to 3, leader changed from eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193) to 073792a321ca4850993a30440770651b (127.8.137.195). New cstate: current_term: 3 leader_uuid: "073792a321ca4850993a30440770651b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 } health_report { overall_health: HEALTHY } } }
I20260709 12:22:51.185286  9493 raft_consensus.cc:1275] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Refusing update from remote peer 073792a321ca4850993a30440770651b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:51.186467  9788 consensus_queue.cc:1048] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:22:51.228083  9569 raft_consensus.cc:1275] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Refusing update from remote peer c391b1d7ed4a499c94f868ca14e87074: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:51.229230  9777 consensus_queue.cc:1048] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:51.238376  9419 raft_consensus.cc:1275] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Refusing update from remote peer c391b1d7ed4a499c94f868ca14e87074: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:51.241088  9785 consensus_queue.cc:1048] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:22:51.282891  9419 raft_consensus.cc:1275] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Refusing update from remote peer c391b1d7ed4a499c94f868ca14e87074: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:51.285166  9785 consensus_queue.cc:1048] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:51.299957  9569 raft_consensus.cc:1275] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Refusing update from remote peer c391b1d7ed4a499c94f868ca14e87074: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:22:51.301582  9797 consensus_queue.cc:1048] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073792a321ca4850993a30440770651b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 37873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:22:51.618968  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:51.625465  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:51.625953  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:51.665894  9493 raft_consensus.cc:1275] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Refusing update from remote peer 073792a321ca4850993a30440770651b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:51.667210  9788 consensus_queue.cc:1048] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c391b1d7ed4a499c94f868ca14e87074" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 38799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:51.670984  9417 raft_consensus.cc:1275] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Refusing update from remote peer 073792a321ca4850993a30440770651b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:22:51.672374  9788 consensus_queue.cc:1048] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb02c76cdbdd4cf190ca05caf642fe88" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:22:52.627019  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:52.632122  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:52.632472  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:53.633322  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:53.639389  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:53.639928  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:54.641029  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:54.647449  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:54.647905  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:55.648803  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:55.654469  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:55.654997  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:56.656076  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:56.661523  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:56.661986  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:57.663060  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:57.668985  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:57.669449  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:58.670485  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:58.675882  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:58.676388  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:22:59.677386  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:22:59.684557  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:22:59.685127  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:00.717070  8743 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:23:00.717607  8743 tablet_server.cc:179] TabletServer@127.8.137.193:0 shutting down...
I20260709 12:23:00.750774  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:00.751502  8743 tablet_replica.cc:333] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.752147  8743 raft_consensus.cc:2243] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.752946  8743 raft_consensus.cc:2272] T 87182730c6014f38bb9b60e4ed2486b1 P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.755393  8743 tablet_replica.cc:333] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.756086  8743 raft_consensus.cc:2243] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.756523  8743 raft_consensus.cc:2272] T 283fdca0e28146e99bab344a8a595fd3 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.758142  8743 tablet_replica.cc:333] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.758761  8743 raft_consensus.cc:2243] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.759191  8743 raft_consensus.cc:2272] T 59b9ed1fc7864936bebdafc1a048f703 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.761029  8743 tablet_replica.cc:333] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.761557  8743 raft_consensus.cc:2243] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.762320  8743 raft_consensus.cc:2272] T 082b54dc8c5b40b5a30cdac841befab9 P eb02c76cdbdd4cf190ca05caf642fe88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.764225  8743 tablet_replica.cc:333] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.764824  8743 raft_consensus.cc:2243] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.765237  8743 raft_consensus.cc:2272] T bcb007f7d565438ba215d4ccad1c2e62 P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.767086  8743 tablet_replica.cc:333] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.767589  8743 raft_consensus.cc:2243] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.768482  8743 raft_consensus.cc:2272] T b87d95fb9055412597d6ea198986cf8e P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.771060  8743 tablet_replica.cc:333] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.771727  8743 raft_consensus.cc:2243] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.772130  8743 raft_consensus.cc:2272] T f6ae3e6fbb274773ab45fee44b483907 P eb02c76cdbdd4cf190ca05caf642fe88 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.773866  8743 tablet_replica.cc:333] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.774396  8743 raft_consensus.cc:2243] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.774871  8743 raft_consensus.cc:2272] T f4b231a1993d47de80e4fb53cca4c3de P eb02c76cdbdd4cf190ca05caf642fe88 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.776594  8743 tablet_replica.cc:333] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.777069  8743 raft_consensus.cc:2243] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.777827  8743 raft_consensus.cc:2272] T 88f20b0eaf3a4f82a482e301b778fdfe P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.779515  8743 tablet_replica.cc:333] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88: stopping tablet replica
I20260709 12:23:00.779982  8743 raft_consensus.cc:2243] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.780396  8743 raft_consensus.cc:2272] T bb471b9e458540c488a45423b29c9882 P eb02c76cdbdd4cf190ca05caf642fe88 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.811868  8743 tablet_server.cc:196] TabletServer@127.8.137.193:0 shutdown complete.
W20260709 12:23:00.829140  9533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.193:43325: connect: Connection refused (error 111) [suppressed 4 similar messages]
W20260709 12:23:00.833469  9533 consensus_peers.cc:597] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b -> Peer eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325): Couldn't send request to peer eb02c76cdbdd4cf190ca05caf642fe88. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:43325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:00.836040  8743 tablet_server.cc:179] TabletServer@127.8.137.194:0 shutting down...
W20260709 12:23:00.839967  9533 consensus_peers.cc:597] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b -> Peer eb02c76cdbdd4cf190ca05caf642fe88 (127.8.137.193:43325): Couldn't send request to peer eb02c76cdbdd4cf190ca05caf642fe88. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:43325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:00.865551  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:00.866313  8743 tablet_replica.cc:333] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.867350  8743 raft_consensus.cc:2243] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.867893  8743 raft_consensus.cc:2272] T 082b54dc8c5b40b5a30cdac841befab9 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.869972  8743 tablet_replica.cc:333] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.870507  8743 raft_consensus.cc:2243] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.871524  8743 raft_consensus.cc:2272] T bcb007f7d565438ba215d4ccad1c2e62 P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.873272  8743 tablet_replica.cc:333] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.873843  8743 raft_consensus.cc:2243] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.874249  8743 raft_consensus.cc:2272] T 87182730c6014f38bb9b60e4ed2486b1 P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.876009  8743 tablet_replica.cc:333] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.876458  8743 raft_consensus.cc:2243] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.876905  8743 raft_consensus.cc:2272] T 283fdca0e28146e99bab344a8a595fd3 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.878968  8743 tablet_replica.cc:333] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.879542  8743 raft_consensus.cc:2243] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.880731  8743 raft_consensus.cc:2272] T f6ae3e6fbb274773ab45fee44b483907 P c391b1d7ed4a499c94f868ca14e87074 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:23:00.880765  9535 consensus_peers.cc:597] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b -> Peer c391b1d7ed4a499c94f868ca14e87074 (127.8.137.194:38799): Couldn't send request to peer c391b1d7ed4a499c94f868ca14e87074. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:00.883482  8743 tablet_replica.cc:333] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.884004  8743 raft_consensus.cc:2243] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.884445  8743 raft_consensus.cc:2272] T b87d95fb9055412597d6ea198986cf8e P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.886113  8743 tablet_replica.cc:333] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.886991  8743 raft_consensus.cc:2243] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.887796  8743 raft_consensus.cc:2272] T f4b231a1993d47de80e4fb53cca4c3de P c391b1d7ed4a499c94f868ca14e87074 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.890133  8743 tablet_replica.cc:333] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.890830  8743 raft_consensus.cc:2243] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.891335  8743 raft_consensus.cc:2272] T 88f20b0eaf3a4f82a482e301b778fdfe P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.893191  8743 tablet_replica.cc:333] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.893708  8743 raft_consensus.cc:2243] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.894250  8743 raft_consensus.cc:2272] T bb471b9e458540c488a45423b29c9882 P c391b1d7ed4a499c94f868ca14e87074 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.895946  8743 tablet_replica.cc:333] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074: stopping tablet replica
I20260709 12:23:00.896430  8743 raft_consensus.cc:2243] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.896833  8743 raft_consensus.cc:2272] T 59b9ed1fc7864936bebdafc1a048f703 P c391b1d7ed4a499c94f868ca14e87074 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.926175  8743 tablet_server.cc:196] TabletServer@127.8.137.194:0 shutdown complete.
I20260709 12:23:00.947458  8743 tablet_server.cc:179] TabletServer@127.8.137.195:0 shutting down...
I20260709 12:23:00.976373  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:23:00.977144  8743 tablet_replica.cc:333] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.978068  8743 raft_consensus.cc:2243] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.979059  8743 raft_consensus.cc:2272] T 283fdca0e28146e99bab344a8a595fd3 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.980830  8743 tablet_replica.cc:333] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.981304  8743 raft_consensus.cc:2243] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:00.982033  8743 raft_consensus.cc:2272] T 59b9ed1fc7864936bebdafc1a048f703 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.983932  8743 tablet_replica.cc:333] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.984431  8743 raft_consensus.cc:2243] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.984869  8743 raft_consensus.cc:2272] T f6ae3e6fbb274773ab45fee44b483907 P 073792a321ca4850993a30440770651b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.986490  8743 tablet_replica.cc:333] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.987109  8743 raft_consensus.cc:2243] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.987537  8743 raft_consensus.cc:2272] T b87d95fb9055412597d6ea198986cf8e P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.989251  8743 tablet_replica.cc:333] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.989744  8743 raft_consensus.cc:2243] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.990159  8743 raft_consensus.cc:2272] T 082b54dc8c5b40b5a30cdac841befab9 P 073792a321ca4850993a30440770651b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.991824  8743 tablet_replica.cc:333] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.992326  8743 raft_consensus.cc:2243] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.992753  8743 raft_consensus.cc:2272] T bcb007f7d565438ba215d4ccad1c2e62 P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.994436  8743 tablet_replica.cc:333] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.994944  8743 raft_consensus.cc:2243] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.995343  8743 raft_consensus.cc:2272] T 88f20b0eaf3a4f82a482e301b778fdfe P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.996934  8743 tablet_replica.cc:333] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.997430  8743 raft_consensus.cc:2243] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:00.997843  8743 raft_consensus.cc:2272] T f4b231a1993d47de80e4fb53cca4c3de P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:00.999495  8743 tablet_replica.cc:333] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:00.999964  8743 raft_consensus.cc:2243] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:23:01.001199  8743 raft_consensus.cc:2272] T bb471b9e458540c488a45423b29c9882 P 073792a321ca4850993a30440770651b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:01.002957  8743 tablet_replica.cc:333] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b: stopping tablet replica
I20260709 12:23:01.003458  8743 raft_consensus.cc:2243] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:01.003919  8743 raft_consensus.cc:2272] T 87182730c6014f38bb9b60e4ed2486b1 P 073792a321ca4850993a30440770651b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:01.052634  8743 tablet_server.cc:196] TabletServer@127.8.137.195:0 shutdown complete.
I20260709 12:23:01.073347  8743 master.cc:562] Master@127.8.137.254:36741 shutting down...
I20260709 12:23:01.099226  8743 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:23:01.099817  8743 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:01.100201  8743 tablet_replica.cc:333] T 00000000000000000000000000000000 P ae4afe2b778048959d9bf0888e13198b: stopping tablet replica
I20260709 12:23:01.121934  8743 master.cc:584] Master@127.8.137.254:36741 shutdown complete.
073792a321ca4850993a30440770651b  1
c391b1d7ed4a499c94f868ca14e87074  4
eb02c76cdbdd4cf190ca05caf642fe88  5
073792a321ca4850993a30440770651b  3
c391b1d7ed4a499c94f868ca14e87074  4
eb02c76cdbdd4cf190ca05caf642fe88  3
073792a321ca4850993a30440770651b  1
c391b1d7ed4a499c94f868ca14e87074  1
eb02c76cdbdd4cf190ca05caf642fe88  8
073792a321ca4850993a30440770651b  3
c391b1d7ed4a499c94f868ca14e87074  3
eb02c76cdbdd4cf190ca05caf642fe88  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28684 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:23:01.166612  8743 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.137.254:38721
I20260709 12:23:01.168849  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:01.175097  9818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:01.175504  9819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:01.175794  9821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:01.177317  8743 server_base.cc:1061] running on GCE node
I20260709 12:23:01.178241  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:01.178463  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:01.178653  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599781178629 us; error 0 us; skew 500 ppm
I20260709 12:23:01.179193  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:01.181639  8743 webserver.cc:533] Webserver started at http://127.8.137.254:44477/ using document root <none> and password file <none>
I20260709 12:23:01.182144  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:01.182344  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:01.182682  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:01.183852  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/master-0-root/instance:
uuid: "2121bfede85b447c9f51ef23233606cc"
format_stamp: "Formatted at 2026-07-09 12:23:01 on dist-test-slave-kpwv"
I20260709 12:23:01.188588  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:23:01.192281  9826 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.193190  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 12:23:01.193593  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/master-0-root
uuid: "2121bfede85b447c9f51ef23233606cc"
format_stamp: "Formatted at 2026-07-09 12:23:01 on dist-test-slave-kpwv"
I20260709 12:23:01.193905  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:01.214394  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:01.215690  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:01.259347  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.254:38721
I20260709 12:23:01.259492  9879 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.254:38721 every 8 connection(s)
I20260709 12:23:01.265357  9880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:01.278038  9880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Bootstrap starting.
I20260709 12:23:01.296765  9880 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:01.303768  9880 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: No bootstrap required, opened a new log
I20260709 12:23:01.307744  9880 raft_consensus.cc:359] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2121bfede85b447c9f51ef23233606cc" member_type: VOTER }
I20260709 12:23:01.308530  9880 raft_consensus.cc:385] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:01.308794  9880 raft_consensus.cc:740] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2121bfede85b447c9f51ef23233606cc, State: Initialized, Role: FOLLOWER
I20260709 12:23:01.309528  9880 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2121bfede85b447c9f51ef23233606cc" member_type: VOTER }
I20260709 12:23:01.310062  9880 raft_consensus.cc:399] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:23:01.310323  9880 raft_consensus.cc:493] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:23:01.310676  9880 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:01.315771  9880 raft_consensus.cc:515] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2121bfede85b447c9f51ef23233606cc" member_type: VOTER }
I20260709 12:23:01.316370  9880 leader_election.cc:304] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2121bfede85b447c9f51ef23233606cc; no voters: 
I20260709 12:23:01.317585  9880 leader_election.cc:290] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:23:01.318045  9884 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:01.319473  9884 raft_consensus.cc:697] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 1 LEADER]: Becoming Leader. State: Replica: 2121bfede85b447c9f51ef23233606cc, State: Running, Role: LEADER
I20260709 12:23:01.320289  9884 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2121bfede85b447c9f51ef23233606cc" member_type: VOTER }
I20260709 12:23:01.321173  9880 sys_catalog.cc:565] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:23:01.327762  9885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2121bfede85b447c9f51ef23233606cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2121bfede85b447c9f51ef23233606cc" member_type: VOTER } }
I20260709 12:23:01.328985  9886 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2121bfede85b447c9f51ef23233606cc. Latest consensus state: current_term: 1 leader_uuid: "2121bfede85b447c9f51ef23233606cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2121bfede85b447c9f51ef23233606cc" member_type: VOTER } }
I20260709 12:23:01.329481  9885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:01.330794  9886 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [sys.catalog]: This master's current role is: LEADER
I20260709 12:23:01.333954  9892 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:23:01.342334  9892 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:23:01.343946  8743 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:23:01.353663  9892 catalog_manager.cc:1383] Generated new cluster ID: cbdfb581103a4fa9aeacb0978560949e
I20260709 12:23:01.353984  9892 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:23:01.374281  9892 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:23:01.375680  9892 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:23:01.394693  9892 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Generated new TSK 0
I20260709 12:23:01.395547  9892 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:23:01.411777  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:01.421178  9903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:01.423563  9904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:01.424209  8743 server_base.cc:1061] running on GCE node
W20260709 12:23:01.424211  9906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:01.425621  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:01.425897  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:01.426095  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599781426076 us; error 0 us; skew 500 ppm
I20260709 12:23:01.426784  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:01.429622  8743 webserver.cc:533] Webserver started at http://127.8.137.193:41031/ using document root <none> and password file <none>
I20260709 12:23:01.430167  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:01.430383  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:01.430764  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:01.432072  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/instance:
uuid: "cd983415389444e292b2330ef630850a"
format_stamp: "Formatted at 2026-07-09 12:23:01 on dist-test-slave-kpwv"
I20260709 12:23:01.437788  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:23:01.442137  9911 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.443302  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:23:01.443609  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "cd983415389444e292b2330ef630850a"
format_stamp: "Formatted at 2026-07-09 12:23:01 on dist-test-slave-kpwv"
I20260709 12:23:01.443903  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:01.470808  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:01.472148  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:01.473671  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:01.476331  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:01.476524  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.476723  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:01.477221  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.527633  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:34909
I20260709 12:23:01.527818  9973 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:34909 every 8 connection(s)
I20260709 12:23:01.533357  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:01.541064  9978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:01.543219  9979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:01.546154  8743 server_base.cc:1061] running on GCE node
W20260709 12:23:01.546284  9981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:01.548213  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:01.548578  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:01.548799  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599781548778 us; error 0 us; skew 500 ppm
I20260709 12:23:01.549465  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:01.552595  9974 heartbeater.cc:344] Connected to a master server at 127.8.137.254:38721
I20260709 12:23:01.552807  8743 webserver.cc:533] Webserver started at http://127.8.137.194:36319/ using document root <none> and password file <none>
I20260709 12:23:01.553048  9974 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:01.553458  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:01.553726  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:01.553936  9974 heartbeater.cc:507] Master 127.8.137.254:38721 requested a full tablet report, sending...
I20260709 12:23:01.554047  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:01.555560  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/instance:
uuid: "9aa345bcd43b44088114b7b6f2830b17"
format_stamp: "Formatted at 2026-07-09 12:23:01 on dist-test-slave-kpwv"
I20260709 12:23:01.556790  9843 ts_manager.cc:194] Registered new tserver with Master: cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:01.558842  9843 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44152
I20260709 12:23:01.561048  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.003s
I20260709 12:23:01.565310  9986 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.566146  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 12:23:01.566494  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root
uuid: "9aa345bcd43b44088114b7b6f2830b17"
format_stamp: "Formatted at 2026-07-09 12:23:01 on dist-test-slave-kpwv"
I20260709 12:23:01.566895  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:01.582093  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:01.583422  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:01.584998  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:01.587366  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:01.587594  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.587860  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:01.588032  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.632736  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.194:35115
I20260709 12:23:01.632957 10049 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.194:35115 every 8 connection(s)
I20260709 12:23:01.638200  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:01.646009 10053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:01.647248 10054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:01.652168  8743 server_base.cc:1061] running on GCE node
W20260709 12:23:01.653115 10056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:01.654325  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:01.654631  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:01.654641 10050 heartbeater.cc:344] Connected to a master server at 127.8.137.254:38721
I20260709 12:23:01.654906  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599781654886 us; error 0 us; skew 500 ppm
I20260709 12:23:01.655160 10050 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:01.655690  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:01.655988 10050 heartbeater.cc:507] Master 127.8.137.254:38721 requested a full tablet report, sending...
I20260709 12:23:01.658456  9843 ts_manager.cc:194] Registered new tserver with Master: 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:01.658567  8743 webserver.cc:533] Webserver started at http://127.8.137.195:41031/ using document root <none> and password file <none>
I20260709 12:23:01.659341  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:01.659626  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:01.659956  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:01.660498  9843 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44160
I20260709 12:23:01.661636  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/instance:
uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
format_stamp: "Formatted at 2026-07-09 12:23:01 on dist-test-slave-kpwv"
I20260709 12:23:01.666497  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.002s
I20260709 12:23:01.670346 10061 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.671401  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:23:01.671995  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root
uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
format_stamp: "Formatted at 2026-07-09 12:23:01 on dist-test-slave-kpwv"
I20260709 12:23:01.672394  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:01.683890  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:01.685210  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:01.686851  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:01.689164  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:01.689426  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.689689  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:01.689870  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:01.733796  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.195:33983
I20260709 12:23:01.733916 10123 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.195:33983 every 8 connection(s)
I20260709 12:23:01.749728 10124 heartbeater.cc:344] Connected to a master server at 127.8.137.254:38721
I20260709 12:23:01.750260 10124 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:01.751153 10124 heartbeater.cc:507] Master 127.8.137.254:38721 requested a full tablet report, sending...
I20260709 12:23:01.753474  9843 ts_manager.cc:194] Registered new tserver with Master: 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:01.754454  8743 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017359958s
I20260709 12:23:01.755093  9843 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44164
I20260709 12:23:01.784533  9843 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44176:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:23:01.791446  9843 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:23:02.011595 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 53edc906d1af41e0992f01434995f9d1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:02.012331 10015 tablet_service.cc:1511] Processing CreateTablet for tablet 26e02c55bc5648e0baa2a4216f02f955 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:02.013756 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53edc906d1af41e0992f01434995f9d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.014899  9939 tablet_service.cc:1511] Processing CreateTablet for tablet 26e02c55bc5648e0baa2a4216f02f955 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:02.016686  9939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e02c55bc5648e0baa2a4216f02f955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.016840 10013 tablet_service.cc:1511] Processing CreateTablet for tablet 912ffd65ac564403ac65b7a5d005d367 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:02.018504 10013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ffd65ac564403ac65b7a5d005d367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.019407 10015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e02c55bc5648e0baa2a4216f02f955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.019055  9938 tablet_service.cc:1511] Processing CreateTablet for tablet 53edc906d1af41e0992f01434995f9d1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:02.020606  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53edc906d1af41e0992f01434995f9d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.022091  9937 tablet_service.cc:1511] Processing CreateTablet for tablet 912ffd65ac564403ac65b7a5d005d367 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:02.023540  9937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ffd65ac564403ac65b7a5d005d367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.025313 10012 tablet_service.cc:1511] Processing CreateTablet for tablet 181ca8c0fd384c8c909b095284d9e12c (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:02.026865 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181ca8c0fd384c8c909b095284d9e12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.030143 10011 tablet_service.cc:1511] Processing CreateTablet for tablet ab118912551248c1985fce40a2e815b0 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:02.031739 10011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab118912551248c1985fce40a2e815b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.035468 10010 tablet_service.cc:1511] Processing CreateTablet for tablet d8453bea911c4756add721edebd5d124 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:02.036885 10010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8453bea911c4756add721edebd5d124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.040270 10009 tablet_service.cc:1511] Processing CreateTablet for tablet bf49747e9a8542e6b75e3545a18c0981 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:02.041754 10009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf49747e9a8542e6b75e3545a18c0981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.045033 10008 tablet_service.cc:1511] Processing CreateTablet for tablet 208adf3e1124402cb1e6c43ba7306b7b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:02.050177 10007 tablet_service.cc:1511] Processing CreateTablet for tablet aa5a16c274db44a5aa641b51f71f7ec6 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:02.052397 10008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208adf3e1124402cb1e6c43ba7306b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.058786 10007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa5a16c274db44a5aa641b51f71f7ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.061488 10006 tablet_service.cc:1511] Processing CreateTablet for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:02.067950 10086 tablet_service.cc:1511] Processing CreateTablet for tablet 181ca8c0fd384c8c909b095284d9e12c (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:02.056191  9939 tablet_service.cc:1511] Processing CreateTablet for tablet 181ca8c0fd384c8c909b095284d9e12c (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:23:02.069487 10086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181ca8c0fd384c8c909b095284d9e12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.069161 10084 tablet_service.cc:1511] Processing CreateTablet for tablet d8453bea911c4756add721edebd5d124 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:02.063817  9936 tablet_service.cc:1511] Processing CreateTablet for tablet ab118912551248c1985fce40a2e815b0 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:02.070730 10084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8453bea911c4756add721edebd5d124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.072562 10085 tablet_service.cc:1511] Processing CreateTablet for tablet ab118912551248c1985fce40a2e815b0 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:23:02.073980 10085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab118912551248c1985fce40a2e815b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.074869  9935 tablet_service.cc:1511] Processing CreateTablet for tablet d8453bea911c4756add721edebd5d124 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:23:02.075728 10087 tablet_service.cc:1511] Processing CreateTablet for tablet 912ffd65ac564403ac65b7a5d005d367 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:23:02.076174 10083 tablet_service.cc:1511] Processing CreateTablet for tablet bf49747e9a8542e6b75e3545a18c0981 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:02.077095 10087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ffd65ac564403ac65b7a5d005d367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.077214  9934 tablet_service.cc:1511] Processing CreateTablet for tablet bf49747e9a8542e6b75e3545a18c0981 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:23:02.078174 10083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf49747e9a8542e6b75e3545a18c0981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.078959 10006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.079463  9933 tablet_service.cc:1511] Processing CreateTablet for tablet 208adf3e1124402cb1e6c43ba7306b7b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:02.081816  9932 tablet_service.cc:1511] Processing CreateTablet for tablet aa5a16c274db44a5aa641b51f71f7ec6 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:02.083441 10088 tablet_service.cc:1511] Processing CreateTablet for tablet 53edc906d1af41e0992f01434995f9d1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:23:02.084787 10088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53edc906d1af41e0992f01434995f9d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.064033 10089 tablet_service.cc:1511] Processing CreateTablet for tablet 26e02c55bc5648e0baa2a4216f02f955 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:23:02.087658 10089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26e02c55bc5648e0baa2a4216f02f955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.088342  9931 tablet_service.cc:1511] Processing CreateTablet for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:02.069646  9939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181ca8c0fd384c8c909b095284d9e12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.090765  9930 tablet_service.cc:1511] Processing CreateTablet for tablet e346f82bc2694f9e968337cc4be9e53e (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:02.092062  9930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e346f82bc2694f9e968337cc4be9e53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.092904 10082 tablet_service.cc:1511] Processing CreateTablet for tablet 208adf3e1124402cb1e6c43ba7306b7b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:23:02.094347 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208adf3e1124402cb1e6c43ba7306b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.094964  9936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab118912551248c1985fce40a2e815b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.095613  9934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf49747e9a8542e6b75e3545a18c0981. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.098141  9933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 208adf3e1124402cb1e6c43ba7306b7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.100194  9931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.103655  9935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8453bea911c4756add721edebd5d124. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.104502 10081 tablet_service.cc:1511] Processing CreateTablet for tablet aa5a16c274db44a5aa641b51f71f7ec6 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:23:02.107111  9932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa5a16c274db44a5aa641b51f71f7ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.115003 10081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa5a16c274db44a5aa641b51f71f7ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.110829 10015 tablet_service.cc:1511] Processing CreateTablet for tablet e346f82bc2694f9e968337cc4be9e53e (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:02.119676 10015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e346f82bc2694f9e968337cc4be9e53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.123919 10139 tablet_bootstrap.cc:492] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:02.136437 10080 tablet_service.cc:1511] Processing CreateTablet for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:23:02.138093 10080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.151444 10087 tablet_service.cc:1511] Processing CreateTablet for tablet e346f82bc2694f9e968337cc4be9e53e (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:23:02.153862 10011 tablet_service.cc:1511] Processing CreateTablet for tablet 32a9a1d2d5fc4243bf902da3519cd483 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:02.157859 10011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32a9a1d2d5fc4243bf902da3519cd483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.157948 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 78df0ad0c1454828a838cd88472c8553 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:02.160265 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78df0ad0c1454828a838cd88472c8553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.158773 10087 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e346f82bc2694f9e968337cc4be9e53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.182039 10141 tablet_bootstrap.cc:492] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:02.191893 10141 tablet_bootstrap.cc:654] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.179062 10139 tablet_bootstrap.cc:654] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.210747 10008 tablet_service.cc:1511] Processing CreateTablet for tablet 1892c18d4e1644109d95b077e989b10f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:02.212206 10089 tablet_service.cc:1511] Processing CreateTablet for tablet 1892c18d4e1644109d95b077e989b10f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:02.213284 10008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1892c18d4e1644109d95b077e989b10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.213698 10089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1892c18d4e1644109d95b077e989b10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.220072  9938 tablet_service.cc:1511] Processing CreateTablet for tablet 78df0ad0c1454828a838cd88472c8553 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:02.220505  9931 tablet_service.cc:1511] Processing CreateTablet for tablet 32a9a1d2d5fc4243bf902da3519cd483 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:02.221603  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78df0ad0c1454828a838cd88472c8553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.224454  9931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32a9a1d2d5fc4243bf902da3519cd483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.228869  9939 tablet_service.cc:1511] Processing CreateTablet for tablet 1892c18d4e1644109d95b077e989b10f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:23:02.229746 10080 tablet_service.cc:1511] Processing CreateTablet for tablet 32a9a1d2d5fc4243bf902da3519cd483 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:23:02.230484  9939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1892c18d4e1644109d95b077e989b10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.231199 10080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32a9a1d2d5fc4243bf902da3519cd483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.242045 10082 tablet_service.cc:1511] Processing CreateTablet for tablet 78df0ad0c1454828a838cd88472c8553 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:23:02.243548 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78df0ad0c1454828a838cd88472c8553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.259259 10088 tablet_service.cc:1511] Processing CreateTablet for tablet d864b424b78d4fffb51785ea7d3b74ed (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:02.267887 10082 tablet_service.cc:1511] Processing CreateTablet for tablet d43a22dba58d423d907a568238e85284 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:02.269317 10088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d864b424b78d4fffb51785ea7d3b74ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.271550 10084 tablet_service.cc:1511] Processing CreateTablet for tablet eb955e753b7546bf8612b6ed1c3fc342 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:02.275983 10140 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:02.297875 10083 tablet_service.cc:1511] Processing CreateTablet for tablet 1154fd85c1e440b7bb5ee236ac1e0a21 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:02.299818 10083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1154fd85c1e440b7bb5ee236ac1e0a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.302148 10084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb955e753b7546bf8612b6ed1c3fc342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.305035 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d43a22dba58d423d907a568238e85284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.304641 10086 tablet_service.cc:1511] Processing CreateTablet for tablet d4909dfc06b24d70bfd398607779f5ae (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:02.317726 10086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4909dfc06b24d70bfd398607779f5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.301764 10012 tablet_service.cc:1511] Processing CreateTablet for tablet d864b424b78d4fffb51785ea7d3b74ed (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:02.330626 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d864b424b78d4fffb51785ea7d3b74ed. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:02.349037  9829 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52884.8us
W20260709 12:23:02.352658  9827 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62428.9us
W20260709 12:23:02.353874  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:02.400266 10008 tablet_service.cc:1511] Processing CreateTablet for tablet d43a22dba58d423d907a568238e85284 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:02.401639 10008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d43a22dba58d423d907a568238e85284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.407708 10080 tablet_service.cc:1511] Processing CreateTablet for tablet a1e3e80f2cad43f08196a18ad2bee6a4 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:02.410125 10086 tablet_service.cc:1511] Processing CreateTablet for tablet 59e61711ca80440e9c13392c6b6b25fa (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:02.414806 10141 tablet_bootstrap.cc:492] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:02.415295 10141 ts_tablet_manager.cc:1403] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.234s	user 0.049s	sys 0.065s
I20260709 12:23:02.415488 10086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e61711ca80440e9c13392c6b6b25fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.421300 10140 tablet_bootstrap.cc:654] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.424826  9931 tablet_service.cc:1511] Processing CreateTablet for tablet d4909dfc06b24d70bfd398607779f5ae (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:02.426151  9931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4909dfc06b24d70bfd398607779f5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.431514 10080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e3e80f2cad43f08196a18ad2bee6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.432341 10088 tablet_service.cc:1511] Processing CreateTablet for tablet bb6fd99043b344de885ff64fd4c81132 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:02.433616 10088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb6fd99043b344de885ff64fd4c81132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.434203  9932 tablet_service.cc:1511] Processing CreateTablet for tablet eb955e753b7546bf8612b6ed1c3fc342 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:02.382182  9934 tablet_service.cc:1511] Processing CreateTablet for tablet d864b424b78d4fffb51785ea7d3b74ed (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:23:02.438699 10006 tablet_service.cc:1511] Processing CreateTablet for tablet eb955e753b7546bf8612b6ed1c3fc342 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:23:02.439033 10009 tablet_service.cc:1511] Processing CreateTablet for tablet d4909dfc06b24d70bfd398607779f5ae (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:23:02.439404  9934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d864b424b78d4fffb51785ea7d3b74ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.440214 10006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb955e753b7546bf8612b6ed1c3fc342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.441946 10085 tablet_service.cc:1511] Processing CreateTablet for tablet afb4bd9b5d6240b19d09014b985ea7f7 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:02.443334 10085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb4bd9b5d6240b19d09014b985ea7f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.433403  9935 tablet_service.cc:1511] Processing CreateTablet for tablet d43a22dba58d423d907a568238e85284 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:23:02.438702 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 1154fd85c1e440b7bb5ee236ac1e0a21 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:02.456782  9939 tablet_service.cc:1511] Processing CreateTablet for tablet 1154fd85c1e440b7bb5ee236ac1e0a21 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:23:02.456923 10012 tablet_service.cc:1511] Processing CreateTablet for tablet a1e3e80f2cad43f08196a18ad2bee6a4 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:02.458009 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1154fd85c1e440b7bb5ee236ac1e0a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.458702 10009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4909dfc06b24d70bfd398607779f5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.453780 10010 tablet_service.cc:1511] Processing CreateTablet for tablet 59e61711ca80440e9c13392c6b6b25fa (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:02.460150 10010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e61711ca80440e9c13392c6b6b25fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.461545 10139 tablet_bootstrap.cc:492] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:02.461751 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e3e80f2cad43f08196a18ad2bee6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.462033 10139 ts_tablet_manager.cc:1403] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.338s	user 0.006s	sys 0.011s
I20260709 12:23:02.465206 10139 raft_consensus.cc:359] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.465895 10139 raft_consensus.cc:385] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.466176 10139 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.466619  9932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb955e753b7546bf8612b6ed1c3fc342. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.469130 10139 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.418462 10141 raft_consensus.cc:359] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.475251 10141 raft_consensus.cc:385] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.475641 10141 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.476447 10141 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.499862  9935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d43a22dba58d423d907a568238e85284. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.521344 10081 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd550066b4447f8848484defc31b9d1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:02.523598 10081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd550066b4447f8848484defc31b9d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.528828  9937 tablet_service.cc:1511] Processing CreateTablet for tablet 59e61711ca80440e9c13392c6b6b25fa (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:23:02.537606  9939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1154fd85c1e440b7bb5ee236ac1e0a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.538139 10050 heartbeater.cc:499] Master 127.8.137.254:38721 was elected leader, sending a full tablet report...
I20260709 12:23:02.543292 10145 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.543491 10139 ts_tablet_manager.cc:1434] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.081s	user 0.009s	sys 0.018s
I20260709 12:23:02.543855 10145 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.544566 10139 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:02.546741 10141 ts_tablet_manager.cc:1434] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.131s	user 0.015s	sys 0.012s
I20260709 12:23:02.548264 10007 tablet_service.cc:1511] Processing CreateTablet for tablet bb6fd99043b344de885ff64fd4c81132 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:02.556859 10089 tablet_service.cc:1511] Processing CreateTablet for tablet 15458ba5804c4c36b01b169af77b2691 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:02.557655 10124 heartbeater.cc:499] Master 127.8.137.254:38721 was elected leader, sending a full tablet report...
I20260709 12:23:02.558307 10089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15458ba5804c4c36b01b169af77b2691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.563256 10007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb6fd99043b344de885ff64fd4c81132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.569582 10012 tablet_service.cc:1511] Processing CreateTablet for tablet d6726f0982834ecf8987472ed6c32a73 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:02.517796 10084 tablet_service.cc:1511] Processing CreateTablet for tablet d6726f0982834ecf8987472ed6c32a73 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:02.562410  9974 heartbeater.cc:499] Master 127.8.137.254:38721 was elected leader, sending a full tablet report...
I20260709 12:23:02.596709 10015 tablet_service.cc:1511] Processing CreateTablet for tablet 15458ba5804c4c36b01b169af77b2691 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:02.598762  9930 tablet_service.cc:1511] Processing CreateTablet for tablet 15458ba5804c4c36b01b169af77b2691 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:23:02.550014 10145 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:02.603598 10141 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:02.604375 10140 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:02.609628 10140 ts_tablet_manager.cc:1403] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.334s	user 0.011s	sys 0.016s
I20260709 12:23:02.578958  9937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e61711ca80440e9c13392c6b6b25fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.613126 10140 raft_consensus.cc:359] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.613910 10140 raft_consensus.cc:385] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.614207 10140 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.608258 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6726f0982834ecf8987472ed6c32a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.607939 10146 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.615379 10146 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.618502 10146 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:02.602578 10088 tablet_service.cc:1511] Processing CreateTablet for tablet 447b5d26fe15480dbb8fedf0f7a44777 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:02.620935 10140 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.627555  9930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15458ba5804c4c36b01b169af77b2691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.586300 10083 tablet_service.cc:1511] Processing CreateTablet for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:02.592296 10084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6726f0982834ecf8987472ed6c32a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.643157 10015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15458ba5804c4c36b01b169af77b2691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.655432  9938 tablet_service.cc:1511] Processing CreateTablet for tablet afb4bd9b5d6240b19d09014b985ea7f7 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:02.658433 10082 tablet_service.cc:1511] Processing CreateTablet for tablet 93686c27d2cf4edea0d5da8d0c1d81ee (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:02.660331 10088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447b5d26fe15480dbb8fedf0f7a44777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.661916 10141 tablet_bootstrap.cc:654] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.665562  9936 tablet_service.cc:1511] Processing CreateTablet for tablet a1e3e80f2cad43f08196a18ad2bee6a4 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:23:02.658083 10009 tablet_service.cc:1511] Processing CreateTablet for tablet afb4bd9b5d6240b19d09014b985ea7f7 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:23:02.680866 10009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb4bd9b5d6240b19d09014b985ea7f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.683380 10139 tablet_bootstrap.cc:654] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:02.698140  9933 tablet_service.cc:1511] Processing CreateTablet for tablet bb6fd99043b344de885ff64fd4c81132 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:23:02.698990 10140 ts_tablet_manager.cc:1434] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.089s	user 0.025s	sys 0.027s
I20260709 12:23:02.699914 10086 tablet_service.cc:1511] Processing CreateTablet for tablet 062c203e13aa4527a094ec24339001e5 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:02.700555 10140 tablet_bootstrap.cc:492] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:02.693697  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afb4bd9b5d6240b19d09014b985ea7f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.688483 10006 tablet_service.cc:1511] Processing CreateTablet for tablet 062c203e13aa4527a094ec24339001e5 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:02.715200 10011 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd550066b4447f8848484defc31b9d1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:02.715718 10013 tablet_service.cc:1511] Processing CreateTablet for tablet 447b5d26fe15480dbb8fedf0f7a44777 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:02.716671 10011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd550066b4447f8848484defc31b9d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.725685  9934 tablet_service.cc:1511] Processing CreateTablet for tablet 9bd550066b4447f8848484defc31b9d1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:23:02.727421 10013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447b5d26fe15480dbb8fedf0f7a44777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.734812 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:02.735951 10099 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:02.738134  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:02.739421 10148 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:02.740025 10148 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:02.740563 10148 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.742094  9936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a1e3e80f2cad43f08196a18ad2bee6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.707540 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93686c27d2cf4edea0d5da8d0c1d81ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.752334  9932 tablet_service.cc:1511] Processing CreateTablet for tablet d6726f0982834ecf8987472ed6c32a73 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:23:02.757551 10148 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.762867 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:02.763574 10099 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:02.763732  9934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bd550066b4447f8848484defc31b9d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.765991  9932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6726f0982834ecf8987472ed6c32a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.767181  9933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bb6fd99043b344de885ff64fd4c81132. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.779296 10148 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:02.787729 10086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 062c203e13aa4527a094ec24339001e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.791623 10010 tablet_service.cc:1511] Processing CreateTablet for tablet 93686c27d2cf4edea0d5da8d0c1d81ee (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:02.803053 10083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.804073  9937 tablet_service.cc:1511] Processing CreateTablet for tablet 062c203e13aa4527a094ec24339001e5 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:23:02.806882  9937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 062c203e13aa4527a094ec24339001e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.814814 10006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 062c203e13aa4527a094ec24339001e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.815728 10010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93686c27d2cf4edea0d5da8d0c1d81ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.819991  9938 tablet_service.cc:1511] Processing CreateTablet for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:02.828748  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.823421 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:23:02.826287  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:02.832196 10065 leader_election.cc:343] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.833609 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.837244 10141 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:02.837942 10141 ts_tablet_manager.cc:1403] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.258s	user 0.028s	sys 0.030s
I20260709 12:23:02.868813 10089 tablet_service.cc:1511] Processing CreateTablet for tablet 6c669b03539649e4a4164ca56748f662 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:02.870486 10089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c669b03539649e4a4164ca56748f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.873306  9931 tablet_service.cc:1511] Processing CreateTablet for tablet 93686c27d2cf4edea0d5da8d0c1d81ee (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:23:02.881863  9937 tablet_service.cc:1511] Processing CreateTablet for tablet 6c669b03539649e4a4164ca56748f662 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:02.886962 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:02.887835 10025 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 1: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:02.889595  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:02.886226 10099 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:02.890993  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:02.886247 10012 tablet_service.cc:1511] Processing CreateTablet for tablet 6c669b03539649e4a4164ca56748f662 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:23:02.892133 10153 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:02.892572 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c669b03539649e4a4164ca56748f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.892635 10153 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.893616  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:02.895277 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:02.898262 10150 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:02.897712 10153 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:02.909653  9991 leader_election.cc:343] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.909965  9935 tablet_service.cc:1511] Processing CreateTablet for tablet 447b5d26fe15480dbb8fedf0f7a44777 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:23:02.899695 10148 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:02.912055 10148 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:23:02.915122  9991 leader_election.cc:343] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:02.917426 10141 raft_consensus.cc:359] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.921293 10007 tablet_service.cc:1511] Processing CreateTablet for tablet 51528ebbf4884121ad7c7d564100498f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:02.922500 10084 tablet_service.cc:1511] Processing CreateTablet for tablet 51528ebbf4884121ad7c7d564100498f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:02.927395 10010 tablet_service.cc:1511] Processing CreateTablet for tablet d7b809cb22d54c31ad5985fb9f01e6ab (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:02.928212 10085 tablet_service.cc:1511] Processing CreateTablet for tablet d7b809cb22d54c31ad5985fb9f01e6ab (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:02.937196  9932 tablet_service.cc:1511] Processing CreateTablet for tablet 51528ebbf4884121ad7c7d564100498f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:23:02.957643 10085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b809cb22d54c31ad5985fb9f01e6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.913177 10148 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.907950  9931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93686c27d2cf4edea0d5da8d0c1d81ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.944828 10139 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:02.965312 10139 ts_tablet_manager.cc:1403] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.421s	user 0.031s	sys 0.042s
I20260709 12:23:02.970371 10139 raft_consensus.cc:359] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:02.971521 10139 raft_consensus.cc:385] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:02.972062 10139 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:02.975626 10010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b809cb22d54c31ad5985fb9f01e6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.987119 10084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51528ebbf4884121ad7c7d564100498f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.952978 10141 raft_consensus.cc:385] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.007899  9939 tablet_service.cc:1511] Processing CreateTablet for tablet d7b809cb22d54c31ad5985fb9f01e6ab (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:23:03.006836  9935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447b5d26fe15480dbb8fedf0f7a44777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.012964  9932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51528ebbf4884121ad7c7d564100498f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.015990  9939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b809cb22d54c31ad5985fb9f01e6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:02.994275 10141 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.019398 10141 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.021631 10141 ts_tablet_manager.cc:1434] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.183s	user 0.007s	sys 0.003s
I20260709 12:23:03.022768 10007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 51528ebbf4884121ad7c7d564100498f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.003644 10139 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.032985 10140 tablet_bootstrap.cc:654] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.037407 10013 tablet_service.cc:1511] Processing CreateTablet for tablet 773b4422370c4cc6b9e7477bdaf29261 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:03.022584 10141 tablet_bootstrap.cc:492] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:03.055645 10150 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.023070  9937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6c669b03539649e4a4164ca56748f662. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.056432 10150 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.041682 10145 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.059986 10145 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.062273 10150 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:03.041525 10013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773b4422370c4cc6b9e7477bdaf29261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.052795 10139 ts_tablet_manager.cc:1434] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.087s	user 0.008s	sys 0.001s
I20260709 12:23:03.064553  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:03.065970 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:03.074229 10145 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:03.066291  9930 tablet_service.cc:1511] Processing CreateTablet for tablet 7e7b0b17a4d646809d4dd46a0268f2eb (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:03.070020 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:03.074661 10065 leader_election.cc:343] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.071213  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:03.075572  9930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e7b0b17a4d646809d4dd46a0268f2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.075652  9949 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:03.076954  9991 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:03.078248 10062 leader_election.cc:343] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.078730 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:03.079360 10139 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:03.024405  9934 tablet_service.cc:1511] Processing CreateTablet for tablet 773b4422370c4cc6b9e7477bdaf29261 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:03.066301 10082 tablet_service.cc:1511] Processing CreateTablet for tablet 773b4422370c4cc6b9e7477bdaf29261 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:23:03.072525 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:03.080986  9934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773b4422370c4cc6b9e7477bdaf29261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.082435 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773b4422370c4cc6b9e7477bdaf29261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.080925 10150 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:03.087360  9914 leader_election.cc:343] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.089885 10084 tablet_service.cc:1511] Processing CreateTablet for tablet 7e7b0b17a4d646809d4dd46a0268f2eb (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:03.091286 10084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e7b0b17a4d646809d4dd46a0268f2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.084618 10145 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.095945 10024 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:03.098280 10145 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.098719 10145 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.104120  9912 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:03.104799 10140 tablet_bootstrap.cc:492] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:03.105289 10140 ts_tablet_manager.cc:1403] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.405s	user 0.016s	sys 0.006s
I20260709 12:23:03.105413 10153 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.105903 10153 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.106242 10153 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.108824 10140 raft_consensus.cc:359] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.109563 10140 raft_consensus.cc:385] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.109875 10140 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.110677 10140 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.112217  9932 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe681129eb440b4925ce53e8c68aaeb (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:03.113510  9932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe681129eb440b4925ce53e8c68aaeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.115722 10009 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe681129eb440b4925ce53e8c68aaeb (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:03.117865 10088 tablet_service.cc:1511] Processing CreateTablet for tablet 0fe681129eb440b4925ce53e8c68aaeb (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:23:03.122447 10009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe681129eb440b4925ce53e8c68aaeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.137616 10006 tablet_service.cc:1511] Processing CreateTablet for tablet d80ed93cd18a4a1095bdd6f9fd0fca80 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:03.138986 10006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d80ed93cd18a4a1095bdd6f9fd0fca80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.141770 10085 tablet_service.cc:1511] Processing CreateTablet for tablet d80ed93cd18a4a1095bdd6f9fd0fca80 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:03.143129 10085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d80ed93cd18a4a1095bdd6f9fd0fca80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.144160  9935 tablet_service.cc:1511] Processing CreateTablet for tablet a7196ffc873e4e7e84c76068d5b95904 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:03.146258 10014 tablet_service.cc:1511] Processing CreateTablet for tablet a7196ffc873e4e7e84c76068d5b95904 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:03.146970 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:03.147545 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7196ffc873e4e7e84c76068d5b95904. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:03.151207  9990 leader_election.cc:343] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.151587 10083 tablet_service.cc:1511] Processing CreateTablet for tablet a7196ffc873e4e7e84c76068d5b95904 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:23:03.152830 10083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7196ffc873e4e7e84c76068d5b95904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.152946 10008 tablet_service.cc:1511] Processing CreateTablet for tablet 4cd31c96c4854a87a361975b579bc1c2 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:03.154137 10008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cd31c96c4854a87a361975b579bc1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.162894  9935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7196ffc873e4e7e84c76068d5b95904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.166139 10086 tablet_service.cc:1511] Processing CreateTablet for tablet 4cd31c96c4854a87a361975b579bc1c2 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:03.167655 10086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cd31c96c4854a87a361975b579bc1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.169123  9938 tablet_service.cc:1511] Processing CreateTablet for tablet 4cd31c96c4854a87a361975b579bc1c2 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:23:03.170635 10088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe681129eb440b4925ce53e8c68aaeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.074576 10012 tablet_service.cc:1511] Processing CreateTablet for tablet 7e7b0b17a4d646809d4dd46a0268f2eb (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:23:03.172154 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e7b0b17a4d646809d4dd46a0268f2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.132123 10139 tablet_bootstrap.cc:654] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.125691 10153 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.177876 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:03.180737 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:23:03.182173  9914 leader_election.cc:343] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.182802 10153 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:03.166186 10141 tablet_bootstrap.cc:654] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.191632  9933 tablet_service.cc:1511] Processing CreateTablet for tablet d80ed93cd18a4a1095bdd6f9fd0fca80 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:23:03.198932  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cd31c96c4854a87a361975b579bc1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.201169  9842 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.210465 10145 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.212502 10024 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:03.211030  9933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d80ed93cd18a4a1095bdd6f9fd0fca80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.214180  9912 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:03.215517 10153 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.216626  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:03.217415  9949 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:03.218787  9930 tablet_service.cc:1511] Processing CreateTablet for tablet 6dfe4ce70e5943448d384222d18a1215 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:03.220062  9930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfe4ce70e5943448d384222d18a1215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.221452 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:03.222698 10145 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:03.223016  9990 leader_election.cc:343] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.223469  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:03.230154 10141 tablet_bootstrap.cc:492] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:03.230901 10141 ts_tablet_manager.cc:1403] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.209s	user 0.015s	sys 0.000s
I20260709 12:23:03.243520 10145 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.251703 10141 raft_consensus.cc:359] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.252539 10141 raft_consensus.cc:385] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.252877 10141 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.262876 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 6dfe4ce70e5943448d384222d18a1215 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:03.264328 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfe4ce70e5943448d384222d18a1215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.270972 10139 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:03.271601 10139 ts_tablet_manager.cc:1403] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.192s	user 0.012s	sys 0.003s
I20260709 12:23:03.290084 10141 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.305621 10083 tablet_service.cc:1511] Processing CreateTablet for tablet 6dfe4ce70e5943448d384222d18a1215 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:23:03.303342 10140 ts_tablet_manager.cc:1434] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.198s	user 0.030s	sys 0.060s
I20260709 12:23:03.308899  9930 tablet_service.cc:1511] Processing CreateTablet for tablet 7253570248b8481791bf2187007dd5df (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:03.309684 10083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6dfe4ce70e5943448d384222d18a1215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.310698 10140 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:03.314920  9932 tablet_service.cc:1511] Processing CreateTablet for tablet a78ccdb241a54c0398e2539a827e5d22 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:03.316710  9930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7253570248b8481791bf2187007dd5df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.317040 10008 tablet_service.cc:1511] Processing CreateTablet for tablet a78ccdb241a54c0398e2539a827e5d22 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:03.317157 10009 tablet_service.cc:1511] Processing CreateTablet for tablet 7253570248b8481791bf2187007dd5df (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:03.317745  9932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a78ccdb241a54c0398e2539a827e5d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.318522 10009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7253570248b8481791bf2187007dd5df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.319125 10008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a78ccdb241a54c0398e2539a827e5d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.321126  9935 tablet_service.cc:1511] Processing CreateTablet for tablet fc2205f3947147b6ad9c3ac2b9bd5058 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:03.322456  9935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2205f3947147b6ad9c3ac2b9bd5058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.327636 10010 tablet_service.cc:1511] Processing CreateTablet for tablet fc2205f3947147b6ad9c3ac2b9bd5058 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:03.328104 10139 raft_consensus.cc:359] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.328856 10139 raft_consensus.cc:385] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.329054 10010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2205f3947147b6ad9c3ac2b9bd5058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.329170 10139 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.331665 10141 ts_tablet_manager.cc:1434] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.100s	user 0.008s	sys 0.007s
I20260709 12:23:03.331822 10088 tablet_service.cc:1511] Processing CreateTablet for tablet a78ccdb241a54c0398e2539a827e5d22 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:23:03.332773 10141 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:03.335940 10086 tablet_service.cc:1511] Processing CreateTablet for tablet 7253570248b8481791bf2187007dd5df (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:23:03.344584 10088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a78ccdb241a54c0398e2539a827e5d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.346565 10082 tablet_service.cc:1511] Processing CreateTablet for tablet fc2205f3947147b6ad9c3ac2b9bd5058 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:23:03.347916 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2205f3947147b6ad9c3ac2b9bd5058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.354297 10086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7253570248b8481791bf2187007dd5df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.356175 10140 tablet_bootstrap.cc:654] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.358016  9932 tablet_service.cc:1511] Processing CreateTablet for tablet 9f126305b66248b59cd9ce87bc952694 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:03.359328  9932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f126305b66248b59cd9ce87bc952694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.360091 10139 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.363083 10008 tablet_service.cc:1511] Processing CreateTablet for tablet 9f126305b66248b59cd9ce87bc952694 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:03.364500 10008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f126305b66248b59cd9ce87bc952694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.378942 10139 ts_tablet_manager.cc:1434] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.107s	user 0.005s	sys 0.009s
I20260709 12:23:03.380486 10139 tablet_bootstrap.cc:492] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:03.382865 10083 tablet_service.cc:1511] Processing CreateTablet for tablet 9f126305b66248b59cd9ce87bc952694 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:23:03.384302 10083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f126305b66248b59cd9ce87bc952694. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:03.386468  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:03.387234 10139 tablet_bootstrap.cc:654] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.386852  9938 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2ff46f3f264308a58e49e2819b1d5b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:03.388160  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2ff46f3f264308a58e49e2819b1d5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.413930 10012 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2ff46f3f264308a58e49e2819b1d5b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:03.415320 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2ff46f3f264308a58e49e2819b1d5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.418367 10145 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.418985 10145 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.421155 10145 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:03.421916 10150 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.421916  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:03.422225 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:03.422521 10150 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:03.423527  9991 leader_election.cc:343] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:03.424010  9990 leader_election.cc:343] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.424373 10141 tablet_bootstrap.cc:654] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.424469  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:03.424791 10150 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:03.425653 10145 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.427781 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:03.429514 10062 leader_election.cc:343] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.430437  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:03.431988 10065 leader_election.cc:343] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.432430 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:03.433578 10150 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.437389 10140 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:03.438112 10140 ts_tablet_manager.cc:1403] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.128s	user 0.014s	sys 0.023s
I20260709 12:23:03.446926  9932 tablet_service.cc:1511] Processing CreateTablet for tablet 886f3cb002374eae906fdbe2d392f3e7 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:03.447713 10009 tablet_service.cc:1511] Processing CreateTablet for tablet 886f3cb002374eae906fdbe2d392f3e7 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:03.448343  9932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 886f3cb002374eae906fdbe2d392f3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.448998 10009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 886f3cb002374eae906fdbe2d392f3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.449589 10088 tablet_service.cc:1511] Processing CreateTablet for tablet 886f3cb002374eae906fdbe2d392f3e7 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:23:03.456293  9949 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.456616 10014 tablet_service.cc:1511] Processing CreateTablet for tablet b5ca4a138b9345ab87fc4055395e0488 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:03.457930 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5ca4a138b9345ab87fc4055395e0488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.459686 10084 tablet_service.cc:1511] Processing CreateTablet for tablet b5ca4a138b9345ab87fc4055395e0488 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:03.460317 10083 tablet_service.cc:1511] Processing CreateTablet for tablet 6a2ff46f3f264308a58e49e2819b1d5b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:23:03.462080  9938 tablet_service.cc:1511] Processing CreateTablet for tablet b5ca4a138b9345ab87fc4055395e0488 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:23:03.462565 10085 tablet_service.cc:1511] Processing CreateTablet for tablet 8b34dc95d0414e70879dd34eaa8213de (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:03.463457  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5ca4a138b9345ab87fc4055395e0488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.465895 10145 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.473606 10141 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:03.474357 10141 ts_tablet_manager.cc:1403] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.142s	user 0.007s	sys 0.011s
I20260709 12:23:03.475621  9930 tablet_service.cc:1511] Processing CreateTablet for tablet 230693f05eb74e8095e27d669f16c8ee (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:03.476965  9930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 230693f05eb74e8095e27d669f16c8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.477459 10089 tablet_service.cc:1511] Processing CreateTablet for tablet 230693f05eb74e8095e27d669f16c8ee (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:03.478978 10006 tablet_service.cc:1511] Processing CreateTablet for tablet 8b34dc95d0414e70879dd34eaa8213de (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:03.480453 10006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b34dc95d0414e70879dd34eaa8213de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.482390 10088 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 886f3cb002374eae906fdbe2d392f3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.483666 10007 tablet_service.cc:1511] Processing CreateTablet for tablet 230693f05eb74e8095e27d669f16c8ee (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:23:03.488832 10085 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b34dc95d0414e70879dd34eaa8213de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.489131 10007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 230693f05eb74e8095e27d669f16c8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.474015  9935 tablet_service.cc:1511] Processing CreateTablet for tablet 8b34dc95d0414e70879dd34eaa8213de (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:23:03.493989  9935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b34dc95d0414e70879dd34eaa8213de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.494990 10089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 230693f05eb74e8095e27d669f16c8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.496709 10140 raft_consensus.cc:359] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.497467 10140 raft_consensus.cc:385] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.497646 10084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5ca4a138b9345ab87fc4055395e0488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.497795 10140 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.487609 10139 tablet_bootstrap.cc:492] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:03.499243 10083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a2ff46f3f264308a58e49e2819b1d5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.499742 10139 ts_tablet_manager.cc:1403] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.009s
I20260709 12:23:03.509852  9933 tablet_service.cc:1511] Processing CreateTablet for tablet db738483c1444059b752648d302dc0d9 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:03.511358  9933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db738483c1444059b752648d302dc0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.514982 10139 raft_consensus.cc:359] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.515784 10139 raft_consensus.cc:385] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.516088 10139 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.516899 10139 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.524423 10139 ts_tablet_manager.cc:1434] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 12:23:03.525542 10139 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:03.531998 10140 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.534464 10140 ts_tablet_manager.cc:1434] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.096s	user 0.005s	sys 0.002s
I20260709 12:23:03.535717 10140 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:03.536201 10145 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:03.497998 10141 raft_consensus.cc:359] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.549645 10082 tablet_service.cc:1511] Processing CreateTablet for tablet db738483c1444059b752648d302dc0d9 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:03.542449 10141 raft_consensus.cc:385] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.551002 10141 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.551196 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db738483c1444059b752648d302dc0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.554461 10012 tablet_service.cc:1511] Processing CreateTablet for tablet db738483c1444059b752648d302dc0d9 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:23:03.554621  9937 tablet_service.cc:1511] Processing CreateTablet for tablet 3e4ebd0dc053435ea316f01d854592c9 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:03.555946  9937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e4ebd0dc053435ea316f01d854592c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.555954 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db738483c1444059b752648d302dc0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.553897 10141 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.572386 10141 ts_tablet_manager.cc:1434] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.098s	user 0.009s	sys 0.002s
I20260709 12:23:03.580195 10139 tablet_bootstrap.cc:654] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.581751 10086 tablet_service.cc:1511] Processing CreateTablet for tablet 3e4ebd0dc053435ea316f01d854592c9 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:03.587531 10086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e4ebd0dc053435ea316f01d854592c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.583055 10140 tablet_bootstrap.cc:654] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.572744 10008 tablet_service.cc:1511] Processing CreateTablet for tablet 3e4ebd0dc053435ea316f01d854592c9 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:23:03.583698 10141 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:03.591602 10008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e4ebd0dc053435ea316f01d854592c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.604657  9938 tablet_service.cc:1511] Processing CreateTablet for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:03.611034 10007 tablet_service.cc:1511] Processing CreateTablet for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:03.612748 10007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.617552 10141 tablet_bootstrap.cc:654] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.630329  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.650473 10141 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:03.651185 10141 ts_tablet_manager.cc:1403] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.068s	user 0.009s	sys 0.007s
I20260709 12:23:03.654970 10141 raft_consensus.cc:359] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.657300 10141 raft_consensus.cc:385] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.657701 10141 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.658663 10141 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.664427 10148 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.665107 10148 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.668757  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:03.670832  9991 leader_election.cc:343] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.672636 10089 tablet_service.cc:1511] Processing CreateTablet for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:23:03.677249 10141 ts_tablet_manager.cc:1434] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.026s	user 0.009s	sys 0.000s
I20260709 12:23:03.677436 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:03.678398 10099 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:03.682294  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:03.683478 10145 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.683882 10145 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.683825  9933 tablet_service.cc:1511] Processing CreateTablet for tablet 0df211fae593420e88f685abc8d618ec (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:03.684329 10145 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.688942  9933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df211fae593420e88f685abc8d618ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.694380 10012 tablet_service.cc:1511] Processing CreateTablet for tablet 0df211fae593420e88f685abc8d618ec (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:03.696033 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df211fae593420e88f685abc8d618ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.702683 10089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.706807 10145 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.709127 10145 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:03.710549  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:03.707165 10084 tablet_service.cc:1511] Processing CreateTablet for tablet 0df211fae593420e88f685abc8d618ec (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:23:03.676090 10148 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:03.721832 10140 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:03.722338 10140 ts_tablet_manager.cc:1403] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.187s	user 0.008s	sys 0.023s
I20260709 12:23:03.722929  9937 tablet_service.cc:1511] Processing CreateTablet for tablet 8bab9d80dad949d8afde49b123fbac1a (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:03.725234  9937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bab9d80dad949d8afde49b123fbac1a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:03.727538  9991 leader_election.cc:343] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.729986 10084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df211fae593420e88f685abc8d618ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.729791 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:03.719916 10141 tablet_bootstrap.cc:492] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:03.736406 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 8bab9d80dad949d8afde49b123fbac1a (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:03.746349 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bab9d80dad949d8afde49b123fbac1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.738577 10099 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.734305 10140 raft_consensus.cc:359] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.740801 10082 tablet_service.cc:1511] Processing CreateTablet for tablet 8bab9d80dad949d8afde49b123fbac1a (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:23:03.742205 10139 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:03.751026 10140 raft_consensus.cc:385] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.751511 10140 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.751742 10139 ts_tablet_manager.cc:1403] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.226s	user 0.018s	sys 0.014s
I20260709 12:23:03.752346 10140 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.755605 10139 raft_consensus.cc:359] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.756454 10139 raft_consensus.cc:385] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.756817 10139 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.757686 10139 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.758090 10099 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:03.761124  9930 tablet_service.cc:1511] Processing CreateTablet for tablet 840d8356270745f492c7d2c7099794c1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:03.762094 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8bab9d80dad949d8afde49b123fbac1a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.762240  9930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840d8356270745f492c7d2c7099794c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.769093  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:03.770438 10148 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.770959 10148 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:03.771450 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 840d8356270745f492c7d2c7099794c1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:03.771720 10148 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.772578 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840d8356270745f492c7d2c7099794c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.777974 10140 ts_tablet_manager.cc:1434] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 12:23:03.786646 10139 ts_tablet_manager.cc:1434] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 12:23:03.789191 10082 tablet_service.cc:1511] Processing CreateTablet for tablet 840d8356270745f492c7d2c7099794c1 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:23:03.792292 10141 tablet_bootstrap.cc:654] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.792678 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840d8356270745f492c7d2c7099794c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.792958 10139 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:03.791314 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 86c102903b3d4742a0b9ac1df1132361 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:03.795890 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c102903b3d4742a0b9ac1df1132361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.798139 10140 tablet_bootstrap.cc:492] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:03.791314  9938 tablet_service.cc:1511] Processing CreateTablet for tablet 86c102903b3d4742a0b9ac1df1132361 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:03.803294  9938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c102903b3d4742a0b9ac1df1132361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.811451 10141 tablet_bootstrap.cc:492] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:03.812054 10141 ts_tablet_manager.cc:1403] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.100s	user 0.012s	sys 0.002s
I20260709 12:23:03.812083 10139 tablet_bootstrap.cc:654] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.815446 10141 raft_consensus.cc:359] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.816337 10141 raft_consensus.cc:385] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.816682 10141 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.817427 10141 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.818360  9841 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:03.819955 10141 ts_tablet_manager.cc:1434] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:03.822109 10140 tablet_bootstrap.cc:654] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.822341 10014 tablet_service.cc:1511] Processing CreateTablet for tablet 41bba71b483942ed97a9e1276a90514f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:03.823594 10014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bba71b483942ed97a9e1276a90514f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.823819 10141 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:03.824048  9937 tablet_service.cc:1511] Processing CreateTablet for tablet 41bba71b483942ed97a9e1276a90514f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:03.825284  9937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bba71b483942ed97a9e1276a90514f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.830801 10141 tablet_bootstrap.cc:654] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.835446 10012 tablet_service.cc:1511] Processing CreateTablet for tablet 978cf11d4cd54b67a4fa39b2b089f3da (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:03.836583 10012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978cf11d4cd54b67a4fa39b2b089f3da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.837502 10150 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.838101 10150 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.841401  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:03.842042  9948 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:03.842542 10084 tablet_service.cc:1511] Processing CreateTablet for tablet 41bba71b483942ed97a9e1276a90514f (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:23:03.842938 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:03.843906 10084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41bba71b483942ed97a9e1276a90514f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.846236 10082 tablet_service.cc:1511] Processing CreateTablet for tablet 86c102903b3d4742a0b9ac1df1132361 (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:23:03.846711 10150 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:03.847371 10082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86c102903b3d4742a0b9ac1df1132361. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.848120 10089 tablet_service.cc:1511] Processing CreateTablet for tablet 978cf11d4cd54b67a4fa39b2b089f3da (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:03.849330 10089 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978cf11d4cd54b67a4fa39b2b089f3da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.849967  9930 tablet_service.cc:1511] Processing CreateTablet for tablet 978cf11d4cd54b67a4fa39b2b089f3da (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:23:03.850692 10139 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:03.851210  9930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978cf11d4cd54b67a4fa39b2b089f3da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.851589  9933 tablet_service.cc:1511] Processing CreateTablet for tablet ed7d787bc5ad4e8db7719c178ffbc59b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:03.852762  9933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7d787bc5ad4e8db7719c178ffbc59b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.853325 10086 tablet_service.cc:1511] Processing CreateTablet for tablet ed7d787bc5ad4e8db7719c178ffbc59b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:03.854509 10086 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7d787bc5ad4e8db7719c178ffbc59b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.851204 10139 ts_tablet_manager.cc:1403] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.059s	user 0.012s	sys 0.001s
W20260709 12:23:03.855099 10062 leader_election.cc:343] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.856304 10007 tablet_service.cc:1511] Processing CreateTablet for tablet ed7d787bc5ad4e8db7719c178ffbc59b (DEFAULT_TABLE table=test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:23:03.857888 10007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7d787bc5ad4e8db7719c178ffbc59b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:03.879225 10164 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.879033 10139 raft_consensus.cc:359] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.879871 10139 raft_consensus.cc:385] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.879707 10164 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.880177 10139 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.881863 10164 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:03.883867 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:23:03.885478  9914 leader_election.cc:343] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.886766 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:03.887449 10024 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:03.888759  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:03.889245 10140 tablet_bootstrap.cc:492] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:03.889683 10140 ts_tablet_manager.cc:1403] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.092s	user 0.017s	sys 0.000s
I20260709 12:23:03.890064 10164 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.890439 10164 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.890810 10164 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.891608 10150 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.892050 10150 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.892848 10140 raft_consensus.cc:359] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.893528 10140 raft_consensus.cc:385] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.893821 10140 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.893966 10150 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:03.895905 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:03.897455 10062 leader_election.cc:343] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.898730 10139 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.901496 10139 ts_tablet_manager.cc:1434] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.004s
I20260709 12:23:03.902551 10139 tablet_bootstrap.cc:492] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:03.907258 10140 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.911044 10139 tablet_bootstrap.cc:654] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:03.912808 10141 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:03.916837 10150 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:03.920004 10141 ts_tablet_manager.cc:1403] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.096s	user 0.006s	sys 0.012s
I20260709 12:23:03.919909 10150 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.922825 10150 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:03.918855 10065 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:03.924826 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:03.925962 10024 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 1: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:03.925872 10150 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:03.926512 10150 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:03.927035 10150 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.935184  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:03.939810  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:03.940963  9948 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:03.946425 10164 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.949784 10164 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:03.950186 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:03.950758 10024 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.950659 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:23:03.952688  9914 leader_election.cc:343] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:03.946375 10065 leader_election.cc:343] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.956115 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:03.960165 10141 raft_consensus.cc:359] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.960943 10141 raft_consensus.cc:385] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.961301 10141 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.964603 10141 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.964330 10024 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:03.966328 10150 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.967453 10065 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:03.967509  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:03.969251 10188 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:03.970741 10140 ts_tablet_manager.cc:1434] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.081s	user 0.038s	sys 0.008s
I20260709 12:23:03.970741 10188 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:03.971877 10188 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.972800 10139 tablet_bootstrap.cc:492] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:03.973285 10139 ts_tablet_manager.cc:1403] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.071s	user 0.021s	sys 0.034s
I20260709 12:23:03.969712 10150 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:03.974119 10150 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:03.975217 10150 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:03.976416 10140 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:03.976248 10139 raft_consensus.cc:359] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.987164 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:03.987295 10139 raft_consensus.cc:385] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:03.987829 10139 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:03.988668 10140 tablet_bootstrap.cc:654] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:03.988668 10062 leader_election.cc:343] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:03.988890 10139 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:03.992311 10139 ts_tablet_manager.cc:1434] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 12:23:04.001967 10141 ts_tablet_manager.cc:1434] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.081s	user 0.015s	sys 0.024s
I20260709 12:23:04.003033 10141 tablet_bootstrap.cc:492] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.004506 10150 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:04.005926 10125 tablet.cc:2406] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:23:04.009145 10141 tablet_bootstrap.cc:654] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:04.011485 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44130.1us
I20260709 12:23:04.013795  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:04.014631  9948 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.020102 10141 tablet_bootstrap.cc:492] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.020603 10141 ts_tablet_manager.cc:1403] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.009s
I20260709 12:23:04.023535 10139 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.023422 10141 raft_consensus.cc:359] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.024103 10141 raft_consensus.cc:385] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.024400 10141 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.025272  9948 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:04.025156 10141 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.028275 10065 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.028949 10141 ts_tablet_manager.cc:1434] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:23:04.029197 10193 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.029690 10193 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:04.029855 10141 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.030627 10139 tablet_bootstrap.cc:654] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.030512 10193 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.036837 10141 tablet_bootstrap.cc:654] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.033735  9841 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.045145 10139 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.045667 10141 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.045825 10139 ts_tablet_manager.cc:1403] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 12:23:04.046188 10141 ts_tablet_manager.cc:1403] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.017s	user 0.011s	sys 0.000s
I20260709 12:23:04.049059 10139 raft_consensus.cc:359] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.049340 10141 raft_consensus.cc:359] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.049903 10139 raft_consensus.cc:385] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.050215 10141 raft_consensus.cc:385] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.050360 10139 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.050661 10141 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.051391 10139 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.051609 10141 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.053848 10139 ts_tablet_manager.cc:1434] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:23:04.054028 10141 ts_tablet_manager.cc:1434] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:04.055121 10141 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.055685 10139 tablet_bootstrap.cc:492] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.057366  9842 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.060967 10141 tablet_bootstrap.cc:654] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.061837 10140 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.062358 10140 ts_tablet_manager.cc:1403] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.086s	user 0.018s	sys 0.025s
I20260709 12:23:04.062744 10139 tablet_bootstrap.cc:654] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.067934 10140 raft_consensus.cc:359] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.069367 10140 raft_consensus.cc:385] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.069792 10140 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.071074 10140 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.081080 10139 tablet_bootstrap.cc:492] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.081938 10141 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.082111 10139 ts_tablet_manager.cc:1403] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.027s	user 0.013s	sys 0.011s
I20260709 12:23:04.082474 10141 ts_tablet_manager.cc:1403] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.028s	user 0.016s	sys 0.008s
I20260709 12:23:04.084911 10139 raft_consensus.cc:359] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.085192 10141 raft_consensus.cc:359] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.085629 10139 raft_consensus.cc:385] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.086011 10141 raft_consensus.cc:385] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.086058 10139 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.081559 10140 ts_tablet_manager.cc:1434] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:23:04.086436 10141 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.087477 10140 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.087065 10139 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.087232 10141 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.089309 10141 ts_tablet_manager.cc:1434] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:04.090548 10141 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.091192 10139 ts_tablet_manager.cc:1434] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:04.092128 10139 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.096565 10141 tablet_bootstrap.cc:654] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.097748 10139 tablet_bootstrap.cc:654] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.108065 10141 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.108783 10141 ts_tablet_manager.cc:1403] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.007s
I20260709 12:23:04.109536 10139 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.110152 10139 ts_tablet_manager.cc:1403] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.004s
I20260709 12:23:04.112051 10141 raft_consensus.cc:359] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.112819 10141 raft_consensus.cc:385] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.113111 10141 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.112985 10139 raft_consensus.cc:359] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.113879 10139 raft_consensus.cc:385] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.114181 10139 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.114045 10141 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.115202 10140 tablet_bootstrap.cc:654] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.115895 10141 ts_tablet_manager.cc:1434] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:04.116859 10141 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.114847 10139 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.119621 10139 ts_tablet_manager.cc:1434] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:23:04.120926 10139 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.122138 10140 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.122217 10141 tablet_bootstrap.cc:654] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.122841 10140 ts_tablet_manager.cc:1403] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.002s
I20260709 12:23:04.126260 10140 raft_consensus.cc:359] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.127391 10140 raft_consensus.cc:385] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.127859 10140 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.129015 10140 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.132757 10140 ts_tablet_manager.cc:1434] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:23:04.134243 10140 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.141311 10139 tablet_bootstrap.cc:654] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.144088 10140 tablet_bootstrap.cc:654] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.146118 10141 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.146703 10141 ts_tablet_manager.cc:1403] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.030s	user 0.025s	sys 0.003s
I20260709 12:23:04.150265 10141 raft_consensus.cc:359] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.151015 10141 raft_consensus.cc:385] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.151298 10141 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.152522 10140 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.152127 10141 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.153024 10140 ts_tablet_manager.cc:1403] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.004s
I20260709 12:23:04.153038 10139 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.153599 10139 ts_tablet_manager.cc:1403] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.001s
I20260709 12:23:04.154219 10141 ts_tablet_manager.cc:1434] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:23:04.156558 10141 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.155792 10140 raft_consensus.cc:359] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.156927 10140 raft_consensus.cc:385] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.157168 10140 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.157882 10140 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.159410 10139 raft_consensus.cc:359] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.160020 10188 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.160516 10188 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.162456 10141 tablet_bootstrap.cc:654] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.163415 10188 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.164376 10140 ts_tablet_manager.cc:1434] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 12:23:04.165043 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.160050 10139 raft_consensus.cc:385] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.165886 10139 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.166100 10193 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.166787 10150 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.166740 10139 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:04.167440  9912 leader_election.cc:343] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.167382 10150 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.166577 10193 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.169581 10150 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.170650 10193 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.165293 10140 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.171721 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.172354 10025 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:04.172644  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.172730  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.173648 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:04.174067 10148 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.174479 10193 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.174574 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.174959 10193 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.174510 10148 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.175352 10099 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:04.176620  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.176836 10148 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.177381 10140 tablet_bootstrap.cc:654] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.177484 10139 ts_tablet_manager.cc:1434] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:23:04.170642 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.177727 10188 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.178107 10188 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.178498 10188 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.178570 10139 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.183043 10141 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.183516 10141 ts_tablet_manager.cc:1403] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.027s	user 0.008s	sys 0.007s
I20260709 12:23:04.175354 10193 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.185205 10139 tablet_bootstrap.cc:654] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.186244 10141 raft_consensus.cc:359] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.187021 10141 raft_consensus.cc:385] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.187296 10141 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.187187  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.188411 10141 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:04.189242  9991 leader_election.cc:343] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.192342 10141 ts_tablet_manager.cc:1434] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:04.178216 10024 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:04.194552 10140 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.195156 10140 ts_tablet_manager.cc:1403] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.010s
I20260709 12:23:04.195015 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:04.196405 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:23:04.193677 10065 leader_election.cc:343] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.198580 10139 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.198319 10140 raft_consensus.cc:359] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.198984 10140 raft_consensus.cc:385] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:04.199020  9990 leader_election.cc:343] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.199064 10139 ts_tablet_manager.cc:1403] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.008s
I20260709 12:23:04.199335 10140 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.199643  9990 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:04.200486 10148 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.200065 10140 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.202410 10140 ts_tablet_manager.cc:1434] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:23:04.208910 10140 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.203295 10141 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.209308 10193 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.216087 10145 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.216584 10145 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.217818 10193 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.209210 10188 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.220511 10188 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.221450 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:04.221709 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.222105 10099 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.222530 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.223168 10025 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.223526 10141 tablet_bootstrap.cc:654] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.223680 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.224843  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:23:04.225102  9912 leader_election.cc:343] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.218726 10145 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.226145  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.226969  9949 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:23:04.226944 10065 leader_election.cc:343] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.228220  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:04.229521 10145 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.232540 10141 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.233029 10141 ts_tablet_manager.cc:1403] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.005s
I20260709 12:23:04.233062 10099 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:04.234480  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.235723 10188 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.236146 10141 raft_consensus.cc:359] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.236976 10188 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:04.237205 10193 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.237706 10193 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.237092 10141 raft_consensus.cc:385] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.238399 10141 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.238063 10188 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.239228 10141 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.240945 10140 tablet_bootstrap.cc:654] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:04.242564  9991 consensus_peers.cc:597] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:04.244370 10025 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:04.203138 10139 raft_consensus.cc:359] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.211483 10148 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.246039 10148 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.204583 10150 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.249966 10141 ts_tablet_manager.cc:1434] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 12:23:04.251091 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.251971 10024 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:04.253255  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.253818 10141 tablet_bootstrap.cc:492] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.254093  9949 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:04.254736 10148 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.256353 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.257002 10099 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:04.258385  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:04.259461 10150 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.256347 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:04.263029 10148 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.263451 10148 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.263842 10141 tablet_bootstrap.cc:654] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.265309  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.263767 10148 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.268092 10150 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.269071 10191 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.269697 10191 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.272253 10191 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.274145 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.275581  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.277093 10139 raft_consensus.cc:385] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:04.277359 10065 leader_election.cc:343] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.277607 10139 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
W20260709 12:23:04.267365  9991 leader_election.cc:343] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.278875 10193 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.278374 10150 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.282294 10140 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.279850 10150 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.282725 10140 ts_tablet_manager.cc:1403] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.074s	user 0.030s	sys 0.023s
I20260709 12:23:04.288082 10140 raft_consensus.cc:359] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.288758 10140 raft_consensus.cc:385] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.289052 10140 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.288959 10148 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.283648 10145 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.285081 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:23:04.291492 10062 leader_election.cc:343] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.292042 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:04.292857 10191 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.293386 10191 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:04.293644 10190 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.294307 10191 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.295379 10148 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.298508 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:04.298909 10141 tablet_bootstrap.cc:492] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.299180 10099 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.299228  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:04.299460 10141 ts_tablet_manager.cc:1403] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.046s	user 0.027s	sys 0.008s
W20260709 12:23:04.300918  9991 leader_election.cc:343] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.301505 10140 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.302731 10141 raft_consensus.cc:359] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.303632 10141 raft_consensus.cc:385] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.303906 10141 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.303910 10139 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.304852 10141 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.305310  9840 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.304872 10140 ts_tablet_manager.cc:1434] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.000s
I20260709 12:23:04.307333 10141 ts_tablet_manager.cc:1434] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:04.307607 10140 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.308208 10141 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.314774 10099 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:04.320117 10150 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.315790 10141 tablet_bootstrap.cc:654] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.317035 10140 tablet_bootstrap.cc:654] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.318774 10139 ts_tablet_manager.cc:1434] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.119s	user 0.012s	sys 0.000s
I20260709 12:23:04.322113  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:04.324043 10148 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.324530 10148 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:04.325356 10148 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.329061 10191 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.329594 10191 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.331905 10191 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.332690  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.333379  9949 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:04.333434 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.334339 10025 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:04.330772  9842 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.336123 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:04.337381 10191 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.337865 10191 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.338317 10191 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.340636 10193 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.341322 10193 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.344367 10193 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.346477  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.347066 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.347401  9949 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:04.347957 10025 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:04.349750 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:04.351222 10193 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.351765 10193 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.352223 10193 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.364688 10139 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.365170 10191 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.319612 10145 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.371439  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:04.372323  9949 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.371447 10140 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.379395 10140 ts_tablet_manager.cc:1403] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.004s
I20260709 12:23:04.383332 10188 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:04.384413 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.385020 10025 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:04.390996  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:04.395118 10150 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.396627 10193 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.383363 10140 raft_consensus.cc:359] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.405090  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:04.405411  9949 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:04.401222  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:04.410082  9948 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.405807  9947 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.400138 10025 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:04.405114 10140 raft_consensus.cc:385] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.419101 10140 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.408488 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.419960 10024 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.428423 10188 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.428977 10188 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.430037 10024 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:04.431354 10188 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.433578 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:04.433725 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.435022 10190 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.434911 10140 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.437412 10140 ts_tablet_manager.cc:1434] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.002s
W20260709 12:23:04.438057  9914 leader_election.cc:343] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.438738  9947 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:04.440421 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:04.405004  9842 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.433693 10150 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.442829 10150 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.448274 10140 tablet_bootstrap.cc:492] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.450809 10164 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.452819 10164 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.453433 10164 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.459865 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.460638 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.465050 10022 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:04.465097 10097 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:04.462870 10190 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:04.468799 10190 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.455575 10150 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.466670  9914 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:04.473713 10188 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.474190 10188 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.475260 10145 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.475345 10188 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.475791 10145 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.481681 10145 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.459652 10164 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.467600 10193 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.491379  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.485509 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.496150  9948 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
W20260709 12:23:04.497337  9990 leader_election.cc:343] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.486826 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.487886 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.498956 10022 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:04.499909  9991 leader_election.cc:343] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.498919 10188 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.498564 10023 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:04.467183 10191 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.465487 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.494558 10141 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.504927 10141 ts_tablet_manager.cc:1403] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.197s	user 0.078s	sys 0.071s
I20260709 12:23:04.485373 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:04.500370  9991 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:04.507100 10145 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.507776 10141 raft_consensus.cc:359] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.508474 10141 raft_consensus.cc:385] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.508615 10190 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.508769 10141 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.509390 10190 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:04.509435 10141 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.443302 10139 tablet_bootstrap.cc:654] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.488785 10140 tablet_bootstrap.cc:654] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:04.490701 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50405.7us
I20260709 12:23:04.462785 10222 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.513900 10222 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:04.515213 10022 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:04.503242 10188 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.513473 10190 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.516149  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.516881  9948 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:04.518301 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:04.514809 10222 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.519306 10191 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.519699 10191 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.520056 10191 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.521159 10223 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.533236 10223 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.519277 10141 ts_tablet_manager.cc:1434] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:23:04.535326 10141 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
W20260709 12:23:04.542169  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64471.1us
I20260709 12:23:04.545398 10191 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.553517 10191 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.554979  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:23:04.551002  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63885.8us
I20260709 12:23:04.556325 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.556917 10024 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.549453 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:04.558718 10098 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:04.558974  9949 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:23:04.566627  9912 leader_election.cc:343] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.567117  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.567847 10024 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:04.570863 10164 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.576865 10139 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.577517 10139 ts_tablet_manager.cc:1403] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.213s	user 0.025s	sys 0.000s
I20260709 12:23:04.591048 10141 tablet_bootstrap.cc:654] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.587397 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.596501 10140 tablet_bootstrap.cc:492] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.596912 10140 ts_tablet_manager.cc:1403] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.149s	user 0.014s	sys 0.010s
I20260709 12:23:04.600255 10140 raft_consensus.cc:359] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.600999 10140 raft_consensus.cc:385] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.601267 10140 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.601936 10140 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.604615 10139 raft_consensus.cc:359] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.605118 10139 raft_consensus.cc:385] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.604210 10025 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:04.593459 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:04.605324 10139 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.607300 10140 ts_tablet_manager.cc:1434] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:23:04.609128 10140 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.611778 10190 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.612591 10190 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:04.613405 10190 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.617103 10139 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.619259 10139 ts_tablet_manager.cc:1434] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.041s	user 0.007s	sys 0.001s
I20260709 12:23:04.620173 10139 tablet_bootstrap.cc:492] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.611089  9912 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.630092 10188 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.631917  9840 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.634064  9840 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.638759  9840 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.640965 10139 tablet_bootstrap.cc:654] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.643436 10140 tablet_bootstrap.cc:654] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.647662 10139 tablet_bootstrap.cc:492] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.648454 10139 ts_tablet_manager.cc:1403] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.002s
I20260709 12:23:04.647873 10141 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.650485 10141 ts_tablet_manager.cc:1403] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.115s	user 0.011s	sys 0.003s
I20260709 12:23:04.650817 10140 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.651279 10140 ts_tablet_manager.cc:1403] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.003s
I20260709 12:23:04.652539 10188 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.653079 10188 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.653468 10141 raft_consensus.cc:359] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.654175 10141 raft_consensus.cc:385] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.654449 10141 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.655128 10188 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.655236 10141 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.655846 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.656731 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:23:04.657295  9912 leader_election.cc:343] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.657377 10098 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:04.657624 10141 ts_tablet_manager.cc:1434] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:04.658624 10141 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.658288 10139 raft_consensus.cc:359] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.658936 10139 raft_consensus.cc:385] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.659185 10139 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.659915 10139 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.661260  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.662232 10139 ts_tablet_manager.cc:1434] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:23:04.662384 10188 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.662824 10188 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.663158 10188 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.666136 10141 tablet_bootstrap.cc:654] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:04.676738  9912 consensus_peers.cc:597] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:04.676993 10139 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.677527 10141 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.677992 10141 ts_tablet_manager.cc:1403] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.006s
I20260709 12:23:04.680815 10141 raft_consensus.cc:359] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.681506 10141 raft_consensus.cc:385] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.681774 10141 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.682173  9840 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:04.682632 10188 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.684532 10141 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.686879 10141 ts_tablet_manager.cc:1434] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:23:04.687322 10139 tablet_bootstrap.cc:654] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.687141 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:04.687697 10098 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.687759 10190 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:04.688107 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.689621 10141 tablet_bootstrap.cc:492] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
W20260709 12:23:04.690111  9912 leader_election.cc:343] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.695940 10141 tablet_bootstrap.cc:654] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.684563 10140 raft_consensus.cc:359] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.699148 10140 raft_consensus.cc:385] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.699559 10140 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.700467 10140 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.701488 10188 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.698515 10098 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:04.703634  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.711649 10191 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.712114 10191 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.713860 10191 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.716092  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.716289 10188 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:23:04.717876 10065 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:04.718544 10065 leader_election.cc:343] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.718936 10188 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:04.721087 10141 tablet_bootstrap.cc:492] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.719427 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.721635 10141 ts_tablet_manager.cc:1403] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.002s
I20260709 12:23:04.722082 10140 ts_tablet_manager.cc:1434] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.002s
W20260709 12:23:04.722437 10062 leader_election.cc:343] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.722918 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:04.722981 10230 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.724069 10193 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.723660 10230 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.727810 10141 raft_consensus.cc:359] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.728487 10141 raft_consensus.cc:385] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.728750 10230 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.729174 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.728745 10141 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.730662 10141 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.731632 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.733091 10141 ts_tablet_manager.cc:1434] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:23:04.733577  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.734081 10141 tablet_bootstrap.cc:492] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.734719 10230 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.736060 10140 tablet_bootstrap.cc:492] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.741954 10141 tablet_bootstrap.cc:654] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.720207 10188 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.747640 10145 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.748350 10145 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.752168  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.752893 10145 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:04.753634  9991 leader_election.cc:343] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.754616 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:23:04.756290  9990 leader_election.cc:343] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.756748  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:04.757838 10145 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.760027 10139 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.760493 10139 ts_tablet_manager.cc:1403] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.084s	user 0.040s	sys 0.024s
I20260709 12:23:04.763211 10139 raft_consensus.cc:359] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.763870 10139 raft_consensus.cc:385] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.764107 10139 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.764778 10139 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.766362 10140 tablet_bootstrap.cc:654] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.766819 10139 ts_tablet_manager.cc:1434] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:23:04.767913 10139 tablet_bootstrap.cc:492] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.771384 10141 tablet_bootstrap.cc:492] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.771872 10141 ts_tablet_manager.cc:1403] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.017s
I20260709 12:23:04.772330  9840 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.774379 10139 tablet_bootstrap.cc:654] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.775177 10141 raft_consensus.cc:359] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.775837 10141 raft_consensus.cc:385] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.776098 10141 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.776914 10141 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.779292 10141 ts_tablet_manager.cc:1434] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:04.780412 10141 tablet_bootstrap.cc:492] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.788269 10141 tablet_bootstrap.cc:654] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:04.799422  9991 consensus_peers.cc:597] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:04.802086 10141 tablet_bootstrap.cc:492] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.802739 10141 ts_tablet_manager.cc:1403] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.023s	user 0.020s	sys 0.000s
I20260709 12:23:04.803426  9949 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.804173 10188 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.804857 10188 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.808969 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.809693 10188 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.809801 10022 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:04.810329 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.811821  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:04.812629  9914 leader_election.cc:343] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.813032 10188 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.813560 10188 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.814036 10188 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.825321 10141 raft_consensus.cc:359] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.826344 10141 raft_consensus.cc:385] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.826711 10141 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.827935 10141 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.830492 10141 ts_tablet_manager.cc:1434] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:23:04.831871 10141 tablet_bootstrap.cc:492] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:04.832850 10145 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.833559 10145 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.834384 10188 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.835990 10145 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.837555  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.838033 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:04.838647 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.838837 10022 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.839234 10097 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:04.838215  9948 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:04.842155  9990 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:04.844277 10212 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:04.842190 10145 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.844666 10212 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:04.845039 10212 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.845121 10193 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.845584 10193 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.847886 10193 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.849591 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.849603 10022 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
W20260709 12:23:04.851240 10062 leader_election.cc:343] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.851348  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:04.852584  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:04.853044 10065 leader_election.cc:343] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.853658 10065 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:04.855190 10230 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.855861 10230 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:04.856663 10212 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.857103 10230 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.877818 10139 tablet_bootstrap.cc:492] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:04.878407 10139 ts_tablet_manager.cc:1403] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.111s	user 0.024s	sys 0.043s
I20260709 12:23:04.858244 10193 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.861671 10188 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.867830 10141 tablet_bootstrap.cc:654] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.868378 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:04.868770 10140 tablet_bootstrap.cc:492] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:04.882956 10140 ts_tablet_manager.cc:1403] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.147s	user 0.069s	sys 0.021s
W20260709 12:23:04.884047  9914 leader_election.cc:343] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.885965 10140 raft_consensus.cc:359] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.886744 10140 raft_consensus.cc:385] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.887076 10140 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.887856 10140 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.889416 10164 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:04.891521 10140 ts_tablet_manager.cc:1434] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:23:04.892040 10164 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.892549 10164 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.893903 10140 tablet_bootstrap.cc:492] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:04.895838 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:04.899626 10164 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.899849 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:04.900938  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:04.901479  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:04.901726  9948 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.902781 10164 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.902618 10139 raft_consensus.cc:359] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.903329 10139 raft_consensus.cc:385] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:04.903617 10139 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:04.904542 10139 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.907562 10139 ts_tablet_manager.cc:1434] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.029s	user 0.004s	sys 0.003s
I20260709 12:23:04.909710 10139 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:04.910435 10193 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:04.916275 10139 tablet_bootstrap.cc:654] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.918694 10140 tablet_bootstrap.cc:654] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:04.930284 10193 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:04.930958  9948 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:04.932566  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:04.933684 10145 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:04.934326 10145 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:04.935634 10193 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.935487 10145 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.936131 10193 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.939536  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:04.941298 10065 leader_election.cc:343] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.946832 10212 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:04.948397 10190 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:04.948748 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:04.949319 10098 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:04.956775 10098 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:04.955865  9840 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:04.959168 10223 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:04.973519 10141 tablet_bootstrap.cc:492] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:04.974187 10141 ts_tablet_manager.cc:1403] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.143s	user 0.011s	sys 0.000s
I20260709 12:23:04.975677 10193 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:04.976974 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:04.982920 10062 leader_election.cc:343] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:04.983573 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:04.985133 10193 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:04.987757 10212 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:04.988277 10212 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:04.990363 10212 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.008329 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:05.009267 10098 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:05.015628  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:05.018016 10212 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:05.018527 10212 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.018956 10212 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:05.025522  9841 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.027124 10191 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.027745 10191 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.028415  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:05.029052  9949 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:05.033309 10191 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:05.045809  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:05.047390 10212 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:05.049068 10065 leader_election.cc:343] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.050527 10212 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.051858  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:05.052505  9948 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:05.057399 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:05.058071 10098 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:05.058585 10141 raft_consensus.cc:359] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.059353 10141 raft_consensus.cc:385] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.059679 10141 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.059507 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:05.065950 10191 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:05.066515 10264 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:23:05.056758 10193 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:23:05.069430 10193 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:05.071375 10098 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:05.072573  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:23:05.069554 10062 leader_election.cc:343] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.073491 10212 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:05.073815 10062 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:05.074772 10212 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:05.075320 10191 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:05.075817 10212 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.082108 10139 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:05.082636 10139 ts_tablet_manager.cc:1403] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.173s	user 0.021s	sys 0.005s
I20260709 12:23:05.083869 10140 tablet_bootstrap.cc:492] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:05.084385 10140 ts_tablet_manager.cc:1403] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.191s	user 0.039s	sys 0.049s
I20260709 12:23:05.085491 10139 raft_consensus.cc:359] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.086212 10139 raft_consensus.cc:385] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.097803 10141 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.101032 10141 ts_tablet_manager.cc:1434] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.126s	user 0.012s	sys 0.000s
I20260709 12:23:05.102264 10141 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:05.102842 10193 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:05.088835  9948 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:05.086575 10139 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.107476 10139 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.109045 10139 ts_tablet_manager.cc:1434] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.026s	user 0.003s	sys 0.002s
I20260709 12:23:05.109761 10139 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:05.089440 10140 raft_consensus.cc:359] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.111474 10140 raft_consensus.cc:385] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.111800 10140 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.112676 10140 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.114895 10140 ts_tablet_manager.cc:1434] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.003s
I20260709 12:23:05.117976 10140 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:05.115880 10139 tablet_bootstrap.cc:654] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.125030 10141 tablet_bootstrap.cc:654] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.132328 10140 tablet_bootstrap.cc:654] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.139369 10190 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.139882 10190 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.143357  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:05.147764 10141 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:05.148275 10141 ts_tablet_manager.cc:1403] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.000s
I20260709 12:23:05.149767 10139 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:05.150370 10139 ts_tablet_manager.cc:1403] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.000s
I20260709 12:23:05.151094 10141 raft_consensus.cc:359] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.151630 10141 raft_consensus.cc:385] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.151875 10141 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.153843 10139 raft_consensus.cc:359] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.154428 10190 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:23:05.154707 10065 leader_election.cc:343] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.155385  9840 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 2, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.156517 10139 raft_consensus.cc:385] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.156922 10139 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.157970 10139 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.160257 10139 ts_tablet_manager.cc:1434] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:23:05.161172 10139 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:05.162014 10140 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:05.162425 10140 ts_tablet_manager.cc:1403] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.003s
I20260709 12:23:05.165017 10140 raft_consensus.cc:359] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.165652 10140 raft_consensus.cc:385] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.165937 10140 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.166554 10140 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.167905 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:05.168090 10141 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.168653 10025 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:05.169425 10140 ts_tablet_manager.cc:1434] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:23:05.170331 10140 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:05.170888 10141 ts_tablet_manager.cc:1434] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:23:05.170958 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:05.171680 10139 tablet_bootstrap.cc:654] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.172187 10141 tablet_bootstrap.cc:492] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:05.172113 10193 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.172597 10193 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.170280  9947 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.172923 10193 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.177270 10140 tablet_bootstrap.cc:654] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.178915 10212 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.179430 10212 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.179550 10193 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.181571 10212 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.182394  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:05.182569  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:05.183164 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:05.183809 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
W20260709 12:23:05.183949 10065 leader_election.cc:343] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:05.184393  9991 leader_election.cc:343] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.185089  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:05.186262 10212 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.185442 10193 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:05.186687 10139 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:05.186648 10212 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.187157 10212 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.187244 10139 ts_tablet_manager.cc:1403] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.004s
I20260709 12:23:05.187798 10140 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:05.188230 10140 ts_tablet_manager.cc:1403] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.018s	user 0.005s	sys 0.009s
I20260709 12:23:05.189939 10139 raft_consensus.cc:359] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.190641 10139 raft_consensus.cc:385] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.190975 10139 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.190722 10140 raft_consensus.cc:359] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.191354 10140 raft_consensus.cc:385] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.191630 10140 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.191759 10139 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.192152 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:05.192759 10025 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.192477 10140 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.194527 10141 tablet_bootstrap.cc:654] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.194774 10140 ts_tablet_manager.cc:1434] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:23:05.195927 10140 tablet_bootstrap.cc:492] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:05.196233 10139 ts_tablet_manager.cc:1434] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:05.197104 10139 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:05.201924 10140 tablet_bootstrap.cc:654] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.205808  9948 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.207796 10193 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.208453 10193 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.213124 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:05.213335  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:05.214643 10062 leader_election.cc:343] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.215188 10193 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:05.215855 10139 tablet_bootstrap.cc:654] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.220733 10212 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.225142  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:23:05.227241  9991 leader_election.cc:343] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:05.230944 10065 leader_election.cc:343] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.231086 10212 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.231520 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:05.238840 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:05.239061 10025 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:05.239445 10098 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.230280 10193 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.240793 10025 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.232520 10188 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.244786 10188 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.234918 10191 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:05.242803 10193 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.243957 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:05.255023 10188 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.259631 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:05.260514 10097 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:05.252643 10193 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:05.263547  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:05.264034 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:05.265165 10065 leader_election.cc:343] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.265029 10022 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:05.261325 10191 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.267017 10191 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:05.267232  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:05.268181 10188 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.268042 10191 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.268600 10188 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.269099 10188 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.264271 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:05.282821 10062 leader_election.cc:343] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.283102 10022 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.283325 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:05.287753 10223 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:05.295265 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:05.296000 10230 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:05.294431 10141 tablet_bootstrap.cc:492] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:05.305161 10141 ts_tablet_manager.cc:1403] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.133s	user 0.020s	sys 0.012s
I20260709 12:23:05.308971 10141 raft_consensus.cc:359] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.309921 10141 raft_consensus.cc:385] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.310243 10141 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.311713  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:05.314045 10141 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.333984 10141 ts_tablet_manager.cc:1434] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:23:05.335587 10141 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:05.336373 10188 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.340466 10230 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.358444 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:05.360575 10140 tablet_bootstrap.cc:492] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:05.360697 10023 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.361070 10140 ts_tablet_manager.cc:1403] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.165s	user 0.003s	sys 0.030s
I20260709 12:23:05.359422 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:05.357201 10188 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.358394 10229 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:23:05.365236 10099 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.368160 10140 raft_consensus.cc:359] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.369165 10140 raft_consensus.cc:385] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.369578 10140 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.351908 10230 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.370487 10140 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.381378 10139 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:05.382269 10139 ts_tablet_manager.cc:1403] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.185s	user 0.011s	sys 0.011s
I20260709 12:23:05.378911 10212 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.389164 10229 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:05.394809 10023 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
W20260709 12:23:05.397115 10280 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:23:05.399118 10230 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.399758 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:05.400127  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:05.400439 10097 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:05.401422 10140 ts_tablet_manager.cc:1434] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.040s	user 0.007s	sys 0.011s
I20260709 12:23:05.402289 10140 tablet_bootstrap.cc:492] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:05.402904 10212 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:05.407171 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:05.408336 10230 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.408823 10230 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:05.417985  9914 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:05.418164 10099 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:05.417630 10230 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.413517 10212 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.419432 10229 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.419898 10229 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.419365  9840 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:05.420363 10229 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.431567 10099 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.467512 10140 tablet_bootstrap.cc:654] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:05.469663  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:05.479436 10139 raft_consensus.cc:359] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.480439 10139 raft_consensus.cc:385] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.480793 10139 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.491003 10139 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.492138 10223 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.492604 10223 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.487516 10285 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:05.489723 10164 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:23:05.497967 10223 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:05.503872 10139 ts_tablet_manager.cc:1434] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.121s	user 0.006s	sys 0.002s
I20260709 12:23:05.505330 10139 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:05.500960 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:05.507655  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:05.509959 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:05.512104 10223 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:05.514282 10285 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:05.525897 10223 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.526559 10223 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.495451 10229 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.535657 10141 tablet_bootstrap.cc:654] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.539628 10290 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.540066  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:05.540207 10290 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.541193 10229 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.542285  9947 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
W20260709 12:23:05.543530  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 84814.7us
W20260709 12:23:05.547080  9912 leader_election.cc:343] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.555310 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:05.555886 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:05.556470 10193 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.556701 10098 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.556975 10193 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.557484 10193 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.559307 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:05.564597 10022 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.559878 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:05.566489 10140 tablet_bootstrap.cc:492] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:05.567471 10140 ts_tablet_manager.cc:1403] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.165s	user 0.017s	sys 0.025s
I20260709 12:23:05.550905 10223 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:05.589051 10140 raft_consensus.cc:359] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.589917 10140 raft_consensus.cc:385] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.590349 10140 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
W20260709 12:23:05.591668 10062 leader_election.cc:343] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.589054  9843 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.591029 10140 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.600526 10140 ts_tablet_manager.cc:1434] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.000s
I20260709 12:23:05.601861 10140 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:05.607980 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:05.610297 10212 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.597048 10290 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:05.611037  9912 leader_election.cc:343] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.621644  9948 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:05.623123 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:05.625391 10141 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:05.625962 10141 ts_tablet_manager.cc:1403] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.291s	user 0.023s	sys 0.011s
I20260709 12:23:05.639274 10141 raft_consensus.cc:359] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.640010 10141 raft_consensus.cc:385] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.640252 10141 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.640743 10141 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.642166 10141 ts_tablet_manager.cc:1434] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.000s
I20260709 12:23:05.642989 10141 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:05.624836 10212 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.645597 10212 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.651551 10139 tablet_bootstrap.cc:654] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.654908 10287 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:23:05.648814  9840 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.627311 10098 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:05.684981 10094 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:05.710639 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:05.708499  9914 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:05.722951 10230 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.723449 10230 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:05.724349 10230 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.721910 10287 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:05.727303 10193 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.724658  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:05.729691 10193 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:05.729781 10095 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:05.711483 10093 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
W20260709 12:23:05.749604  9991 leader_election.cc:343] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:05.752630 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 135665us
I20260709 12:23:05.761790 10140 tablet_bootstrap.cc:654] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.796070  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:23:05.797155  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44400.4us
I20260709 12:23:05.797744 10022 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:05.800561 10287 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.801007 10287 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.801339 10287 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.809484 10141 tablet_bootstrap.cc:654] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.814252 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:23:05.814862  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58164.6us
I20260709 12:23:05.815927  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:23:05.818012 10062 leader_election.cc:343] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.820673 10292 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.832074  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:05.839187 10292 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:05.840166 10140 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:05.840788 10140 ts_tablet_manager.cc:1403] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.239s	user 0.016s	sys 0.002s
I20260709 12:23:05.844956 10291 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:05.845497 10291 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:05.847504 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:05.854974 10291 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.856022 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:05.879557 10140 raft_consensus.cc:359] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.880584 10140 raft_consensus.cc:385] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.880946 10140 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.882926 10140 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.863927 10099 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:05.872239 10301 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:23:05.889483 10141 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:05.889962 10141 ts_tablet_manager.cc:1403] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.247s	user 0.020s	sys 0.012s
I20260709 12:23:05.890064 10140 ts_tablet_manager.cc:1434] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 12:23:05.890254  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:05.891573 10140 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:05.884479 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:05.894874 10291 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:05.892663  9945 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:05.897449 10287 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.898205 10139 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:05.899132 10139 ts_tablet_manager.cc:1403] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.394s	user 0.019s	sys 0.037s
I20260709 12:23:05.896431 10291 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:05.900640 10291 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:05.900871  9912 leader_election.cc:343] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.902700 10141 raft_consensus.cc:359] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.903508 10141 raft_consensus.cc:385] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.903760 10141 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.904399 10141 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.912787 10139 raft_consensus.cc:359] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.913609 10139 raft_consensus.cc:385] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.913985 10139 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.914866 10139 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.917727 10139 ts_tablet_manager.cc:1434] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:23:05.917935 10287 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.921793 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:05.922504 10140 tablet_bootstrap.cc:654] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.923293 10300 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.923889 10300 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:05.922977 10139 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:05.923207 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:05.925284 10097 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:05.924872 10300 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.926792 10291 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.931775 10291 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:05.919504 10141 ts_tablet_manager.cc:1434] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.003s
I20260709 12:23:05.932492 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:05.932922 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:05.933223 10141 tablet_bootstrap.cc:492] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:05.933548 10099 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:05.933923  9912 leader_election.cc:343] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.940363 10097 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:05.941910  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:05.942967 10287 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.941975  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:05.943490 10287 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:05.945499 10139 tablet_bootstrap.cc:654] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.944852 10287 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:05.948520  9991 leader_election.cc:343] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:05.940789 10141 tablet_bootstrap.cc:654] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:05.950815 10099 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:05.952175  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:05.953240 10291 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:05.954468 10291 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:05.955379 10291 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.960850 10292 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:05.966879 10140 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:05.967381 10140 ts_tablet_manager.cc:1403] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.076s	user 0.016s	sys 0.002s
I20260709 12:23:05.981406 10140 raft_consensus.cc:359] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.982141 10140 raft_consensus.cc:385] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:05.982402 10140 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:05.983026 10140 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:05.983233  9843 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:05.989113 10140 ts_tablet_manager.cc:1434] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:23:05.990074 10140 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:05.990831 10141 tablet_bootstrap.cc:492] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:05.991498 10141 ts_tablet_manager.cc:1403] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.000s
I20260709 12:23:06.001101 10141 raft_consensus.cc:359] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.001897 10141 raft_consensus.cc:385] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.002168 10141 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.002141  9945 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.003139 10141 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.005825 10141 ts_tablet_manager.cc:1434] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.001s
I20260709 12:23:06.007794 10141 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.014703 10141 tablet_bootstrap.cc:654] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.015581 10287 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.017441 10291 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.017082  9842 catalog_manager.cc:5697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:06.021416 10140 tablet_bootstrap.cc:654] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.026285 10139 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:06.026770 10139 ts_tablet_manager.cc:1403] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.108s	user 0.013s	sys 0.012s
I20260709 12:23:06.029844 10139 raft_consensus.cc:359] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.031164 10139 raft_consensus.cc:385] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.031515 10139 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.032866 10139 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.035432 10139 ts_tablet_manager.cc:1434] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:06.036346 10139 tablet_bootstrap.cc:492] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:06.038578 10148 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.036095  9840 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.051000  9843 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.043531 10139 tablet_bootstrap.cc:654] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.066938 10141 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.067622 10141 ts_tablet_manager.cc:1403] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.060s	user 0.011s	sys 0.008s
I20260709 12:23:06.071343 10141 raft_consensus.cc:359] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.072402 10141 raft_consensus.cc:385] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.072817 10140 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.072826 10141 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.073477 10140 ts_tablet_manager.cc:1403] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.084s	user 0.018s	sys 0.007s
I20260709 12:23:06.074079 10141 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.076297 10140 raft_consensus.cc:359] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.076997 10140 raft_consensus.cc:385] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.077275 10140 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.077917 10140 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.078401 10141 ts_tablet_manager.cc:1434] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:23:06.080173 10141 tablet_bootstrap.cc:492] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.080236 10140 ts_tablet_manager.cc:1434] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:06.081483 10140 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:06.087461 10139 tablet_bootstrap.cc:492] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:06.087833 10140 tablet_bootstrap.cc:654] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.088178 10139 ts_tablet_manager.cc:1403] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.052s	user 0.028s	sys 0.009s
I20260709 12:23:06.090502 10141 tablet_bootstrap.cc:654] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.100888 10139 raft_consensus.cc:359] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.101634 10139 raft_consensus.cc:385] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.101917 10139 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.102478 10300 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.103030 10300 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.104907 10140 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.104979 10141 tablet_bootstrap.cc:492] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.105239 10300 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.105432 10140 ts_tablet_manager.cc:1403] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.006s
I20260709 12:23:06.105525 10141 ts_tablet_manager.cc:1403] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.000s
I20260709 12:23:06.106989  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.107631 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.108302 10024 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:06.108290 10141 raft_consensus.cc:359] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.102499 10139 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.109804 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:06.110203 10140 raft_consensus.cc:359] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.110698 10300 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.110946 10140 raft_consensus.cc:385] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.111086 10300 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.111280 10140 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.111508 10300 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.109095 10141 raft_consensus.cc:385] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.111677 10139 ts_tablet_manager.cc:1434] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:23:06.111989 10141 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.112972 10139 tablet_bootstrap.cc:492] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:06.112880 10141 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.114972 10141 ts_tablet_manager.cc:1434] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:23:06.115826 10141 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.118758 10300 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.120008 10139 tablet_bootstrap.cc:654] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.121395 10141 tablet_bootstrap.cc:654] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.122138 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:06.122766 10024 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.122861 10140 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.125767 10140 ts_tablet_manager.cc:1434] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 12:23:06.126744 10140 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:06.130847 10229 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:06.132537 10065 leader_election.cc:343] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.134423 10140 tablet_bootstrap.cc:654] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.131453 10229 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.143160 10291 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.145373 10024 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:06.129812 10300 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.149191  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:06.148412 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:06.149698 10300 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.151602 10300 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.152693 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.153357 10097 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
W20260709 12:23:06.154300 10065 leader_election.cc:343] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.154731  9914 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:06.151062 10191 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:06.155457 10191 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:06.155666 10230 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.156080 10230 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.156373 10191 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.156880 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.156415 10230 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:06.160637 10062 leader_election.cc:343] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.157991 10229 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.162815  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.163594 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:06.164556 10065 leader_election.cc:343] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.167850 10139 tablet_bootstrap.cc:492] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:06.168594 10139 ts_tablet_manager.cc:1403] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.056s	user 0.016s	sys 0.019s
I20260709 12:23:06.170392 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:06.162170 10300 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.171806 10193 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.176796 10139 raft_consensus.cc:359] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.177534 10139 raft_consensus.cc:385] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.177834 10139 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.178465 10139 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.194686 10230 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.197592 10230 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.199302 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:06.200178 10097 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.203467 10139 ts_tablet_manager.cc:1434] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 12:23:06.204449 10139 tablet_bootstrap.cc:492] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
W20260709 12:23:06.207043  9912 leader_election.cc:343] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.209227 10097 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:06.210781  9914 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:06.210367  9840 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:06.212069 10191 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.212752 10191 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.213285 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:06.215651  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.216837 10191 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:23:06.217162 10065 leader_election.cc:343] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.218132 10230 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:23:06.218881  9912 leader_election.cc:343] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.219449 10230 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:06.218897 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.222877 10022 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:06.227530 10139 tablet_bootstrap.cc:654] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.229059 10292 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.232082 10229 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.232707 10229 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.234800 10141 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.235431 10141 ts_tablet_manager.cc:1403] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.120s	user 0.016s	sys 0.052s
I20260709 12:23:06.249584 10229 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.251771 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.252578 10140 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.252691 10098 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:06.253153 10141 raft_consensus.cc:359] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.254073 10141 raft_consensus.cc:385] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.253134 10140 ts_tablet_manager.cc:1403] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.127s	user 0.023s	sys 0.023s
I20260709 12:23:06.254514 10141 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.255314 10141 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.254446 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:06.256805 10191 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.257570 10141 ts_tablet_manager.cc:1434] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:23:06.257038 10140 raft_consensus.cc:359] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.258359 10140 raft_consensus.cc:385] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.258682 10141 tablet_bootstrap.cc:492] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.258665 10140 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.259697 10140 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.256989  9914 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:06.262416 10140 ts_tablet_manager.cc:1434] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:06.262795 10292 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.263196 10292 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.263430 10140 tablet_bootstrap.cc:492] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:06.263571 10292 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.265233 10141 tablet_bootstrap.cc:654] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.267362 10148 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.267859 10148 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.269990 10148 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.270928 10140 tablet_bootstrap.cc:654] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.272985  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.257202 10191 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.274222 10191 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.274899 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.275574 10098 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:06.277518  9990 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:06.278256 10141 tablet_bootstrap.cc:492] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.281358 10141 ts_tablet_manager.cc:1403] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.004s
I20260709 12:23:06.282148 10291 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.282706 10291 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.284925 10291 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.286357 10212 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.286789 10212 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.287114 10212 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.288136 10141 raft_consensus.cc:359] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.289147 10141 raft_consensus.cc:385] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.289582 10141 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.290522 10141 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.292727 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.302845 10141 ts_tablet_manager.cc:1434] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.021s	user 0.011s	sys 0.005s
I20260709 12:23:06.303879 10141 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.304881 10140 tablet_bootstrap.cc:492] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.305320 10140 ts_tablet_manager.cc:1403] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.042s	user 0.010s	sys 0.019s
I20260709 12:23:06.293354 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.222461 10230 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.308482 10140 raft_consensus.cc:359] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.309238 10140 raft_consensus.cc:385] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.309553 10140 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.310045 10141 tablet_bootstrap.cc:654] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.310451 10140 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.298633 10191 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.313858 10140 ts_tablet_manager.cc:1434] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:23:06.314826  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:06.315172 10140 tablet_bootstrap.cc:492] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Bootstrap starting.
W20260709 12:23:06.316309 10065 leader_election.cc:343] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.318711 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:06.319320 10024 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.321900 10141 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.322365 10141 ts_tablet_manager.cc:1403] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.019s	user 0.009s	sys 0.007s
I20260709 12:23:06.323628 10140 tablet_bootstrap.cc:654] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.302667 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.324941 10098 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:06.325148 10141 raft_consensus.cc:359] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.325842 10141 raft_consensus.cc:385] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.326107 10141 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.326207  9914 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:06.326946 10191 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.326889 10141 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.327572 10024 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:06.328110 10291 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.328584 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:06.329028 10141 ts_tablet_manager.cc:1434] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:06.329679 10300 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:06.329977 10141 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.330157 10300 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:06.331107 10300 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.333218 10139 tablet_bootstrap.cc:492] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:06.333668 10139 ts_tablet_manager.cc:1403] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.129s	user 0.009s	sys 0.021s
I20260709 12:23:06.336762 10291 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.337281 10141 tablet_bootstrap.cc:654] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.336989 10139 raft_consensus.cc:359] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.337824 10139 raft_consensus.cc:385] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.318701 10212 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.341574  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:06.344172 10022 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:06.345978 10140 tablet_bootstrap.cc:492] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.346731 10140 ts_tablet_manager.cc:1403] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.006s
I20260709 12:23:06.349355 10292 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.349823 10140 raft_consensus.cc:359] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.350754 10140 raft_consensus.cc:385] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.350924  9945 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:06.351105 10140 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.353022 10212 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.352053 10140 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.356125 10140 ts_tablet_manager.cc:1434] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 12:23:06.355607  9842 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:06.357185 10140 tablet_bootstrap.cc:492] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:06.358534 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:06.359154 10098 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.338120 10139 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.366043 10139 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.368681 10139 ts_tablet_manager.cc:1434] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 12:23:06.370193 10139 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:06.368237 10292 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.368814 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:06.376505 10022 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:06.376346 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:06.373226  9840 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.378552 10141 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.379127 10141 ts_tablet_manager.cc:1403] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.049s	user 0.020s	sys 0.007s
I20260709 12:23:06.379976  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:06.384613 10292 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.384955 10140 tablet_bootstrap.cc:654] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.386679 10098 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:06.387573 10141 raft_consensus.cc:359] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.388120 10141 raft_consensus.cc:385] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.388090  9990 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:06.389531 10212 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:06.389901 10141 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.390240 10212 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:06.391707 10139 tablet_bootstrap.cc:654] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.391390 10212 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.400223 10292 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.404029 10148 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.390902 10141 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.404770 10148 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.407579 10141 ts_tablet_manager.cc:1434] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 12:23:06.408366 10148 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.408872 10141 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.409564 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.409885  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.411250 10292 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.412281 10292 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.415390 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.417284 10230 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:06.417388  9912 leader_election.cc:343] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.417922 10230 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.420644 10230 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.420764 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.421521 10024 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:06.422410 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.425168 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.425803 10095 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:06.426875 10141 tablet_bootstrap.cc:654] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.427209 10285 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.432328 10292 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.436620  9914 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:06.447877  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:06.448827 10291 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.452412 10229 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.452837 10229 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.453153 10229 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.457314 10291 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.468524 10229 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.474246 10140 tablet_bootstrap.cc:492] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.474994 10140 ts_tablet_manager.cc:1403] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.118s	user 0.018s	sys 0.027s
I20260709 12:23:06.479691 10140 raft_consensus.cc:359] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.480677 10140 raft_consensus.cc:385] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.481026 10140 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.481940 10140 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.485476 10300 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.485845 10141 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.485989 10300 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.486549 10141 ts_tablet_manager.cc:1403] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.018s
I20260709 12:23:06.489003  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:06.490361 10065 leader_election.cc:343] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.490713 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:06.491498  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:23:06.492213  9912 leader_election.cc:343] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.493127 10309 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.493244 10229 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.493304 10141 raft_consensus.cc:359] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.494282 10141 raft_consensus.cc:385] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.497895 10309 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:06.501955 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:06.502842 10097 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.510013 10024 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.512499 10140 ts_tablet_manager.cc:1434] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:23:06.494547 10141 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.513705 10140 tablet_bootstrap.cc:492] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:06.514041 10141 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.516330 10141 ts_tablet_manager.cc:1434] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 12:23:06.516479 10139 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:06.519656 10139 ts_tablet_manager.cc:1403] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.150s	user 0.055s	sys 0.074s
W20260709 12:23:06.521274  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:06.521827 10140 tablet_bootstrap.cc:654] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.517268 10141 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.525806  9841 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.519140 10097 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:06.526820 10292 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.522330 10139 raft_consensus.cc:359] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.530436 10300 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.530567 10139 raft_consensus.cc:385] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.531067 10139 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.531898 10139 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.532702 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.535142 10141 tablet_bootstrap.cc:654] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.535045  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:23:06.536262 10062 leader_election.cc:343] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.536727 10062 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:06.540773 10193 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.540697 10139 ts_tablet_manager.cc:1434] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.000s
I20260709 12:23:06.542367 10193 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.543004 10139 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:06.549348 10191 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.550100 10191 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.553102 10191 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.555750 10212 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.555989 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.556470 10212 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.559027 10022 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:06.561707 10212 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.562168 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:23:06.563699 10062 consensus_peers.cc:597] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.564267 10191 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.564749 10191 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.564370  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.566534 10140 tablet_bootstrap.cc:492] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.566788 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.567147 10140 ts_tablet_manager.cc:1403] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.054s	user 0.002s	sys 0.046s
I20260709 12:23:06.567085  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:06.568332  9991 leader_election.cc:343] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:06.568583 10065 leader_election.cc:343] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.569284 10292 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:06.569545 10191 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.570042 10292 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:06.570820 10229 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.571321 10229 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.572604 10140 raft_consensus.cc:359] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.573473 10140 raft_consensus.cc:385] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.573792 10140 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.574612 10140 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.575315 10229 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.576936 10140 ts_tablet_manager.cc:1434] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.002s
I20260709 12:23:06.576872 10191 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.578086 10140 tablet_bootstrap.cc:492] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:06.578167 10097 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:06.578922 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:06.580704  9912 leader_election.cc:343] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.581243 10141 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.581678 10141 ts_tablet_manager.cc:1403] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.065s	user 0.006s	sys 0.007s
I20260709 12:23:06.571281 10292 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.584309 10141 raft_consensus.cc:359] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.586347 10141 raft_consensus.cc:385] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.586669 10141 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.586792 10140 tablet_bootstrap.cc:654] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.587531 10141 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.589900 10141 ts_tablet_manager.cc:1434] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:06.592308  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:06.596518 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:06.597739 10140 tablet_bootstrap.cc:492] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
W20260709 12:23:06.597684 10065 leader_election.cc:343] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.596002 10139 tablet_bootstrap.cc:654] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.598287 10140 ts_tablet_manager.cc:1403] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.020s	user 0.005s	sys 0.010s
I20260709 12:23:06.597517 10023 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.596261 10191 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.600986 10140 raft_consensus.cc:359] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.601495 10141 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.601653 10140 raft_consensus.cc:385] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.601905 10140 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.602248 10212 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.602545 10140 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.602773 10212 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.604902 10140 ts_tablet_manager.cc:1434] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:23:06.605080 10212 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.605885 10140 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:06.610981 10141 tablet_bootstrap.cc:654] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.611904  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.612792  9943 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:06.615039 10140 tablet_bootstrap.cc:654] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.615098  9943 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:06.626219  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 91289us
I20260709 12:23:06.628715 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.629645 10097 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:06.630226 10141 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.631130 10141 ts_tablet_manager.cc:1403] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.000s
I20260709 12:23:06.636957 10141 raft_consensus.cc:359] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.637959 10141 raft_consensus.cc:385] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.639825  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:06.640784 10212 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.641153 10212 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.638321 10141 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.641487 10212 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.642290 10141 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.642994 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.643657 10097 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:06.645500  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:06.647773 10140 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.648627 10140 ts_tablet_manager.cc:1403] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.007s
I20260709 12:23:06.648355 10212 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.649605 10300 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.650470 10212 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.649372 10141 ts_tablet_manager.cc:1434] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:23:06.651635 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:06.652704 10141 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.652606 10097 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.664211 10141 tablet_bootstrap.cc:654] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.673276 10141 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.673749 10141 ts_tablet_manager.cc:1403] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.001s
I20260709 12:23:06.676700 10300 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.676700 10141 raft_consensus.cc:359] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.677559 10141 raft_consensus.cc:385] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.677325 10300 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.677831 10141 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.678673 10141 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.680869 10141 ts_tablet_manager.cc:1434] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 12:23:06.681155 10300 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.683357 10141 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.683121 10140 raft_consensus.cc:359] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.683830 10140 raft_consensus.cc:385] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.684227 10140 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.684928 10140 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.687527 10140 ts_tablet_manager.cc:1434] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 12:23:06.688799 10140 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:06.693636 10097 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:06.694934  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:06.695842 10212 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:06.696439 10212 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:06.697330 10212 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.689621 10141 tablet_bootstrap.cc:654] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.696226  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.703433 10191 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.708206 10191 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:06.711344 10065 leader_election.cc:343] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.712657  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.712815 10191 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:23:06.720867 10065 leader_election.cc:343] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:06.702981  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 68735.1us
I20260709 12:23:06.741261 10140 tablet_bootstrap.cc:654] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.739807 10300 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:23:06.745924  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:06.751909  9947 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.760604 10141 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:06.761260 10141 ts_tablet_manager.cc:1403] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.078s	user 0.006s	sys 0.006s
I20260709 12:23:06.756570  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:06.763868 10230 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:06.764251 10230 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:06.764639 10230 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.772215 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:06.773944 10062 leader_election.cc:343] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.774459 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:06.775545 10361 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.777377 10023 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
W20260709 12:23:06.779846  9912 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:06.781459 10230 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.782723 10285 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.786813 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:06.787755 10300 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:06.788228 10300 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:06.788951 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:06.789045 10300 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:06.790254  9912 leader_election.cc:343] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.757210 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.806283 10139 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:06.806821 10139 ts_tablet_manager.cc:1403] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.264s	user 0.034s	sys 0.053s
I20260709 12:23:06.807408 10230 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.808523  9947 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
W20260709 12:23:06.809494 10062 leader_election.cc:343] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:06.811137 10062 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:06.811801 10230 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.812306 10230 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.877245 10188 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.886446 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:06.887143 10141 raft_consensus.cc:359] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.891430 10141 raft_consensus.cc:385] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.891820 10141 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.891515 10098 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.892536 10141 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.916963 10230 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.917575 10139 raft_consensus.cc:359] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.918262 10139 raft_consensus.cc:385] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:06.918527 10139 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:06.919121 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.919178 10139 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.920025 10024 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:06.921942 10139 ts_tablet_manager.cc:1434] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.115s	user 0.007s	sys 0.000s
I20260709 12:23:06.923524 10139 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:06.894491 10193 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.894922 10188 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.929929 10188 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:06.933001 10212 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:06.937227 10191 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.938189 10191 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:06.926288 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116799us
I20260709 12:23:06.952219 10229 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:06.955912 10229 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:06.954289 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.958664  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:06.959873 10292 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:06.964942 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:06.969612 10140 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:06.970120 10140 ts_tablet_manager.cc:1403] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.282s	user 0.025s	sys 0.014s
I20260709 12:23:06.971009 10141 ts_tablet_manager.cc:1434] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.209s	user 0.026s	sys 0.026s
I20260709 12:23:06.972393 10141 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:06.978536  9840 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:06.989657  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:06.976935 10098 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:06.990320 10292 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:06.983697 10368 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:06.992650 10139 tablet_bootstrap.cc:654] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:06.977591 10191 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:06.991250 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:06.999018 10229 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:07.011934 10065 leader_election.cc:343] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.012010 10140 raft_consensus.cc:359] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.018743 10140 raft_consensus.cc:385] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.013648 10024 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:07.019706  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:07.022833 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.023459 10230 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:06.997071 10368 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.019080 10140 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.036849 10140 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.037052 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.041349 10230 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:07.045890 10309 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:07.048465 10140 ts_tablet_manager.cc:1434] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.078s	user 0.006s	sys 0.002s
I20260709 12:23:07.051473 10140 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:07.056316 10141 tablet_bootstrap.cc:654] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.048448 10300 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:23:07.061450  9912 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:07.063313 10229 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.062152 10188 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:23:07.072021  9840 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:07.075968 10368 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:23:07.071152 10062 leader_election.cc:343] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.079347  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.079605 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.080360 10368 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.080992 10368 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:07.082892 10368 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:07.084165 10065 leader_election.cc:343] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.083343 10368 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.087020 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.090621 10361 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.090793 10368 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.091140 10361 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.094041 10361 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.071148 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:07.096277 10062 leader_election.cc:343] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.096777 10062 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.097888 10300 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.098214  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:07.098690 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.099423 10188 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:07.100076 10188 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:07.102241 10229 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.108368 10139 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:07.102741  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.111081 10139 ts_tablet_manager.cc:1403] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.188s	user 0.016s	sys 0.010s
I20260709 12:23:07.070791 10230 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:07.114998 10062 leader_election.cc:343] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.111310  9841 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:07.118918 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:07.120395 10062 leader_election.cc:343] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:07.125090 10065 leader_election.cc:343] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.125497 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.130237  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.127168 10300 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:07.135656 10065 leader_election.cc:343] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.135937 10140 tablet_bootstrap.cc:654] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.136158 10065 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.137437 10300 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.147133 10141 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.147603 10141 ts_tablet_manager.cc:1403] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.175s	user 0.007s	sys 0.010s
I20260709 12:23:07.153699 10141 raft_consensus.cc:359] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.154369 10141 raft_consensus.cc:385] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.154668 10141 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.155944 10141 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.158385 10139 raft_consensus.cc:359] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.159175 10139 raft_consensus.cc:385] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.159686 10139 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.160456 10139 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:07.159066  9912 consensus_peers.cc:597] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:07.164047 10139 ts_tablet_manager.cc:1434] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 12:23:07.168017 10309 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:07.170534 10309 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.171105 10309 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.174572 10141 ts_tablet_manager.cc:1434] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:23:07.175812 10141 tablet_bootstrap.cc:492] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.177068 10212 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:07.182865 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.183528 10098 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:07.185000 10141 tablet_bootstrap.cc:654] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.185269  9990 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:07.188611 10285 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:07.189028 10285 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.189364 10285 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.189723 10309 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.196743 10285 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.200690  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:07.201530  9947 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.202150 10285 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.205247 10148 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.205901 10148 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.208267 10148 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.208530 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:07.209408 10098 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.211225 10368 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.211330  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.211810 10368 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.213884 10368 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.215075 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.215466  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.215639 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:07.216703 10065 leader_election.cc:343] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.216755  9990 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:23:07.217288 10062 leader_election.cc:343] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.217928 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.218346 10212 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.218937 10193 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.219969  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.224854 10141 tablet_bootstrap.cc:492] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.225354 10141 ts_tablet_manager.cc:1403] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.050s	user 0.021s	sys 0.016s
I20260709 12:23:07.230019 10193 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:07.231001 10141 raft_consensus.cc:359] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.231681 10141 raft_consensus.cc:385] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.231947 10141 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.232681 10141 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.234186 10141 ts_tablet_manager.cc:1434] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:23:07.241556 10098 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:07.243073  9990 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:07.244083 10141 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.244155 10285 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:07.244815 10285 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:07.245679 10285 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.247375  9947 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:07.262208 10361 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.270112 10361 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.272394 10361 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.273806 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.255309 10139 tablet_bootstrap.cc:492] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
W20260709 12:23:07.275527 10062 leader_election.cc:343] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.254578 10141 tablet_bootstrap.cc:654] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.280956  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:07.283308 10065 leader_election.cc:343] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.283727 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.286186 10193 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.297465 10361 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:07.303591  9843 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:07.308390 10139 tablet_bootstrap.cc:654] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.304930 10140 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:07.310936 10140 ts_tablet_manager.cc:1403] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.261s	user 0.016s	sys 0.006s
I20260709 12:23:07.313550 10140 raft_consensus.cc:359] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.317162 10140 raft_consensus.cc:385] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.317438 10140 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.318069 10140 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.320082 10140 ts_tablet_manager.cc:1434] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:23:07.320957 10140 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:07.326073 10141 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.326534 10141 ts_tablet_manager.cc:1403] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.083s	user 0.008s	sys 0.004s
I20260709 12:23:07.328874 10140 tablet_bootstrap.cc:654] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.329330 10141 raft_consensus.cc:359] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.330006 10141 raft_consensus.cc:385] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.330291 10141 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.330961 10141 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.333401 10141 ts_tablet_manager.cc:1434] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:07.333626 10139 tablet_bootstrap.cc:492] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:07.334076 10139 ts_tablet_manager.cc:1403] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.168s	user 0.012s	sys 0.000s
I20260709 12:23:07.334548 10141 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.336488 10139 raft_consensus.cc:359] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.337075 10230 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.337714 10230 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.338459  9843 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:07.339781 10230 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.340489 10141 tablet_bootstrap.cc:654] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.337162 10139 raft_consensus.cc:385] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.341110 10139 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.341697 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.341861 10139 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.342379 10099 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:07.343485 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.343883 10139 ts_tablet_manager.cc:1434] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:23:07.344831 10139 tablet_bootstrap.cc:492] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
W20260709 12:23:07.345209  9912 leader_election.cc:343] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.345922 10141 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.345892  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:07.346349 10023 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.346410 10141 ts_tablet_manager.cc:1403] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.012s	user 0.007s	sys 0.004s
I20260709 12:23:07.346693 10230 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:07.347150 10230 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.347489 10230 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.348726 10140 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:07.349161 10140 ts_tablet_manager.cc:1403] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:23:07.349804 10141 raft_consensus.cc:359] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.350452 10141 raft_consensus.cc:385] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.350724 10141 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.351477 10141 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.351862 10140 raft_consensus.cc:359] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.352541 10140 raft_consensus.cc:385] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.352795 10140 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.353423 10140 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.353828 10141 ts_tablet_manager.cc:1434] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:07.354247 10230 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.355296 10141 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.357043 10230 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.357152 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:07.358974 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:07.359393 10139 tablet_bootstrap.cc:654] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.359607 10099 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:07.362355  9912 leader_election.cc:343] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.362931 10141 tablet_bootstrap.cc:654] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.363214 10140 ts_tablet_manager.cc:1434] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:23:07.364179 10140 tablet_bootstrap.cc:492] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:07.378795 10099 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:07.380386  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:07.381145 10230 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:07.381691 10230 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:07.381858 10140 tablet_bootstrap.cc:654] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.382629 10230 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.389690 10139 tablet_bootstrap.cc:492] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:07.390312 10139 ts_tablet_manager.cc:1403] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.000s
I20260709 12:23:07.393787 10139 raft_consensus.cc:359] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.395612 10139 raft_consensus.cc:385] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.395944 10139 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.396817 10139 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.401933 10140 tablet_bootstrap.cc:492] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:07.402437 10140 ts_tablet_manager.cc:1403] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.000s
I20260709 12:23:07.402909 10141 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.403342 10141 ts_tablet_manager.cc:1403] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.048s	user 0.012s	sys 0.012s
I20260709 12:23:07.405189 10140 raft_consensus.cc:359] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.405858 10140 raft_consensus.cc:385] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.405810 10141 raft_consensus.cc:359] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.406258 10140 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.406539 10141 raft_consensus.cc:385] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.406852 10141 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.408334 10139 ts_tablet_manager.cc:1434] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:23:07.409441 10139 tablet_bootstrap.cc:492] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:07.410398 10361 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.410005  9841 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:07.410952 10361 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.413275 10361 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.416397 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.416286 10141 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.416976 10140 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.417855  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.418458 10141 ts_tablet_manager.cc:1434] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 12:23:07.419360 10140 ts_tablet_manager.cc:1434] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
W20260709 12:23:07.420085 10062 leader_election.cc:343] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.420776 10140 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Bootstrap starting.
W20260709 12:23:07.421131 10065 leader_election.cc:343] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.421518 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.422585 10193 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.426354 10141 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.433024 10141 tablet_bootstrap.cc:654] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.437361 10139 tablet_bootstrap.cc:654] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.438095 10140 tablet_bootstrap.cc:654] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.442464 10141 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.442903 10141 ts_tablet_manager.cc:1403] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.017s	user 0.008s	sys 0.008s
I20260709 12:23:07.443794 10140 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:07.444381 10140 ts_tablet_manager.cc:1403] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.000s
I20260709 12:23:07.444897 10141 raft_consensus.cc:359] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.445441 10141 raft_consensus.cc:385] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.445708 10141 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.446347 10141 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.448057 10141 ts_tablet_manager.cc:1434] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.005s	user 0.002s	sys 0.001s
I20260709 12:23:07.448118 10140 raft_consensus.cc:359] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.448767 10140 raft_consensus.cc:385] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.449020 10140 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.449468 10141 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.449651 10140 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.451797 10140 ts_tablet_manager.cc:1434] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:07.455665 10141 tablet_bootstrap.cc:654] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.452950 10140 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:07.460414 10139 tablet_bootstrap.cc:492] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:07.461086 10139 ts_tablet_manager.cc:1403] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.052s	user 0.010s	sys 0.010s
I20260709 12:23:07.464543 10140 tablet_bootstrap.cc:654] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.465251 10139 raft_consensus.cc:359] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.466614 10139 raft_consensus.cc:385] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.467025 10139 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.468159 10139 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.476845 10140 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:07.477334 10140 ts_tablet_manager.cc:1403] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.025s	user 0.005s	sys 0.011s
I20260709 12:23:07.480098 10140 raft_consensus.cc:359] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.480748 10140 raft_consensus.cc:385] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.481009 10140 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.481586 10140 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.484243 10230 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.484509 10139 ts_tablet_manager.cc:1434] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:23:07.484809 10230 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.486920 10230 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.484113 10140 ts_tablet_manager.cc:1434] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.001s
I20260709 12:23:07.489897 10140 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:07.489699 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.490698 10099 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:07.491778 10285 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.492465 10285 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.495143 10285 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.496009  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.497002 10141 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.498340 10141 ts_tablet_manager.cc:1403] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.049s	user 0.021s	sys 0.024s
I20260709 12:23:07.498203 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.498636 10140 tablet_bootstrap.cc:654] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.498545  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:07.499641  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:07.501998 10285 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.502290 10230 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:07.502811 10230 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.500551 10141 raft_consensus.cc:359] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.503685 10141 raft_consensus.cc:385] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.503935 10141 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.503289 10230 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.504657 10141 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.506742 10141 ts_tablet_manager.cc:1434] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:07.507608 10141 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.513413 10141 tablet_bootstrap.cc:654] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.517728 10230 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.518914 10291 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.540805 10291 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.527582 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.531839 10141 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.545207 10141 ts_tablet_manager.cc:1403] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.038s	user 0.013s	sys 0.009s
I20260709 12:23:07.545569 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.546319 10291 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.547410 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.535064 10139 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:07.548053 10098 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:07.550644 10024 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.552968 10141 raft_consensus.cc:359] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.553640 10141 raft_consensus.cc:385] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.553903 10141 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.554494 10141 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.554889 10291 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.555552 10291 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.561378 10230 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.565828 10291 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:07.566136  9912 leader_election.cc:343] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.569725 10141 ts_tablet_manager.cc:1434] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.000s
I20260709 12:23:07.569713  9914 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:07.570999 10141 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.571417 10291 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:07.572017 10291 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.574030 10291 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.577674 10141 tablet_bootstrap.cc:654] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.582048 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:07.582710 10095 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.586395 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.587410 10096 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:07.596463 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.597213 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:07.597388 10023 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
W20260709 12:23:07.599223  9912 leader_election.cc:343] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.605417 10361 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.606158 10361 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.611892  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:07.613926 10065 leader_election.cc:343] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.614742 10361 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.619325  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:07.620862 10188 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:07.621236 10188 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:07.621572 10188 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.629491 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:07.631611 10062 leader_election.cc:343] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.632277 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.633986 10361 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:07.635141  9912 leader_election.cc:343] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.641647 10139 tablet_bootstrap.cc:654] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.642249 10140 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:07.642769 10140 ts_tablet_manager.cc:1403] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.153s	user 0.057s	sys 0.073s
I20260709 12:23:07.645738 10140 raft_consensus.cc:359] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.646387 10291 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.646426 10140 raft_consensus.cc:385] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.647423 10140 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.648027 10140 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.648324 10292 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:07.650135 10140 ts_tablet_manager.cc:1434] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:23:07.650970 10291 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.651822 10140 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:07.656169 10139 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:07.656697 10139 ts_tablet_manager.cc:1403] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.122s	user 0.012s	sys 0.000s
I20260709 12:23:07.658082 10140 tablet_bootstrap.cc:654] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.659389 10139 raft_consensus.cc:359] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.661664 10139 raft_consensus.cc:385] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.663244 10139 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.663704 10141 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.663515 10309 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:07.664124 10141 ts_tablet_manager.cc:1403] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.093s	user 0.035s	sys 0.044s
I20260709 12:23:07.664198 10139 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.666457 10139 ts_tablet_manager.cc:1434] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:07.666973 10141 raft_consensus.cc:359] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.667913 10141 raft_consensus.cc:385] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.668143 10139 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:07.668184 10141 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.669286 10141 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.671502 10141 ts_tablet_manager.cc:1434] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:07.672380 10141 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.673300 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:07.673965 10097 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.681373 10141 tablet_bootstrap.cc:654] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.683859 10139 tablet_bootstrap.cc:654] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.691500 10097 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:07.707612 10139 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:07.708417 10139 ts_tablet_manager.cc:1403] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.009s
I20260709 12:23:07.709460  9914 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:07.712312 10139 raft_consensus.cc:359] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.713413 10139 raft_consensus.cc:385] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.713743 10139 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.714665 10139 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.715420 10212 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.716090 10212 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.716990 10139 ts_tablet_manager.cc:1434] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:07.718178 10139 tablet_bootstrap.cc:492] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:07.721822  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.722409 10291 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:07.723124 10291 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:07.724231 10291 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.730078 10148 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:07.733210 10212 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.736579 10021 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.743450 10361 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.744277 10361 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.748373  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.749773 10139 tablet_bootstrap.cc:654] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:07.750226 10065 leader_election.cc:343] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.753073 10361 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.753827 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:07.755931 10062 leader_election.cc:343] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.756553 10062 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.758463 10361 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.770988 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:07.782549 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.783751 10097 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:07.787043 10095 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:07.804615  9990 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:07.808282 10212 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:07.799587  9912 leader_election.cc:343] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:07.812409  9912 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:07.823036 10139 tablet_bootstrap.cc:492] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:07.826495 10139 ts_tablet_manager.cc:1403] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.109s	user 0.012s	sys 0.004s
I20260709 12:23:07.829517 10188 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.829823 10139 raft_consensus.cc:359] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.838116 10139 raft_consensus.cc:385] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.838642 10139 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.853101 10139 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.854803 10188 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.856007 10139 ts_tablet_manager.cc:1434] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:23:07.859614 10139 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
W20260709 12:23:07.860903  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47768us
I20260709 12:23:07.865159 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:07.865988 10021 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.882668 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:07.883590 10096 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:07.886778 10193 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.887488 10193 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.888893 10291 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.889616 10291 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.892210 10193 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.898483  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.899564 10021 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
W20260709 12:23:07.901005 10065 leader_election.cc:343] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.892532 10291 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:07.894523 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:07.906569 10062 leader_election.cc:343] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.907266 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:07.908830 10193 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.918265 10139 tablet_bootstrap.cc:654] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.910352 10212 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:07.925259 10140 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:07.925779 10140 ts_tablet_manager.cc:1403] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.274s	user 0.099s	sys 0.150s
I20260709 12:23:07.931471 10141 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:07.932022 10141 ts_tablet_manager.cc:1403] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.260s	user 0.076s	sys 0.110s
I20260709 12:23:07.934674 10141 raft_consensus.cc:359] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.935335 10141 raft_consensus.cc:385] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.935647 10141 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.936105 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:07.936952 10141 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.939543 10141 ts_tablet_manager.cc:1434] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:07.940968 10141 tablet_bootstrap.cc:492] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:07.945143 10193 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.946508 10193 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.947844 10140 raft_consensus.cc:359] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.948593 10140 raft_consensus.cc:385] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:07.948911 10140 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:07.949662 10140 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.951752  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:07.952287 10140 ts_tablet_manager.cc:1434] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:23:07.953693 10140 tablet_bootstrap.cc:492] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:07.954335 10291 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:07.954854 10291 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:07.955642 10291 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.957212  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:07.960088  9842 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:07.961208 10193 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.961794 10300 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.967603 10300 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:07.961941 10361 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:07.973162  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:07.974972 10291 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:23:07.983639 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:07.984617 10095 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:07.962620 10416 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:08.000262 10140 tablet_bootstrap.cc:654] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:07.991891 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:07.994475  9912 leader_election.cc:343] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:08.005635 10065 leader_election.cc:343] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:07.989564 10361 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.015281 10300 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:07.999099 10416 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:08.019506 10139 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:08.020156 10139 ts_tablet_manager.cc:1403] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.161s	user 0.022s	sys 0.015s
I20260709 12:23:08.022740 10292 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.023257 10292 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.023782 10096 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:08.025343 10292 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:08.025602 10062 leader_election.cc:343] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.026152 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:08.029331  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:08.026242 10062 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:08.030295 10141 tablet_bootstrap.cc:654] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:08.030753 10062 leader_election.cc:343] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.031597 10193 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:08.037739  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.043978  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.035099 10361 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:23:08.049105 10065 leader_election.cc:343] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.050045 10291 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:08.050875 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:08.052381 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:08.053965  9912 leader_election.cc:343] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.054064 10139 raft_consensus.cc:359] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.054777 10139 raft_consensus.cc:385] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:08.054785 10062 leader_election.cc:343] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.055110 10139 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.055305 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:08.055925 10139 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.056399 10422 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:08.049614 10212 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.059029 10212 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.059307 10139 ts_tablet_manager.cc:1434] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.038s	user 0.005s	sys 0.002s
I20260709 12:23:08.081154 10139 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
W20260709 12:23:08.066996  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:08.035125 10292 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:23:08.050676 10065 leader_election.cc:343] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.085302 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:08.086191 10422 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:08.085428 10292 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.095904 10188 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.099351 10309 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.095079 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.120049  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.121120 10309 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.121811 10188 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.101509 10212 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.127589 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.061900 10416 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.100466 10292 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.137558 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:08.134188 10098 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:08.145831  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:08.147063 10212 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:08.149324  9912 leader_election.cc:343] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.196424 10361 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.197028 10361 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.201604 10309 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.239252 10139 tablet_bootstrap.cc:700] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent opening tablet: real 0.121s	user 0.003s	sys 0.000s
W20260709 12:23:08.210034  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64332.3us
I20260709 12:23:08.241336 10139 tablet_bootstrap.cc:654] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.250432 10361 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:08.260001 10140 tablet_bootstrap.cc:492] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:08.260561 10140 ts_tablet_manager.cc:1403] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.307s	user 0.024s	sys 0.013s
I20260709 12:23:08.252713 10188 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.266059  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.266448  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.274866 10141 tablet_bootstrap.cc:492] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:08.275362 10141 ts_tablet_manager.cc:1403] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.335s	user 0.019s	sys 0.017s
W20260709 12:23:08.272710 10065 leader_election.cc:343] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:08.277596  9991 leader_election.cc:343] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.260756  9914 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:08.281025 10140 raft_consensus.cc:359] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.281381 10292 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.282291 10290 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:08.282697 10290 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.283038 10290 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.281764 10140 raft_consensus.cc:385] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.286350 10140 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.286386 10141 raft_consensus.cc:359] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.287226 10141 raft_consensus.cc:385] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.287568 10141 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.287289 10140 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.288700 10141 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:08.276585 10425 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:23:08.295359 10141 ts_tablet_manager.cc:1434] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.000s
I20260709 12:23:08.296381 10141 tablet_bootstrap.cc:492] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:08.299337 10140 ts_tablet_manager.cc:1434] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 12:23:08.299993 10140 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:08.300921 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.301571 10096 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:08.306917 10292 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.307307  9990 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:08.307801 10290 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.308081 10285 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:08.308578 10285 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.308919 10285 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.310185 10290 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.312235 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.312865 10096 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:08.323763 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:08.330336 10285 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.336519 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:08.337075 10097 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.344717  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:08.345927 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:08.346503 10096 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:08.348695  9912 consensus_peers.cc:597] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:08.350818 10285 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:08.353101 10062 leader_election.cc:343] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.353601 10062 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:08.354313 10097 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:08.356278 10422 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:08.359553 10139 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:08.360072 10139 ts_tablet_manager.cc:1403] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.279s	user 0.018s	sys 0.012s
I20260709 12:23:08.362637 10139 raft_consensus.cc:359] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.363144 10141 tablet_bootstrap.cc:654] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.365303  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:08.375921 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:08.376565 10098 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.376843 10290 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:08.377228 10290 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.377677 10290 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.379532 10188 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:08.381969  9991 leader_election.cc:343] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.385717 10098 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:08.367645 10096 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:08.387550 10290 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.391804 10139 raft_consensus.cc:385] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.392249 10139 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.393054 10139 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.374154 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:08.395040 10023 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.387295  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:08.395813 10139 ts_tablet_manager.cc:1434] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.000s
I20260709 12:23:08.396770 10292 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:08.388115 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:08.399051 10292 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:08.367870 10140 tablet_bootstrap.cc:654] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.399942 10292 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.402068 10023 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:08.403349 10025 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.394323 10290 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.405903 10139 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:08.393497  9842 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.411621  9842 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.418452 10139 tablet_bootstrap.cc:654] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.424507  9990 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:08.425693 10432 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:08.426283 10432 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:08.427320 10139 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:08.428006 10139 ts_tablet_manager.cc:1403] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.001s
I20260709 12:23:08.427330 10432 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.430727 10139 raft_consensus.cc:359] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.431427 10139 raft_consensus.cc:385] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.431692 10139 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.432289 10139 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.434635 10139 ts_tablet_manager.cc:1434] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:23:08.436311 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:08.436910 10099 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.437244 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:08.437835 10024 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.438187 10139 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:08.447073 10139 tablet_bootstrap.cc:654] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.451438  9914 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:08.452363 10292 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:08.452834 10292 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:08.453686 10292 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.454270  9839 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.458879 10139 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:08.464758 10139 ts_tablet_manager.cc:1403] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.003s
I20260709 12:23:08.460424 10140 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:08.475051 10140 ts_tablet_manager.cc:1403] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.175s	user 0.008s	sys 0.003s
I20260709 12:23:08.463068 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
W20260709 12:23:08.463589  9912 leader_election.cc:343] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.477689 10139 raft_consensus.cc:359] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.478391 10139 raft_consensus.cc:385] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.478689 10139 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.479324 10139 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.481436 10139 ts_tablet_manager.cc:1434] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.000s
I20260709 12:23:08.482299 10139 tablet_bootstrap.cc:492] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
W20260709 12:23:08.484935  9912 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:08.485535  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:08.486413 10290 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:08.487715 10290 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:08.487510 10140 raft_consensus.cc:359] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.488201 10140 raft_consensus.cc:385] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.488479 10140 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.488564 10290 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.489270 10140 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.491370 10140 ts_tablet_manager.cc:1434] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 12:23:08.492307 10140 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:08.492026  9843 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.493878 10024 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:08.499678 10140 tablet_bootstrap.cc:654] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.504840 10139 tablet_bootstrap.cc:654] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.505339  9842 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.511252 10290 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.512027 10290 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.511245 10140 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:08.513080 10140 ts_tablet_manager.cc:1403] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.000s
I20260709 12:23:08.515121 10141 tablet_bootstrap.cc:492] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:08.515662 10141 ts_tablet_manager.cc:1403] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.219s	user 0.025s	sys 0.025s
I20260709 12:23:08.515450 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:08.516696 10140 raft_consensus.cc:359] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.517383 10140 raft_consensus.cc:385] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.517674 10140 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.517752 10139 tablet_bootstrap.cc:492] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:08.518250 10139 ts_tablet_manager.cc:1403] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.000s
I20260709 12:23:08.518476 10140 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.520752 10141 raft_consensus.cc:359] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.521085 10140 ts_tablet_manager.cc:1434] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:08.521085 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.522123 10141 raft_consensus.cc:385] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.522462 10099 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:08.522790 10140 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:08.522488 10141 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.523715  9914 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:08.523787 10141 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.524608 10292 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.521138 10139 raft_consensus.cc:359] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.526773 10139 raft_consensus.cc:385] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.527070 10139 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.527976 10139 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.529964 10140 tablet_bootstrap.cc:654] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.530316 10139 ts_tablet_manager.cc:1434] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.012s	user 0.001s	sys 0.003s
I20260709 12:23:08.531307 10139 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:08.533347 10292 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:08.535054  9843 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.527046 10290 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.540390 10139 tablet_bootstrap.cc:654] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.546872 10141 ts_tablet_manager.cc:1434] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 12:23:08.551602 10141 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:08.552459 10309 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.552971 10309 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.555805  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.556442  9947 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:08.562690  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:08.563537 10290 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:08.565330 10432 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:08.565742 10432 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.566072 10432 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.567286 10141 tablet_bootstrap.cc:654] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.570533 10140 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:08.571025 10140 ts_tablet_manager.cc:1403] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.048s	user 0.019s	sys 0.025s
I20260709 12:23:08.572819 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.572973 10309 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.573443 10097 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:08.573382 10140 raft_consensus.cc:359] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.574101 10140 raft_consensus.cc:385] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.574364 10140 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.574990 10140 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.576380 10140 ts_tablet_manager.cc:1434] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:23:08.577122 10140 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:08.578027 10432 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.583133 10140 tablet_bootstrap.cc:654] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.584874 10141 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:08.585318 10141 ts_tablet_manager.cc:1403] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.004s
I20260709 12:23:08.588033 10141 raft_consensus.cc:359] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.588593 10141 raft_consensus.cc:385] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.588815 10141 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.589399 10141 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.590955 10141 ts_tablet_manager.cc:1434] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:23:08.591060  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:08.591693 10141 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:08.592042  9948 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.593547 10309 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.594182 10309 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.599392  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.600248 10141 tablet_bootstrap.cc:654] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.609119 10141 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:08.611660 10140 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:08.612125 10140 ts_tablet_manager.cc:1403] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.035s	user 0.011s	sys 0.001s
I20260709 12:23:08.614703 10140 raft_consensus.cc:359] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.615356 10140 raft_consensus.cc:385] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.615624 10140 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.616221 10140 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.609561 10141 ts_tablet_manager.cc:1403] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:23:08.621207 10141 raft_consensus.cc:359] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.621893 10141 raft_consensus.cc:385] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.622151 10141 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.622803 10141 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.624799 10141 ts_tablet_manager.cc:1434] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:08.625042 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:08.625600 10099 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.625765 10141 tablet_bootstrap.cc:492] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:08.627804  9948 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:08.629078  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:08.629848 10383 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:08.630301 10383 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:08.618497 10140 ts_tablet_manager.cc:1434] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:23:08.631099 10383 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.631871 10099 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:08.632051 10140 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:08.623965 10432 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.638217 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.638664 10140 tablet_bootstrap.cc:654] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.639079 10099 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:08.640283  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:08.641163 10309 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.641324 10432 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.644418 10141 tablet_bootstrap.cc:654] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.663427 10140 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:08.663918 10140 ts_tablet_manager.cc:1403] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.004s
I20260709 12:23:08.666141 10422 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.666697 10422 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.666960 10140 raft_consensus.cc:359] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.667599 10140 raft_consensus.cc:385] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.667878 10140 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.668594 10140 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.670719 10140 ts_tablet_manager.cc:1434] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:08.670849 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:08.671728 10024 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.672178 10422 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:08.675892 10140 tablet_bootstrap.cc:492] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Bootstrap starting.
W20260709 12:23:08.682804 10062 leader_election.cc:343] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.695364 10188 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.695827 10422 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.695936 10188 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.696326 10422 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.698160 10422 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:08.702913 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:08.707224 10062 leader_election.cc:343] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.719802 10188 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.720602 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:08.724856  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.732981 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.733645 10097 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:08.734221  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.735745  9914 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:08.736589  9912 leader_election.cc:343] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.737514 10290 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:08.737890 10290 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.738236 10290 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:08.745011 10065 leader_election.cc:343] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.745647 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:08.746781 10422 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:08.752110  9843 catalog_manager.cc:5697] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:08.756748 10292 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:08.768550 10423 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.769098 10423 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.771770 10423 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:08.773041 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:08.780653 10062 leader_election.cc:343] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.782661 10141 tablet_bootstrap.cc:492] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
W20260709 12:23:08.782879 10065 leader_election.cc:343] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.783133 10141 ts_tablet_manager.cc:1403] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.158s	user 0.056s	sys 0.051s
I20260709 12:23:08.783332 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:08.785267 10140 tablet_bootstrap.cc:654] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.788926 10139 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:08.789397 10139 ts_tablet_manager.cc:1403] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.258s	user 0.053s	sys 0.065s
I20260709 12:23:08.780565 10290 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.785665 10422 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:08.790648  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:08.793574 10432 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:08.794014 10065 leader_election.cc:343] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.794425 10065 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:08.794704 10290 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.794862 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:08.796262 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:23:08.796456  9912 leader_election.cc:343] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.796801 10099 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.798041 10139 raft_consensus.cc:359] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.798739 10139 raft_consensus.cc:385] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.798987 10139 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.799751 10139 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.802281 10423 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:08.802582 10099 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:08.803023 10141 raft_consensus.cc:359] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.803658 10141 raft_consensus.cc:385] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.803928 10141 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.804075  9914 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:08.804553 10141 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.804966 10230 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:08.807040 10139 ts_tablet_manager.cc:1434] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 12:23:08.807421 10230 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:08.808566 10416 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.808238 10230 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.810148 10141 ts_tablet_manager.cc:1434] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.002s
I20260709 12:23:08.811173 10141 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:08.809053 10416 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.817314 10416 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.821949 10139 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:08.822140 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:08.824321  9912 leader_election.cc:343] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.829617 10023 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:08.830168 10291 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.826877 10188 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:23:08.834131 10291 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.881762 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.910910 10099 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:08.922816 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:08.924490  9912 leader_election.cc:343] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.916795 10361 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.928818 10361 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.921326 10291 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.934448 10141 tablet_bootstrap.cc:654] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.943102 10139 tablet_bootstrap.cc:654] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:08.947453 10140 tablet_bootstrap.cc:492] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:08.948104 10140 ts_tablet_manager.cc:1403] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.273s	user 0.017s	sys 0.058s
I20260709 12:23:08.945487 10230 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:08.950443 10230 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.951836 10361 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:08.952292 10230 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:08.926298 10188 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:23:08.935802  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:08.952894 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:08.953377 10230 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:23:08.972929  9912 leader_election.cc:343] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:08.960608 10230 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:08.977587 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:08.978435 10099 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:08.981383 10140 raft_consensus.cc:359] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.982124 10140 raft_consensus.cc:385] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:08.982398 10140 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:08.985405 10140 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:08.976991 10230 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:09.089185 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:09.090138 10097 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:09.090396 10139 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:09.090965 10139 ts_tablet_manager.cc:1403] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.271s	user 0.008s	sys 0.026s
I20260709 12:23:09.093677 10140 ts_tablet_manager.cc:1434] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.145s	user 0.006s	sys 0.017s
I20260709 12:23:09.094583 10140 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: Bootstrap starting.
W20260709 12:23:09.011569 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64649.9us
I20260709 12:23:09.094485  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:09.110675 10139 raft_consensus.cc:359] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.111083  9914 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:09.111447 10139 raft_consensus.cc:385] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.111805 10139 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.111848 10188 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:09.116184 10139 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.117602 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:09.112501 10290 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 12:23:09.128047 10062 leader_election.cc:343] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.112604 10291 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:09.112286 10188 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.141652 10139 ts_tablet_manager.cc:1434] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.000s
I20260709 12:23:09.138119 10291 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.147027  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:09.122620 10309 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.116547 10463 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.156478 10139 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
W20260709 12:23:09.159935  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58687.7us
I20260709 12:23:09.140750 10188 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:09.154493 10291 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:09.159034 10309 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.181773 10309 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:09.169023  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55057.4us
I20260709 12:23:09.155201 10290 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 12:23:09.184975 10432 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:23:09.199110 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:09.199807 10096 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:09.139130  9839 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:09.204061 10462 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:23:09.205684 10416 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:09.206231 10230 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.212710 10230 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:09.213328 10141 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:09.213801 10141 ts_tablet_manager.cc:1403] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.403s	user 0.012s	sys 0.008s
I20260709 12:23:09.214291 10140 tablet_bootstrap.cc:654] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:09.230690 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44741.4us
I20260709 12:23:09.247705 10291 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.218945 10462 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:23:09.235831  9990 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:09.264880 10141 raft_consensus.cc:359] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.280857 10141 raft_consensus.cc:385] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.272986 10188 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:09.281056  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45349us
I20260709 12:23:09.268927 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:09.282574 10099 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:09.160902 10463 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.281188 10141 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.253681 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:09.283959 10141 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.286545 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:23:09.291431  9912 leader_election.cc:343] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.287809  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:09.294956  9943 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:09.295920 10099 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:09.284897 10383 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:09.296540 10383 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:09.296907 10383 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:09.298575 10188 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:09.305213  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:09.313848 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:09.314298 10188 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:09.314489 10099 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:09.314797 10188 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:09.315501 10188 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:09.319270 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 134593us
I20260709 12:23:09.321597 10141 ts_tablet_manager.cc:1434] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.108s	user 0.011s	sys 0.000s
W20260709 12:23:09.322890 10065 leader_election.cc:343] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.323354 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:09.328239 10099 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:09.268301 10432 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.039s
I20260709 12:23:09.336875 10141 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:09.343166  9914 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:09.345355 10139 tablet_bootstrap.cc:654] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.351179 10472 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:09.351599 10467 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:09.350782 10471 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.054s
I20260709 12:23:09.360395 10472 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:09.402400 10291 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:09.401775 10472 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.414301 10463 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:09.437430 10140 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:09.442768 10140 ts_tablet_manager.cc:1403] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.348s	user 0.017s	sys 0.011s
I20260709 12:23:09.440634 10467 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.463639 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:09.480996 10093 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:09.495368  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:23:09.508788 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 114434us
I20260709 12:23:09.484962 10467 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.517580 10141 tablet_bootstrap.cc:654] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.518345 10383 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:09.527774 10479 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.176s	user 0.004s	sys 0.000s
I20260709 12:23:09.529348 10139 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:09.537590 10093 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:09.543231 10139 ts_tablet_manager.cc:1403] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.387s	user 0.028s	sys 0.024s
I20260709 12:23:09.546089 10139 raft_consensus.cc:359] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.546838 10139 raft_consensus.cc:385] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.547107 10139 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.547747 10139 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.548940 10467 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:09.549530 10139 ts_tablet_manager.cc:1434] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:09.551668  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:09.554059 10291 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:09.554567 10291 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:09.555459 10291 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.557149 10383 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:09.561020 10023 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:09.566910 10140 raft_consensus.cc:359] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.569396 10140 raft_consensus.cc:385] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.569890 10140 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.571780 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:09.572664 10140 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:09.604081 10062 leader_election.cc:343] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.678484 10139 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:09.681895 10140 ts_tablet_manager.cc:1434] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.238s	user 0.011s	sys 0.001s
W20260709 12:23:09.690124  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 84284.8us
I20260709 12:23:09.676015 10472 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.639374 10292 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.052s
I20260709 12:23:09.697188 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:09.699322 10092 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:09.706897 10141 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:09.707403 10141 ts_tablet_manager.cc:1403] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.371s	user 0.018s	sys 0.010s
I20260709 12:23:09.708439 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:09.709913 10062 leader_election.cc:343] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.715631  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:09.703603 10472 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.716513  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:09.698463 10491 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:09.767967  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61210.4us
I20260709 12:23:09.773674 10020 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:09.782353 10065 leader_election.cc:343] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:09.782845 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:09.868685 10497 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:09.869513 10484 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.086s
I20260709 12:23:09.768748 10472 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:09.881817  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:09.884742 10141 raft_consensus.cc:359] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.885685 10141 raft_consensus.cc:385] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:09.886029 10141 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:09.891536 10141 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.765852 10491 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.893553 10141 ts_tablet_manager.cc:1434] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.186s	user 0.025s	sys 0.018s
W20260709 12:23:09.939945  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 154895us
W20260709 12:23:09.941627  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 171794us
I20260709 12:23:09.939226  9991 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:09.949049 10478 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:09.941556 10472 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.958760 10472 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.966987 10491 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:09.968430 10464 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:09.968701 10472 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:09.969017 10464 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:09.971377 10139 tablet_bootstrap.cc:654] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:09.976378 10464 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:09.979681 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:09.982717 10141 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:09.983289 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:10.025892  9912 leader_election.cc:343] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:10.028934  9912 leader_election.cc:343] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.048897 10383 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.049431 10383 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.060951 10141 tablet_bootstrap.cc:654] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.085813 10139 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:10.086442 10139 ts_tablet_manager.cc:1403] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.408s	user 0.032s	sys 0.016s
I20260709 12:23:10.093156 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:10.099551 10140 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:10.103619 10139 raft_consensus.cc:359] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.106437 10139 raft_consensus.cc:385] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.098508 10383 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.106730 10139 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.106930  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.100894 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:10.111236 10097 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:10.112519  9990 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:10.113428 10478 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:10.110937 10139 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.116298 10139 ts_tablet_manager.cc:1434] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.002s
I20260709 12:23:10.117142 10139 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:10.113889 10478 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.119621 10478 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:10.122812 10141 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:10.123313 10141 ts_tablet_manager.cc:1403] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.141s	user 0.015s	sys 0.015s
I20260709 12:23:10.124708 10140 tablet_bootstrap.cc:654] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.124821 10022 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.126997 10141 raft_consensus.cc:359] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.127724 10141 raft_consensus.cc:385] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.127991 10141 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.128619 10141 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.138464 10139 tablet_bootstrap.cc:654] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.139969 10423 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.140607 10423 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.141656 10478 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.142696 10423 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:10.143666 10423 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.144147 10423 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.144479 10478 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.146023 10423 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:10.140081 10510 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.146718 10510 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.147660  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.149225 10510 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:10.150959 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:10.151558 10093 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:10.151957 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:10.152546 10098 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:10.154495  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.154961 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.155196  9944 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:10.156744  9839 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:10.158978 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.160494 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:10.138350 10065 leader_election.cc:343] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.162549 10093 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:10.186825 10092 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
W20260709 12:23:10.165243  9912 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:10.188635  9912 leader_election.cc:343] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.166492 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.193343  9990 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:10.195441 10141 ts_tablet_manager.cc:1434] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.072s	user 0.008s	sys 0.000s
I20260709 12:23:10.195685 10478 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:10.196386 10478 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:10.197583 10478 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.198961 10188 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:10.167240 10062 leader_election.cc:343] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.210470 10140 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
W20260709 12:23:10.210649 10062 leader_election.cc:343] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.211009 10140 ts_tablet_manager.cc:1403] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.112s	user 0.041s	sys 0.011s
W20260709 12:23:10.211323 10062 leader_election.cc:343] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:10.212193 10062 leader_election.cc:343] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.169059 10065 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:10.170300  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:10.170634 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.218044 10140 raft_consensus.cc:359] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:10.218405 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 80087.3us
I20260709 12:23:10.218850 10140 raft_consensus.cc:385] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.219136 10140 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.175482 10139 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:10.219760 10139 ts_tablet_manager.cc:1403] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.103s	user 0.013s	sys 0.001s
I20260709 12:23:10.220029 10475 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:10.220440 10065 leader_election.cc:343] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.220851 10065 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:10.170986  9839 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:10.222275 10285 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.222564 10463 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.222795 10285 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.199718 10462 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:23:10.223829 10065 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:10.180985  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:10.199683 10141 tablet_bootstrap.cc:492] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
W20260709 12:23:10.225571  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44672.1us
I20260709 12:23:10.167064 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:10.226269 10096 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:10.226387  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.226562 10504 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:10.201272 10461 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:23:10.228147 10062 leader_election.cc:343] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.228598 10285 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.233457 10462 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:10.234905 10462 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:10.235831  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.236145 10462 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:10.238845 10188 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:10.239183 10188 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:10.228957 10139 raft_consensus.cc:359] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.239866 10139 raft_consensus.cc:385] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.240139 10139 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.240052 10188 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.241019 10188 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:10.240975 10139 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.243337 10139 ts_tablet_manager.cc:1434] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.001s
I20260709 12:23:10.240988  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.244259 10139 tablet_bootstrap.cc:492] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:10.229658 10475 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:10.242264  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.244843 10475 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.246197 10475 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.229535 10504 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.246783 10504 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.257128  9914 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:23:10.257941 10065 leader_election.cc:343] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.258386 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:23:10.262152 10065 leader_election.cc:343] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.262666 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:10.265998 10502 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:10.266425 10502 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.271538 10140 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:10.272691 10065 leader_election.cc:343] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.273110 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:10.274514 10502 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.267603 10464 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.274093 10463 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.276422 10422 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.284888 10140 ts_tablet_manager.cc:1434] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.074s	user 0.006s	sys 0.000s
I20260709 12:23:10.280916 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:10.288746 10024 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:23:10.292971 10096 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:10.304534  9990 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:10.312108 10140 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:10.316721 10475 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.305858 10422 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.284257 10504 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.317565 10285 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:10.338938 10422 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.321043 10500 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:10.353376 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:10.354041 10024 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.343406 10285 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:10.359323 10285 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:10.360363 10024 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:23:10.361975 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:10.362370 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.362803 10463 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:10.363389 10463 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:10.332890 10464 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:23:10.363941 10062 leader_election.cc:343] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.342491 10475 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:10.349651 10422 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:10.374583 10504 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.374418  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:10.372840 10463 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.378756 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:23:10.380033  9912 leader_election.cc:343] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.380259 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:10.389043 10096 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.389920  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.368147 10464 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.393345 10464 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:10.395848 10139 tablet_bootstrap.cc:654] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.397316 10141 tablet_bootstrap.cc:654] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.399191 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.402761 10140 tablet_bootstrap.cc:654] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:10.405823 10065 leader_election.cc:343] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.406302 10065 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:10.409423  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.411722 10423 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:23:10.415889 10065 leader_election.cc:343] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:10.416383 10062 leader_election.cc:343] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.418004 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:10.419448 10475 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.415385 10502 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.427755  9948 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:10.428709 10502 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.431161 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:10.444367 10020 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:10.444954 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:10.429754 10509 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.449713  9914 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:10.450887 10188 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.447211 10509 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.429461 10285 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.456491 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:10.460934  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:10.462373 10188 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:10.462994 10188 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:10.464006 10188 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.457038 10023 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:10.495287 10509 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.502297 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:10.507503 10285 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.483899 10504 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:23:10.513299 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:10.514221 10097 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:10.518121  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:10.515630 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:10.521003  9912 leader_election.cc:343] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.525648  9991 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:10.525871  9914 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:10.529729 10478 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:10.527621 10460 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:10.532071 10460 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:10.532416 10460 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.550411 10141 tablet_bootstrap.cc:492] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:10.550951 10141 ts_tablet_manager.cc:1403] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.354s	user 0.023s	sys 0.023s
I20260709 12:23:10.617576 10141 raft_consensus.cc:359] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.618351 10141 raft_consensus.cc:385] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.625823 10460 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.634910 10140 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:10.635432 10140 ts_tablet_manager.cc:1403] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.323s	user 0.027s	sys 0.046s
I20260709 12:23:10.636812 10500 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:10.637318 10500 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:10.638074 10500 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.639219 10500 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:10.639752 10140 raft_consensus.cc:359] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.640461 10140 raft_consensus.cc:385] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.640733 10140 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.641386 10140 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.643652 10140 ts_tablet_manager.cc:1434] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:10.644606 10140 tablet_bootstrap.cc:492] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Bootstrap starting.
W20260709 12:23:10.642460  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:10.651022 10140 tablet_bootstrap.cc:654] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.667430 10139 tablet_bootstrap.cc:492] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:10.668064 10139 ts_tablet_manager.cc:1403] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.424s	user 0.030s	sys 0.043s
I20260709 12:23:10.668982 10141 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.670291 10141 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.672565 10141 ts_tablet_manager.cc:1434] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.121s	user 0.006s	sys 0.000s
I20260709 12:23:10.673511 10141 tablet_bootstrap.cc:492] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:10.690359 10460 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.695747  9839 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:23:10.698930 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:10.697803  9839 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:10.699261 10139 raft_consensus.cc:359] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.700294 10139 raft_consensus.cc:385] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.700582 10139 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
W20260709 12:23:10.701663  9912 leader_election.cc:343] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.701973 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:10.705513 10139 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.704260 10097 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.708590 10139 ts_tablet_manager.cc:1434] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.040s	user 0.011s	sys 0.002s
I20260709 12:23:10.714311 10141 tablet_bootstrap.cc:654] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.721545 10139 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:10.723973  9842 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:10.728183 10097 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:10.732643 10140 tablet_bootstrap.cc:492] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:10.744815  9914 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:10.746105 10460 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:10.746641 10460 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:10.747782 10141 tablet_bootstrap.cc:492] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:10.747537 10460 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.748324 10141 ts_tablet_manager.cc:1403] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.003s
I20260709 12:23:10.747705  9843 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:10.751207 10139 tablet_bootstrap.cc:654] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.755273 10140 ts_tablet_manager.cc:1403] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.111s	user 0.017s	sys 0.023s
I20260709 12:23:10.757678 10141 raft_consensus.cc:359] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.761358 10141 raft_consensus.cc:385] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.761682 10141 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.762347 10141 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.768072 10140 raft_consensus.cc:359] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.768989 10140 raft_consensus.cc:385] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.769135 10141 ts_tablet_manager.cc:1434] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:23:10.769407 10140 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.770066 10141 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Bootstrap starting.
I20260709 12:23:10.770223 10140 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.773842 10140 ts_tablet_manager.cc:1434] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:10.774919 10140 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:10.777977 10139 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:10.778455 10139 ts_tablet_manager.cc:1403] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.069s	user 0.014s	sys 0.003s
I20260709 12:23:10.781276 10139 raft_consensus.cc:359] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.782013 10139 raft_consensus.cc:385] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.782339 10139 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.783529 10139 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.825495 10141 tablet_bootstrap.cc:654] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.847571  9943 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:23:10.848018  9943 pending_rounds.cc:85] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 0
I20260709 12:23:10.848330  9943 pending_rounds.cc:107] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:23:10.848641  9943 raft_consensus.cc:2889] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:23:10.856364 10140 tablet_bootstrap.cc:654] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.857625 10485 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:10.859762 10097 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:10.863404 10023 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.863934 10139 ts_tablet_manager.cc:1434] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.085s	user 0.011s	sys 0.002s
I20260709 12:23:10.865078 10139 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:10.866492 10416 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.866993 10416 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.869820 10416 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.871034  9840 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:10.876700 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.878458 10020 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.881011 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:10.881685 10095 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:10.887239 10285 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:10.888368  9912 leader_election.cc:343] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.890304 10416 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:10.892139 10422 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.893281 10422 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.895346 10422 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:10.901520  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:10.901794 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.903825 10141 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: No bootstrap required, opened a new log
I20260709 12:23:10.903986  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:10.904354 10141 ts_tablet_manager.cc:1403] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent bootstrapping tablet: real 0.134s	user 0.015s	sys 0.003s
I20260709 12:23:10.911633 10416 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:10.912205 10416 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:10.908746 10062 leader_election.cc:343] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:10.913353 10065 leader_election.cc:343] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.913787 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:10.913697 10416 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:10.918138 10422 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:10.925352 10141 raft_consensus.cc:359] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.926338 10141 raft_consensus.cc:385] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.926644 10141 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.927577 10141 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.937064 10141 ts_tablet_manager.cc:1434] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:23:10.937214 10140 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:10.937944 10140 ts_tablet_manager.cc:1403] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.163s	user 0.017s	sys 0.006s
I20260709 12:23:10.949539 10139 tablet_bootstrap.cc:654] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:10.938046 10500 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:23:10.959311 10464 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:23:10.964615 10463 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:10.973868 10140 raft_consensus.cc:359] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.978672 10140 raft_consensus.cc:385] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:10.978991 10140 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:10.983139 10463 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.983882 10463 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.986037 10463 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:10.987984  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:10.989487 10065 leader_election.cc:343] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:10.989401 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:10.994952 10500 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:10.995477 10500 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.997675 10500 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:10.998401 10140 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:10.998993 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.014494 10095 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
W20260709 12:23:11.000197 10062 leader_election.cc:343] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.015498 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:11.002653 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.016182 10422 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.010474 10463 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.017198 10463 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.019171 10140 ts_tablet_manager.cc:1434] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.081s	user 0.008s	sys 0.003s
I20260709 12:23:11.019831 10463 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:11.022543  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:11.023588 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:11.029857 10065 leader_election.cc:343] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.016134  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:11.030514 10062 leader_election.cc:343] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:23:11.030851  9912 leader_election.cc:343] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.030951 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:11.031782 10140 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:11.031929 10463 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.039266 10140 tablet_bootstrap.cc:654] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.002270 10416 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.046195 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:23:11.048056  9912 leader_election.cc:343] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.049065 10500 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.049467 10500 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.049813 10500 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:11.060021 10544 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.101s	user 0.000s	sys 0.005s
I20260709 12:23:11.063654 10139 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.064131 10139 ts_tablet_manager.cc:1403] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.199s	user 0.021s	sys 0.004s
I20260709 12:23:11.067699 10416 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.068446 10139 raft_consensus.cc:359] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.069125 10139 raft_consensus.cc:385] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.069397 10139 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.070044 10139 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.072134 10139 ts_tablet_manager.cc:1434] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:11.081980 10140 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:11.087116 10140 ts_tablet_manager.cc:1403] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.056s	user 0.017s	sys 0.021s
I20260709 12:23:11.078241 10500 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.089596 10139 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.085251 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:11.090000 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:11.091254 10097 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.090257 10500 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.094331 10140 raft_consensus.cc:359] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.095103 10140 raft_consensus.cc:385] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.097378 10140 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.097819 10140 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.094542 10096 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.104162 10097 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.105301  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.106865 10229 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.107339 10229 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:11.108132 10229 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.104918 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:11.114789 10460 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.115319 10460 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:11.121829  9912 leader_election.cc:343] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.122521 10140 ts_tablet_manager.cc:1434] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 12:23:11.123608 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.124505 10140 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:11.124629 10097 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:11.128087 10139 tablet_bootstrap.cc:654] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.128080  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.121374 10460 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.129554 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.130822 10416 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.133704 10416 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:23:11.136656  9912 leader_election.cc:343] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.137557 10416 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.143946 10096 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.146556  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:11.147763 10500 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.148222 10500 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:11.149037 10500 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.150542 10140 tablet_bootstrap.cc:654] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.149391 10416 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.161059 10416 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.165671 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:11.166155 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:11.166782 10097 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:11.167304  9912 leader_election.cc:343] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.173663 10140 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:11.174220 10140 ts_tablet_manager.cc:1403] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.051s	user 0.012s	sys 0.005s
I20260709 12:23:11.176919 10097 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.177539 10140 raft_consensus.cc:359] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.163384  9842 catalog_manager.cc:5697] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.184760 10139 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.185596 10139 ts_tablet_manager.cc:1403] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.096s	user 0.009s	sys 0.011s
I20260709 12:23:11.187520  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:11.188310 10139 raft_consensus.cc:359] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.188997 10139 raft_consensus.cc:385] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.189137 10500 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.189303 10139 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.190694 10500 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:11.190956 10139 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.192209 10500 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.193156 10139 ts_tablet_manager.cc:1434] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:23:11.185213 10140 raft_consensus.cc:385] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.193694 10140 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.194361 10140 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.196750 10139 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.211200 10140 ts_tablet_manager.cc:1434] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.037s	user 0.008s	sys 0.000s
I20260709 12:23:11.212222 10140 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:11.206187 10416 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:23:11.223517 10139 tablet_bootstrap.cc:654] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.223719  9843 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.226190 10140 tablet_bootstrap.cc:654] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.225567  9843 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.235427 10285 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.236161 10285 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.243925  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:11.259163 10023 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.262686 10139 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.263265 10139 ts_tablet_manager.cc:1403] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.067s	user 0.030s	sys 0.008s
I20260709 12:23:11.264758 10285 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.266063 10139 raft_consensus.cc:359] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.268033 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.270686 10416 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:11.271487 10139 raft_consensus.cc:385] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.271811 10139 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.272464 10139 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.274844 10139 ts_tablet_manager.cc:1434] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 12:23:11.274814  9990 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:11.277109 10139 tablet_bootstrap.cc:492] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.277217 10285 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.280216 10140 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:11.280670 10140 ts_tablet_manager.cc:1403] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.069s	user 0.016s	sys 0.008s
I20260709 12:23:11.282405 10285 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.282976 10285 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.292796 10140 raft_consensus.cc:359] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.293536 10140 raft_consensus.cc:385] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.293816 10140 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.293295 10139 tablet_bootstrap.cc:654] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.295733  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:11.294399 10140 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.298954 10140 ts_tablet_manager.cc:1434] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.018s	user 0.014s	sys 0.004s
I20260709 12:23:11.299930 10140 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:11.304832 10416 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.305361 10416 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.305920 10140 tablet_bootstrap.cc:654] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.308501 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.309053 10139 tablet_bootstrap.cc:492] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.309168 10020 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:11.309521 10139 ts_tablet_manager.cc:1403] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.033s	user 0.008s	sys 0.002s
I20260709 12:23:11.312477  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.312480 10139 raft_consensus.cc:359] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.314035 10504 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.314393 10139 raft_consensus.cc:385] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.314465 10504 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.314913 10139 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.315006 10504 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.316493 10285 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.316448 10416 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.318908 10139 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.319630 10140 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:11.320077 10140 ts_tablet_manager.cc:1403] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.020s	user 0.010s	sys 0.008s
I20260709 12:23:11.320672 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.321066 10139 ts_tablet_manager.cc:1434] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:23:11.321316 10096 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:11.322397 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.322542 10140 raft_consensus.cc:359] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.322909 10463 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.323140 10097 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:11.323241 10140 raft_consensus.cc:385] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.323633 10140 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.323374 10463 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.324250 10140 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.325415 10463 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:11.325563  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:11.326574 10485 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.327260 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.327600 10139 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
W20260709 12:23:11.328677 10062 leader_election.cc:343] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.331123 10140 ts_tablet_manager.cc:1434] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:23:11.332123 10140 tablet_bootstrap.cc:492] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:11.334429 10500 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.335223  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:11.335042 10500 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.337319 10500 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.337726 10485 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.340353 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.340992 10097 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:11.343163  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.343482 10504 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.343979 10416 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.344352 10416 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.344671 10416 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.345677 10504 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.348863 10139 tablet_bootstrap.cc:654] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.349169 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:11.349736 10097 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.354904 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:11.358652 10020 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.357267 10140 tablet_bootstrap.cc:654] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:11.357766 10065 leader_election.cc:343] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.367296 10065 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:11.368062 10139 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.368531 10139 ts_tablet_manager.cc:1403] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.041s	user 0.010s	sys 0.005s
I20260709 12:23:11.368991 10464 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.371520 10139 raft_consensus.cc:359] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.372291 10139 raft_consensus.cc:385] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.372560 10139 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.372630 10285 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.373086 10285 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.373272 10139 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.376039 10139 ts_tablet_manager.cc:1434] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:11.376410 10285 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.376332 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.377487 10096 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:11.356185 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.379631  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:11.379958  9912 leader_election.cc:343] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.380749 10097 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.381224  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:11.382231 10485 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.383333  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.384135 10504 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.384634 10504 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:11.385399 10504 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.387589 10140 tablet_bootstrap.cc:492] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:11.388038 10140 ts_tablet_manager.cc:1403] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.002s
I20260709 12:23:11.389660 10139 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.396270 10020 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.394251 10416 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.400808 10139 tablet_bootstrap.cc:654] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.406466 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:11.407117 10097 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.406837 10140 raft_consensus.cc:359] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.407506 10140 raft_consensus.cc:385] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.408221 10416 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.408695 10140 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.409793 10140 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.414853 10097 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.417088 10140 ts_tablet_manager.cc:1434] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.002s
I20260709 12:23:11.417543  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.418021 10140 tablet_bootstrap.cc:492] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:11.418687 10416 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.419231 10416 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:11.420495 10416 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.424894 10139 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.425383 10139 ts_tablet_manager.cc:1403] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.011s
I20260709 12:23:11.428000 10140 tablet_bootstrap.cc:654] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.428524 10139 raft_consensus.cc:359] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.429593 10139 raft_consensus.cc:385] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.430016 10139 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.430624 10485 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.430820 10139 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.433193 10139 ts_tablet_manager.cc:1434] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:11.434037 10139 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.441042 10139 tablet_bootstrap.cc:654] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.443275 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:11.443961 10020 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.446403 10140 tablet_bootstrap.cc:492] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:11.447199 10140 ts_tablet_manager.cc:1403] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.029s	user 0.010s	sys 0.004s
I20260709 12:23:11.406457  9843 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.499397 10139 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.501116 10285 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.505575 10423 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.501188 10139 ts_tablet_manager.cc:1403] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.067s	user 0.011s	sys 0.003s
I20260709 12:23:11.502357 10285 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.530278 10504 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.531334 10504 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.537703 10285 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.541400 10500 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.541282 10423 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.542346 10500 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.544608 10423 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:11.545724 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.554190 10504 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.556138 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.557008 10140 raft_consensus.cc:359] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.557926 10140 raft_consensus.cc:385] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.558301 10140 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.559334 10140 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.565491 10020 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.558713 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.571602 10097 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:11.577663 10096 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:11.584010 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.584578  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:11.583895 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.582679 10504 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:23:11.590268 10229 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.585265  9943 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 0.
I20260709 12:23:11.589784  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.591065 10229 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.595395 10460 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.595970 10460 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.597090 10229 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:11.601729 10062 leader_election.cc:343] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.605321 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.606208 10092 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:11.610352 10460 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.613451 10500 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.613754  9842 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.615247  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:11.619407 10139 raft_consensus.cc:359] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.620401 10139 raft_consensus.cc:385] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.620738 10139 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.621466 10139 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.623288  9914 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.631693 10571 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.632143 10571 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.632493 10571 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.633893 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.634764 10092 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:11.640712 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:11.641397  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.641898 10423 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.642288 10423 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.642642 10423 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.645545 10571 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.647996 10188 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:11.648360 10188 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.648486 10571 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.648650 10188 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:23:11.654053  9912 leader_election.cc:343] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.654407 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:23:11.654987  9912 leader_election.cc:343] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.655098 10095 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.663950 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.664769 10023 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:11.664791 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:11.666745 10017 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.677757 10571 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.694458 10139 ts_tablet_manager.cc:1434] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.155s	user 0.009s	sys 0.000s
I20260709 12:23:11.694737  9991 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:11.695412 10140 ts_tablet_manager.cc:1434] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.248s	user 0.010s	sys 0.000s
I20260709 12:23:11.701154 10095 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.694048 10423 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.715575 10571 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.717564 10423 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:11.718475  9914 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.719188 10571 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.719658 10571 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:11.720631 10571 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.685576 10188 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.708837 10481 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.730165 10188 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.709118 10017 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.736590 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:11.749840 10460 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.767164 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:11.743173 10140 tablet_bootstrap.cc:492] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:11.768038 10092 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:11.744437 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:23:11.751466 10062 leader_election.cc:343] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.740022 10481 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.746304  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:11.772184  9943 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:11.764928 10188 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:11.758451 10139 tablet_bootstrap.cc:492] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.775398 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:11.776453 10423 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:23:11.782040  9912 leader_election.cc:343] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.780287 10460 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.789189 10140 tablet_bootstrap.cc:654] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.793203 10229 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:11.794137  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:11.794862 10139 tablet_bootstrap.cc:654] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.805251 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:11.805919 10095 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:11.812808 10188 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:23:11.821328 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:11.821062 10504 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.831367 10416 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.831934 10416 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.832983 10140 tablet_bootstrap.cc:492] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:11.833451 10140 ts_tablet_manager.cc:1403] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.001s
W20260709 12:23:11.836686  9912 leader_election.cc:343] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.838905 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.833992 10416 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.839562 10016 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:11.842461 10139 tablet_bootstrap.cc:492] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.842969 10139 ts_tablet_manager.cc:1403] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.085s	user 0.009s	sys 0.005s
I20260709 12:23:11.843273  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.844466 10504 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:11.844844 10504 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:11.845176 10504 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:11.845808 10139 raft_consensus.cc:359] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.846537 10139 raft_consensus.cc:385] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.846846 10139 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.846033  9840 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:11.847486 10139 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.854719 10140 raft_consensus.cc:359] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.855458 10140 raft_consensus.cc:385] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.856041 10140 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.856770 10140 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.857110 10504 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.861443 10504 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.859501 10139 ts_tablet_manager.cc:1434] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:23:11.863085 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:11.863642 10016 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:11.865818 10139 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.878664 10139 tablet_bootstrap.cc:654] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.900843 10016 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:11.902896  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:11.903683 10416 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:11.904209 10416 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:11.905071 10416 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.909132 10095 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:11.914971 10139 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.915489 10139 ts_tablet_manager.cc:1403] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.005s
I20260709 12:23:11.918224 10139 raft_consensus.cc:359] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.918933 10139 raft_consensus.cc:385] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:11.919198 10139 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.919956 10139 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.923326 10140 ts_tablet_manager.cc:1434] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.090s	user 0.009s	sys 0.000s
I20260709 12:23:11.924717 10140 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:11.926964 10570 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.927561 10570 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.930593 10570 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.931025 10139 ts_tablet_manager.cc:1434] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:23:11.931154  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:11.932132  9944 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:11.932133 10139 tablet_bootstrap.cc:492] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.942384 10461 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:11.942866 10139 tablet_bootstrap.cc:654] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:11.946899 10500 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.947463 10500 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.949026  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:11.949851 10500 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:11.950294 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.951586 10096 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:23:11.952713 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:11.954250  9912 leader_election.cc:343] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.954470 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:11.957947 10504 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:11.958658 10504 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:11.959973 10504 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.973970 10500 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:11.975517 10139 tablet_bootstrap.cc:492] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:11.975970 10139 ts_tablet_manager.cc:1403] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.044s	user 0.009s	sys 0.006s
I20260709 12:23:11.979231 10464 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:11.979795 10464 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.982698 10464 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:11.982895  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:11.978331 10139 raft_consensus.cc:359] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.983623 10139 raft_consensus.cc:385] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:23:11.984565 10065 leader_election.cc:343] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.983916 10139 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:11.985374 10139 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:11.987463 10139 ts_tablet_manager.cc:1434] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:23:11.988435 10139 tablet_bootstrap.cc:492] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:11.990609 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:11.984469 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:11.996650 10139 tablet_bootstrap.cc:654] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
W20260709 12:23:11.999082 10062 leader_election.cc:343] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:23:11.999540 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:12.000273 10423 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.001981 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:12.002655 10095 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:12.009063 10139 tablet_bootstrap.cc:492] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:12.009725 10139 ts_tablet_manager.cc:1403] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.001s
I20260709 12:23:12.009141 10140 tablet_bootstrap.cc:654] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.012552 10139 raft_consensus.cc:359] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.013303 10139 raft_consensus.cc:385] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.013638 10139 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.014451 10139 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.014382  9839 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.015254  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:12.016372 10570 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.016062  9839 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.017656 10139 ts_tablet_manager.cc:1434] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:12.018776 10139 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:12.024468 10570 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.025122 10139 tablet_bootstrap.cc:654] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.032073  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:12.033062 10188 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.033432 10188 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.033741 10188 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.033790 10416 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:12.040037 10188 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.042253 10188 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:12.043028 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:12.043651 10021 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.043803 10188 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.050216 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:12.050853 10097 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.059309 10140 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:12.059984 10140 ts_tablet_manager.cc:1403] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.135s	user 0.008s	sys 0.003s
I20260709 12:23:12.066135 10097 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:12.068553  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:12.069895 10504 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.069842 10140 raft_consensus.cc:359] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.070650 10504 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:12.071413 10140 raft_consensus.cc:385] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.071800 10140 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.071997 10504 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.072751 10140 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.071267 10021 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:12.087036 10140 ts_tablet_manager.cc:1434] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.003s
I20260709 12:23:12.084647 10139 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:12.090121 10140 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: Bootstrap starting.
I20260709 12:23:12.090981 10139 ts_tablet_manager.cc:1403] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.072s	user 0.018s	sys 0.005s
W20260709 12:23:12.098104  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:12.100206 10139 raft_consensus.cc:359] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.101055 10139 raft_consensus.cc:385] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.101377 10139 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.102090 10139 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.109148 10140 tablet_bootstrap.cc:654] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.108765  9843 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.109413 10139 ts_tablet_manager.cc:1434] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:23:12.110834 10139 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Bootstrap starting.
I20260709 12:23:12.115231 10416 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.115732 10416 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.118955 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:12.119787 10098 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:12.120404 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:12.121057 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:12.122213  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:12.123044 10188 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.123391 10188 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.123503 10139 tablet_bootstrap.cc:654] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Neither blocks nor log segments found. Creating new log.
I20260709 12:23:12.123733 10188 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.143836 10140 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: No bootstrap required, opened a new log
I20260709 12:23:12.144495 10139 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: No bootstrap required, opened a new log
I20260709 12:23:12.144950 10139 ts_tablet_manager.cc:1403] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Time spent bootstrapping tablet: real 0.034s	user 0.009s	sys 0.009s
I20260709 12:23:12.146077 10188 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.147920 10188 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:12.148744 10416 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:12.149688 10423 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:12.149778 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:12.150435 10021 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.150321 10423 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.152410 10423 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:12.154579 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:12.155215 10093 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.155539 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:12.156255 10024 raft_consensus.cc:2368] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3.
I20260709 12:23:12.156162  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:12.157127 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:12.157665 10062 leader_election.cc:400] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3.
I20260709 12:23:12.157953 10062 leader_election.cc:403] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:23:12.158391 10139 raft_consensus.cc:359] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.158875  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:12.159057 10139 raft_consensus.cc:385] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.159650 10139 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.159902 10504 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.160392 10140 ts_tablet_manager.cc:1403] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: Time spent bootstrapping tablet: real 0.070s	user 0.017s	sys 0.017s
I20260709 12:23:12.160758 10504 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:12.162850 10423 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:23:12.162540 10139 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.161594 10504 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.163403 10140 raft_consensus.cc:359] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.165815 10140 raft_consensus.cc:385] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:23:12.166085 10140 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Initialized, Role: FOLLOWER
I20260709 12:23:12.166805 10140 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.169360 10140 ts_tablet_manager.cc:1434] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:23:12.173394 10416 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:12.175835 10423 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3.
I20260709 12:23:12.178114 10139 ts_tablet_manager.cc:1434] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.000s
I20260709 12:23:12.190225 10416 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:23:12.189783 10093 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:12.211690 10188 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.212299 10188 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.214768 10570 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.215349 10570 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.218056  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:12.218263 10570 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:12.220721 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:12.221519 10024 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:12.226748 10188 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:12.226984  9912 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:12.228799 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:12.229537 10097 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 1: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:12.230317 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:12.230909 10097 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 0.
I20260709 12:23:12.231783  9990 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:12.228077 10188 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.233596 10188 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.234035 10188 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.262785 10188 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.243229 10570 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.277721 10188 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:12.281929 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:12.282574 10098 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.285741 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:12.286801 10016 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.292976 10098 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:12.294485 10478 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:12.295044 10478 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.297185 10478 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:12.299144  9914 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:12.300017 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:12.300720 10098 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:12.300626  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:12.301416  9942 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 0.
I20260709 12:23:12.305145  9991 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:12.309378 10500 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.309908 10500 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:12.297957 10570 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:23:12.310732 10500 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.311522 10570 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:23:12.311923 10570 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:12.312348 10570 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.307502  9843 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.297870 10016 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:12.324234 10570 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.326442 10570 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:12.330987 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:12.331791 10098 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.340809 10098 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:12.342217  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:12.344569 10478 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:23:12.345103 10478 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:12.345986 10478 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:12.347476  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:12.348114  9943 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:23:12.355782  9943 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:12.358500  9843 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 1, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.370440  9839 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:12.452875 10016 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:12.454152 10188 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.464455 10571 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.490996 10093 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:12.495163 10571 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:23:12.504997 10460 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.547086 10500 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.558712 10571 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.581986 10571 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.592844 10460 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.697856 10460 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.709596 10416 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.743211 10604 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:12.757789 10604 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:23:13.131183  9894 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:23:13.655128 10571 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:13.655646 10571 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:13.657490 10571 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:13.658263 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:13.658443 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:13.658967 10016 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:13.659183 10098 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:13.660198  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:13.660913 10571 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:13.661504 10571 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:13.661885 10571 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:13.666801 10571 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:13.668401 10571 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:13.669498 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:13.669528 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:13.670282 10016 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:13.670233 10093 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:13.676364 10016 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:13.676486 10093 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:13.677459  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:13.678087 10460 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:13.678469 10460 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 2 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:13.679095 10460 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:13.687847  9840 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a reported cstate change: term changed from 0 to 2, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 2 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:14.065100 10098 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:14.066733 10571 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:14.090548 10021 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:23:14.093374 10188 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:23:14.660749  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:23:14.688969 10632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.690117 10631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:23:14.690518 10634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:23:14.693559  8743 server_base.cc:1061] running on GCE node
I20260709 12:23:14.694911  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:23:14.695344  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:23:14.695585  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783599794695565 us; error 0 us; skew 500 ppm
I20260709 12:23:14.696378  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:23:14.700089  8743 webserver.cc:533] Webserver started at http://127.8.137.196:45481/ using document root <none> and password file <none>
I20260709 12:23:14.700630  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:23:14.700841  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:23:14.701175  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:23:14.702464  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-3-root/instance:
uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-kpwv"
I20260709 12:23:14.707026  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:23:14.710952 10639 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.711942  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.001s
I20260709 12:23:14.712272  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-3-root
uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
format_stamp: "Formatted at 2026-07-09 12:23:14 on dist-test-slave-kpwv"
I20260709 12:23:14.712599  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:23:14.728454  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:23:14.729844  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:23:14.731441  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:23:14.733902  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:23:14.734139  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.734411  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:23:14.734633  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:23:14.785136  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.196:36979
I20260709 12:23:14.785305 10702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.196:36979 every 8 connection(s)
I20260709 12:23:14.790529  8743 auto_leader_rebalancer-test.cc:439] add a tserver: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:14.805629 10703 heartbeater.cc:344] Connected to a master server at 127.8.137.254:38721
I20260709 12:23:14.806141 10703 heartbeater.cc:461] Registering TS with master...
I20260709 12:23:14.806967 10703 heartbeater.cc:507] Master 127.8.137.254:38721 requested a full tablet report, sending...
I20260709 12:23:14.808992  9840 ts_manager.cc:194] Registered new tserver with Master: 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:14.811019  9840 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:42270
I20260709 12:23:15.264904  9942 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:15.274072 10098 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:15.275902 10571 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:15.284178 10016 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:15.286134 10571 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:15.301211  9912 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 062c203e13aa4527a094ec24339001e5. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:15.304725 10096 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.307053 10021 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.309064 10710 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.328986  9840 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.372639 10096 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:15.387800  9944 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:15.389300 10619 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:15.391234 10021 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:15.392743 10619 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:15.400256 10619 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.401420  9944 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:15.418692 10062 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d864b424b78d4fffb51785ea7d3b74ed. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:15.425729  9839 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.455683 10016 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.461587 10021 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:15.473104 10096 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:15.477880 10478 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:15.478749  9944 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:15.480450 10715 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:15.497169  9987 consensus_peers.cc:597] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d4909dfc06b24d70bfd398607779f5ae. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:15.499370 10604 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.502534  9944 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.503391 10098 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.518801 10024 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:15.520606  9843 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:15.531241  9987 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aa5a16c274db44a5aa641b51f71f7ec6. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:15.531507 10098 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:15.532439  9944 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:15.532838 10604 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:15.533766 10715 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:15.545734 10715 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.551594  9944 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.573135  9840 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:15.579910 10098 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:15.736277 10737 ts_tablet_manager.cc:933] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:15.741055 10737 tablet_copy_client.cc:323] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:15.762039  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 062c203e13aa4527a094ec24339001e5 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:15.762705  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 062c203e13aa4527a094ec24339001e5 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-062c203e13aa4527a094ec24339001e5
I20260709 12:23:15.771188  9959 tablet_copy_source_session.cc:215] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:15.776414 10737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 062c203e13aa4527a094ec24339001e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.790262 10737 tablet_copy_client.cc:806] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:15.791056 10737 tablet_copy_client.cc:670] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:15.796344 10737 tablet_copy_client.cc:538] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:15.805615 10737 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:15.814491 10703 heartbeater.cc:499] Master 127.8.137.254:38721 was elected leader, sending a full tablet report...
I20260709 12:23:15.830101 10737 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:15.831275 10737 tablet_bootstrap.cc:492] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:15.831990 10737 ts_tablet_manager.cc:1403] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.027s	user 0.016s	sys 0.010s
I20260709 12:23:15.835119 10737 raft_consensus.cc:359] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:15.836006 10737 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:15.836681 10737 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:15.839455 10737 ts_tablet_manager.cc:1434] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:23:15.841401  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-062c203e13aa4527a094ec24339001e5 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:15.841912  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-062c203e13aa4527a094ec24339001e5 on tablet 062c203e13aa4527a094ec24339001e5 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:15.855707 10737 ts_tablet_manager.cc:933] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:15.857402 10737 tablet_copy_client.cc:323] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:15.869215 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet d864b424b78d4fffb51785ea7d3b74ed from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:15.869848 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet d864b424b78d4fffb51785ea7d3b74ed from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-d864b424b78d4fffb51785ea7d3b74ed
I20260709 12:23:15.878218 10109 tablet_copy_source_session.cc:215] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:15.881544 10737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d864b424b78d4fffb51785ea7d3b74ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:15.897766 10737 tablet_copy_client.cc:806] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:15.898397 10737 tablet_copy_client.cc:670] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:15.916687 10737 tablet_copy_client.cc:538] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:15.924974 10737 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:15.943984 10737 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:15.945003 10737 tablet_bootstrap.cc:492] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:15.945721 10737 ts_tablet_manager.cc:1403] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.021s	user 0.021s	sys 0.000s
I20260709 12:23:15.950260 10737 raft_consensus.cc:359] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:15.951577 10737 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:15.952273 10737 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:15.954497 10737 ts_tablet_manager.cc:1434] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:23:15.956301 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d864b424b78d4fffb51785ea7d3b74ed received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:15.956749 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d864b424b78d4fffb51785ea7d3b74ed on tablet d864b424b78d4fffb51785ea7d3b74ed with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:15.974339 10737 ts_tablet_manager.cc:933] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:15.977270 10737 tablet_copy_client.cc:323] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:15.988945 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet d4909dfc06b24d70bfd398607779f5ae from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:15.989548 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet d4909dfc06b24d70bfd398607779f5ae from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-d4909dfc06b24d70bfd398607779f5ae
I20260709 12:23:15.996711 10035 tablet_copy_source_session.cc:215] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:15.999646 10737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4909dfc06b24d70bfd398607779f5ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.013361 10737 tablet_copy_client.cc:806] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:16.014026 10737 tablet_copy_client.cc:670] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:16.018393 10737 tablet_copy_client.cc:538] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:16.026880 10737 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:16.041144 10737 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:16.043614 10737 tablet_bootstrap.cc:492] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:16.044337 10737 ts_tablet_manager.cc:1403] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 12:23:16.047394 10737 raft_consensus.cc:359] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:16.048188 10737 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:16.048810 10737 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:16.051190 10737 ts_tablet_manager.cc:1434] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:23:16.052961 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d4909dfc06b24d70bfd398607779f5ae received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:16.053385 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d4909dfc06b24d70bfd398607779f5ae on tablet d4909dfc06b24d70bfd398607779f5ae with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:16.104377 10737 ts_tablet_manager.cc:933] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:16.105933 10737 tablet_copy_client.cc:323] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:16.107623 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet aa5a16c274db44a5aa641b51f71f7ec6 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:16.108160 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet aa5a16c274db44a5aa641b51f71f7ec6 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-aa5a16c274db44a5aa641b51f71f7ec6
I20260709 12:23:16.115617 10035 tablet_copy_source_session.cc:215] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:16.127507 10737 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa5a16c274db44a5aa641b51f71f7ec6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:16.139178 10737 tablet_copy_client.cc:806] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:16.139791 10737 tablet_copy_client.cc:670] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:16.143303 10737 tablet_copy_client.cc:538] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:16.150360 10737 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:16.167291 10737 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:16.168200 10737 tablet_bootstrap.cc:492] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:16.168915 10737 ts_tablet_manager.cc:1403] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.019s	user 0.015s	sys 0.005s
I20260709 12:23:16.171712 10737 raft_consensus.cc:359] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:16.172458 10737 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:16.173033 10737 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:16.174567 10737 ts_tablet_manager.cc:1434] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 12:23:16.180212 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-aa5a16c274db44a5aa641b51f71f7ec6 received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:16.180675 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-aa5a16c274db44a5aa641b51f71f7ec6 on tablet aa5a16c274db44a5aa641b51f71f7ec6 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:16.202584 10677 raft_consensus.cc:1217] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:16.387858 10677 raft_consensus.cc:1217] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:16.406097 10677 raft_consensus.cc:1217] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:16.565187 10677 raft_consensus.cc:1217] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:16.801123 10751 raft_consensus.cc:1064] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:16.804029 10751 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:16.817410 10677 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.817723 10024 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.820185 10727 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:16.822849  9942 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.829144 10727 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:16.831228 10727 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:16.836162 10710 raft_consensus.cc:1064] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:16.839187 10710 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:16.860083 10098 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.862710 10676 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.865631 10710 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:23:16.876230 10628 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:16.885187 10751 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:16.889874 10677 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:16.903795 10024 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.907151 10016 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:16.909547 10715 raft_consensus.cc:1064] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:16.915258 10628 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:16.917454 10715 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:16.912129  9943 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:16.939046 10677 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.932078  9840 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:16.941363 10098 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.941661  9944 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:16.942785 10736 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:16.945504 10736 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:16.954551 10752 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:16.970494 10753 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:16.972012 10024 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:16.984598 10715 raft_consensus.cc:1064] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:16.991962 10715 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:17.013055 10096 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:17.044493  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 91019.3us
I20260709 12:23:17.067603 10752 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.075693 10677 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:17.076479  9943 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:17.079201 10771 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:17.080644 10736 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:17.096092 10093 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:17.091665 10677 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.109359 10715 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:17.130359  9943 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.133749 10093 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:17.135102  9839 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:17.151975  9842 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.152415 10736 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.157799 10677 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.176719 10675 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:17.179363 10759 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:17.182026  9943 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.184770  9942 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:17.187024 10751 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:17.192438  9841 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.197649 10727 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.203723  9942 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.208222  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d864b424b78d4fffb51785ea7d3b74ed with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:17.211524 10677 raft_consensus.cc:2955] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.224471 10016 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:17.235885  9839 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:17.237663 10093 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.258127 10676 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.268303 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet d864b424b78d4fffb51785ea7d3b74ed with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:17.289602 10675 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:17.295543  9944 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:17.303537 10024 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:17.305518 10752 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:17.318430 10780 tablet_replica.cc:333] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:17.318655  9943 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:17.319528 10780 raft_consensus.cc:2243] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.325317 10780 raft_consensus.cc:2272] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.325688 10673 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:17.328297 10768 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:17.329591 10022 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:17.335208 10093 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:17.335515 10098 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.336637 10752 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:17.338502 10771 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.343807 10736 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.345333 10674 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.348878 10673 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.350574 10780 ts_tablet_manager.cc:1916] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:17.355710  9943 raft_consensus.cc:2955] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.358431 10675 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:17.385808  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet aa5a16c274db44a5aa641b51f71f7ec6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:17.387885  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d4909dfc06b24d70bfd398607779f5ae with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:17.389310 10761 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:17.396807 10761 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:17.405587  9840 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.410156 10736 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:17.426801  9843 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.440244  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet d4909dfc06b24d70bfd398607779f5ae with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:17.439618 10098 raft_consensus.cc:2955] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.442555 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet aa5a16c274db44a5aa641b51f71f7ec6 with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:17.448613 10628 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.458467 10674 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.473062  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 062c203e13aa4527a094ec24339001e5 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:17.475824 10024 raft_consensus.cc:2955] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.479722 10786 tablet_replica.cc:333] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:17.481976 10787 tablet_replica.cc:333] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:17.483188 10787 raft_consensus.cc:2243] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.482781  9840 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:17.484052 10787 raft_consensus.cc:2272] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.485356 10786 raft_consensus.cc:2243] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.486174 10786 raft_consensus.cc:2272] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.491545 10786 ts_tablet_manager.cc:1916] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:17.495407 10787 ts_tablet_manager.cc:1916] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:17.504670 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 062c203e13aa4527a094ec24339001e5 with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:17.517822 10787 ts_tablet_manager.cc:1929] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:17.518489 10787 log.cc:1199] T aa5a16c274db44a5aa641b51f71f7ec6 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/aa5a16c274db44a5aa641b51f71f7ec6
I20260709 12:23:17.519158 10786 ts_tablet_manager.cc:1929] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:17.519639 10786 log.cc:1199] T d4909dfc06b24d70bfd398607779f5ae P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/d4909dfc06b24d70bfd398607779f5ae
I20260709 12:23:17.520260  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet aa5a16c274db44a5aa641b51f71f7ec6 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:17.521486  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet d4909dfc06b24d70bfd398607779f5ae (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:17.522818 10787 tablet_replica.cc:333] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:17.524480 10787 raft_consensus.cc:2243] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:17.525169 10787 raft_consensus.cc:2272] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:17.531955 10787 ts_tablet_manager.cc:1916] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:17.533360 10780 ts_tablet_manager.cc:1929] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:17.533792 10780 log.cc:1199] T d864b424b78d4fffb51785ea7d3b74ed P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/d864b424b78d4fffb51785ea7d3b74ed
I20260709 12:23:17.535564  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet d864b424b78d4fffb51785ea7d3b74ed (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:17.544872 10787 ts_tablet_manager.cc:1929] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:17.545346 10787 log.cc:1199] T 062c203e13aa4527a094ec24339001e5 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/062c203e13aa4527a094ec24339001e5
I20260709 12:23:17.546716  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 062c203e13aa4527a094ec24339001e5 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:18.578047  9943 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:18.584810  9912 consensus_peers.cc:597] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 32a9a1d2d5fc4243bf902da3519cd483. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.585583 10097 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:18.586541 10022 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:18.587143 10628 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:18.588182 10788 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:18.596207 10628 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.598106 10098 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.599368 10022 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.611297 10097 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:18.613186  9840 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:18.636551 10062 consensus_peers.cc:597] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 15458ba5804c4c36b01b169af77b2691. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.637763  9943 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:18.638536 10016 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:18.639566 10759 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:18.640415 10751 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:18.664889 10759 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.671805  9943 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.680218 10016 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.688994  9942 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:18.698019  9839 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.718453 10093 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:18.719727 10628 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:18.720693 10022 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:18.719738  9912 consensus_peers.cc:597] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 773b4422370c4cc6b9e7477bdaf29261. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.724996 10761 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:18.738847 10628 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.743266 10022 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.760703 10093 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.771624  9942 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:18.772912  9839 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:18.781225 10016 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:18.784145  9912 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9f126305b66248b59cd9ce87bc952694. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:18.786412 10785 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:18.787441 10093 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:18.790088 10785 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:18.798359 10628 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.801827 10016 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.811157 10093 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:18.822304  9839 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:19.037652 10808 ts_tablet_manager.cc:933] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:19.041446 10808 tablet_copy_client.cc:323] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:19.043394  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 32a9a1d2d5fc4243bf902da3519cd483 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:19.043946  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 32a9a1d2d5fc4243bf902da3519cd483 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-32a9a1d2d5fc4243bf902da3519cd483
I20260709 12:23:19.052220  9959 tablet_copy_source_session.cc:215] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:19.057291 10808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 32a9a1d2d5fc4243bf902da3519cd483. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:19.076016 10810 ts_tablet_manager.cc:933] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:19.075984 10808 tablet_copy_client.cc:806] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:19.076948 10808 tablet_copy_client.cc:670] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:19.081370 10808 tablet_copy_client.cc:538] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:19.083853 10810 tablet_copy_client.cc:323] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:19.085755 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet 15458ba5804c4c36b01b169af77b2691 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:19.086282 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet 15458ba5804c4c36b01b169af77b2691 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-15458ba5804c4c36b01b169af77b2691
I20260709 12:23:19.090366 10808 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:19.097957 10109 tablet_copy_source_session.cc:215] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:19.108003 10810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15458ba5804c4c36b01b169af77b2691. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:19.114251 10808 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:19.115302 10808 tablet_bootstrap.cc:492] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:19.116034 10808 ts_tablet_manager.cc:1403] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.018s
I20260709 12:23:19.119115 10808 raft_consensus.cc:359] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:19.120021 10808 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:19.120723 10808 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:19.131201 10808 ts_tablet_manager.cc:1434] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.002s
I20260709 12:23:19.133297  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-32a9a1d2d5fc4243bf902da3519cd483 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:19.133740  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-32a9a1d2d5fc4243bf902da3519cd483 on tablet 32a9a1d2d5fc4243bf902da3519cd483 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:19.142171 10810 tablet_copy_client.cc:806] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:19.142856 10810 tablet_copy_client.cc:670] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:19.146970 10810 tablet_copy_client.cc:538] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:19.156177 10810 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:19.177111 10810 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:19.178023 10810 tablet_bootstrap.cc:492] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:19.178730 10810 ts_tablet_manager.cc:1403] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.023s	user 0.023s	sys 0.000s
I20260709 12:23:19.181355 10810 raft_consensus.cc:359] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:19.182042 10810 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:19.182641 10810 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:19.185163 10810 ts_tablet_manager.cc:1434] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:23:19.186914 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-15458ba5804c4c36b01b169af77b2691 received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:19.187356 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-15458ba5804c4c36b01b169af77b2691 on tablet 15458ba5804c4c36b01b169af77b2691 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:19.310278 10810 ts_tablet_manager.cc:933] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:19.312822 10810 tablet_copy_client.cc:323] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:19.314786  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 9f126305b66248b59cd9ce87bc952694 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:19.315408  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 9f126305b66248b59cd9ce87bc952694 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-9f126305b66248b59cd9ce87bc952694
I20260709 12:23:19.320071 10808 ts_tablet_manager.cc:933] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:19.325835 10808 tablet_copy_client.cc:323] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:19.326550  9959 tablet_copy_source_session.cc:215] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:19.327706  9958 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 773b4422370c4cc6b9e7477bdaf29261 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:19.328346  9958 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 773b4422370c4cc6b9e7477bdaf29261 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-773b4422370c4cc6b9e7477bdaf29261
I20260709 12:23:19.335521 10810 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f126305b66248b59cd9ce87bc952694. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:19.336488  9958 tablet_copy_source_session.cc:215] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:19.340210 10808 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 773b4422370c4cc6b9e7477bdaf29261. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:19.358106 10808 tablet_copy_client.cc:806] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:19.358152 10810 tablet_copy_client.cc:806] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:19.358964 10808 tablet_copy_client.cc:670] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:19.359251 10810 tablet_copy_client.cc:670] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:19.364802 10810 tablet_copy_client.cc:538] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:19.364948 10808 tablet_copy_client.cc:538] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:19.373982 10810 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:19.375857 10808 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:19.403334 10810 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:19.404592 10810 tablet_bootstrap.cc:492] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:19.405325 10810 ts_tablet_manager.cc:1403] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.032s	user 0.019s	sys 0.009s
I20260709 12:23:19.408423 10810 raft_consensus.cc:359] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:19.410228 10810 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:19.410730 10808 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:19.411211 10810 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:19.411866 10808 tablet_bootstrap.cc:492] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:19.412894 10808 ts_tablet_manager.cc:1403] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.011s
I20260709 12:23:19.419790 10810 ts_tablet_manager.cc:1434] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:23:19.420703 10808 raft_consensus.cc:359] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:19.421662 10808 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:19.422364 10808 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:19.425298 10808 ts_tablet_manager.cc:1434] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:23:19.428321  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-9f126305b66248b59cd9ce87bc952694 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:19.428741  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-9f126305b66248b59cd9ce87bc952694 on tablet 9f126305b66248b59cd9ce87bc952694 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:19.431418  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-773b4422370c4cc6b9e7477bdaf29261 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:19.431850  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-773b4422370c4cc6b9e7477bdaf29261 on tablet 773b4422370c4cc6b9e7477bdaf29261 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:19.466321 10674 raft_consensus.cc:1217] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:19.597970 10674 raft_consensus.cc:1217] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:19.710714 10674 raft_consensus.cc:1217] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:19.943310 10674 raft_consensus.cc:1217] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:19.972954 10761 raft_consensus.cc:1064] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:19.975260 10761 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:19.981061 10674 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:19.983109 10016 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:19.983429 10098 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:19.983482 10788 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:19.984876 10785 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:19.988591 10785 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:20.004782 10788 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.006680 10674 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.010341 10098 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.020073 10024 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.033362  9843 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.072299  9943 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:20.079351 10759 raft_consensus.cc:1064] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:20.081995 10759 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:20.091836 10674 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:20.093514 10785 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:20.095346 10674 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.096997 10795 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.101505 10785 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.105409 10096 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:20.108958 10674 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.121184  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 32a9a1d2d5fc4243bf902da3519cd483 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:20.121559 10016 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.121559  9943 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.123981 10795 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.125010 10751 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.127337 10761 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:20.169835 10751 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.186659  9841 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.195266  9942 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.216876 10098 raft_consensus.cc:2955] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.216032  9839 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.218886 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet 32a9a1d2d5fc4243bf902da3519cd483 with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:20.226760 10834 tablet_replica.cc:333] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:20.229116 10834 raft_consensus.cc:2243] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:23:20.228394 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52581.1us
I20260709 12:23:20.230264 10834 raft_consensus.cc:2272] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.232314 10674 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.241312 10834 ts_tablet_manager.cc:1916] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:20.271832 10016 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.278964 10834 ts_tablet_manager.cc:1929] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:20.279572 10834 log.cc:1199] T 32a9a1d2d5fc4243bf902da3519cd483 P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/32a9a1d2d5fc4243bf902da3519cd483
I20260709 12:23:20.281220  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet 32a9a1d2d5fc4243bf902da3519cd483 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:20.284548 10096 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:20.292429 10674 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:20.293988 10832 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:20.296119 10761 raft_consensus.cc:1064] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:20.301249 10016 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:20.301970 10761 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:20.303102 10832 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:20.304420 10831 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.308738 10677 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.319521 10024 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.321382 10096 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.323092 10785 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.323907 10788 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.325853  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 15458ba5804c4c36b01b169af77b2691 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:20.330492 10024 raft_consensus.cc:2955] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.340014 10674 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.341774 10785 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.346074  9839 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:20.363248  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet 15458ba5804c4c36b01b169af77b2691 with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:20.365639 10788 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.366912 10016 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.377895 10674 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.393316 10761 raft_consensus.cc:1064] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:20.394167 10840 tablet_replica.cc:333] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:20.394697  9841 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.395208 10840 raft_consensus.cc:2243] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.397377 10840 raft_consensus.cc:2272] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.397429 10098 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.397830 10761 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:20.406466 10840 ts_tablet_manager.cc:1916] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:20.426069  9942 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:20.429003 10674 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.431629 10628 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.437256 10098 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.439661 10628 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.441885 10016 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:20.451068 10840 ts_tablet_manager.cc:1929] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:20.451622 10016 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:20.451622 10677 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:20.451635 10840 log.cc:1199] T 15458ba5804c4c36b01b169af77b2691 P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/15458ba5804c4c36b01b169af77b2691
I20260709 12:23:20.455729 10785 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:20.463400  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet 15458ba5804c4c36b01b169af77b2691 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:20.463326 10785 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:20.465538 10785 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:20.468083 10844 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.483446 10628 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.485971 10677 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.493489 10021 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.497658 10674 raft_consensus.cc:2955] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.503736  9841 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.525259 10098 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.530112  9841 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.553915 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 9f126305b66248b59cd9ce87bc952694 with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
W20260709 12:23:20.563985  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52705.1us
I20260709 12:23:20.565294  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 9f126305b66248b59cd9ce87bc952694 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:20.584463 10850 tablet_replica.cc:333] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:20.585599 10850 raft_consensus.cc:2243] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.590551 10850 raft_consensus.cc:2272] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.596117 10850 ts_tablet_manager.cc:1916] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:20.597055  9942 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:20.605078 10098 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:20.607551 10016 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.611956 10788 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:20.618165 10850 ts_tablet_manager.cc:1929] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:20.619520 10850 log.cc:1199] T 9f126305b66248b59cd9ce87bc952694 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/9f126305b66248b59cd9ce87bc952694
I20260709 12:23:20.619257 10847 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.622345  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 9f126305b66248b59cd9ce87bc952694 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:20.625496 10674 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:20.626959 10788 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:20.626736 10098 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.630985 10674 raft_consensus.cc:2955] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.643488  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 773b4422370c4cc6b9e7477bdaf29261 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:20.651510  9841 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:20.678507 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet 773b4422370c4cc6b9e7477bdaf29261 with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:20.679593 10834 tablet_replica.cc:333] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:20.680708 10834 raft_consensus.cc:2243] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:20.681579 10834 raft_consensus.cc:2272] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:20.686360 10834 ts_tablet_manager.cc:1916] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:20.700475 10834 ts_tablet_manager.cc:1929] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:20.700874 10834 log.cc:1199] T 773b4422370c4cc6b9e7477bdaf29261 P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/773b4422370c4cc6b9e7477bdaf29261
I20260709 12:23:20.702260  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet 773b4422370c4cc6b9e7477bdaf29261 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:21.777698 10096 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:21.785611 10016 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:21.790333 10795 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:21.793304  9942 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:21.794070 10062 consensus_peers.cc:597] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d8453bea911c4756add721edebd5d124. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:21.795388 10854 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:21.804298 10795 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.805985 10024 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.806740  9942 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.822980 10016 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:21.835366  9987 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4cd31c96c4854a87a361975b579bc1c2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:21.836910 10098 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:21.841596 10807 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:21.841627  9841 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.853349  9942 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:21.855461 10807 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:21.865609 10806 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.867362 10096 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.886421  9942 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.892210 10096 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:21.901698 10016 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:21.902278  9944 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:21.904335 10854 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:21.908036 10832 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:23:21.931739  9840 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.933036  9942 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:21.936905 10062 consensus_peers.cc:597] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ab118912551248c1985fce40a2e815b0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:21.929296 10854 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.965893  9942 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:21.971457  9839 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.978914 10021 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:21.984820 10024 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:21.987704 10852 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:21.989601 10096 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:21.991706 10788 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:21.991699  9912 consensus_peers.cc:597] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fc2205f3947147b6ad9c3ac2b9bd5058. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:22.019325 10852 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:22.024852 10098 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:22.035491 10016 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Committing config change with OpId 3.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:22.062687  9840 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:22.314399 10873 ts_tablet_manager.cc:933] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:22.316380 10873 tablet_copy_client.cc:323] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:22.318274 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet 4cd31c96c4854a87a361975b579bc1c2 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:22.318835 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet 4cd31c96c4854a87a361975b579bc1c2 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-4cd31c96c4854a87a361975b579bc1c2
I20260709 12:23:22.324795 10035 tablet_copy_source_session.cc:215] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:22.328433 10873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4cd31c96c4854a87a361975b579bc1c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:22.348541 10873 tablet_copy_client.cc:806] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:22.348683 10875 ts_tablet_manager.cc:933] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:22.349255 10873 tablet_copy_client.cc:670] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:22.350975 10875 tablet_copy_client.cc:323] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:22.353070 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet ab118912551248c1985fce40a2e815b0 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:22.353648 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet ab118912551248c1985fce40a2e815b0 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-ab118912551248c1985fce40a2e815b0
I20260709 12:23:22.353947 10873 tablet_copy_client.cc:538] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:22.366262 10109 tablet_copy_source_session.cc:215] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:22.369650 10873 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:22.417534 10875 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab118912551248c1985fce40a2e815b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:22.445701 10875 tablet_copy_client.cc:806] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:22.445950 10873 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:22.446444 10875 tablet_copy_client.cc:670] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:22.446934 10873 tablet_bootstrap.cc:492] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:22.447625 10873 ts_tablet_manager.cc:1403] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.078s	user 0.030s	sys 0.041s
I20260709 12:23:22.451253 10873 raft_consensus.cc:359] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:22.452242 10873 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:22.453039 10873 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:22.457340 10873 ts_tablet_manager.cc:1434] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.005s
I20260709 12:23:22.472528 10875 tablet_copy_client.cc:538] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:22.497942 10879 ts_tablet_manager.cc:933] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:22.504988 10879 tablet_copy_client.cc:323] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:22.518386 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet d8453bea911c4756add721edebd5d124 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:22.519019 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet d8453bea911c4756add721edebd5d124 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-d8453bea911c4756add721edebd5d124
I20260709 12:23:22.535245 10875 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:22.581717 10109 tablet_copy_source_session.cc:215] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:23:22.743288  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81769.7us
I20260709 12:23:22.759935 10882 ts_tablet_manager.cc:933] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
W20260709 12:23:22.806504  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62245.8us
I20260709 12:23:22.808516 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-4cd31c96c4854a87a361975b579bc1c2 received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:22.809048 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-4cd31c96c4854a87a361975b579bc1c2 on tablet 4cd31c96c4854a87a361975b579bc1c2 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:22.818802 10882 tablet_copy_client.cc:323] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:22.931150 10875 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:22.972931 10677 raft_consensus.cc:1217] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:23.000669 10875 tablet_bootstrap.cc:492] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:23.004248 10875 ts_tablet_manager.cc:1403] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.497s	user 0.053s	sys 0.045s
I20260709 12:23:23.036921 10875 raft_consensus.cc:359] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:23.093478  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 156747us
I20260709 12:23:23.125280 10879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d8453bea911c4756add721edebd5d124. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:23:23.176385  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 238143us
W20260709 12:23:23.152112 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 95238.3us
I20260709 12:23:23.183070 10875 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:23.185514 10875 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:23.200361 10875 ts_tablet_manager.cc:1434] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.167s	user 0.002s	sys 0.006s
I20260709 12:23:23.222810 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-ab118912551248c1985fce40a2e815b0 received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:23.223322 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-ab118912551248c1985fce40a2e815b0 on tablet ab118912551248c1985fce40a2e815b0 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:23.235695 10879 tablet_copy_client.cc:806] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:23.236392 10879 tablet_copy_client.cc:670] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:23.240636 10879 tablet_copy_client.cc:538] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:23.244395  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet fc2205f3947147b6ad9c3ac2b9bd5058 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:23.244966  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet fc2205f3947147b6ad9c3ac2b9bd5058 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-fc2205f3947147b6ad9c3ac2b9bd5058
I20260709 12:23:23.253201 10879 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:23.253510  9959 tablet_copy_source_session.cc:215] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:23.260274 10882 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc2205f3947147b6ad9c3ac2b9bd5058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:23.341890 10882 tablet_copy_client.cc:806] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:23.347836 10882 tablet_copy_client.cc:670] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:23.351801 10879 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:23.351878 10882 tablet_copy_client.cc:538] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:23.353017 10879 tablet_bootstrap.cc:492] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:23.353794 10879 ts_tablet_manager.cc:1403] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.101s	user 0.036s	sys 0.051s
I20260709 12:23:23.357157 10879 raft_consensus.cc:359] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:23.358045 10879 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:23.358765 10879 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:23.364813 10879 ts_tablet_manager.cc:1434] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:23:23.366042 10896 raft_consensus.cc:1064] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:23.366536 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d8453bea911c4756add721edebd5d124 received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:23.367036 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d8453bea911c4756add721edebd5d124 on tablet d8453bea911c4756add721edebd5d124 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:23.368800 10896 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:23.370579 10882 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:23.401348 10677 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:23.404006 10898 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:23.428231 10674 raft_consensus.cc:1217] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:23.451835 10095 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:23.453553  9944 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:23.455310 10900 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:23.456650 10898 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:23.469426 10882 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:23.470484 10882 tablet_bootstrap.cc:492] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:23.471210 10882 ts_tablet_manager.cc:1403] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.101s	user 0.019s	sys 0.025s
I20260709 12:23:23.474931 10882 raft_consensus.cc:359] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:23.475817 10882 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:23.476501 10882 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:23.479043 10882 ts_tablet_manager.cc:1434] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 12:23:23.486178  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-fc2205f3947147b6ad9c3ac2b9bd5058 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:23.486819  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-fc2205f3947147b6ad9c3ac2b9bd5058 on tablet fc2205f3947147b6ad9c3ac2b9bd5058 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:23.493834 10899 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.497138 10095 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.509438 10677 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.521261  9841 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.550998  9944 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:23.685313 10913 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:23:23.720780 10677 raft_consensus.cc:1217] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.2->[3.3-3.3]   Dedup: 3.3->[]
W20260709 12:23:23.752703  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08000646a0 after lost signal to thread 8746
W20260709 12:23:23.758410  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800064d80 after lost signal to thread 9879
W20260709 12:23:23.759588  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800288000 after lost signal to thread 9973
W20260709 12:23:23.760614  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800240f20 after lost signal to thread 10049
W20260709 12:23:23.761642  9876 debug-util.cc:398] Leaking SignalData structure 0x7b080030b300 after lost signal to thread 10123
W20260709 12:23:23.768569  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08002fe7a0 after lost signal to thread 10702
I20260709 12:23:23.814551 10677 raft_consensus.cc:1217] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:23.852787 10021 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:23.865813 10677 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:23.867816  9942 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:23.867700 10888 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:23.877218 10888 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:23.878809 10897 raft_consensus.cc:1064] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:23.881188 10897 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:23.883584 10886 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.896773 10677 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.906543  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4cd31c96c4854a87a361975b579bc1c2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:23.916507 10674 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:23.918165 10901 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:23.926606 10021 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:23.928108  9942 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:23.928206 10897 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:23.934427 10919 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:23.917812  9839 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:23.944238  9942 raft_consensus.cc:2955] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.960072 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 4cd31c96c4854a87a361975b579bc1c2 with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:23.976806 10919 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.978073  9943 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.976274 10025 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:23.983745 10926 tablet_replica.cc:333] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:23.990562 10926 raft_consensus.cc:2243] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:23.994658 10926 raft_consensus.cc:2272] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:24.008754 10677 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.020222  9841 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.028716 10926 ts_tablet_manager.cc:1916] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:24.054445 10096 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:24.067839 10677 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:24.068329 10025 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:24.069705 10897 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:24.071513 10919 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:24.083976 10919 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.090176 10677 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.098066 10025 raft_consensus.cc:2955] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.104302  9840 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.108320 10926 ts_tablet_manager.cc:1929] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:24.108822 10926 log.cc:1199] T 4cd31c96c4854a87a361975b579bc1c2 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/4cd31c96c4854a87a361975b579bc1c2
I20260709 12:23:24.110571  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 4cd31c96c4854a87a361975b579bc1c2 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:24.111541  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ab118912551248c1985fce40a2e815b0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:24.128049  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet ab118912551248c1985fce40a2e815b0 with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:24.132854 10927 tablet_replica.cc:333] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:24.134146 10927 raft_consensus.cc:2243] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:24.134928 10927 raft_consensus.cc:2272] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:24.138779 10927 ts_tablet_manager.cc:1916] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:24.155646 10927 ts_tablet_manager.cc:1929] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:24.156069 10927 log.cc:1199] T ab118912551248c1985fce40a2e815b0 P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/ab118912551248c1985fce40a2e815b0
I20260709 12:23:24.157918  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet ab118912551248c1985fce40a2e815b0 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:24.237182 10919 raft_consensus.cc:1064] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:24.240175 10919 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:24.256088 10889 raft_consensus.cc:1064] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:24.258952 10889 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 1, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:24.280980 10674 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:24.283459 10914 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:24.285871 10016 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:24.287801  9943 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:24.288395 10677 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:24.289490 10901 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:24.290432 10897 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:24.295205 10914 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:24.298024 10097 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:24.299608 10016 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:24.300724 10891 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:24.305320 10901 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:24.447237 10919 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.448943  9944 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.459688 10914 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.468272 10095 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.491925  9841 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.499166 10025 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.532516 10674 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:24.560729  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64105.1us
I20260709 12:23:24.572269  9841 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:24.573732 10674 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.577359 10017 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.614514 10096 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:24.627622  9944 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:24.628012 10674 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:24.631413 10897 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:24.637956 10673 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:24.638504 10096 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:24.639941 10017 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:24.639430 10917 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.640285 10914 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:24.641552 10795 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:24.642160 10914 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:24.645964 10021 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.657334 10936 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.660806  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d8453bea911c4756add721edebd5d124 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:24.667783 10673 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.671444  9841 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:24.678133  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fc2205f3947147b6ad9c3ac2b9bd5058 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:24.651726 10674 raft_consensus.cc:2955] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.685897  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet d8453bea911c4756add721edebd5d124 with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:24.687793 10096 raft_consensus.cc:2955] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:24.689226  9843 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 4 to 5, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:24.717870 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet fc2205f3947147b6ad9c3ac2b9bd5058 with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:24.737876 10938 tablet_replica.cc:333] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:24.743309 10938 raft_consensus.cc:2243] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:24.744326 10938 raft_consensus.cc:2272] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:24.739055 10937 tablet_replica.cc:333] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:24.749408 10937 raft_consensus.cc:2243] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:24.750285 10937 raft_consensus.cc:2272] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:24.752851 10938 ts_tablet_manager.cc:1916] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:24.755009 10937 ts_tablet_manager.cc:1916] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:24.776224 10937 ts_tablet_manager.cc:1929] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:24.777657 10937 log.cc:1199] T d8453bea911c4756add721edebd5d124 P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/d8453bea911c4756add721edebd5d124
I20260709 12:23:24.779420  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet d8453bea911c4756add721edebd5d124 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:24.787278 10938 ts_tablet_manager.cc:1929] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.4
I20260709 12:23:24.787748 10938 log.cc:1199] T fc2205f3947147b6ad9c3ac2b9bd5058 P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/fc2205f3947147b6ad9c3ac2b9bd5058
I20260709 12:23:24.790050  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet fc2205f3947147b6ad9c3ac2b9bd5058 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:25.824451 10095 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:25.831928  9942 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:25.832043 10062 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1892c18d4e1644109d95b077e989b10f. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:25.832942 10023 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:25.833710 10897 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:25.834579 10917 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:25.844934 10897 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:25.847860 10023 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:25.848526  9942 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:25.866236  9943 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:25.866796  9841 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:25.900818  9912 consensus_peers.cc:597] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0df211fae593420e88f685abc8d618ec. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:25.904635 10095 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:25.905175 10023 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:25.906128 10936 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:25.907305 10914 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:25.928122 10939 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:25.931294 10023 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:25.931596 10095 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:25.955039 10017 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:25.976348  9840 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:25.989437  9942 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:25.991668  9987 consensus_peers.cc:597] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 53edc906d1af41e0992f01434995f9d1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:25.994925 10888 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:26.028678 10096 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:26.038764 10886 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:26.038411 10954 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:26.045013  9943 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:26.062287 10098 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.071547  9841 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:26.073956 10096 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:26.120190 10062 consensus_peers.cc:597] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 94aa8e92ea8d4f81a4ed4dbb1e99635d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:26.120981 10017 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:26.123124  9943 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:26.124433 10897 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:26.125334 10795 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:26.136382 10917 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:26.137995  9943 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:26.138983 10017 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:26.160825  9841 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:26.306437 10961 ts_tablet_manager.cc:933] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:26.308911 10961 tablet_copy_client.cc:323] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:26.310734  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 0df211fae593420e88f685abc8d618ec from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:26.311304  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 0df211fae593420e88f685abc8d618ec from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-0df211fae593420e88f685abc8d618ec
I20260709 12:23:26.322026  9959 tablet_copy_source_session.cc:215] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:26.327195 10961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0df211fae593420e88f685abc8d618ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:26.353077 10961 tablet_copy_client.cc:806] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:26.353714 10961 tablet_copy_client.cc:670] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:26.371870 10961 tablet_copy_client.cc:538] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:26.382586 10961 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:26.419135 10961 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:26.420110 10961 tablet_bootstrap.cc:492] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:26.420809 10961 ts_tablet_manager.cc:1403] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.041s	user 0.021s	sys 0.008s
I20260709 12:23:26.425472 10961 raft_consensus.cc:359] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.426299 10961 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:26.428979 10963 ts_tablet_manager.cc:933] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:26.429400 10961 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.438649 10963 tablet_copy_client.cc:323] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:26.440534 10961 ts_tablet_manager.cc:1434] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.008s
I20260709 12:23:26.440899 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet 1892c18d4e1644109d95b077e989b10f from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:26.441491 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet 1892c18d4e1644109d95b077e989b10f from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-1892c18d4e1644109d95b077e989b10f
I20260709 12:23:26.447438  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-0df211fae593420e88f685abc8d618ec received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:26.447970  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-0df211fae593420e88f685abc8d618ec on tablet 0df211fae593420e88f685abc8d618ec with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:26.462152 10109 tablet_copy_source_session.cc:215] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:26.466006 10963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1892c18d4e1644109d95b077e989b10f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:26.619741 10961 ts_tablet_manager.cc:933] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:26.633404 10961 tablet_copy_client.cc:323] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:26.647531 10979 ts_tablet_manager.cc:933] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:26.654232 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:26.654964 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-94aa8e92ea8d4f81a4ed4dbb1e99635d
I20260709 12:23:26.665282 10109 tablet_copy_source_session.cc:215] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:26.692734 10979 tablet_copy_client.cc:323] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:26.694530 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet 53edc906d1af41e0992f01434995f9d1 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:26.695099 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet 53edc906d1af41e0992f01434995f9d1 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-53edc906d1af41e0992f01434995f9d1
I20260709 12:23:26.705265 10961 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:26.705332 10035 tablet_copy_source_session.cc:215] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:26.725731 10979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53edc906d1af41e0992f01434995f9d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:26.735785 10963 tablet_copy_client.cc:806] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:26.736450 10963 tablet_copy_client.cc:670] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:26.740944 10963 tablet_copy_client.cc:538] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:26.756189 10963 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:26.758397 10979 tablet_copy_client.cc:806] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:26.759052 10979 tablet_copy_client.cc:670] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:26.762265 10961 tablet_copy_client.cc:806] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:26.762876 10961 tablet_copy_client.cc:670] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:26.763494 10979 tablet_copy_client.cc:538] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:26.767156 10961 tablet_copy_client.cc:538] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:26.784863 10961 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:26.787845 10979 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:26.811487 10673 raft_consensus.cc:1217] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:26.822391 10963 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:26.824637 10963 tablet_bootstrap.cc:492] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:26.825570 10963 ts_tablet_manager.cc:1403] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.070s	user 0.016s	sys 0.011s
I20260709 12:23:26.838013 10961 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:26.839181 10961 tablet_bootstrap.cc:492] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:26.840076 10961 ts_tablet_manager.cc:1403] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.055s	user 0.022s	sys 0.003s
I20260709 12:23:26.844174 10963 raft_consensus.cc:359] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.845099 10963 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:26.845800 10963 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.849203 10963 ts_tablet_manager.cc:1434] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.004s
I20260709 12:23:26.851332 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-1892c18d4e1644109d95b077e989b10f received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:26.851832 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-1892c18d4e1644109d95b077e989b10f on tablet 1892c18d4e1644109d95b077e989b10f with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:26.856874 10979 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:26.859339 10979 tablet_bootstrap.cc:492] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:26.860250 10979 ts_tablet_manager.cc:1403] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.073s	user 0.022s	sys 0.009s
I20260709 12:23:26.863581 10979 raft_consensus.cc:359] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.857615 10961 raft_consensus.cc:359] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.864775 10961 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:26.865547 10961 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.867880 10961 ts_tablet_manager.cc:1434] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 12:23:26.870137 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-94aa8e92ea8d4f81a4ed4dbb1e99635d received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:26.870558 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-94aa8e92ea8d4f81a4ed4dbb1e99635d on tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:26.873284 10979 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:26.874187 10979 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:26.877498 10979 ts_tablet_manager.cc:1434] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:23:26.879686 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-53edc906d1af41e0992f01434995f9d1 received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:26.880115 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-53edc906d1af41e0992f01434995f9d1 on tablet 53edc906d1af41e0992f01434995f9d1 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:26.903774 10673 raft_consensus.cc:1217] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:27.040437 10673 raft_consensus.cc:1217] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:23:27.114121 10673 raft_consensus.cc:1217] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:27.293057 10978 raft_consensus.cc:1064] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:27.296056 10978 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:27.320407 10673 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.321193 10096 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.322805 10984 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.325534 10983 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.326337 10021 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.333446 10984 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:27.343801 10984 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.348364 10021 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.345665 10674 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.372854  9841 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.407409 10096 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.426924 10970 raft_consensus.cc:1064] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:27.429503 10970 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:27.457932 10674 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.457932 10021 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.459985 10897 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.461637 10897 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.476953 10970 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.482945 10674 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.489230 10021 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.497268  9942 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.504124 10998 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.564703  9942 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.710039 10970 raft_consensus.cc:1064] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:27.712551 10970 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:27.737277  9843 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.755036  9944 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:27.765372 11003 raft_consensus.cc:1064] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:27.777503 11003 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:27.780874 10674 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.782501 10897 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.798389  9942 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.800194 10967 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.810364 10676 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:27.810628 10021 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:27.812716 11003 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:27.813843 10967 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:23:27.825730  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55407.6us
I20260709 12:23:27.836681  9943 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:27.844730 10956 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:27.961885 10970 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.968436  9944 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:27.988968  9843 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:27.989835 10674 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.005759 10021 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:28.012706  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 239696us
W20260709 12:23:28.019336  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 94567.4us
I20260709 12:23:28.055073 10676 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:28.064199 10021 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:28.066828 10888 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.074191 10095 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:28.080808 10023 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:28.082262 10970 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:28.096095 10676 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.102249  9943 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.107203 10967 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.115590  9839 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:28.144361 10095 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:23:28.226172 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66163.6us
I20260709 12:23:28.262691 10673 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:28.267279 10017 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.293036 10998 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:28.314109 10674 raft_consensus.cc:2955] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.319621  9839 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.327950 10099 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:28.341120  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:28.354928 10983 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:28.356664 10674 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:23:28.357707  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 119813us
I20260709 12:23:28.379038 10956 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:28.388197  9943 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:23:28.393818  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 98058us
I20260709 12:23:28.437923 10984 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:28.389683 10897 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
W20260709 12:23:28.463027  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 82608.2us
I20260709 12:23:28.476450 10970 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:28.516249 10998 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:28.527550  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 132280us
W20260709 12:23:28.532183  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 137190us
I20260709 12:23:28.539233 10674 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.543903  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1892c18d4e1644109d95b077e989b10f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:28.546761  9947 raft_consensus.cc:2955] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:28.564314  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08001e7a80 after lost signal to thread 8746
W20260709 12:23:28.577172  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08000cde80 after lost signal to thread 9879
W20260709 12:23:28.578347  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080030c860 after lost signal to thread 9973
W20260709 12:23:28.584875  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080033bbc0 after lost signal to thread 10049
W20260709 12:23:28.586045  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08002ce5c0 after lost signal to thread 10123
I20260709 12:23:28.588843  9840 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:28.553010 10099 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:28.591331  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800315620 after lost signal to thread 10702
W20260709 12:23:28.600406  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08000b5740 after lost signal to thread 10952
I20260709 12:23:28.615437 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet 1892c18d4e1644109d95b077e989b10f with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:28.672312 11032 tablet_replica.cc:333] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:28.673493 11032 raft_consensus.cc:2243] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:28.674328 11032 raft_consensus.cc:2272] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:28.676587  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:28.714494 11034 tablet_replica.cc:333] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:28.715626 11034 raft_consensus.cc:2243] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:28.716460 11034 raft_consensus.cc:2272] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:28.725504 11034 ts_tablet_manager.cc:1916] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:28.740211 11032 ts_tablet_manager.cc:1916] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:28.750833 11028 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.768448 11034 ts_tablet_manager.cc:1929] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:28.776693 11034 log.cc:1199] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/94aa8e92ea8d4f81a4ed4dbb1e99635d
I20260709 12:23:28.834409  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet 94aa8e92ea8d4f81a4ed4dbb1e99635d (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:28.838662 10020 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.862527 10674 raft_consensus.cc:2955] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.904488  9840 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:28.920589 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 0df211fae593420e88f685abc8d618ec with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:28.963963 11036 tablet_replica.cc:333] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:28.965317 11028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41118 (request call id 68) took 1212 ms. Trace:
I20260709 12:23:28.974086 11036 raft_consensus.cc:2243] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:28.975523 11036 raft_consensus.cc:2272] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:28.979594 10023 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:28.973464 11028 rpcz_store.cc:276] 0709 12:23:27.752758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:27.752879 (+   121us) service_pool.cc:224] Handling call
0709 12:23:27.758252 (+  5373us) log.cc:844] Serialized 501 byte log entry
0709 12:23:28.965208 (+1206956us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67112,"raft.queue_time_us":74260,"raft.run_cpu_time_us":1198,"raft.run_wall_time_us":1196,"thread_start_us":126012,"threads_started":1}
I20260709 12:23:28.994010  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0df211fae593420e88f685abc8d618ec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:29.006821 10099 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:29.008589 11030 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:23:29.015031 11036 ts_tablet_manager.cc:1916] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:29.027617 11032 ts_tablet_manager.cc:1929] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:29.028092 11032 log.cc:1199] T 1892c18d4e1644109d95b077e989b10f P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/1892c18d4e1644109d95b077e989b10f
I20260709 12:23:29.028880 10676 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:29.029944  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet 1892c18d4e1644109d95b077e989b10f (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:29.038780 11019 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:23:29.051755 11026 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:29.061090 10092 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:29.067232 11036 ts_tablet_manager.cc:1929] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:29.067744 11036 log.cc:1199] T 0df211fae593420e88f685abc8d618ec P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/0df211fae593420e88f685abc8d618ec
I20260709 12:23:29.082566  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 0df211fae593420e88f685abc8d618ec (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:29.087143 10676 raft_consensus.cc:2955] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:29.091794  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 53edc906d1af41e0992f01434995f9d1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:29.147357  9839 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:29.174618  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet 53edc906d1af41e0992f01434995f9d1 with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:29.175588 11034 tablet_replica.cc:333] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:29.193115 11034 raft_consensus.cc:2243] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:29.193931 11034 raft_consensus.cc:2272] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:29.197350 11034 ts_tablet_manager.cc:1916] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:29.233052 11034 ts_tablet_manager.cc:1929] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:23:29.233454 11034 log.cc:1199] T 53edc906d1af41e0992f01434995f9d1 P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/53edc906d1af41e0992f01434995f9d1
W20260709 12:23:29.234894  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54411.1us
I20260709 12:23:29.244002  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet 53edc906d1af41e0992f01434995f9d1 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:30.390771 10023 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:30.410488 10099 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:30.414637 11019 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:30.417980  9947 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:30.434769 11044 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:23:30.434743  9987 consensus_peers.cc:597] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 447b5d26fe15480dbb8fedf0f7a44777. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:23:30.467675  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47969.6us
I20260709 12:23:30.466849 11038 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.475683 10099 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.489060  9947 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:30.495204  9946 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.493031  9843 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:30.515050  9912 consensus_peers.cc:597] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a78ccdb241a54c0398e2539a827e5d22. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:30.517307 10025 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:30.520368 11012 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.528143 10099 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:30.534765 11012 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.565001 10981 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.584869 10096 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.584693 10020 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.634522  9839 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:30.663386  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79203.9us
I20260709 12:23:30.689963  9942 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:30.744462  9912 consensus_peers.cc:597] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 840d8356270745f492c7d2c7099794c1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:30.747028 10017 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:30.751821 10096 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:30.759673 11012 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.769801 11056 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.798964 11012 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.800762 10017 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.821259 10096 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.826431  9840 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.849660 10090 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:30.942121  9942 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:30.943411 11061 ts_tablet_manager.cc:933] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:30.943471 10023 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:30.944710 10062 consensus_peers.cc:597] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 886f3cb002374eae906fdbe2d392f3e7. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:30.945536 11041 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.946462 11014 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:30.977308 11014 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.980773  9947 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:30.986909 10017 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:31.034797  9839 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:31.036208 11061 tablet_copy_client.cc:323] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:31.039166 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet 447b5d26fe15480dbb8fedf0f7a44777 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:31.039825 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet 447b5d26fe15480dbb8fedf0f7a44777 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-447b5d26fe15480dbb8fedf0f7a44777
I20260709 12:23:31.067418 10035 tablet_copy_source_session.cc:215] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:31.104801 11061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 447b5d26fe15480dbb8fedf0f7a44777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:31.139262 11061 tablet_copy_client.cc:806] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:31.140026 11061 tablet_copy_client.cc:670] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
W20260709 12:23:31.140309 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 132461us
I20260709 12:23:31.145267 11061 tablet_copy_client.cc:538] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:31.160662 11069 ts_tablet_manager.cc:933] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
W20260709 12:23:31.161509 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 98627.3us
I20260709 12:23:31.170030 11061 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:31.249593 11069 tablet_copy_client.cc:323] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:31.251402  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet a78ccdb241a54c0398e2539a827e5d22 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:31.251977  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet a78ccdb241a54c0398e2539a827e5d22 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-a78ccdb241a54c0398e2539a827e5d22
I20260709 12:23:31.260947  9959 tablet_copy_source_session.cc:215] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:31.274333 11069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a78ccdb241a54c0398e2539a827e5d22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:31.292722 11061 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:31.293782 11061 tablet_bootstrap.cc:492] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:31.294519 11061 ts_tablet_manager.cc:1403] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.125s	user 0.023s	sys 0.003s
I20260709 12:23:31.297920 11061 raft_consensus.cc:359] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:31.326782 11061 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:31.349779 11075 ts_tablet_manager.cc:933] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:31.329967 11061 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:31.373667 11074 ts_tablet_manager.cc:933] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:31.374879 11075 tablet_copy_client.cc:323] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:31.376899 11069 tablet_copy_client.cc:806] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:31.377552 11069 tablet_copy_client.cc:670] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:31.384114  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 840d8356270745f492c7d2c7099794c1 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:31.384699  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 840d8356270745f492c7d2c7099794c1 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-840d8356270745f492c7d2c7099794c1
I20260709 12:23:31.386801 11069 tablet_copy_client.cc:538] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:31.392493  9959 tablet_copy_source_session.cc:215] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:31.396165 11069 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:31.397223 11075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 840d8356270745f492c7d2c7099794c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:31.410143 11061 ts_tablet_manager.cc:1434] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.115s	user 0.010s	sys 0.009s
I20260709 12:23:31.424919 11074 tablet_copy_client.cc:323] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:31.425340 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-447b5d26fe15480dbb8fedf0f7a44777 received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:31.425762 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-447b5d26fe15480dbb8fedf0f7a44777 on tablet 447b5d26fe15480dbb8fedf0f7a44777 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:31.427199 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet 886f3cb002374eae906fdbe2d392f3e7 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:31.437291 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet 886f3cb002374eae906fdbe2d392f3e7 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-886f3cb002374eae906fdbe2d392f3e7
I20260709 12:23:31.446354 10109 tablet_copy_source_session.cc:215] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:31.453331 11075 tablet_copy_client.cc:806] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:31.453913 11075 tablet_copy_client.cc:670] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:31.473311 11075 tablet_copy_client.cc:538] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:31.476838 11074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 886f3cb002374eae906fdbe2d392f3e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:31.497962 11069 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:31.507419 11069 tablet_bootstrap.cc:492] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:31.508232 11069 ts_tablet_manager.cc:1403] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.112s	user 0.027s	sys 0.009s
I20260709 12:23:31.511475 11069 raft_consensus.cc:359] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:31.515650 11069 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:31.516396 11069 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:31.519579 11069 ts_tablet_manager.cc:1434] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.005s
I20260709 12:23:31.517316 11074 tablet_copy_client.cc:806] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:31.526451 11074 tablet_copy_client.cc:670] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:31.535559  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-a78ccdb241a54c0398e2539a827e5d22 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:31.536067  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-a78ccdb241a54c0398e2539a827e5d22 on tablet a78ccdb241a54c0398e2539a827e5d22 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:31.538123 11074 tablet_copy_client.cc:538] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:31.549700 11075 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:31.574214 10674 raft_consensus.cc:1217] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:31.579836 11074 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:31.628536 11075 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:31.632936 11075 tablet_bootstrap.cc:492] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:31.631062 11074 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:31.639171 11075 ts_tablet_manager.cc:1403] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.090s	user 0.018s	sys 0.013s
I20260709 12:23:31.639559 11074 tablet_bootstrap.cc:492] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:31.640702 11074 ts_tablet_manager.cc:1403] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.061s	user 0.020s	sys 0.008s
I20260709 12:23:31.643849 11074 raft_consensus.cc:359] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:31.644701 11074 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:31.645326 11074 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:31.648779 11074 ts_tablet_manager.cc:1434] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:23:31.650697 11075 raft_consensus.cc:359] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:31.652797 11075 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:31.653584 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-886f3cb002374eae906fdbe2d392f3e7 received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:31.654004 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-886f3cb002374eae906fdbe2d392f3e7 on tablet 886f3cb002374eae906fdbe2d392f3e7 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:31.654404 11075 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:31.658274 11075 ts_tablet_manager.cc:1434] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:23:31.698344  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-840d8356270745f492c7d2c7099794c1 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:31.699710  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-840d8356270745f492c7d2c7099794c1 on tablet 840d8356270745f492c7d2c7099794c1 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:31.843657 10674 raft_consensus.cc:1217] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:31.864461 10676 raft_consensus.cc:1217] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:31.958343 10676 raft_consensus.cc:1217] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:32.060509 11057 raft_consensus.cc:1064] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:32.063891 11057 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:32.098157 10676 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.100759 11089 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:32.106264 10092 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.110237 11086 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:32.139226 10016 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.141175 11089 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:32.167640 11085 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.192117 10021 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.215461 10090 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.221262 10676 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.237272  9839 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:32.355459 11019 raft_consensus.cc:1064] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:32.364416 11019 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:32.415105 11066 raft_consensus.cc:1064] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:32.456506 11066 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:23:32.510437  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83995.7us
I20260709 12:23:32.513178 10676 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.520757  9944 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.525317 11057 raft_consensus.cc:1064] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:32.537290 11019 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:32.571448 10099 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.575337 11100 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:32.595636 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 72734.5us
I20260709 12:23:32.596026 11099 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
W20260709 12:23:32.614305 11095 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.110s	user 0.003s	sys 0.002s
I20260709 12:23:32.615231 11057 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:32.622112 10673 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.624868 10016 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.640540 11098 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:32.645488  9947 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.648177 11104 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
W20260709 12:23:32.655501  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56775us
I20260709 12:23:32.658535 11088 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
W20260709 12:23:32.664240  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 139987us
I20260709 12:23:32.723189 10672 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.740657 11099 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.754215 10024 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.754689  9942 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:32.773053 11084 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:23:32.788316 10095 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.795589 10092 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:32.790843 10676 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.813925  9843 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.827700 11012 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:32.836732 10095 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:32.841656 11012 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:32.846766 11089 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:32.867429 10676 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:32.886507 11085 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:23:32.924980 11088 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.954337 10025 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:23:32.957741  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67967.4us
I20260709 12:23:32.953824 11102 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.959924  9944 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:32.967238 10018 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:32.997105  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 123033us
I20260709 12:23:32.981750 10674 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.037132 10676 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.040905 11105 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.068120 10672 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.076355  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a78ccdb241a54c0398e2539a827e5d22 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.081517 11087 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:33.088969  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48289.4us
I20260709 12:23:33.093544 10019 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.087750 11099 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.098052  9947 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.104971  9840 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:33.110924 11019 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.127349  9947 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.146238 10095 raft_consensus.cc:2955] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.147718  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 447b5d26fe15480dbb8fedf0f7a44777 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.152491 10098 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.142340  9841 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.158324 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet a78ccdb241a54c0398e2539a827e5d22 with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:33.193815 10671 raft_consensus.cc:2955] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.213063  9843 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.216517 10676 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.225435  9947 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.241923  9840 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.249531 11122 tablet_replica.cc:333] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:33.253087 11122 raft_consensus.cc:2243] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:33.277345 10090 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:33.285761 10019 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.285912 10671 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.287070 11122 raft_consensus.cc:2272] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:33.287645 11104 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.288709 11041 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:33.296123 11122 ts_tablet_manager.cc:1916] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:33.297262 11098 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.299228 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 447b5d26fe15480dbb8fedf0f7a44777 with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:33.299540 10671 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.313373 10019 raft_consensus.cc:2955] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.326032  9947 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:33.332139 10098 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.333583 11084 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.333998  9840 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 1 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.332141 10676 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:33.336732 11123 tablet_replica.cc:333] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:33.350092 10090 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.349412 11037 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.360028 11087 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:33.368127 10672 raft_consensus.cc:2955] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.380007  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet 886f3cb002374eae906fdbe2d392f3e7 with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:33.373427 11123 raft_consensus.cc:2243] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:33.380437  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 886f3cb002374eae906fdbe2d392f3e7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.381263 11123 raft_consensus.cc:2272] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:33.383610  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 840d8356270745f492c7d2c7099794c1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:33.413110 11124 tablet_replica.cc:333] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:33.411288 11123 ts_tablet_manager.cc:1916] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:33.424959  9843 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:33.437116 11124 raft_consensus.cc:2243] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:33.438390 11124 raft_consensus.cc:2272] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:33.443471 11124 ts_tablet_manager.cc:1916] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:33.467118 11122 ts_tablet_manager.cc:1929] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:33.467758 11122 log.cc:1199] T a78ccdb241a54c0398e2539a827e5d22 P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/a78ccdb241a54c0398e2539a827e5d22
I20260709 12:23:33.469900  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet a78ccdb241a54c0398e2539a827e5d22 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:33.472817 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet 840d8356270745f492c7d2c7099794c1 with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:33.483726 11122 tablet_replica.cc:333] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:33.486428 11122 raft_consensus.cc:2243] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:33.513804 11122 raft_consensus.cc:2272] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:33.517818 11122 ts_tablet_manager.cc:1916] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:33.522634 11124 ts_tablet_manager.cc:1929] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:33.523300 11124 log.cc:1199] T 886f3cb002374eae906fdbe2d392f3e7 P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/886f3cb002374eae906fdbe2d392f3e7
I20260709 12:23:33.528018  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet 886f3cb002374eae906fdbe2d392f3e7 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:33.535766 11123 ts_tablet_manager.cc:1929] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:33.536388 11123 log.cc:1199] T 447b5d26fe15480dbb8fedf0f7a44777 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/447b5d26fe15480dbb8fedf0f7a44777
I20260709 12:23:33.538782  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 447b5d26fe15480dbb8fedf0f7a44777 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:33.546126 11122 ts_tablet_manager.cc:1929] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:33.546727 11122 log.cc:1199] T 840d8356270745f492c7d2c7099794c1 P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/840d8356270745f492c7d2c7099794c1
I20260709 12:23:33.548372  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet 840d8356270745f492c7d2c7099794c1 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:34.574043 10019 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:34.582482 10090 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:34.584432 11126 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:34.585881  9944 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:34.587590 11126 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:34.588150  9987 consensus_peers.cc:597] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7253570248b8481791bf2187007dd5df. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:34.595285 11105 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.598017 10090 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.599897  9944 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.625555  9944 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:34.625949  9843 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:34.656634 10019 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.661525 10098 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:34.663808  9912 consensus_peers.cc:597] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 59e61711ca80440e9c13392c6b6b25fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:34.669412 11084 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:23:34.693635 11012 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.699548 11012 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.701189 10018 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.725370 10019 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:34.726936 10090 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.768440  9944 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.769970 11126 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.770480 10098 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.765486  9840 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:34.774729  9987 consensus_peers.cc:597] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ed7d787bc5ad4e8db7719c178ffbc59b. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:34.776129 11126 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.806196  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:34.811317 11126 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.815033  9947 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.821362 10095 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.850560  9840 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.871753  9942 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:34.904510 10095 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.904999 10019 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:34.907016 11087 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:34.909394  9912 consensus_peers.cc:597] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3e4ebd0dc053435ea316f01d854592c9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:34.910406 11087 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:34.933792 11087 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.943347 10095 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.952440 10019 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:34.966101  9840 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:34.985560  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:34.986240  9942 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 2 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:34.996425  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:34.997113  9942 raft_consensus.cc:606] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:35.005402  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:35.006116  9942 raft_consensus.cc:606] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:35.009848  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:35.010500  9942 raft_consensus.cc:606] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:35.013098  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:35.013667  9942 raft_consensus.cc:606] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:35.015897  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:35.017520  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:35.018517  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:35.159576 11148 ts_tablet_manager.cc:933] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:35.171105 11150 ts_tablet_manager.cc:933] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:35.175567 11148 tablet_copy_client.cc:323] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:35.178158 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet 7253570248b8481791bf2187007dd5df from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:35.178807 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet 7253570248b8481791bf2187007dd5df from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-7253570248b8481791bf2187007dd5df
I20260709 12:23:35.190347 10035 tablet_copy_source_session.cc:215] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.198742 11147 raft_consensus.cc:993] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:35.199858 11147 raft_consensus.cc:1081] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:35.201768 10095 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:35.202340 10095 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.202669 10095 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.196372 11148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7253570248b8481791bf2187007dd5df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.211838 10095 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:35.216972  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:35.217940 11150 tablet_copy_client.cc:323] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:35.218101  9942 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:35.218521  9942 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:35.219367  9942 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:35.219959 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet ed7d787bc5ad4e8db7719c178ffbc59b from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:35.220892  9942 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.221187 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet ed7d787bc5ad4e8db7719c178ffbc59b from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-ed7d787bc5ad4e8db7719c178ffbc59b
I20260709 12:23:35.228741 10095 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:35.229729 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:35.231402 10019 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.237408 10035 tablet_copy_source_session.cc:215] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.241238 11150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed7d787bc5ad4e8db7719c178ffbc59b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.241366 10019 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:35.242810 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:35.243592 11127 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:35.244059 11127 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:35.244887 11127 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:35.263391  9942 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:35.277177 11153 ts_tablet_manager.cc:933] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:35.282867  9840 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:35.287067 11150 tablet_copy_client.cc:806] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.287797 11150 tablet_copy_client.cc:670] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.291970 11150 tablet_copy_client.cc:538] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.297103 11153 tablet_copy_client.cc:323] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:35.299235  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 59e61711ca80440e9c13392c6b6b25fa from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:35.299810  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 59e61711ca80440e9c13392c6b6b25fa from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-59e61711ca80440e9c13392c6b6b25fa
I20260709 12:23:35.300482 11148 tablet_copy_client.cc:806] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.301098 11148 tablet_copy_client.cc:670] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.301519 11084 raft_consensus.cc:993] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:35.302043 11084 raft_consensus.cc:1081] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:35.303781 10676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42028
I20260709 12:23:35.304349 10676 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.304687 10676 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.310868 11148 tablet_copy_client.cc:538] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.311198  9959 tablet_copy_source_session.cc:215] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.318058 11153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59e61711ca80440e9c13392c6b6b25fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.320477 10676 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:35.330968 11125 raft_consensus.cc:993] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:35.331502 11125 raft_consensus.cc:1081] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:35.334941 10674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42028
I20260709 12:23:35.335511 10674 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.335881 10674 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.350687 10674 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:35.353994  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:35.354569  9942 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:35.354875  9942 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:35.355647 10676 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:35.355573  9942 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:35.356281 11155 ts_tablet_manager.cc:933] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:35.356709  9942 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.357551 10674 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:35.358388 11150 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:35.360044  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:35.360610  9944 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:35.360914  9944 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:35.361625  9944 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:35.371173 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:35.371802 10097 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:35.372166 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:35.372463 11148 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:35.374622  9944 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.378829 10095 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.386933 10097 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:23:35.395421 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:23:35.397673 11158 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:35.387036 11153 tablet_copy_client.cc:806] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.403146 11153 tablet_copy_client.cc:670] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.404898 11158 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:35.406677 11158 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:35.408005 11153 tablet_copy_client.cc:538] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:35.390456 10095 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:35.431267 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:23:35.432094 11159 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:35.434296  9944 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:35.436307  9942 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:23:35.441293 11150 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.445159 11150 tablet_bootstrap.cc:492] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:35.445986 11150 ts_tablet_manager.cc:1403] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.088s	user 0.014s	sys 0.011s
I20260709 12:23:35.444109 11153 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:35.455487 11150 raft_consensus.cc:359] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:35.465412 11159 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:35.466485 11159 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:35.469872 11155 tablet_copy_client.cc:323] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:35.464264 11150 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:35.474254  9840 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 4, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 4 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:35.477896 11150 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:35.481515  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 3e4ebd0dc053435ea316f01d854592c9 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:35.482127  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 3e4ebd0dc053435ea316f01d854592c9 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-3e4ebd0dc053435ea316f01d854592c9
I20260709 12:23:35.490141 11150 ts_tablet_manager.cc:1434] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 12:23:35.492260 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-ed7d787bc5ad4e8db7719c178ffbc59b received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:35.492894 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-ed7d787bc5ad4e8db7719c178ffbc59b on tablet ed7d787bc5ad4e8db7719c178ffbc59b with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:35.614992  9959 tablet_copy_source_session.cc:215] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:35.615895  9840 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:35.625830 11155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3e4ebd0dc053435ea316f01d854592c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:35.636317 11169 raft_consensus.cc:993] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:35.636967 11125 raft_consensus.cc:1081] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 2 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:35.642889 10095 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:35.643241  9944 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:35.643504 10095 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.643908 10095 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.645972 11127 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:35.653406 10095 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:35.670748 11087 raft_consensus.cc:993] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:35.702219 11147 raft_consensus.cc:1081] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:35.704674 10095 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:35.707484 10090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:35.708113 10090 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:35.708472 10090 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.729689 10090 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:35.777339  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:35.778918  9942 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:35.779263  9942 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:35.788892 10090 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:35.801800  9942 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:35.750622 11148 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.823657 10671 raft_consensus.cc:1217] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:35.849833  9942 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.860566 11148 tablet_bootstrap.cc:492] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:35.864281 11148 ts_tablet_manager.cc:1403] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.492s	user 0.027s	sys 0.011s
I20260709 12:23:35.863353  9943 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:23:35.866384 11159 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:35.858538  9942 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:35.873420 11148 raft_consensus.cc:359] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:35.876644 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 163524us
I20260709 12:23:35.879586 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:35.883234 10019 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:35.903143 10019 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:35.906427  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:35.907140  9942 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:35.907526  9942 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:35.908258  9942 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:35.890369 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:35.913391  9942 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:35.915179 11153 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:35.915647 11155 tablet_copy_client.cc:806] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:35.916131 11153 tablet_bootstrap.cc:492] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:35.916224 11155 tablet_copy_client.cc:670] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:35.916940 11153 ts_tablet_manager.cc:1403] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.473s	user 0.050s	sys 0.043s
I20260709 12:23:35.921484  9942 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:35.922844  9943 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:35.884114 11148 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:35.924901 11160 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
W20260709 12:23:35.941177  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56782.3us
I20260709 12:23:35.929366 11148 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:35.958189 10096 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:23:35.961220 11153 raft_consensus.cc:359] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:35.962232 11153 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:35.938347 11098 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:35.963326 11098 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:35.947731 11148 ts_tablet_manager.cc:1434] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.077s	user 0.006s	sys 0.001s
W20260709 12:23:35.966996 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 76704.2us
I20260709 12:23:35.983471 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:35.962962 11153 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:35.964197 11098 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:36.015355  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:36.001821 11155 tablet_copy_client.cc:538] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:36.003374 11127 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:36.001524 11160 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:23:36.036726 11127 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:36.053915 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-7253570248b8481791bf2187007dd5df received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:36.082345 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-7253570248b8481791bf2187007dd5df on tablet 7253570248b8481791bf2187007dd5df with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:36.095309  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:36.114464 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 116438us
I20260709 12:23:36.118731 10022 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:36.107574 11153 ts_tablet_manager.cc:1434] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.190s	user 0.005s	sys 0.005s
I20260709 12:23:36.146402 10674 raft_consensus.cc:1217] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 12:23:36.149225 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:36.149950 10017 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:36.154902 11127 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:36.177176 10017 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:36.188126 11155 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
W20260709 12:23:36.191934 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49063.1us
W20260709 12:23:36.246759  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47411.8us
W20260709 12:23:36.283483  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 88004.3us
I20260709 12:23:36.300621  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-59e61711ca80440e9c13392c6b6b25fa received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:36.301059  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-59e61711ca80440e9c13392c6b6b25fa on tablet 59e61711ca80440e9c13392c6b6b25fa with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:36.324496 11205 raft_consensus.cc:1064] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:36.331921 10671 raft_consensus.cc:1217] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:36.342962 11205 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:23:36.400710 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 176702us
I20260709 12:23:36.409018 11199 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:36.410567 10096 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:23:36.421142  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52378.5us
I20260709 12:23:36.423454 11159 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:36.431555  9840 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 2 to 3, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:36.448462 10671 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.449646 10095 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.450073  9942 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:36.451381 11126 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:36.453308 11177 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.454512 11182 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.529381 11155 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:36.556737 11155 tablet_bootstrap.cc:492] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:36.557495 11155 ts_tablet_manager.cc:1403] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.370s	user 0.022s	sys 0.006s
I20260709 12:23:36.562536 11177 raft_consensus.cc:1064] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:36.578339 11126 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:36.567003 10095 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:36.571384 11177 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:36.599468 11155 raft_consensus.cc:359] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:36.600404 11155 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:36.601099 11155 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:36.608152 11155 ts_tablet_manager.cc:1434] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.050s	user 0.003s	sys 0.003s
I20260709 12:23:36.609969  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-3e4ebd0dc053435ea316f01d854592c9 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:36.610433  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-3e4ebd0dc053435ea316f01d854592c9 on tablet 3e4ebd0dc053435ea316f01d854592c9 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:36.698372  9944 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:36.706151  9947 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:36.700034  9841 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:36.708451  9841 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:36.766505 11184 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:23:36.789183  9947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:36.789886  9947 raft_consensus.cc:606] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:36.760504 11194 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:23:36.815236 10671 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:36.816864  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:36.817468  9944 raft_consensus.cc:606] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:36.823897 10676 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:36.828227 10022 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:36.845932  9944 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:36.832223 11184 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:23:36.838274 11182 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
W20260709 12:23:36.875073 11220 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:23:36.877580 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55415us
I20260709 12:23:36.954581 11182 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:36.973351 10095 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:36.979790 11180 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:23:36.966172  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 66635.9us
I20260709 12:23:37.013401  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c669b03539649e4a4164ca56748f662"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:37.014228  9944 raft_consensus.cc:606] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:37.028187  9942 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.041090  9946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d43a22dba58d423d907a568238e85284"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:37.041787  9946 raft_consensus.cc:606] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:37.052544  9946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:37.053237  9946 raft_consensus.cc:606] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:37.054677  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:37.056412  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:37.057307  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:37.069211 10019 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:37.071741 11165 raft_consensus.cc:1064] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:37.074489 11165 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:37.095830 10094 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.098644 10022 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:23:37.119328 10676 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:37.127928 10672 raft_consensus.cc:1217] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:37.133082 11184 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:37.139734 11176 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.154297 11182 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:37.153723  9947 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.165064 11169 raft_consensus.cc:993] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:37.162299 10671 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.165602 11169 raft_consensus.cc:1081] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:37.168999 11126 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:37.187914 11180 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.198797 10671 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.208628 11147 raft_consensus.cc:993] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:37.211364  9843 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.213100 11147 raft_consensus.cc:1081] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:37.214880 10021 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45930
I20260709 12:23:37.215410 10021 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:37.215758 10021 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.221964 10021 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.225907 10022 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.228628 10096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:37.229192 10096 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:37.229528 10096 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.231210  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:37.232333  9947 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:37.232623  9947 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:37.233451  9947 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.235124  9947 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.237162 11171 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:37.241199  9947 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:37.258611 10021 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:37.267560 10095 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.273623 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:37.274212 10090 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.269879  9991 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:37.272105 10096 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.272413 10097 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.292922 10090 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:37.293375 10096 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:37.298297 11084 raft_consensus.cc:993] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
W20260709 12:23:37.298885 11233 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:23:37.301008 11147 raft_consensus.cc:1081] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:37.303093 10090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d43a22dba58d423d907a568238e85284"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:37.303704 10090 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:37.304579 10090 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.311740  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:37.312309  9947 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:37.312572  9947 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:37.313295  9947 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.314268  9947 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.333957  9947 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:37.345322 10090 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.335748 10094 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.381268 10065 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:37.391692 10090 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:37.395539 11193 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:37.399597  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:37.401686  9943 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:37.402083  9943 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:37.384219 11164 raft_consensus.cc:993] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:37.404516  9943 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.397912 11171 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:23:37.413125 11168 raft_consensus.cc:1081] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:37.418164 11181 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:37.418774 11181 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:37.419788 11181 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.413240 11231 raft_consensus.cc:993] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:37.423185 11231 raft_consensus.cc:1081] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:37.425450 10094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:37.426048 10094 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:37.426407 10094 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.439869 10094 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.442103 10094 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:23:37.464622  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.476075  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:37.476881  9946 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:37.477279  9946 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:37.478361  9946 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.479905  9946 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.488019  9943 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.491205  9946 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:37.535372 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:37.547760 11173 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:37.548616 11173 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:37.551023 11173 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:37.587579  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 252253us
I20260709 12:23:37.674844  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ed7d787bc5ad4e8db7719c178ffbc59b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:23:37.580818  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 270207us
I20260709 12:23:37.579778 11193 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:37.679656 11193 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.676173 11172 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:37.601714  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.686789 11168 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:37.679708 10672 raft_consensus.cc:2955] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:37.700348 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48979us
I20260709 12:23:37.709128 11084 raft_consensus.cc:1064] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:37.710129 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:37.710723 10017 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.711364 11084 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:37.715000 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:37.715592 10022 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.716410 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:37.716979 10024 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.722446 10017 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:37.725983 10024 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:37.727222 10022 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:37.731694 10671 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.742231 10017 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:37.744767 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43737.2us
I20260709 12:23:37.746235 10019 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:37.558491  9943 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:37.751217 10017 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c669b03539649e4a4164ca56748f662"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45930
I20260709 12:23:37.754408 10017 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:37.758217 10017 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.772284  9842 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.801435 10017 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.802117 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:37.816383  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:37.817211  9942 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:37.817596  9942 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:37.829187 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:37.830772 10096 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:37.826880  9942 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:37.829504  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:37.823916 10017 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:37.831794  9942 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:37.830787  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54813.1us
W20260709 12:23:37.834148  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48576.6us
I20260709 12:23:37.831925 10019 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:37.842721  9942 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:37.851081 10096 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:37.855432  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:37.831091 11173 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:37.868912 11239 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:37.869479 11239 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:37.870335 11239 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.880734 11173 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:37.881798 11173 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:37.899511 10096 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:37.918020 11245 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.076s	user 0.002s	sys 0.000s
W20260709 12:23:37.937633  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 102651us
W20260709 12:23:37.938416 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 106588us
I20260709 12:23:37.955322  9839 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:37.966009 10094 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:37.969841 11171 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:23:37.988719  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49953.7us
I20260709 12:23:37.990406  9840 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:23:37.994302  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:38.032881 10019 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.035250  9944 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.036284  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet ed7d787bc5ad4e8db7719c178ffbc59b with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:38.038503 11255 tablet_replica.cc:333] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:38.038389 11127 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.040633 11255 raft_consensus.cc:2243] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:38.038913 10676 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:38.045537 11255 raft_consensus.cc:2272] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:38.045028  9843 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:38.055655  9948 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.057785 11235 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.059741 10096 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.061209 11239 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.065840  9840 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:38.070245  9840 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:38.098989 10017 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:38.103389 11171 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:23:38.112802 11164 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:38.114930  9943 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.115422  9947 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:38.116888 11182 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:38.122252 10671 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:38.126478 11195 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.129664  9947 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.126292 11239 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.141386 10025 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.146519 11098 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:38.148902 11236 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:23:38.143193 11255 ts_tablet_manager.cc:1916] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:38.173508  9843 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:38.223295  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7253570248b8481791bf2187007dd5df with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:38.309584  9948 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.316396 10022 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.340157  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:38.330435 11194 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 12:23:38.329345 11235 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:23:38.361531 10670 raft_consensus.cc:2955] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.384097  9843 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:38.414947 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55551.9us
I20260709 12:23:38.477025 11171 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.482427 10019 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.482981 10090 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.495086 10676 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.533845 11193 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:23:38.540782 10019 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.544643 11255 ts_tablet_manager.cc:1929] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:38.622833 11255 log.cc:1199] T ed7d787bc5ad4e8db7719c178ffbc59b P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/ed7d787bc5ad4e8db7719c178ffbc59b
I20260709 12:23:38.588420 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 7253570248b8481791bf2187007dd5df with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:38.595232  9947 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:38.611007 10095 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.531908 11265 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:23:38.641959 11098 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:38.644963  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet ed7d787bc5ad4e8db7719c178ffbc59b (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:38.657635  9839 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.711977 10095 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:38.718732 10095 raft_consensus.cc:606] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:38.721859 10095 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:38.722623 10095 raft_consensus.cc:606] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:38.726895 11283 tablet_replica.cc:333] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:38.737133 11283 raft_consensus.cc:2243] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:38.738204 11283 raft_consensus.cc:2272] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:38.742242 11283 ts_tablet_manager.cc:1916] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:38.776829  9948 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:38.778370  9944 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:38.783903 10095 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:38.787623 11283 ts_tablet_manager.cc:1929] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 12:23:38.788152 11283 log.cc:1199] T 7253570248b8481791bf2187007dd5df P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/7253570248b8481791bf2187007dd5df
I20260709 12:23:38.789810  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 7253570248b8481791bf2187007dd5df (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:38.793229 10676 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:38.798875 11278 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:38.813880 11278 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:38.831820 11271 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.843719 10676 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.844094 10096 raft_consensus.cc:2955] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.851168 11098 raft_consensus.cc:993] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:38.851902 11237 raft_consensus.cc:1081] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:38.864336 10676 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:38.875605  9840 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.876793 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45922
I20260709 12:23:38.884079 10024 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:38.884555 10024 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:38.877658 11278 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:38.896485 10024 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:38.897341 10019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:38.898516 10019 raft_consensus.cc:606] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:38.901494  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:38.902400 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:38.902871 10095 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:38.903038 10090 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:38.903436 10095 raft_consensus.cc:606] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:38.903434 10090 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:38.904618 10090 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:38.905716 10090 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:38.906384  9948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db738483c1444059b752648d302dc0d9"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:38.907003  9948 raft_consensus.cc:606] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:38.908689  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:38.908898 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet 3e4ebd0dc053435ea316f01d854592c9 with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:38.910432  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:38.902093  9944 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:38.915766  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:38.928277 11181 raft_consensus.cc:993] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:38.928910 11181 raft_consensus.cc:1081] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:38.932721 10024 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:38.936053  9944 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:38.937368 10019 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:38.937292 11168 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.939333  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3e4ebd0dc053435ea316f01d854592c9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:38.946488 10671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42042
I20260709 12:23:38.969429 10671 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:38.969859 10671 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:38.976667 10671 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:38.981624  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:38.982221  9948 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:38.988914  9948 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:38.990705 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:38.991510 11279 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:38.991957 11279 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:38.992791 11279 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:38.949069 10090 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:38.949441 10676 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:38.952240 11290 tablet_replica.cc:333] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:38.965459 11098 raft_consensus.cc:993] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:38.985991  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:39.003085 11098 raft_consensus.cc:1081] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:39.004941 11239 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:39.005489 11239 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:39.006292 10671 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:39.006407 11239 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:39.008250 11290 raft_consensus.cc:2243] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:39.009039 11290 raft_consensus.cc:2272] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:39.013376 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:39.013490 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45922
I20260709 12:23:39.014058 10024 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:39.014007 10019 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:39.014580 10019 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:39.014554 10024 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.015501 10019 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:39.016776 10019 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.017011 11290 ts_tablet_manager.cc:1916] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:39.025283 10019 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:39.022003  9842 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.022547 10024 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:39.013653  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46432.6us
I20260709 12:23:39.031003  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:39.031630  9948 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.033406 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:39.033988 10090 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:39.034282 10090 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:39.035048 10090 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:39.036298 10090 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.030789 10024 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:39.039050  9948 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:39.040371  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:39.042294  9840 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.045615 11274 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:39.046205 11274 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:39.047390 11274 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:39.056543  9839 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:39.060532 10090 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
W20260709 12:23:39.080644 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52448.3us
I20260709 12:23:39.086113 11168 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:39.080971 11290 ts_tablet_manager.cc:1929] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:39.085402  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 59e61711ca80440e9c13392c6b6b25fa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:39.088686 11290 log.cc:1199] T 3e4ebd0dc053435ea316f01d854592c9 P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/3e4ebd0dc053435ea316f01d854592c9
I20260709 12:23:39.086123 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 59e61711ca80440e9c13392c6b6b25fa with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:39.091560 11283 tablet_replica.cc:333] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:39.090361  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet 3e4ebd0dc053435ea316f01d854592c9 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:39.105132 11283 raft_consensus.cc:2243] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:39.105893 11283 raft_consensus.cc:2272] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:39.125569 11283 ts_tablet_manager.cc:1916] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:39.129540 10019 raft_consensus.cc:2955] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:39.134413  9839 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:39.158924 11283 ts_tablet_manager.cc:1929] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:39.159482 11283 log.cc:1199] T 59e61711ca80440e9c13392c6b6b25fa P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/59e61711ca80440e9c13392c6b6b25fa
I20260709 12:23:39.161131  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 59e61711ca80440e9c13392c6b6b25fa (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:39.184221 11169 raft_consensus.cc:993] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:39.184841 11169 raft_consensus.cc:1081] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:39.186518 10090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db738483c1444059b752648d302dc0d9"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:39.187151 10090 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:39.187508 10090 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.195227 11237 raft_consensus.cc:993] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:39.195811 11237 raft_consensus.cc:1081] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:39.195317 10090 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:39.197794 10676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42034
I20260709 12:23:39.198769 10090 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:39.198700  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:39.199443  9944 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:39.198382 10676 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:39.200146 10676 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.199733  9944 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:39.203329  9944 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:39.208120  9944 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.209911 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:39.210572 10022 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.218429 10676 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:39.221055  9944 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:39.222254 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:39.222690 10022 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:39.223109 11237 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:39.223673 11237 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:39.224741 11237 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:39.232820 10676 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:39.234083 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:39.234730 10095 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:39.235040 10095 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:39.235777 10095 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:39.236805 10095 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.247390 10095 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:39.248416 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:23:39.249298 11279 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:39.249768 11279 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:39.250913 11279 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:39.254662 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:39.255491 10022 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:39.264987 10022 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:39.270051  9840 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.270721  9839 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:39.473192 10090 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:39.474794 11295 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:39.509819  9944 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:39.512209 11295 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:39.540094  9948 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:39.541617 10022 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:39.542438 11279 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:39.543104 11223 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:39.630523 10090 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:39.632593 11239 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:39.635382  9948 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:39.637041 11235 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:39.688657 10022 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:39.690340 11306 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:39.711529 10090 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:39.712925 11306 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:39.720275  9947 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:39.721705 11275 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:39.796756 10022 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:39.798264 11314 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.120815  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:40.172068 10022 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:40.178807 10090 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:40.179767  9987 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d6726f0982834ecf8987472ed6c32a73. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:40.180305 11317 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.180886  9948 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.181279  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df211fae593420e88f685abc8d618ec"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:40.181844  9944 raft_consensus.cc:606] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:40.182350 11235 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.185065 10095 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:40.185757 10095 raft_consensus.cc:606] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:40.188968 10022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:40.189554 10022 raft_consensus.cc:606] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:40.189277 11239 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.191205 10095 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.194331 10022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:40.194962 10022 raft_consensus.cc:606] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:40.211488  9947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:40.217981  9947 raft_consensus.cc:606] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:40.221489  9842 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.222425  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:40.222996 11235 raft_consensus.cc:993] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:40.223421 11235 raft_consensus.cc:1081] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:40.224236  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:40.225322  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:40.225432 10090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37528
I20260709 12:23:40.225942 10090 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:40.226320 10090 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.228685 11169 raft_consensus.cc:993] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:40.229182 11169 raft_consensus.cc:1081] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:40.228955 10097 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:40.230891 10671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0df211fae593420e88f685abc8d618ec"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42028
I20260709 12:23:40.231393 10671 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:40.231720 10671 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.238548 11295 raft_consensus.cc:993] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:40.239089 11295 raft_consensus.cc:1081] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:40.241500 10676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42042
I20260709 12:23:40.242080 10676 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:40.242398 10676 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.243224 10671 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.248502  9944 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.251737 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:40.252363 10022 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.253099  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:40.253722  9947 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:40.254065  9947 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:40.258219  9947 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.265723 10025 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:40.267155 10022 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
W20260709 12:23:40.267481 10062 consensus_peers.cc:597] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fe681129eb440b4925ce53e8c68aaeb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:40.268363 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:40.268692 11237 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:40.269140 11279 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:40.269605 11279 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:40.270494 11279 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.274163  9948 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:40.275828 11237 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:40.290807  9947 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:40.308990  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 51542.3us
I20260709 12:23:40.308388  9948 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.297664 11237 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.320775 10676 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.324014  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:40.324733  9944 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.325963 10090 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:40.331302  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:40.331887 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:40.332552 10019 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:40.332887 10019 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:40.333429 10090 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:40.333683 10019 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:40.334997 10019 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.338263 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:40.338804 10671 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:40.338920 10024 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:40.339457 10024 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:40.340207 10024 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.341188 10024 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.331998  9946 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.349010  9946 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:40.349918 10019 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:40.351368 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:40.352372 11314 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:40.353098 11314 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:40.354034 11314 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:40.358717 10676 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:40.361446  9947 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:40.368175 10024 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:40.369804 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:40.371201 11313 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:40.373953  9944 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:40.375638 11313 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:40.377519 11313 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.388515 10024 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.401849  9840 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:40.415961  9944 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:40.431463  9912 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d80ed93cd18a4a1095bdd6f9fd0fca80. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:40.431645  9842 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:40.433568  9842 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:40.434501 10095 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.435001 10024 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.435899 11169 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.438748 11169 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.457796  9840 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:40.466341 11314 raft_consensus.cc:993] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:40.466881 11275 raft_consensus.cc:1081] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:40.480696 10671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42034
I20260709 12:23:40.481281 10671 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:40.481673 10671 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.491993 11169 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.501922 10024 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.503984 10095 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.507149 10671 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.510881 10671 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:40.512560 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:40.513192 10024 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.513259  9948 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:40.515594 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:40.516134 10090 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:40.516429 10090 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:40.520787 10024 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:40.517910 10090 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.522104 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:40.522899 10090 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.522974 11279 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:40.523510 11279 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:40.524397 11279 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:40.521453  9842 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.531404 10090 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:40.539804 10095 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.542418 11324 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:40.545075  9912 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 181ca8c0fd384c8c909b095284d9e12c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:40.545254  9843 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:40.556537 11169 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.552819 10024 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:40.560787 10095 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.560815 11324 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:23:40.564978 11324 raft_consensus.cc:993] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:40.565491 11324 raft_consensus.cc:1081] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:40.567206 10022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45930
I20260709 12:23:40.567723 10022 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:40.568066 10022 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.576318 10022 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:40.580027  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:40.580852  9948 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:40.581255  9948 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:40.581979  9948 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:40.579991  9843 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.584144 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:40.583817  9948 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.584712 10095 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:40.592999 10022 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:40.595100 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:40.596503  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:40.597229  9948 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:40.597759 11317 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:40.598322 11317 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:40.598979 11338 ts_tablet_manager.cc:933] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:40.599213 11317 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:40.608273 10024 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:40.623411 11338 tablet_copy_client.cc:323] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:40.625676 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet d6726f0982834ecf8987472ed6c32a73 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:40.626222 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet d6726f0982834ecf8987472ed6c32a73 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-d6726f0982834ecf8987472ed6c32a73
I20260709 12:23:40.629302  9843 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:40.634285 10035 tablet_copy_source_session.cc:215] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:40.642668 11338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d6726f0982834ecf8987472ed6c32a73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:40.662352 11338 tablet_copy_client.cc:806] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:40.663055 11338 tablet_copy_client.cc:670] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:40.669458 11338 tablet_copy_client.cc:538] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:40.683106  9948 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:40.685503 11279 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:40.692373 11338 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:40.718771 11341 ts_tablet_manager.cc:933] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:40.733758 10024 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:40.738065 11313 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:40.757272 11338 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:40.758344 11338 tablet_bootstrap.cc:492] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:40.759125 11338 ts_tablet_manager.cc:1403] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.067s	user 0.037s	sys 0.024s
I20260709 12:23:40.759209 11341 tablet_copy_client.cc:323] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:40.762009 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet 0fe681129eb440b4925ce53e8c68aaeb from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:40.762737 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet 0fe681129eb440b4925ce53e8c68aaeb from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-0fe681129eb440b4925ce53e8c68aaeb
I20260709 12:23:40.774185 10109 tablet_copy_source_session.cc:215] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:40.762149 11338 raft_consensus.cc:359] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:40.776717 11338 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:40.777448 11338 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:40.779677 11341 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fe681129eb440b4925ce53e8c68aaeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:40.781812 11338 ts_tablet_manager.cc:1434] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 12:23:40.785061 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d6726f0982834ecf8987472ed6c32a73 received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:40.785490 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d6726f0982834ecf8987472ed6c32a73 on tablet d6726f0982834ecf8987472ed6c32a73 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:40.800941 11341 tablet_copy_client.cc:806] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:40.801515 11341 tablet_copy_client.cc:670] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:40.806403 11341 tablet_copy_client.cc:538] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:40.819977 11341 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:40.831066  9948 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:40.832481 11279 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:40.844009  9944 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:40.846038 11341 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:40.847311 11341 tablet_bootstrap.cc:492] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:40.848201 11341 ts_tablet_manager.cc:1403] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.028s	user 0.025s	sys 0.001s
I20260709 12:23:40.848898 11314 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:40.851714 11341 raft_consensus.cc:359] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:40.852779 11341 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:40.853434 11341 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:40.856037 11341 ts_tablet_manager.cc:1434] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:23:40.858654 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-0fe681129eb440b4925ce53e8c68aaeb received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:40.859087 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-0fe681129eb440b4925ce53e8c68aaeb on tablet 0fe681129eb440b4925ce53e8c68aaeb with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:40.863890 11338 ts_tablet_manager.cc:933] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:40.868777 10024 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:40.873106 10022 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:40.875121 11313 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:40.881464 11314 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:40.945082 11338 tablet_copy_client.cc:323] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:40.946831  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet d80ed93cd18a4a1095bdd6f9fd0fca80 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:40.947357  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet d80ed93cd18a4a1095bdd6f9fd0fca80 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-d80ed93cd18a4a1095bdd6f9fd0fca80
I20260709 12:23:40.961264  9959 tablet_copy_source_session.cc:215] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:40.982992 11338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d80ed93cd18a4a1095bdd6f9fd0fca80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.002061 11338 tablet_copy_client.cc:806] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:41.002657 11338 tablet_copy_client.cc:670] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:41.006304 10676 raft_consensus.cc:1217] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:41.012925 10024 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:41.014202 11313 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:41.017246 11338 tablet_copy_client.cc:538] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:41.033741 11338 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:41.058102 10090 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:41.059846 11338 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:41.061126 11338 tablet_bootstrap.cc:492] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:41.062067 11338 ts_tablet_manager.cc:1403] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.029s	user 0.019s	sys 0.007s
I20260709 12:23:41.063062 11313 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:41.076164 10095 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:41.069710 11338 raft_consensus.cc:359] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:41.079893 11338 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:41.080752 11338 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:41.083343 11338 ts_tablet_manager.cc:1434] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.001s
I20260709 12:23:41.085125  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d80ed93cd18a4a1095bdd6f9fd0fca80 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:41.085562  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d80ed93cd18a4a1095bdd6f9fd0fca80 on tablet d80ed93cd18a4a1095bdd6f9fd0fca80 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:41.095127 11356 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:41.117359  9944 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:41.119091 11239 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:41.139555 11338 ts_tablet_manager.cc:933] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:41.155182 11338 tablet_copy_client.cc:323] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:41.157141  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 181ca8c0fd384c8c909b095284d9e12c from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:41.157779  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 181ca8c0fd384c8c909b095284d9e12c from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-181ca8c0fd384c8c909b095284d9e12c
I20260709 12:23:41.163581  9959 tablet_copy_source_session.cc:215] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:41.167037 11338 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 181ca8c0fd384c8c909b095284d9e12c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:41.181715 11338 tablet_copy_client.cc:806] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:41.182245 11338 tablet_copy_client.cc:670] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:41.185925 11338 tablet_copy_client.cc:538] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:41.194820 11338 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:41.194820 10676 raft_consensus.cc:1217] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:41.220337 11338 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:41.221272 11338 tablet_bootstrap.cc:492] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:41.221935 11338 ts_tablet_manager.cc:1403] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.027s	user 0.023s	sys 0.003s
I20260709 12:23:41.224835 11338 raft_consensus.cc:359] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:41.225598 11338 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:41.226163 11338 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:41.228250 11338 ts_tablet_manager.cc:1434] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:23:41.231948  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-181ca8c0fd384c8c909b095284d9e12c received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:41.232448  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-181ca8c0fd384c8c909b095284d9e12c on tablet 181ca8c0fd384c8c909b095284d9e12c with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:41.324543 10676 raft_consensus.cc:1217] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:41.502140  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:41.540413 10676 raft_consensus.cc:1217] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:41.590586  9948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:41.591238  9948 raft_consensus.cc:606] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:41.607268  9948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:41.607897  9948 raft_consensus.cc:606] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:41.610751 11235 raft_consensus.cc:1064] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:41.613534 11235 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.628194 10676 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:41.630015 11235 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:41.632773 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c669b03539649e4a4164ca56748f662"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:41.633111  9944 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:41.633419 10024 raft_consensus.cc:606] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:41.635931 11366 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:41.638785  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "840d8356270745f492c7d2c7099794c1"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:41.639405  9944 raft_consensus.cc:606] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:41.651140 10097 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:41.651984 10097 raft_consensus.cc:606] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:41.655174  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:23:41.656986  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:41.657835  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:41.669544 11366 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:41.679518 10676 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:41.689271  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 76615.6us
I20260709 12:23:41.693543 10097 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:41.698688 11368 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:41.705269  9948 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:41.706741  9842 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:41.719218 10097 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:41.721503 11169 raft_consensus.cc:993] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:41.722002 11364 raft_consensus.cc:1081] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:41.724287 10095 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:41.724864 10095 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:41.725229 10095 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.749475 10095 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:41.755445 10095 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:41.755671  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:41.756425  9948 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:41.756767  9948 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:41.757629  9948 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:41.759320  9948 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.762738 10022 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.766124  9948 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:41.768534 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:41.769246  9947 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:41.770166 11275 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:41.770696 11275 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:41.771205 11366 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:41.771543 11275 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:41.775913 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:41.776473 10024 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.779815 10676 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:41.784230 11171 raft_consensus.cc:993] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:41.784708 11171 raft_consensus.cc:1081] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:41.786137 10671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "840d8356270745f492c7d2c7099794c1"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42028
I20260709 12:23:41.786677 10671 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:41.786998 10671 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.787684 10024 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:41.796099 11366 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:41.800029  9947 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:41.805675 10671 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.808266  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:41.808859  9944 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:41.809135  9944 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:41.809805  9944 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.810968  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d6726f0982834ecf8987472ed6c32a73 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:41.812354  9944 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.812889 11364 raft_consensus.cc:993] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:41.813521 11364 raft_consensus.cc:1081] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:41.821115 10670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42028
I20260709 12:23:41.821700 10670 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:41.822067 10670 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.831604 11324 raft_consensus.cc:1064] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:41.808080 11317 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:41.835090 11324 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.844525 10670 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.844755 10676 raft_consensus.cc:2955] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:41.846230 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48285.3us
I20260709 12:23:41.846637  9839 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:41.852579  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:41.853327  9948 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:41.847466  9944 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:41.854533 10670 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:41.856139 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:41.857210 11279 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:41.853770  9948 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:41.862215  9948 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.863515  9948 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.858752 11279 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:41.864660 11279 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.849339 10671 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:41.869086  9843 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:41.870448  9948 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:41.872040 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet d6726f0982834ecf8987472ed6c32a73 with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:41.872589 10670 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:41.875957 10097 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:41.876540 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:41.878574 11329 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:41.879138 11329 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:41.879845 11171 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:41.879944 11329 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.884232 11171 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:41.895972 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:41.896653 10090 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.897912 11314 raft_consensus.cc:993] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:41.898375 11314 raft_consensus.cc:1081] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:41.898933 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:41.903266 10096 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.908584 10096 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:41.910319 10090 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:41.917948 10022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45922
I20260709 12:23:41.919157 10022 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:41.919205 10019 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:41.919605 11381 tablet_replica.cc:333] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:41.919761 10022 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.920682 11381 raft_consensus.cc:2243] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:41.921366 11381 raft_consensus.cc:2272] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:41.923878 11364 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:41.925793 11275 raft_consensus.cc:1064] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:41.934378 11275 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.941041 11381 ts_tablet_manager.cc:1916] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:41.951664 11364 raft_consensus.cc:1064] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:41.956229 11364 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:41.938220 10022 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:41.961335 10022 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:41.961762 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:41.962356 10090 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:41.962668 10090 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:41.963416 10090 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:41.964627 10090 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:41.972429 10676 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:41.972888 10019 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:41.973742 11317 raft_consensus.cc:993] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:41.974188 11317 raft_consensus.cc:1081] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:41.975063 11314 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:41.977246 10090 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:41.978374 11314 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:41.982987 11381 ts_tablet_manager.cc:1929] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:41.983502 11381 log.cc:1199] T d6726f0982834ecf8987472ed6c32a73 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/d6726f0982834ecf8987472ed6c32a73
I20260709 12:23:42.027074  9944 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:42.028708  9843 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.040567 10019 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.041704  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:42.046506  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet d6726f0982834ecf8987472ed6c32a73 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:42.050475 10019 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.057821 10676 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.061357 11169 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.111979 11171 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:23:42.126636 11324 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:42.139074 11239 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:42.160996  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:42.162181  9948 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:42.167337 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46437.2us
I20260709 12:23:42.175448 11239 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:42.177222 11239 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:42.138042 11314 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:23:42.184573 10090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c669b03539649e4a4164ca56748f662"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37528
I20260709 12:23:42.185599 10090 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:42.186053 10090 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:42.191844 10096 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:42.211979 10097 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.219803 11392 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:23:42.230418  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 91532us
I20260709 12:23:42.230564 10090 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:42.247911 10090 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:42.261466 10672 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.258253  9948 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
W20260709 12:23:42.279935  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49690.3us
I20260709 12:23:42.316170  9843 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.346182  9842 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
W20260709 12:23:42.382660  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78071.6us
I20260709 12:23:42.384703  9839 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.399592 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:42.400559 10022 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:42.400995 10022 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:42.403388  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:42.402091 10022 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:42.411270  9944 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:42.414734  9947 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:23:42.415210 11401 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:23:42.440768 10092 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:42.450541 10024 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:23:42.462787  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 89320.6us
I20260709 12:23:42.470888 11237 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:42.472357 10022 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:42.402209 11394 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.474117  9947 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:42.474243  9946 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:42.478931 11169 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.484894 10019 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.497381  9944 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:42.495863 11329 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:23:42.503770 11279 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:42.526467 10022 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:42.536721  9948 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:42.537294 11313 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:42.529031 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:42.538694 11237 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:42.539153 11237 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:23:42.541668  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108275us
I20260709 12:23:42.533540  9945 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.590430 10676 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.602581 11237 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:42.609731 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51838.8us
W20260709 12:23:42.612365 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43591.3us
I20260709 12:23:42.626093 10022 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:42.628249 10672 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.639359 10671 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:42.645779 10097 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:42.666401 11409 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:23:42.679816 11394 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:23:42.681399 11279 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:42.704357  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55938.4us
I20260709 12:23:42.710322 10671 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.743791 10099 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:42.743489 10094 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.738057 11324 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.772428  9842 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.819070  9842 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:42.828560 11171 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 12:23:42.854384  9946 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:42.863900 11239 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260709 12:23:42.976372 11418 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:23:42.980020  9843 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 2 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:42.998773 10099 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:43.001013 11317 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:43.029039  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d80ed93cd18a4a1095bdd6f9fd0fca80 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.002326 10094 raft_consensus.cc:2955] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.034677  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:43.068166 11419 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.109s	user 0.004s	sys 0.001s
I20260709 12:23:43.087023 10024 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.099828 11314 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:23:43.129355  9842 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.140837 10096 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:43.156805 10671 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:43.156713 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet d80ed93cd18a4a1095bdd6f9fd0fca80 with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:43.165956  9942 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:43.171948 10099 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.175269 10671 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:43.177088 11390 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.185957 11416 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:43.186297 10090 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab118912551248c1985fce40a2e815b0"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:43.187124 10090 raft_consensus.cc:606] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:43.191460 11364 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.199013 10090 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.201201 10671 raft_consensus.cc:2955] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.197007 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:43.208168 10024 raft_consensus.cc:606] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:43.214426  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
W20260709 12:23:43.215512 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54573.4us
I20260709 12:23:43.208285 11434 tablet_replica.cc:333] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:43.218402  9942 raft_consensus.cc:606] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:43.220243 10024 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:43.221288  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:43.223730  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:43.227007  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 181ca8c0fd384c8c909b095284d9e12c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.228619 11434 raft_consensus.cc:2243] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.229398 11434 raft_consensus.cc:2272] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:43.232860 11379 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:43.233296 11434 ts_tablet_manager.cc:1916] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:43.237680  9942 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:43.239413 11394 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:43.243250 11314 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:43.254878 11434 ts_tablet_manager.cc:1929] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:43.255548 11434 log.cc:1199] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/d80ed93cd18a4a1095bdd6f9fd0fca80
I20260709 12:23:43.255394  9843 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.259027  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet d80ed93cd18a4a1095bdd6f9fd0fca80 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:43.271579 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet 181ca8c0fd384c8c909b095284d9e12c with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:43.272738 11434 tablet_replica.cc:333] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:43.274065 11434 raft_consensus.cc:2243] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.274948 11434 raft_consensus.cc:2272] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:43.281299  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:43.285387 11434 ts_tablet_manager.cc:1916] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:43.305025 11379 raft_consensus.cc:993] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:43.305553 11379 raft_consensus.cc:1081] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:43.313656 11434 ts_tablet_manager.cc:1929] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:43.314257 11434 log.cc:1199] T 181ca8c0fd384c8c909b095284d9e12c P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/181ca8c0fd384c8c909b095284d9e12c
I20260709 12:23:43.316581  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet 181ca8c0fd384c8c909b095284d9e12c (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:43.326241 11394 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.338541 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab118912551248c1985fce40a2e815b0"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45922
I20260709 12:23:43.340193 10024 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:43.340730 10024 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.340128 10671 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.355170 10022 raft_consensus.cc:2955] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.372020  9842 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 4 to 5, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:43.382395  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0fe681129eb440b4925ce53e8c68aaeb with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:43.369997 10024 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:43.391985 10024 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:43.392676 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:43.398147  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fe681129eb440b4925ce53e8c68aaeb with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:43.413709 10096 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:43.414122 10096 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:43.414952 10096 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:43.416390 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:43.416878 10096 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.416972 10676 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.423976 10676 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:43.425668  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:43.435343 11436 tablet_replica.cc:333] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:43.441052 10096 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:43.443517 11399 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:43.444185 11399 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:43.445055 11399 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:43.450510 11436 raft_consensus.cc:2243] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:43.452488 11436 raft_consensus.cc:2272] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:43.456498 11436 ts_tablet_manager.cc:1916] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:43.459048  9838 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.478636 11436 ts_tablet_manager.cc:1929] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:43.479240 11436 log.cc:1199] T 0fe681129eb440b4925ce53e8c68aaeb P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/0fe681129eb440b4925ce53e8c68aaeb
I20260709 12:23:43.481151  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet 0fe681129eb440b4925ce53e8c68aaeb (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:43.491571 11317 raft_consensus.cc:993] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:43.492065 11317 raft_consensus.cc:1081] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:43.493696 10096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37528
I20260709 12:23:43.494269 10096 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:43.494663 10096 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.502117 10096 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:43.504050 10096 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:43.505033 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:43.505666 10671 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.506548 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:43.507181 10024 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:43.507491 10024 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:43.508276 10024 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:43.509461 10024 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.512348 10671 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:43.513518 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:23:43.514519 11394 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:43.515108 11394 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:43.516283 10024 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:43.516109 11394 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:43.527753  9838 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.734155 11409 raft_consensus.cc:993] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:43.734714 11409 raft_consensus.cc:1081] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:43.736263 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45930
I20260709 12:23:43.736781 10024 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:43.737108 10024 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.745203 10024 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:43.747468 10024 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:43.747758  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:43.748344  9942 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:43.748664  9942 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:43.749230  9942 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:43.750110  9942 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.751307 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:43.751852 10096 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:43.756899  9942 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:43.758414 10096 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:43.759140  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:43.759964 11317 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:43.760457 11317 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:43.761492 11317 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:43.775578  9838 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:43.825975 10090 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:43.827361 11399 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:43.841738 10676 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:43.845062 11405 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:43.953303 10676 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:43.954761 11430 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:43.964381 10024 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:43.968159 11386 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:44.162798 10096 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:44.164356 11405 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.180660  9942 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:44.183681 11399 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:44.454479  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:44.482468  9944 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:44.489524 10096 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.490247 10019 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:23:44.490476  9912 consensus_peers.cc:597] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7b809cb22d54c31ad5985fb9f01e6ab. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:44.491242 11452 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.498035 11410 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:44.510826 11410 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.515872 10096 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.524737 10099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6c669b03539649e4a4164ca56748f662"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:44.525381 10099 raft_consensus.cc:606] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:23:44.528481 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7253570248b8481791bf2187007dd5df"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:44.529070 10024 raft_consensus.cc:606] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:44.535955  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:44.536674  9942 raft_consensus.cc:606] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:23:44.539013  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 6c669b03539649e4a4164ca56748f662: Remote error: Service unavailable: leader transfer in progress
I20260709 12:23:44.547780 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:44.548386 10025 raft_consensus.cc:606] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:44.550300 10024 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.552881 11386 raft_consensus.cc:993] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:44.553128  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:44.553398 11386 raft_consensus.cc:1081] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Signalling peer cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:44.554842  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:44.540638  9840 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.555665  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:44.555768  9944 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:44.579653  9912 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b5ca4a138b9345ab87fc4055395e0488. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:44.581804 10096 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.582019 11399 raft_consensus.cc:993] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:44.582530 11399 raft_consensus.cc:1081] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:44.585340 10096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37528
I20260709 12:23:44.585984 10096 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.583829 11409 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:23:44.586493 10096 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.593775 10096 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:44.597596 10024 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.599088 11409 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.599188  9942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6c669b03539649e4a4164ca56748f662"
dest_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:41126
I20260709 12:23:44.601025  9942 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.601516  9942 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:44.608426  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:44.607745 11452 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.609254  9944 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.610859 10096 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:44.612555 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:44.613188 10024 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:44.613492 10024 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:44.614233 10024 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:44.615367 10024 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.617874 10096 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.622716 10024 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:44.624382 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:44.626502 11314 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:44.627019 11314 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:44.627823 11314 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:44.636667  9944 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:44.635807 10024 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:44.646070  9942 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:44.648007 11452 raft_consensus.cc:993] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:44.648629 11452 raft_consensus.cc:1081] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:44.650565 10096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:44.650763 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:44.651448 10025 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:44.651265  9942 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:44.657797 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:44.658763 10094 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:44.658991 10025 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:23:44.660708  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:44.661624 11409 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:44.659170 10094 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:23:44.667164  9987 consensus_peers.cc:597] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1154fd85c1e440b7bb5ee236ac1e0a21. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:44.668285 10094 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:44.671066  9942 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.667346 10099 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:44.672745 11317 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.672583 10019 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.671293 10094 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:44.675767 11317 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:44.684013 10094 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:23:44.690555 10096 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.692451 10096 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.699134 11405 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.703780 11409 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:44.705854 10096 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:44.706387  9838 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 2 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:44.712893  9838 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.704640 10099 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.709949  9942 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:44.711393 11409 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:44.726553 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:44.727171 10671 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.727468  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:44.728024  9944 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:44.728307  9944 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:44.729009  9944 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:44.730052  9944 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:44.734424 10671 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:44.735733 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:23:44.736569 11455 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:44.737265 11455 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:44.737498 10096 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:44.742022 11455 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:44.743997  9944 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:44.752076 11317 raft_consensus.cc:993] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:44.755151 11469 raft_consensus.cc:1081] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:44.759795 10671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7253570248b8481791bf2187007dd5df"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42042
I20260709 12:23:44.760439 10671 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:44.760836 10671 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.766242  9838 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.771446 10671 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:44.774866 10671 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:44.775589  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:44.776204  9944 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.776750 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:44.777333 10019 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:44.777631 10019 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:44.778373 10019 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:44.779752 10019 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:44.785749  9944 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:44.787190 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:44.788100 11408 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:44.788769 11408 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:44.789628 11408 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:44.789951  9840 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.792038  9842 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 4, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 4 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:44.822494 10019 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:44.846522  9840 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 3 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:44.905197 11472 ts_tablet_manager.cc:933] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:44.907537 11472 tablet_copy_client.cc:323] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:44.909478  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet d7b809cb22d54c31ad5985fb9f01e6ab from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:44.910048  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet d7b809cb22d54c31ad5985fb9f01e6ab from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-d7b809cb22d54c31ad5985fb9f01e6ab
I20260709 12:23:44.917541  9959 tablet_copy_source_session.cc:215] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:44.921205 11472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7b809cb22d54c31ad5985fb9f01e6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:44.938489 11472 tablet_copy_client.cc:806] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:44.939119 11472 tablet_copy_client.cc:670] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:44.945006 11472 tablet_copy_client.cc:538] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:44.954825 11472 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:44.976243 11472 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:44.977053 11472 tablet_bootstrap.cc:492] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:44.977644 11472 ts_tablet_manager.cc:1403] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.006s
I20260709 12:23:44.980618 11472 raft_consensus.cc:359] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:44.981654 11472 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:44.982518 11472 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:44.985656 11472 ts_tablet_manager.cc:1434] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:23:44.988667  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d7b809cb22d54c31ad5985fb9f01e6ab received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:44.989169  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-d7b809cb22d54c31ad5985fb9f01e6ab on tablet d7b809cb22d54c31ad5985fb9f01e6ab with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:45.071148 11472 ts_tablet_manager.cc:933] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:45.072819 11472 tablet_copy_client.cc:323] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:45.074330  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet b5ca4a138b9345ab87fc4055395e0488 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:45.074916  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet b5ca4a138b9345ab87fc4055395e0488 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-b5ca4a138b9345ab87fc4055395e0488
I20260709 12:23:45.081012  9942 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:45.083149 11455 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:45.112493  9959 tablet_copy_source_session.cc:215] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.118187 11472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b5ca4a138b9345ab87fc4055395e0488. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.139706 10024 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:45.141605 11455 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:45.145864 11472 tablet_copy_client.cc:806] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.146634 11472 tablet_copy_client.cc:670] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.151671 11472 tablet_copy_client.cc:538] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.167687 11472 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:45.170804 11481 ts_tablet_manager.cc:933] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:45.179716 11481 tablet_copy_client.cc:323] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:45.182015 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet 1154fd85c1e440b7bb5ee236ac1e0a21 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:45.182643 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet 1154fd85c1e440b7bb5ee236ac1e0a21 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-1154fd85c1e440b7bb5ee236ac1e0a21
I20260709 12:23:45.191474 10035 tablet_copy_source_session.cc:215] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:45.195271 11481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1154fd85c1e440b7bb5ee236ac1e0a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:45.196449 11472 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:45.197536 11472 tablet_bootstrap.cc:492] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:45.198292 11472 ts_tablet_manager.cc:1403] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.031s	user 0.021s	sys 0.003s
I20260709 12:23:45.201792 11472 raft_consensus.cc:359] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:45.202838 11472 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:45.203533 11472 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:45.206568 11472 ts_tablet_manager.cc:1434] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:23:45.209398 10024 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:45.211856  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-b5ca4a138b9345ab87fc4055395e0488 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:45.211722 11464 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:45.212401  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-b5ca4a138b9345ab87fc4055395e0488 on tablet b5ca4a138b9345ab87fc4055395e0488 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:45.218209 11481 tablet_copy_client.cc:806] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:45.218963 11481 tablet_copy_client.cc:670] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:45.224893  9942 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:45.228746 11408 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:45.232231 11481 tablet_copy_client.cc:538] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:45.244585 11481 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:45.270134 10025 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:45.272588 11447 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:45.278891 11481 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:45.280256 11481 tablet_bootstrap.cc:492] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:45.288108 10099 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:45.290573 11409 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:45.295706 10676 raft_consensus.cc:1217] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:45.300417 11481 ts_tablet_manager.cc:1403] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.056s	user 0.028s	sys 0.003s
I20260709 12:23:45.305186 11481 raft_consensus.cc:359] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:45.315133 11481 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:45.315985 11481 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:45.317857 11481 ts_tablet_manager.cc:1434] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:23:45.319015  9944 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:45.321048 11394 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:45.378850 10672 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:45.387893 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-1154fd85c1e440b7bb5ee236ac1e0a21 received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:45.388381 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-1154fd85c1e440b7bb5ee236ac1e0a21 on tablet 1154fd85c1e440b7bb5ee236ac1e0a21 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:45.402488 11455 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:23:45.658447 10676 raft_consensus.cc:1217] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:45.698199  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:45.732842 10676 raft_consensus.cc:1217] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:45.756618  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:45.757226  9944 raft_consensus.cc:606] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:45.760021 10099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8453bea911c4756add721edebd5d124"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:45.760627 10099 raft_consensus.cc:606] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:45.763334 10099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db738483c1444059b752648d302dc0d9"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:45.763911 10099 raft_consensus.cc:606] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:23:45.765158  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:45.767001  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:45.768939  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:45.783475 11314 raft_consensus.cc:993] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:45.783982 11314 raft_consensus.cc:1081] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:45.786366 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8453bea911c4756add721edebd5d124"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45922
I20260709 12:23:45.787001 10024 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:45.787365 10024 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.794942 10024 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:45.797665 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:45.798513 10099 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:45.798947 10099 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:45.800637 10099 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:45.801769 10099 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.804128 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:45.804724 10676 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:45.809355 10099 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:45.810830  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:45.811699 11469 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:45.811862 10024 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:45.812361 11469 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:45.811591 10676 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:45.813292 11469 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:45.830060  9842 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:45.870276 11410 raft_consensus.cc:1064] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:45.872861 11410 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:45.884845 10676 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:45.886451 11409 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:45.892558 10099 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:45.894322 10024 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:45.895713 11464 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:45.899215 11409 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:45.934084 11409 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:45.936057 10096 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:45.947464 10671 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:45.952913 10024 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:45.955646  9842 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:45.978574  9944 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:45.993320 10024 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:45.994765 11464 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.011015 10676 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:46.013612 11409 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.016875 11410 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.019381 10019 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.021404 10676 raft_consensus.cc:2955] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.026172  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d7b809cb22d54c31ad5985fb9f01e6ab with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:46.034092  9842 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.045823 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet d7b809cb22d54c31ad5985fb9f01e6ab with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:46.083770 11507 tablet_replica.cc:333] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:46.085357 11507 raft_consensus.cc:2243] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:46.086098 11507 raft_consensus.cc:2272] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:46.090051 11507 ts_tablet_manager.cc:1916] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:46.092857 11464 raft_consensus.cc:1064] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:46.095532 11464 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:46.102783 11410 raft_consensus.cc:993] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:46.103276 11410 raft_consensus.cc:1081] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:46.104923 10099 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:46.105430 10099 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.105754 10099 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.113263 11507 ts_tablet_manager.cc:1929] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:46.113897 11507 log.cc:1199] T d7b809cb22d54c31ad5985fb9f01e6ab P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/d7b809cb22d54c31ad5985fb9f01e6ab
I20260709 12:23:46.117163 10676 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.118075  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet d7b809cb22d54c31ad5985fb9f01e6ab (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:46.118920 11464 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.119417 10024 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.123505 11394 raft_consensus.cc:993] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:46.124102 11394 raft_consensus.cc:1081] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Signalling peer cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:46.124794 10096 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.124181 11464 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.126732 11410 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.123904 10099 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:46.129226  9944 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "db738483c1444059b752648d302dc0d9"
dest_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:41126
I20260709 12:23:46.129797  9944 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.130118  9944 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.135211  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:46.135999  9942 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:46.136317  9942 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:46.137219  9942 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:46.138320  9942 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.145476 10099 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:46.146642 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:46.147244 10019 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:46.149341  9942 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:46.138154  9944 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:46.155321 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:46.155946 10025 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.154063 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:46.158988 11394 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:46.159523 11394 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:46.160382 11394 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:46.163152 10025 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:46.164403 10019 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:46.165304  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:46.165521 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:46.166190 10099 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:46.166532 10099 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:46.167052 11410 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:46.167558 11410 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:46.168682 11410 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:46.167519 10099 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:46.176469 10099 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:46.173578  9944 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:46.186841 11409 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.188115  9838 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:46.205914 10099 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:46.206954 10676 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.219350 10024 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.226452 10099 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.230309  9842 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:46.226451  9840 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.276083  9944 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:46.280407 10024 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:46.280875 10676 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:46.281638 11514 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.282562 11410 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.289619 11410 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.297327 10671 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.308553  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b5ca4a138b9345ab87fc4055395e0488 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:46.315917 10025 raft_consensus.cc:2955] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.317546 10099 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:46.319010 11469 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:46.323455 10676 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:46.333127  9843 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 3 to 4, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.338928 11468 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:46.352085 11468 raft_consensus.cc:1064] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:46.354646 11468 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:46.361543 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet b5ca4a138b9345ab87fc4055395e0488 with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:46.365278 11507 tablet_replica.cc:333] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:46.366362 11507 raft_consensus.cc:2243] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:46.367022 11507 raft_consensus.cc:2272] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:46.373050 11507 ts_tablet_manager.cc:1916] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:46.395440  9942 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.396049 10096 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.406241 11468 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.407994 11469 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.414649 10671 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:46.421365 11522 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.430342 11507 ts_tablet_manager.cc:1929] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:46.430965 11507 log.cc:1199] T b5ca4a138b9345ab87fc4055395e0488 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/b5ca4a138b9345ab87fc4055395e0488
I20260709 12:23:46.432622  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet b5ca4a138b9345ab87fc4055395e0488 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:46.439116 11522 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.446934 10671 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.449600 10096 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.468536  9838 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.511402  9942 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.519531 10024 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:46.535456 10676 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:46.538307 10099 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:46.540331 11522 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.543192 11469 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:46.585628 10024 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.574463 11522 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.587096 10676 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.589766 11394 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:46.605468  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1154fd85c1e440b7bb5ee236ac1e0a21 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:46.610941  9838 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 1 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.616036 10096 raft_consensus.cc:2955] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:46.624630  9944 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:46.625749  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet 1154fd85c1e440b7bb5ee236ac1e0a21 with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:46.629714 11495 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:46.638566 10025 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:46.641983 11410 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:46.652293 11529 tablet_replica.cc:333] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:46.653734 11529 raft_consensus.cc:2243] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:46.654455 11529 raft_consensus.cc:2272] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:46.667855 11529 ts_tablet_manager.cc:1916] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:46.674268 10096 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:46.675650 11409 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:46.688943 11529 ts_tablet_manager.cc:1929] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:46.689689 11529 log.cc:1199] T 1154fd85c1e440b7bb5ee236ac1e0a21 P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/1154fd85c1e440b7bb5ee236ac1e0a21
I20260709 12:23:46.691856  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet 1154fd85c1e440b7bb5ee236ac1e0a21 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:46.828958  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:46.870951 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:46.871762 10025 raft_consensus.cc:606] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:46.874810 10096 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:46.875440 10096 raft_consensus.cc:606] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:46.877959 10096 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:46.878435 10096 raft_consensus.cc:606] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:23:46.880692  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:46.881155  9944 raft_consensus.cc:606] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:46.882269  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:23:46.883862  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:46.884583  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:46.992643 11469 raft_consensus.cc:993] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:46.993306 11469 raft_consensus.cc:1081] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:46.995065 10096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37528
I20260709 12:23:46.995534 10096 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:46.995814 10096 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.000329 10096 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:47.001991 10096 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:47.003018 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:47.002911 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:47.003646 10671 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.003736 10025 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.004081 10025 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:47.004833 10025 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:47.006022 10025 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.010336 10671 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:47.011662 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:23:47.012408 10025 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:47.012975 11497 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.013499 11497 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:47.014338 11497 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:47.017279 11394 raft_consensus.cc:993] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:47.017738 11394 raft_consensus.cc:1081] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Signalling peer cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:47.019306  9942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1"
dest_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:41126
I20260709 12:23:47.019842  9942 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.020165  9942 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.027293  9942 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:47.029844  9842 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.031278 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:47.031792  9942 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:47.031852 10025 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.030761 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:47.033406 10096 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:47.033712 10096 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:47.034802 11495 raft_consensus.cc:993] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:47.034559 10096 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:47.035284 11495 raft_consensus.cc:1081] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:47.036073 10096 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.040961 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45922
I20260709 12:23:47.041530 10024 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.041843 10024 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.043705 10025 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:47.043696 10096 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:47.047178  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:47.048506 10024 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:47.048799 11514 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:47.049576 11514 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:47.050402 11514 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:47.054412 10024 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:47.054692  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:47.055336  9942 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.054878 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:47.057538 10096 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:47.057845 10096 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:47.058576 10096 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:47.059700 10096 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:47.062033  9942 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:47.063366  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:47.064329 11469 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:47.063870  9842 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.064832 11469 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:47.065688 11469 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:47.066167 10096 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:47.079612  9842 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:47.239049 11409 raft_consensus.cc:993] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:47.239568 11410 raft_consensus.cc:1081] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:47.240981 10671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42028
I20260709 12:23:47.241361 10671 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:47.241578 10671 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.246770 10671 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:47.248175 10671 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:47.249141 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:47.249326  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:47.249697 10024 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.249869  9942 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:47.250211  9942 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:47.250882  9942 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:47.252069  9942 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:47.256842 10024 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:47.258417 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:47.259548 11408 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:47.260090 11408 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:47.260980 11408 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:47.262077  9942 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:47.269721  9842 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 2 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:47.433115 10024 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:47.434376 11410 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.450243 10096 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:47.451601 11514 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.475939 10096 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:47.477332 11468 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.482268 10024 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:47.483801 11495 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:47.487396  9944 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:47.491343 11522 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.498485 10671 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:47.501653 11495 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:47.712126 10096 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:47.719041 10024 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:23:47.719244 10062 consensus_peers.cc:597] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 978cf11d4cd54b67a4fa39b2b089f3da. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:47.719743  9945 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:47.719870  9944 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:47.720386 11495 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:47.721315 11408 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:47.721800 11497 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.732954 10025 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:47.738938 11495 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.740506 11408 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:47.745553  9945 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.747924 10024 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.776290 10094 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:47.794847  9945 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:47.796922 11394 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:47.794898  9843 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.806883 10019 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
W20260709 12:23:47.806885 10062 consensus_peers.cc:597] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e346f82bc2694f9e968337cc4be9e53e. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:47.808564 11495 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:47.808624 11394 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.829550  9944 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.833225 10025 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.844017 10024 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:47.861971  9842 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 4, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 3 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:47.868180  9987 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 03a9b6ec47044f6d8c27ae5f07c64f3a. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:47.871694  9945 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:47.872269 10094 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:47.873591 11468 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:47.874765 11469 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:47.892063 11522 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.893811 10094 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.893934  9945 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.910215  9842 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.917259  9944 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:47.924111  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:47.942739  9912 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 912ffd65ac564403ac65b7a5d005d367. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:47.952376 10094 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:47.953157 10024 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:47.953994 11514 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:47.956981 11409 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:47.968374 11410 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.970336 10096 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:47.991608  9842 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.005528 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:48.006240 10024 raft_consensus.cc:606] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:48.012821  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:48.013497  9944 raft_consensus.cc:606] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:48.016428 10096 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:48.017016 10096 raft_consensus.cc:606] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:48.020268  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:48.010115 10024 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:48.022576  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:48.026144  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:48.137357 11569 ts_tablet_manager.cc:933] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:48.139302 11569 tablet_copy_client.cc:323] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:48.141142 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet 978cf11d4cd54b67a4fa39b2b089f3da from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:48.141618 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet 978cf11d4cd54b67a4fa39b2b089f3da from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-978cf11d4cd54b67a4fa39b2b089f3da
I20260709 12:23:48.147094 10109 tablet_copy_source_session.cc:215] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:48.150010 11569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 978cf11d4cd54b67a4fa39b2b089f3da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:48.165994 11569 tablet_copy_client.cc:806] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:48.166697 11569 tablet_copy_client.cc:670] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:48.171221 11569 tablet_copy_client.cc:538] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:48.179939 11569 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:48.200037 11569 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:48.200783 11569 tablet_bootstrap.cc:492] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:48.201331 11569 ts_tablet_manager.cc:1403] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.005s
I20260709 12:23:48.203591 11569 raft_consensus.cc:359] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:48.204233 11569 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:48.204771 11569 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:48.206818 11569 ts_tablet_manager.cc:1434] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:23:48.208770 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-978cf11d4cd54b67a4fa39b2b089f3da received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:48.209252 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-978cf11d4cd54b67a4fa39b2b089f3da on tablet 978cf11d4cd54b67a4fa39b2b089f3da with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:48.279992 11569 ts_tablet_manager.cc:933] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:48.281661 11569 tablet_copy_client.cc:323] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:48.283255 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet e346f82bc2694f9e968337cc4be9e53e from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:48.283864 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet e346f82bc2694f9e968337cc4be9e53e from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-e346f82bc2694f9e968337cc4be9e53e
I20260709 12:23:48.293022 10109 tablet_copy_source_session.cc:215] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:48.296967 11569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e346f82bc2694f9e968337cc4be9e53e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:48.319708 11569 tablet_copy_client.cc:806] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:48.320283 11569 tablet_copy_client.cc:670] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:48.326987 11569 tablet_copy_client.cc:538] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:48.336665 11569 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:48.344031 11409 raft_consensus.cc:993] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:48.344504 11409 raft_consensus.cc:1081] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:48.345871 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45930
I20260709 12:23:48.346309 10024 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.346530 10024 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.352648 10024 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.355449  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:48.355731 10024 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:48.356091  9944 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.356408  9944 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:48.357182  9944 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.358404  9944 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.358886 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:48.359704 10094 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.365684 11569 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:48.365659  9944 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:48.366912 11569 tablet_bootstrap.cc:492] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:48.367218  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:48.367654 11569 ts_tablet_manager.cc:1403] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.031s	user 0.024s	sys 0.007s
I20260709 12:23:48.368050 11469 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.369402 10094 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:48.371069 11569 raft_consensus.cc:359] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:48.371951 11569 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:48.372592 11569 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:48.374114 11469 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:48.375233 11469 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.375661 11569 ts_tablet_manager.cc:1434] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 12:23:48.377748 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-e346f82bc2694f9e968337cc4be9e53e received from {username='slave'} at 127.0.0.1:57396
I20260709 12:23:48.378155 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-e346f82bc2694f9e968337cc4be9e53e on tablet e346f82bc2694f9e968337cc4be9e53e with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:48.400493 11569 ts_tablet_manager.cc:933] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:48.419531 11522 raft_consensus.cc:993] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:48.420027 11522 raft_consensus.cc:1081] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:48.421669 10096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37528
I20260709 12:23:48.422253 10096 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.422639 10096 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.424934  9842 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.437325 10096 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.440662  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:48.441260  9942 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.443625 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:48.444263 10024 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.444595 10024 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:48.445448 10024 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.446467 10024 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.454030 10024 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:48.454928 10096 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:48.455324 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:48.456455 11394 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.456921 11394 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:48.457751 11394 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.464176 11574 ts_tablet_manager.cc:933] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:48.467016  9942 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:48.469584 11569 tablet_copy_client.cc:323] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:48.471403 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:48.472010 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet 03a9b6ec47044f6d8c27ae5f07c64f3a from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-03a9b6ec47044f6d8c27ae5f07c64f3a
I20260709 12:23:48.477952 11574 tablet_copy_client.cc:323] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:48.479686  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 912ffd65ac564403ac65b7a5d005d367 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:48.480217  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 912ffd65ac564403ac65b7a5d005d367 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-912ffd65ac564403ac65b7a5d005d367
I20260709 12:23:48.480225  9842 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:48.487747  9959 tablet_copy_source_session.cc:215] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:48.488547 10035 tablet_copy_source_session.cc:215] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:48.491850 11569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:48.492720 11574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 912ffd65ac564403ac65b7a5d005d367. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:48.509342 11569 tablet_copy_client.cc:806] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:48.510056 11569 tablet_copy_client.cc:670] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:48.513334 11574 tablet_copy_client.cc:806] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:48.513581 11394 raft_consensus.cc:993] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:48.514081 11569 tablet_copy_client.cc:538] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:48.531401 11574 tablet_copy_client.cc:670] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:48.537529 11574 tablet_copy_client.cc:538] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:48.541509 11569 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:48.551008 11394 raft_consensus.cc:1081] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:48.552634 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45922
I20260709 12:23:48.553215 10024 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:48.553560 10024 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.563215 11574 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:48.564147 10024 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.566943  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:48.567574  9944 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.575752  9944 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:48.577061  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:48.577929 11468 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:48.578405 11468 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:48.579216 11468 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.580623 11569 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:48.581670 11569 tablet_bootstrap.cc:492] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:48.582396 11569 ts_tablet_manager.cc:1403] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.041s	user 0.025s	sys 0.008s
I20260709 12:23:48.585388 11569 raft_consensus.cc:359] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:48.586388 11569 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:48.587199 11569 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:48.588526 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:48.589174 10094 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:48.589495 10094 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:48.590241 10094 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:48.588608 10671 raft_consensus.cc:1217] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:48.591300 10094 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:48.594378 11569 ts_tablet_manager.cc:1434] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.002s
I20260709 12:23:48.596412 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-03a9b6ec47044f6d8c27ae5f07c64f3a received from {username='slave'} at 127.0.0.1:36302
I20260709 12:23:48.596839 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-03a9b6ec47044f6d8c27ae5f07c64f3a on tablet 03a9b6ec47044f6d8c27ae5f07c64f3a with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:48.603431 10094 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:48.606139 10024 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:48.621266  9838 catalog_manager.cc:5697] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:48.627169 11574 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:23:48.628686 11574 tablet_bootstrap.cc:492] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:48.629431 11574 ts_tablet_manager.cc:1403] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.066s	user 0.013s	sys 0.010s
I20260709 12:23:48.633432 11574 raft_consensus.cc:359] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:48.641004 11574 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:48.641711 11574 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:48.644501 11574 ts_tablet_manager.cc:1434] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.004s
I20260709 12:23:48.646741  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-912ffd65ac564403ac65b7a5d005d367 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:23:48.647193  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-912ffd65ac564403ac65b7a5d005d367 on tablet 912ffd65ac564403ac65b7a5d005d367 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:48.828109 10094 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.830669 11469 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.853906  9942 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.859592 10671 raft_consensus.cc:1217] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:23:48.860889 10676 raft_consensus.cc:1217] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:48.862052 11469 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:48.879292  9944 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.881201 11495 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:48.887809 10024 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:48.889295 11495 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:49.092033  9942 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.094053 11468 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:23:49.101446 11588 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:23:49.106679  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:23:49.166989  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 132232us
I20260709 12:23:49.221963 10676 raft_consensus.cc:1217] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:23:49.230335 10096 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:49.256969 10094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:49.257738 10094 raft_consensus.cc:606] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:23:49.247373 11597 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
W20260709 12:23:49.281476  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46642.1us
I20260709 12:23:49.300655 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:49.301273 10025 raft_consensus.cc:606] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
W20260709 12:23:49.306339 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 113057us
I20260709 12:23:49.306862 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8453bea911c4756add721edebd5d124"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:49.307446 10025 raft_consensus.cc:606] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:49.308970  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:49.310623  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:49.311342  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:49.353706 11601 raft_consensus.cc:993] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
W20260709 12:23:49.357070 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 202796us
I20260709 12:23:49.354355 11597 raft_consensus.cc:1081] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:49.363628 10090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d8453bea911c4756add721edebd5d124"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37528
I20260709 12:23:49.364212 10090 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.364563 10090 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.371527 10090 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:49.373304 10090 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:49.427202 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:49.427829 10024 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:49.428140 10024 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:49.428864 10024 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:49.430029 10024 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.437096 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:49.437685 10671 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.442056 11595 raft_consensus.cc:1064] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:49.442220 11522 raft_consensus.cc:1064] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:49.445070 11595 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:49.445079 11522 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:49.469442 10676 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:49.471256 11469 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:49.475287 10024 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:49.482141 10090 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:49.485702 11469 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:49.486043  9945 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:49.490713 11596 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:49.479022 10671 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:49.510133 11596 raft_consensus.cc:993] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:49.511744 11596 raft_consensus.cc:1081] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Signalling peer cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:49.513566  9942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361"
dest_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:41138
I20260709 12:23:49.514308  9942 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.514794  9942 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.565096 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:49.568528 11603 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.570727  9946 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:49.574266 11497 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:49.575068 11605 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:49.574759  9946 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.574812 11497 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:49.582983 11497 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:49.596570 10096 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.598184 10676 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:49.602703 11495 raft_consensus.cc:1064] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:49.605913 11495 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:49.618439 10024 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:23:49.618777 11600 raft_consensus.cc:1064] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:23:49.621443 11600 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:49.537918  9942 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:49.627839 11605 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:23:49.628098  9838 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.631832  9942 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:49.622308 10676 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.632766 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:49.633340 10090 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.638842 11394 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:49.646730 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:49.647305 10025 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:49.647580 10025 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:49.648286 10025 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:49.649317 10025 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.650997 10090 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:49.652364  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:49.669065 11409 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:49.669710 11409 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:49.670708 11409 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:49.675036 10025 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:49.711283 10025 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:49.711795 10676 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:23:49.712514 11614 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:23:49.720213 11497 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:49.721227 11607 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.826452  9943 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:23:49.830356 11604 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.845826 11394 raft_consensus.cc:993] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:49.846280 11595 raft_consensus.cc:1081] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Signalling peer cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:49.846297  9843 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 2 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:49.860373 10022 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.868898 11625 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:49.881186  9946 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:41126
I20260709 12:23:49.887192  9943 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.887219  9946 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:49.895148  9946 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.900523 10670 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.903935  9946 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:49.908130 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:49.908731 10025 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.908914 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:49.909449 10090 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:49.909729 10090 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:49.910454 10090 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:49.911449 10090 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:49.907297  9946 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:49.924989 11629 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:23:49.931393 10025 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:49.932654  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:49.934149 11591 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:49.934684 11591 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:49.935504 11591 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:49.940244 10671 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:49.941908 11592 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:49.949657 10096 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:23:49.950933 11605 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.952677 11409 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:49.966558  9943 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:49.992451 10671 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:50.005059 10019 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:23:50.006201 11597 consensus_queue.cc:1175] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 2 status { last_received { term: 2 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 2 index: 4 } }
I20260709 12:23:50.009166 10090 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:50.019479 11497 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:50.021852 10022 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:50.039896 11409 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:50.039155  9943 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.051486 11597 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:50.061360 10676 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.068231 11603 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.080343 10676 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.084369  9944 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.106914 10096 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:50.108507 11591 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.066030 10024 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.119529  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:50.120064 10670 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.117249 11591 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.139644 10022 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:50.160826 11625 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:50.166494 11612 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:50.167901  9838 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.178525 10099 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.175809  9838 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:50.185840  9838 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.196102 10670 raft_consensus.cc:2955] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.205224 10022 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:50.206852 11600 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:23:50.226475 10024 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.261831 10099 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:50.282385 10676 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.286002 11625 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.291165  9841 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 4 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.293694 10022 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:50.293129  9841 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 3, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.295161 11625 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:50.307003 11625 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.310715 10024 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.311592 10670 raft_consensus.cc:2955] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.319519  9840 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 4 to 5, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.329195  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 978cf11d4cd54b67a4fa39b2b089f3da with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:50.338989 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 03a9b6ec47044f6d8c27ae5f07c64f3a with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:37522
I20260709 12:23:50.340946  9944 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:50.345463 10099 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:50.351315 10090 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:50.353200 11409 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:50.350178 10670 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:23:50.354974 10676 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:50.356709 11625 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:50.356904  9946 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:50.362004 11600 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:50.358090  9842 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 4 to 5, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.362900 11625 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:50.373668 10676 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.375813 11607 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.387990 10670 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.393537  9933 tablet_service.cc:1558] Processing DeleteTablet for tablet 978cf11d4cd54b67a4fa39b2b089f3da with delete_type TABLET_DATA_TOMBSTONED (TS cd983415389444e292b2330ef630850a not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:41118
I20260709 12:23:50.384356  9946 raft_consensus.cc:2955] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.371371 11600 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.390084 10099 raft_consensus.cc:2955] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:23:50.402791  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e346f82bc2694f9e968337cc4be9e53e with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:50.415846 10022 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:50.417847 11409 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.424758  9839 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 5 to 6, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 3 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.431758  9830 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 912ffd65ac564403ac65b7a5d005d367 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:23:50.443246 10090 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:50.445562 11600 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:50.457716 11648 tablet_replica.cc:333] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:23:50.468176 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet e346f82bc2694f9e968337cc4be9e53e with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:50.464701  9843 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 4, VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) evicted. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:50.490257 11648 raft_consensus.cc:2243] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:50.491058 11648 raft_consensus.cc:2272] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:50.508461 11648 ts_tablet_manager.cc:1916] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:50.509407 11647 tablet_replica.cc:333] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:23:50.509809 11651 tablet_replica.cc:333] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:50.517827 10007 tablet_service.cc:1558] Processing DeleteTablet for tablet 912ffd65ac564403ac65b7a5d005d367 with delete_type TABLET_DATA_TOMBSTONED (TS 9aa345bcd43b44088114b7b6f2830b17 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45918
I20260709 12:23:50.520421 11651 raft_consensus.cc:2243] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:50.521667 11651 raft_consensus.cc:2272] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:50.526017 11651 ts_tablet_manager.cc:1916] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:50.544643 11647 raft_consensus.cc:2243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:50.544966 11651 ts_tablet_manager.cc:1929] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260709 12:23:50.545509 11651 log.cc:1199] T e346f82bc2694f9e968337cc4be9e53e P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/e346f82bc2694f9e968337cc4be9e53e
I20260709 12:23:50.545622 11647 raft_consensus.cc:2272] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:50.554874 11647 ts_tablet_manager.cc:1916] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:50.564594 11648 ts_tablet_manager.cc:1929] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:50.565011 11648 log.cc:1199] T 978cf11d4cd54b67a4fa39b2b089f3da P cd983415389444e292b2330ef630850a: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/wals/978cf11d4cd54b67a4fa39b2b089f3da
I20260709 12:23:50.566723  9830 catalog_manager.cc:5028] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): tablet 978cf11d4cd54b67a4fa39b2b089f3da (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:50.566915  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet e346f82bc2694f9e968337cc4be9e53e (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:50.567917 11651 tablet_replica.cc:333] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:23:50.577829 11647 ts_tablet_manager.cc:1929] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:23:50.578331 11647 log.cc:1199] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/03a9b6ec47044f6d8c27ae5f07c64f3a
I20260709 12:23:50.579572 11651 raft_consensus.cc:2243] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:23:50.580298  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 03a9b6ec47044f6d8c27ae5f07c64f3a (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:50.580926 11651 raft_consensus.cc:2272] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:23:50.584287 11651 ts_tablet_manager.cc:1916] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:23:50.596438 11651 ts_tablet_manager.cc:1929] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:23:50.596853 11651 log.cc:1199] T 912ffd65ac564403ac65b7a5d005d367 P 9aa345bcd43b44088114b7b6f2830b17: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/wals/912ffd65ac564403ac65b7a5d005d367
I20260709 12:23:50.598135  9827 catalog_manager.cc:5028] TS 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): tablet 912ffd65ac564403ac65b7a5d005d367 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:23:50.600553  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:50.638490  9946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "062c203e13aa4527a094ec24339001e5"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:50.638995  9946 raft_consensus.cc:606] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:50.641702 10099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:50.642272 10099 raft_consensus.cc:606] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:50.644768  9946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:23:50.645249  9946 raft_consensus.cc:606] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 1 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:50.646272  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:23:50.647440  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:50.648037  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:50.677640 11604 raft_consensus.cc:993] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:50.678220 11604 raft_consensus.cc:1081] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Signalling peer 6c7bd3fa1bfa431c89fc9c9096452a76 to start an election
I20260709 12:23:50.680531 10670 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:42034
I20260709 12:23:50.681107 10670 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:50.681456 10670 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:50.688633 10670 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:50.691399 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:50.692622 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:50.692858 10022 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:50.693207 10099 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:50.693549 10099 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:50.694129 10099 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:50.695052 10099 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:50.698076 10022 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:50.698665 10670 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:50.699443 10640 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:50.699965 10099 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:50.700198 11535 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:50.700711 11535 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:50.701573 11535 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:50.710695  9839 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 3 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:23:51.024638 11610 raft_consensus.cc:993] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:51.025141 11600 raft_consensus.cc:1081] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:23:51.026726 10099 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:23:51.027216 10099 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.027511 10099 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.032557 10099 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:51.035159 10099 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:51.035048  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:51.035712  9946 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:51.036021  9946 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:51.036015 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:51.036791 10022 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.036988  9946 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:51.037930  9946 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.041514 10022 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:51.042232  9946 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:23:51.042536 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:51.043397 11604 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:51.043857 11604 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:51.044559 11604 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:51.062952  9839 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:23:51.152681 11610 raft_consensus.cc:993] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:51.153165 11600 raft_consensus.cc:1081] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:51.154431 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "062c203e13aa4527a094ec24339001e5"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45930
I20260709 12:23:51.154901 10024 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:51.155133 10024 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.160426 10024 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:51.162343 10024 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:51.162829 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:51.163476 10676 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.162829  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:51.164327  9946 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:51.164670  9946 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:51.165377  9946 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:51.166525  9946 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:51.169972 10676 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:51.171172  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:51.171933 11603 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:51.172451 11603 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:51.173318 11603 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:51.177029  9946 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:51.186278  9839 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:51.211897 10090 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:51.213311 11535 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:51.236523 10022 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:51.240144 11553 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:51.565805 10022 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:51.567220 11604 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:51.576982  9944 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:51.580137 11604 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:51.583626  9946 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.585913 11603 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.609472 10670 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:23:51.620760 11469 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:23:51.626317 10022 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:51.651811  9987 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a7196ffc873e4e7e84c76068d5b95904. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:51.653026 10096 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:51.658376 11469 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:51.673106  9945 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:51.674708 11603 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:51.679929 11469 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.683758 10090 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.686559  9945 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.690546  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.698814  9943 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:51.704903  9842 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a reported cstate change: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:51.719847  9912 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:51.720652 10096 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:51.722188 11610 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:51.726722 10022 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:23:51.729018 11610 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:23:51.743943 11610 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.747707 10090 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.758064 10022 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.773948 10096 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:51.783114  9943 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:23:51.786840 11497 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:51.789038  9842 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 2, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 1 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:51.862766 10062 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 230693f05eb74e8095e27d669f16c8ee. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:51.867245 10022 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:23:51.884379  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 115012us
I20260709 12:23:51.901968 10022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:23:51.902556 10022 raft_consensus.cc:606] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:23:51.905268 10090 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:35180
I20260709 12:23:51.905805 10090 raft_consensus.cc:606] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:51.906916  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:23:51.908470  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:23:51.909181  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:51.929718 11604 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:51.931386 11680 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.946871 10023 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.953748  9943 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:51.968427  9945 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:51.969787  9839 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 3, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
W20260709 12:23:51.976442  9912 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7e7b0b17a4d646809d4dd46a0268f2eb. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:23:51.978044 10023 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:51.978044 10090 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:51.979867 11679 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:51.980715 11600 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:52.000679 11610 raft_consensus.cc:2955] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:52.005986 10023 raft_consensus.cc:2955] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:52.007989 10094 raft_consensus.cc:2955] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:52.023926  9842 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index -1 to 4, NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) added. New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:23:52.085464 11686 ts_tablet_manager.cc:933] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:52.087505 11686 tablet_copy_client.cc:323] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.194:35115
I20260709 12:23:52.089107 10035 tablet_copy_service.cc:140] P 9aa345bcd43b44088114b7b6f2830b17: Received BeginTabletCopySession request for tablet a7196ffc873e4e7e84c76068d5b95904 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:36302)
I20260709 12:23:52.089650 10035 tablet_copy_service.cc:161] P 9aa345bcd43b44088114b7b6f2830b17: Beginning new tablet copy session on tablet a7196ffc873e4e7e84c76068d5b95904 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:36302: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-a7196ffc873e4e7e84c76068d5b95904
I20260709 12:23:52.092914 11682 raft_consensus.cc:993] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:52.093441 11680 raft_consensus.cc:1081] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:23:52.095517 10024 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45922
I20260709 12:23:52.096108 10024 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:52.096454 10024 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.099624 10035 tablet_copy_source_session.cc:215] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:52.101615 10024 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:52.103142 11686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a7196ffc873e4e7e84c76068d5b95904. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:52.104333 10024 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:52.104733 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:52.105109 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:52.105630 10676 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.105247 10094 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:52.127682 10094 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:52.131424 11603 raft_consensus.cc:993] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: : Instructing follower cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:52.314738 10094 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:52.320127 10094 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:52.587404 10676 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:52.592448 11669 raft_consensus.cc:1081] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Signalling peer cd983415389444e292b2330ef630850a to start an election
I20260709 12:23:52.713989 11686 tablet_copy_client.cc:806] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:52.715094 11686 tablet_copy_client.cc:670] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:53.131981 11692 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:53.132831 11692 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:53.136044 11692 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:53.238323 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.000s	sys 0.001s
W20260709 12:23:53.238869 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.000s	sys 0.002s
I20260709 12:23:53.258577 11696 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:53.259313 11696 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:53.265529 11696 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:53.268868 11692 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:53.269644 11692 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:53.283895 11696 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:53.284771 11696 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:53.299624 11697 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:53.300450 11697 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:53.307395 11692 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:53.311373 11698 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:53.311827 11692 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:53.312104 11698 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:53.312688 11692 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:53.317216 11697 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:53.318811 11696 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:53.321928 11698 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:53.324076 11692 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.268296 11692 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:54.271250 11696 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:54.272418 11696 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.277319 11696 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.278090 11696 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:54.279021 11696 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.270432 11692 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.467567 11697 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:54.469043 11697 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.387012 11692 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.471903 11698 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:54.472982 11698 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.643774 11698 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:54.648886 11698 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:54.646386 11692 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:54.644688 11701 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:54.391685 11696 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.804376 11699 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:54.805557 11692 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.846433 11699 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.848332 11692 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.849205 11692 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:54.851456 11692 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.808947 11698 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.811434 11697 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:54.808144 11701 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.865170 11701 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:54.849874 11699 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.857199 11698 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:23:54.868841 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.041s	sys 0.058s
W20260709 12:23:54.875211 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.042s	sys 0.058s
I20260709 12:23:54.876304 11702 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:54.877403 11702 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.879026 11698 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:54.880105 11698 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.883179 11700 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:54.884198 11700 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.885607 11699 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:54.886992 11701 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:54.886814 11699 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.887950 11701 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.890198 11699 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.891602 11700 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:54.892361 11700 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:54.893322 11700 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:54.895589 11701 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.897076 11698 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.934634 11702 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.934931 11692 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:54.943795 11700 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:54.979732 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.651s (SENT) [suppressed 2 similar messages]
W20260709 12:23:55.024794 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45285.9us
W20260709 12:23:55.074206  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.484s	user 0.017s	sys 0.047s
W20260709 12:23:55.074873  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.484s	user 0.017s	sys 0.048s
I20260709 12:23:55.075593 11706 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:55.087064 11706 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:55.105419 11597 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:23:52.858341 10094 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.000s
I20260709 12:23:55.122014 10094 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
W20260709 12:23:55.122901 10094 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1297) took 3018 ms (client timeout 1532 ms). Trace:
W20260709 12:23:55.123270 10094 rpcz_store.cc:269] 0709 12:23:52.104381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.104516 (+   135us) service_pool.cc:224] Handling call
0709 12:23:55.122677 (+3018161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:55.106472 11597 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:55.138003 11597 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:55.144743 11704 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.145793 11704 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.152475 11711 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.152789 11713 ts_tablet_manager.cc:933] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:55.151127 11597 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:23:55.160960  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.567s	user 0.010s	sys 0.022s
W20260709 12:23:55.161432  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.570s	user 0.010s	sys 0.022s
I20260709 12:23:55.161118 11597 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.164013 11597 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.166308 11708 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:55.166668 11710 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:55.169307 11704 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.169493 11714 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.174993 11710 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:55.176569 11706 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:55.183897 11704 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:55.191942 11710 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:55.192127 11708 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.194762 11707 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.195820 11707 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.171519 11597 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.208694 11705 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.209566 11705 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.205901 11707 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.201609 11711 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.215207 11597 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.215638 11706 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.211153 11720 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.217028 11706 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.211942 11714 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.247876 11714 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.250017 11714 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:55.214738 11704 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.260746 11708 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.261439 11708 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:23:55.261561 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:55.261749 11597 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.262292 11597 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:55.262439 11712 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:55.262954 11597 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:55.261945 11708 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.272841 11597 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:55.273716 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.275274 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.275148 11708 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.251643 11719 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:55.263254 11712 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:55.275151 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.284930 10098 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:55.291471 11717 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:55.257248 11713 tablet_copy_client.cc:323] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:23:55.283018 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.281108 11597 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:55.297379 11724 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:55.298810 11704 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.301690 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.302528 10098 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:55.284909 11720 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.307440 11720 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.309077 11707 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:55.309845 11707 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:55.311054 11710 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:55.311741 11710 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.313766 11710 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.314250 11710 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:55.315446 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.316039 11706 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.292832 11716 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:55.316532 11719 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:55.292464 11717 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.298300 11724 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.317023 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.332177 11714 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:55.897035 10093 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:55.336027 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.898375 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:23:55.955681  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.004s	sys 0.023s
W20260709 12:23:55.997514  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.005s	sys 0.023s
W20260709 12:23:55.351400 10064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.585s	user 0.008s	sys 0.034s
I20260709 12:23:55.316569 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:55.338199 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.071156 11728 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.071619 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.072252 10093 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:56.071916 11728 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.072922 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.073377 10093 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:56.074488 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.074983 11726 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.075243 10092 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:56.075816 11726 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.076293 11724 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.076833  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:23:56.077261 11716 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.078438 11726 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:55.895536 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.068696 11709 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.079283 10091 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.079485 11709 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.081176 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.081840 11709 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.074252 11728 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:55.874431 11712 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:55.975752 11707 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.095950 10094 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:55.348559 11705 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.348943 11711 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:55.939484 11731 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.108088 11727 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.109066 11730 ts_tablet_manager.cc:933] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:23:56.104254 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.104725 11716 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.110878 11710 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:55.971297 11736 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:55.972378 11733 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:56.088434 11714 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.116627 11709 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.119164  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.118916 11727 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.120095 11723 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.133821 11723 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.113075 11712 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:56.134898 11712 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.136225 11723 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:56.137077  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.008s	sys 0.035s
W20260709 12:23:56.137429  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.008s	sys 0.035s
I20260709 12:23:56.137707 11712 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.120229  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.124804 11711 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.138543  9945 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:56.138705 11711 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.139436 11710 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.121965 11717 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.140064  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.141196 11711 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.123162 11726 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.124185  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:56.126199 10064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.378s	user 0.012s	sys 0.060s
I20260709 12:23:56.131574 11731 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.132504 11736 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.133139 11727 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:56.130544  9915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.016s	sys 0.027s
I20260709 12:23:56.141445 11709 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.143424 11735 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:56.143517 11733 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.130411 11716 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:23:56.149950  9915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.908s	user 0.017s	sys 0.027s
I20260709 12:23:56.150418  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.147621 11726 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.176678 11726 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.152679  9946 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:56.153151 11709 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:56.158921  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.895s	user 0.009s	sys 0.010s
W20260709 12:23:56.180922  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.009s	sys 0.010s
W20260709 12:23:56.147192  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.020s	sys 0.038s
W20260709 12:23:56.181586  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.846s	user 0.021s	sys 0.038s
I20260709 12:23:56.150213 11712 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.153894 11735 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.188009 11732 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.188004 11738 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.179366 11711 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:56.212407  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.185148 11723 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:56.215063 11709 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.219107  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.221920 11715 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.223690  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.222667 11740 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.224282  9946 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:56.215205 11730 tablet_copy_client.cc:323] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
I20260709 12:23:56.218686 11732 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.229795  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.219136 11709 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.230834 11733 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.232718 11737 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.227523 11716 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.233585 11735 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.234247 11719 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.235008  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.216696 11705 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:56.241155 11709 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.241818  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.242043 11727 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.221501 11707 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.216936 11712 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:56.248375  9990 connection.cc:256] RPC call timeout handler was delayed by -1.6795116700000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 12:23:56.246209 11732 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.247951  9991 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:56.236338 11731 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:56.292255 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80074.2us
I20260709 12:23:56.292856  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.223475 11715 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.232216 11711 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.297708  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.298430 11711 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.293684  9945 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.245894 11737 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:56.301003 10063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 3.477s	user 0.014s	sys 0.033s
I20260709 12:23:56.301337  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.289377 11723 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:23:56.302351  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54451.8us
I20260709 12:23:56.302717 11737 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.302861 11742 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:56.303261 11737 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:56.303683 11737 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.303769 11723 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.250656 11710 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:23:56.302747 10063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 3.558s	user 0.014s	sys 0.035s
I20260709 12:23:56.303529 11742 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.292555 11681 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.294468 11739 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:56.298979  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.229089 11738 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.298453 11716 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.308661 11727 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.305078 11737 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.295212 11705 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.311960 11739 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:56.308831 11681 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.318748 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.322932 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.316381 11710 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.325136  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.326072  9941 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.327540 11710 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.327708  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.328554  9941 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:56.329893  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.308681 11707 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.331404 11681 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.332125 11707 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.333953 11705 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.321451 11736 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.295526 11740 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.338093  9946 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:56.340296 11740 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:56.342351 10643 connection.cc:256] RPC call timeout handler was delayed by -1.0807844299997669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 12:23:56.337690  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:56.346062 11735 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:56.350047 11735 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.346825 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.342391 11712 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:56.352480 11715 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.350445 11735 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.353574 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.354579 10097 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:56.361271  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:56.365792 10120 debug-util.cc:398] Leaking SignalData structure 0x7b0800108e20 after lost signal to thread 8746
I20260709 12:23:56.350446 11739 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:56.367621  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.368089 11727 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.368520  9941 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:56.383633  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:56.384414  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.370518 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.384742 11705 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:56.399274 11705 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.399739 11705 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.493181  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.494697  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.494827  9949 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
W20260709 12:23:56.502077 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 149776us
I20260709 12:23:56.503087 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.503830 10025 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:56.503952 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.504925 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.507350 11742 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:56.509706 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.525338  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1296) took 1423 ms. Trace:
I20260709 12:23:56.525481 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.526319 10020 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.526021  9944 rpcz_store.cc:276] 0709 12:23:55.101829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.509005 (+1407176us) service_pool.cc:224] Handling call
0709 12:23:56.524452 (+ 15447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.524687 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.524716 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:56.524787 (+    71us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:23:56.524865 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.524943 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.525096 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.527594 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.527890  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.528319 10018 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.534197  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.535408 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.536051 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.536659 10025 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.545678  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.546388  9944 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.557621  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:23:56.559046 11710 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.533589  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.559252 11735 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.560954 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.561534 11719 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:56.564865 11735 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.565418 11705 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.565550 11719 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.567811 11719 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.568069  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.572335 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.573139  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:56.573632 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.568637 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.574467 10094 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:56.569451  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.559660 11710 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.558501  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-35eb6b5f8b8e4d2f8e0a6a08ded4d6a4
I20260709 12:23:56.573760  9949 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.569546 11737 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:56.575706 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.587558 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.586794 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.590757 10096 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:56.586750 11712 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:56.578227 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.596134 10022 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.597903 11705 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.563284 11743 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.588325 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.577060 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.600023 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.599067  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.587178 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.578899 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.584121 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:56.600893  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:56.598832 11743 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:56.605242 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.609218 10023 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:23:56.614681 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:56.615672 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 105652us
I20260709 12:23:56.617054 10017 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:56.618971 10020 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:56.619496 11710 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.620339 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.622114  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1293) took 4500 ms. Trace:
I20260709 12:23:56.622542  9943 rpcz_store.cc:276] 0709 12:23:52.121758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:52.121957 (+   199us) service_pool.cc:224] Handling call
0709 12:23:56.621406 (+4499449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.621669 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.621697 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:56.621739 (+    42us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:23:56.621772 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:56.621824 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:56.621937 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.623852  9948 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a"
dest_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:41138
I20260709 12:23:56.624109  9941 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.624558  9948 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:23:56.624924  9948 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.604471 11712 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:56.628800 11712 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:56.638660 11743 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:56.638294 11597 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.641808 10092 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:56.598801 11737 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:56.648010 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:56.650856  9948 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:56.652444 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.654075  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.659653  9839 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 325) took 1590 ms. Trace:
I20260709 12:23:56.659997  9839 rpcz_store.cc:276] 0709 12:23:55.069260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.115032 (+ 45772us) service_pool.cc:224] Handling call
0709 12:23:56.659498 (+1544466us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":4}
I20260709 12:23:56.655409 10016 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:56.668135 11597 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.669591 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.670349 10092 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:23:56.671303 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.671897 10098 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:56.662621  9959 tablet_copy_source_session.cc:215] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:56.681214  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:56.682893  9940 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.685464  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.686116  9944 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:56.686951  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.687878  9949 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:56.688056  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.688627  9944 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:56.688869  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:56.689455  9949 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:23:56.683888 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.693768 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.694281 10094 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:56.690533  9944 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:56.698969 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.699234 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.700090 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.700783 11738 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:56.701242 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.701697 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.701853 10099 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:56.702853 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.703487 10090 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:56.700173 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.708983  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:56.709865 11738 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:56.710319 11738 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:56.696146  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:56.711459 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.712661  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:56.713464 10642 connection.cc:256] RPC call timeout handler was delayed by -1.4718632540002545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
W20260709 12:23:56.714072 10642 connection.cc:256] RPC call timeout handler was delayed by -1.4300234050001563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
I20260709 12:23:56.713467 11727 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:56.714849 11727 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:23:56.715026 10642 connection.cc:256] RPC call timeout handler was delayed by -1.0020087950001653s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 12:23:56.715231 11727 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:56.717332  9948 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:56.895638  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:41138 (request call id 1299) took 1754 ms. Trace:
I20260709 12:23:56.895998  9948 rpcz_store.cc:276] 0709 12:23:55.141079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.623589 (+1482510us) service_pool.cc:224] Handling call
0709 12:23:56.895517 (+271928us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:56.898217  9940 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:56.811874 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:23:56.898934 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RunLeaderElection -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 246035us
I20260709 12:23:56.893764 11749 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:56.817065 11737 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:56.718263  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:56.719825  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:56.710758 11738 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:56.812633 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.812333 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:56.814059  9941 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:57.060127 10095 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:57.071926 10094 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
W20260709 12:23:57.071933  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 352156us
I20260709 12:23:57.075953 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.061623 11597 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:57.077838 11597 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.080766 11597 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:57.033771 11715 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.091719  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.081887 11740 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:23:57.101795  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 461420us
I20260709 12:23:57.071528 11737 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:23:57.102799 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 420540us
I20260709 12:23:57.103047 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:23:57.113386  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 395140us
I20260709 12:23:57.118012  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.105070 11740 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.102885 11728 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.123247 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.120069 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.106851 11737 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.093771 11749 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.115445 11705 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:57.122905 11740 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.131587  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:57.132490  9987 connection.cc:256] RPC call timeout handler was delayed by -2.8862829470003817s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 12:23:57.125736 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet 230693f05eb74e8095e27d669f16c8ee from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:23:57.136145 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.122853 11710 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.133832 11597 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:57.141266 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet 230693f05eb74e8095e27d669f16c8ee from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-230693f05eb74e8095e27d669f16c8ee
I20260709 12:23:57.141616 11597 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.142076 11597 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.125901 11715 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:57.163663  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78583.3us
I20260709 12:23:57.165405 11759 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.165834 11759 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.166229 11759 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.147522 11736 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:23:57.157522 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 441181us
I20260709 12:23:57.170533 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.171274 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.148204 11728 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.174747 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.178963 10021 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:57.168625 11736 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:23:57.181316 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64719.5us
I20260709 12:23:57.183228 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.188683 11760 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:23:57.189361 10098 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:57.190371 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:57.167702 11710 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:57.174336 11727 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.193246 11758 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:57.193816 11758 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:57.195226 11758 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.175684 11705 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:57.187000 11736 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.220240 11705 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.200405 11749 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:57.236172  8748 debug-util.cc:398] Leaking SignalData structure 0x7b080000fc00 after lost signal to thread 10065
W20260709 12:23:57.236887  8748 kernel_stack_watchdog.cc:198] Thread 10065 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 602ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 12:23:57.239454 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.223842 11758 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.246434 11710 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.249136 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.249707 10109 tablet_copy_source_session.cc:215] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:23:57.251973 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.255791 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.257922  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 281) took 1115 ms. Trace:
I20260709 12:23:57.257643 11597 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.258282  9945 rpcz_store.cc:276] 0709 12:23:56.142613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.142802 (+   189us) service_pool.cc:224] Handling call
0709 12:23:57.257710 (+1114908us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540042de54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:57.257834 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273246080}
W20260709 12:23:57.261504 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68772.8us
I20260709 12:23:57.263590 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:57.264271 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.264599 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.264974 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:57.265699 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:23:57.281762  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82879.1us
I20260709 12:23:57.267270 11727 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:57.267407 10640 connection.cc:256] RPC call timeout handler was delayed by -2.02402383000026s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
W20260709 12:23:57.285132 10640 connection.cc:256] RPC call timeout handler was delayed by -2.00790609600017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 12:23:57.268452 11597 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.275094 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:57.275378 11728 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.277175  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:23:57.270922 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 90988.2us
I20260709 12:23:57.281962 10099 raft_consensus.cc:2376] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3. Re-sending same reply.
I20260709 12:23:57.287509  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.279260 11715 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.293576  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.297799 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.299098 10025 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:57.296576 11597 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:57.278551 11737 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.309785 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:57.306830 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.304847  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.312136 11715 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.312525 11715 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.323673  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.327538 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:57.296870  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:23:57.334303 10670 thread.cc:641] tablet-copy [worker] (thread pool) Time spent creating pthread: real 2.135s	user 0.018s	sys 0.032s
W20260709 12:23:57.334699 10670 thread.cc:608] tablet-copy [worker] (thread pool) Time spent starting thread: real 2.135s	user 0.018s	sys 0.032s
I20260709 12:23:57.335376 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.302026  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:57.341041 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54344.3us
I20260709 12:23:57.296969 11750 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:57.312903 11715 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.343861  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:57.344970 10640 connection.cc:256] RPC call timeout handler was delayed by -1.9829979520000052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
I20260709 12:23:57.344419 11767 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.340412  9940 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.351016  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.352432 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.362035 11772 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.362632 11772 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.363104 11772 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.347078 11767 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.370275 11759 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.371515 10024 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 2: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:57.372488 11759 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.373193 11767 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.372915 11738 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.350735 11727 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.375393 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.368150 11756 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.376803  9941 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:57.377146 11756 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.377518 11756 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.377611  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:57.378218  9945 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:57.376523  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84697.4us
I20260709 12:23:57.379392  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:57.380800  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:23:57.382326 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49716.5us
I20260709 12:23:57.376143 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:57.378152 10120 debug-util.cc:398] Leaking SignalData structure 0x7b080014a0a0 after lost signal to thread 9879
I20260709 12:23:57.383072 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:57.386351  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:23:57.421305  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:57.420691 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.378712 11727 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.424813 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:57.384186 11759 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:57.350564 11750 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.428954 11773 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.391582 11710 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:57.432070 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.432613 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.432789 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.435014 11752 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.435473 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.436300 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.440523 10017 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:57.440979  9945 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:23:57.442055 10022 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:23:57.441573  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 93057.1us
I20260709 12:23:57.426503 11727 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.425123 11765 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:23:57.442370  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58628.5us
I20260709 12:23:57.445082  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:57.430431 11759 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.447342  9945 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.449766 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.437636 11773 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.450707 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.453050 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.442551 11752 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.453132 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.450245 11759 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.470460 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.471189 10017 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:57.474284 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.452042 11773 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.464094 10018 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:57.482038 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.483747 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.483772 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.451731 11737 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.456625 11752 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.477959 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:57.488948 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.497138 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:57.499089 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.500334 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:57.502909 11738 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.505478 11766 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.505968 11766 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.506191  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1167) took 1176 ms. Trace:
W20260709 12:23:57.507390 10120 debug-util.cc:398] Leaking SignalData structure 0x7b08001131e0 after lost signal to thread 9973
I20260709 12:23:57.514251  9944 rpcz_store.cc:276] 0709 12:23:56.329657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.813199 (+483542us) service_pool.cc:224] Handling call
0709 12:23:57.259138 (+445939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.270725 (+ 11587us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:57.270776 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:57.270830 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:57.270858 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.270922 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.505986 (+235064us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b540042de50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:57.506099 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1024922496}
I20260709 12:23:57.568176 11771 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.568672 11771 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.569034 11771 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.506412 11766 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.511472 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2370) took 1174 ms. Trace:
I20260709 12:23:57.571139 10091 rpcz_store.cc:276] 0709 12:23:56.337387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.338762 (+  1375us) service_pool.cc:224] Handling call
0709 12:23:57.500197 (+1161435us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:23:57.511383 (+ 11186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2336000}
I20260709 12:23:57.577682 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 207) took 1074 ms. Trace:
I20260709 12:23:57.578290 11772 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.578096 10016 rpcz_store.cc:276] 0709 12:23:56.503414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:56.503578 (+   164us) service_pool.cc:224] Handling call
0709 12:23:57.577468 (+1073890us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b54001c7354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:57.577576 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":346836864}
W20260709 12:23:57.580782 10120 debug-util.cc:398] Leaking SignalData structure 0x7b080023d480 after lost signal to thread 10049
I20260709 12:23:57.581650  9945 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:57.584578 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:23:57.586040 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 4.834s	user 0.035s	sys 0.044s
W20260709 12:23:57.586547 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 4.835s	user 0.035s	sys 0.044s
W20260709 12:23:57.592028 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 226133us
I20260709 12:23:57.592944  9945 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.597347  9944 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.599185 10017 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.599908  9944 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.602005 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:57.604816  9940 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:23:57.609966  9991 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.611629 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:23:57.630331 10120 debug-util.cc:398] Leaking SignalData structure 0x7b080036d600 after lost signal to thread 10123
W20260709 12:23:57.630702 10120 debug-util.cc:398] Leaking SignalData structure 0x7b0800298180 after lost signal to thread 10124
I20260709 12:23:57.630945  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.631760  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.632102 11747 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.632157 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:57.632599 11747 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.632984 11747 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.633407 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:57.612865 11773 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.614441 11772 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:57.633350  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.595377 11750 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:57.619446  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1298) took 2516 ms. Trace:
I20260709 12:23:57.587440 11740 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:23:57.636296 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 211558us
I20260709 12:23:57.620049 11686 tablet_copy_client.cc:538] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:57.621541  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1326) took 1051 ms. Trace:
I20260709 12:23:57.638356 10017 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.606379 11736 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.612304 11767 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.629508 11715 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.634418 10020 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:57.616211 11756 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.610323 11759 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.626621 11771 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.649273  9947 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.650357 11759 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.650661 11771 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.651145 10025 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:57.637292 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:57.636155  9942 rpcz_store.cc:276] 0709 12:23:55.131490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.973842 (+842352us) service_pool.cc:224] Handling call
0709 12:23:55.974236 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:56.105377 (+131141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:56.105410 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:56.105469 (+    59us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:23:56.105499 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.619010 (+1513511us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b540003eb54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:23:57.619071 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:57.619231 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262191360}
I20260709 12:23:57.621493 11766 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:23:57.639176  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 202154us
I20260709 12:23:57.639706 11749 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:23:57.653659 11749 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:57.654788 11736 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.654554 11749 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.638104  9941 rpcz_store.cc:276] 0709 12:23:56.569878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.075581 (+505703us) service_pool.cc:224] Handling call
0709 12:23:57.075927 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:57.377233 (+301306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:57.621277 (+244044us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54000b6154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:23:57.621481 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685370624}
I20260709 12:23:57.640000  9943 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.657963  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:57.659049  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:57.637741 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:23:57.631769 10120 debug-util.cc:398] Leaking SignalData structure 0x7b08003056e0 after lost signal to thread 10702
W20260709 12:23:57.641635 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 193168us
I20260709 12:23:57.668036 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:23:57.668390 10120 debug-util.cc:398] Leaking SignalData structure 0x7b08002e7d60 after lost signal to thread 10703
I20260709 12:23:57.668982 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:57.639933  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.636688 11751 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.641306 11597 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:57.669826 11751 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.670051 11597 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:57.636689 11708 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:57.644932  9948 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.670356  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.634979 11743 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:57.646857  9946 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:57.671455 11743 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.648147 10021 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:57.660890 11705 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:57.660996  9949 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:57.661341 11681 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.661442 10017 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.661437  9940 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:57.660347  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:57.662422  9941 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:57.657814 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:57.676715 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:57.680225 10098 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.662882 10020 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.663442 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.683547 11737 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:57.663676 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.662915 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.667021 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.684083 11737 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:57.684548 10025 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:57.667702 11766 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.667740 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.685077 11737 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.669234  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:57.671178 11597 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:57.638449 11765 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.690996 11765 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.691432 11765 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.691524 10098 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:23:57.638257 11768 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.692873  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.692982 11768 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.641592 11750 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:57.677896 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:23:57.670293 11751 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.697999  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:57.703184 11778 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.703681 11778 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:57.704056 11778 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.678689 11708 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:57.709365  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.712975 10022 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:57.713268  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:57.714075 11763 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:57.714288 11762 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:57.714229 11764 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:23:57.711422 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61414.1us
I20260709 12:23:57.674137 11705 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.693513 11768 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.718433 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:57.672009 11779 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:23:57.679625 11681 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:57.699445 11772 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.712225 11710 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.725346 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:57.742978 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:23:57.750535 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:57.718704 11763 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:57.725353  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.721603 11774 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:57.724043 11705 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.721702 11766 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.731441 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:57.720976  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:57.722169 11756 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.715994 11759 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:23:57.734097 11681 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.724462 11762 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:57.768949  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.746235 11710 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:23:57.771310  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50402.8us
I20260709 12:23:57.772094 11753 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:23:57.772787 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47532us
I20260709 12:23:57.744051 11772 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.780557 11773 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:57.782038  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.744465 11727 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.717590 11750 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.783432  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:57.721740 11769 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 12:23:57.773809 11766 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.793658  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.779187 11710 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.795413 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:57.799841 11777 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:57.784505 11771 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:23:57.801043 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57365.4us
I20260709 12:23:57.781927  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.797748  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.787273 11772 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.803583 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:57.812321  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:57.807253 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:57.784228 11774 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:23:57.807384 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64490.9us
I20260709 12:23:57.810672 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:57.790227 11764 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:57.819262  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:57.792557 11753 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:23:57.819653 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 76441.2us
I20260709 12:23:57.819933 11753 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.819686 11764 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.823333 11764 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.817346  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.772760 11708 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.795073 11766 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.846618 11792 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.847132 11792 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.847597 11792 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.846448 11761 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:57.855221 11769 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:57.835300 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:23:57.804126 11771 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:23:57.867136 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46177us
I20260709 12:23:57.868685  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:57.818735 11740 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:57.873960 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63420.2us
I20260709 12:23:57.870025  9943 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
W20260709 12:23:57.876458 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81115.2us
I20260709 12:23:57.876696 11730 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:57.865655 11743 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:57.869459 11771 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.866854 11761 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:57.868333 11767 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.897285 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:57.892009 11773 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:23:57.903121 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:57.871862 11781 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:23:57.898816 11782 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.126s
I20260709 12:23:57.902782 11763 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:57.921361 11763 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:23:57.922770 11785 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:23:57.923717 11738 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:57.924153 11727 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:57.924256 10674 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:57.912708 11773 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:57.924506 11738 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.913198 11791 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.925882 11781 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:23:57.904606 11681 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:23:57.959103  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 256307us
W20260709 12:23:57.960879  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 141886us
I20260709 12:23:57.903986 11743 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:23:57.963114  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:57.879719 11762 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.950570 11773 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:57.914077 11780 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:57.948954 11767 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:57.972826  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:23:57.967672 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79543.1us
I20260709 12:23:57.955776 11761 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:57.975978 11770 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:57.981010  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:57.962433 11781 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:57.922323 11763 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:57.949235 11747 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.992409 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:57.998831 11763 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.995445 11770 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:57.964979 11791 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:57.969686 11750 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:57.976130 11761 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:57.978150 11782 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:23:57.992424 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.016738 11789 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:58.008435 11770 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:58.022295  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:58.023406 11795 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:58.008558 11791 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.024725  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:23:58.025830  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 79100.7us
I20260709 12:23:58.028082  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:58.029945  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:57.910982 11774 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.007117 11767 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.045094 11686 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:23:58.034767  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:58.033205 11704 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:58.027628 11740 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:58.024289 11770 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:57.992203 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:23:58.031633 11784 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:23:58.055635 11704 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:58.056216 11704 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:58.032829 11795 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:58.049427 11777 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:58.052011 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.070073 10675 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:58.070290 11704 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.071213 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.071548 11768 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.073652 11778 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.070127 10673 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:58.103462 11738 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:58.121611 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.076468 11756 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:23:58.102916 11787 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:58.124123 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.130658 10022 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:23:58.066036 11790 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 12:23:58.133930 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:58.131762 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:58.134931 10093 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:58.134827 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:58.114892 11747 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.139678 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.143828 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:58.141763 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.150691 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:58.156733 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:58.137405 11743 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:58.143343 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.145526 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:58.170409 10099 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:58.169492 11753 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.170847 10099 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:58.173671 10099 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.174510 11753 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.175035 10671 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:58.175225 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.158886 11743 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.155974 11747 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:58.192757 11743 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:58.193531 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:58.194175 10095 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.213798  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:58.217561 11743 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:58.218144 11743 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.200083 11713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 230693f05eb74e8095e27d669f16c8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:23:58.190380 11767 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:58.179983 11770 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.212647 11778 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.175968 11800 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.164s
I20260709 12:23:58.156651 11795 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.243853 11795 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.247547 11743 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:58.278470 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:58.325250 10092 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:58.275589 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:58.289939 11797 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:58.292191 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:58.295531 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:58.314709  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:58.305433 11705 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.004s
I20260709 12:23:58.313143 11772 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.313858  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:58.315349 11681 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:58.316149  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:58.316411 11766 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.317509 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.353274 10671 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:58.239872 11761 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:58.308162 11791 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.326436 10093 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.329650  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:58.357935  9948 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.333695  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:58.334131  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:58.359442 10670 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:58.355923 11794 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.136s
W20260709 12:23:58.360909 11764 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.001s
I20260709 12:23:58.360991 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.361334 11768 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:58.362067 11728 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:58.362172 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.362749 11789 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:58.304502 11715 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.343433  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:58.332474  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:58.305040 11781 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.346959 11705 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.355203 10095 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:58.356184 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:23:58.364454 11710 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.002s
I20260709 12:23:58.362473 11728 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:58.363240 11750 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:58.387475 11750 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:23:58.372615 11788 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.403s	user 0.002s	sys 0.000s
I20260709 12:23:58.373711 10669 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:23:58.392369  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.015s	sys 0.022s
I20260709 12:23:58.375844 10668 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:58.376757 11791 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:58.523161 11765 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.831s	user 0.007s	sys 0.007s
W20260709 12:23:58.385444  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53059.2us
W20260709 12:23:58.334565 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 465469us
W20260709 12:23:58.515741 11751 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.794s	user 0.007s	sys 0.007s
I20260709 12:23:58.361353 11764 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:23:58.449265  8748 debug-util.cc:398] Leaking SignalData structure 0x7b080000f500 after lost signal to thread 10065
I20260709 12:23:58.388159 11715 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:58.449594 10673 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:58.531854 11772 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.375532 11789 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:58.389181 11710 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.388468 11750 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:23:58.497526  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 272379us
W20260709 12:23:58.498359  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 258173us
I20260709 12:23:58.537900  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:23:58.529098 11771 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.625s	user 0.012s	sys 0.008s
I20260709 12:23:58.388675 10675 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:58.518930 10093 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:23:58.520546  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.015s	sys 0.022s
W20260709 12:23:58.521054  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.001s	sys 0.032s
W20260709 12:23:58.549126  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.001s	sys 0.032s
W20260709 12:23:58.521543 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 370066us
W20260709 12:23:58.446298 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.010s	sys 0.031s
W20260709 12:23:58.550056 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.010s	sys 0.032s
W20260709 12:23:58.550498 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 677974us
I20260709 12:23:58.523597 11765 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.516813 11760 raft_consensus.cc:2764] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
W20260709 12:23:58.519634 11763 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.520s	user 0.015s	sys 0.004s
W20260709 12:23:58.532336 11773 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.549s	user 0.017s	sys 0.010s
I20260709 12:23:58.531782 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.534510 11730 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:42028 (request call id 543) took 3351 ms. Trace:
I20260709 12:23:58.530418  9948 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
W20260709 12:23:58.390946 11752 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.889s	user 0.007s	sys 0.007s
I20260709 12:23:58.529964 11751 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.520926 11766 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:58.544950 10670 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:23:58.549468  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 587540us
I20260709 12:23:58.570503 11805 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:23:58.597103 11805 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.598739 10671 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:58.585652 11790 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
W20260709 12:23:58.587126  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:58.609897 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.613646 10671 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:58.613997 10099 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.615286 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.616611  9946 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:58.547686 11781 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.627619  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:58.543922 11770 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.658991  9948 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:58.562566 11773 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.668350  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:58.669135 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:58.670006 11773 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.670094  9949 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:23:58.670730 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.670853 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.672997 10676 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:58.673748  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:58.675158  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:58.670369 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.701098 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.674706 11705 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:58.566941 11730 rpcz_store.cc:276] 0709 12:23:55.183212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.187641 (+  4429us) service_pool.cc:224] Handling call
0709 12:23:56.124971 (+937330us) ts_tablet_manager.cc:934] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
0709 12:23:57.791857 (+1666886us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:23:58.291115 (+499258us) tablet_metadata.cc:665] Metadata flushed
0709 12:23:58.291243 (+   128us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:23:58.533525 (+242282us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:23:58.533966 (+   441us) tablet_replica.cc:206] Creating consensus instance
0709 12:23:58.534344 (+   378us) ts_tablet_manager.cc:989] Replica registered.
0709 12:23:58.534415 (+    71us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":54807,"tablet-copy.queue_time_us":909241,"thread_start_us":2155561,"threads_started":2}
I20260709 12:23:58.596263 11752 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.725440 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.709493 11790 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:58.725611  9947 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:58.726581  9946 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:58.727329 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:58.546482 11789 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.537952 11728 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.730448  9948 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.647845 11750 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:58.554142 11771 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.721491 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:58.739435 11795 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.048s
I20260709 12:23:58.731271 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:58.793104 10099 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:58.702576 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:23:58.775759 11792 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.928s	user 0.008s	sys 0.017s
I20260709 12:23:58.775744 11730 tablet_copy_client.cc:806] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:58.606673 11803 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.194s
I20260709 12:23:58.841270 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:23:58.843495 11802 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:23:58.814098 11749 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:58.810509 11790 consensus_queue.cc:1243] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:23:58.836400 11751 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:58.861651 11730 tablet_copy_client.cc:670] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:58.904386 11710 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:58.904639 11752 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.881829 11749 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:58.930974 11782 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:23:58.939927 11782 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:58.940524 11710 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:58.838869 11792 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:58.961447 10672 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:58.961589 11812 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.429s	user 0.001s	sys 0.001s
I20260709 12:23:58.962425 11749 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.963608 11749 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:58.941053 11782 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:58.980183 11782 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:23:58.978858 11809 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:23:58.960999 11765 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:58.945735 11705 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 12:23:58.986403 11765 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:58.986843 11765 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:58.987681 11765 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.012403 11792 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:23:59.058784  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:23:59.061380  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54205.6us
W20260709 12:23:59.064848  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.065289 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 403) took 1094 ms. Trace:
I20260709 12:23:59.063375 11771 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:59.066435 10677 rpcz_store.cc:276] 0709 12:23:57.899328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.910412 (+ 11084us) service_pool.cc:224] Handling call
0709 12:23:58.992797 (+1082385us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:23:58.993010 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:58.993634 (+   624us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:58.993658 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:58.993696 (+    38us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:58.993739 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:58.993792 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:58.993930 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5182848}
W20260709 12:23:59.066816  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.008s	sys 0.011s
W20260709 12:23:59.069265  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.008s	sys 0.011s
I20260709 12:23:59.083745 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.084431 10016 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:23:59.087810 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 414) took 1018 ms. Trace:
I20260709 12:23:59.090077 10677 rpcz_store.cc:276] 0709 12:23:58.071764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.070218 (+998454us) service_pool.cc:224] Handling call
0709 12:23:59.087720 (+ 17502us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.092411 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:23:59.099761 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.019s
W20260709 12:23:59.100873 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.004s	sys 0.019s
W20260709 12:23:59.101305 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.002s	sys 0.020s
W20260709 12:23:59.102581 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.003s	sys 0.021s
W20260709 12:23:59.100991  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 165694us
W20260709 12:23:59.101303 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 539193us
W20260709 12:23:59.100462 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.006s	sys 0.013s
W20260709 12:23:59.103593 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 552939us
W20260709 12:23:59.104161 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.006s	sys 0.014s
I20260709 12:23:59.103799 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:23:59.104481 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 556041us
I20260709 12:23:59.105094 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:59.118969 11686 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:23:59.156256  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.129406 11758 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.212225 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.273108 11758 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:59.273577 11686 tablet_bootstrap.cc:492] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:23:59.273886 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 416) took 1165 ms. Trace:
I20260709 12:23:59.274022  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:59.120524 11713 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:42034 (request call id 475) took 4024 ms. Trace:
I20260709 12:23:59.107923 11792 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:59.124670 11816 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:59.259007 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.115054 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:59.222146 11789 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.246498 10093 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:59.174631 11819 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:59.276154 11686 ts_tablet_manager.cc:1403] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 1.231s	user 0.052s	sys 0.048s
I20260709 12:23:59.278012 11819 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.108336 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:23:59.279438 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 171664us
I20260709 12:23:59.280303 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.280553 11819 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:59.183005 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 412) took 1146 ms. Trace:
I20260709 12:23:59.281951 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:23:59.281659 11686 raft_consensus.cc:359] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:59.282908 11804 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:59.139734 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 418) took 1031 ms. Trace:
I20260709 12:23:59.274928 11713 rpcz_store.cc:276] 0709 12:23:55.120205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.123086 (+  2881us) service_pool.cc:224] Handling call
0709 12:23:55.155528 (+ 32442us) ts_tablet_manager.cc:934] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
0709 12:23:58.017712 (+2862184us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:23:58.780695 (+762983us) tablet_metadata.cc:665] Metadata flushed
0709 12:23:58.780823 (+   128us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:23:59.103610 (+322787us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:23:59.119672 (+ 16062us) tablet_replica.cc:206] Creating consensus instance
0709 12:23:59.120379 (+   707us) ts_tablet_manager.cc:989] Replica registered.
0709 12:23:59.120458 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":41410,"tablet-copy.queue_time_us":28709,"thread_start_us":44338,"threads_started":2}
I20260709 12:23:59.196074 11749 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 12:23:59.282763 11792 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:59.170074 10022 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:23:59.108271  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.245147 10099 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:59.223435 11764 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:59.283916 11713 tablet_copy_client.cc:806] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:23:59.216485 11728 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.285811 11713 tablet_copy_client.cc:670] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:23:59.182124 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.286887 10674 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.287305 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 410) took 1275 ms. Trace:
I20260709 12:23:59.287670 10674 rpcz_store.cc:276] 0709 12:23:58.011811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.181752 (+1169941us) service_pool.cc:224] Handling call
0709 12:23:59.287233 (+105481us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.157665 11765 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.287110 10670 rpcz_store.cc:276] 0709 12:23:58.108810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.361943 (+253133us) service_pool.cc:224] Handling call
0709 12:23:59.139528 (+777585us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54006d01d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:59.139668 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":395764992}
I20260709 12:23:59.289398 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.276383 11816 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:59.144043 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 477) took 1266 ms. Trace:
I20260709 12:23:59.270340  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:59.115567 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.291906 11760 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:59.293339  9948 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:59.293685 11738 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:59.293941 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 545) took 1270 ms. Trace:
I20260709 12:23:59.294359 10677 rpcz_store.cc:276] 0709 12:23:58.024454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.114993 (+1090539us) service_pool.cc:224] Handling call
0709 12:23:59.293874 (+178881us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.195844 11807 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 12:23:59.295320 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.295859 11779 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.296041  9948 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:59.296162 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 555) took 1096 ms. Trace:
I20260709 12:23:59.216423  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:59.292761 10676 rpcz_store.cc:276] 0709 12:23:57.877653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.885389 (+  7736us) service_pool.cc:224] Handling call
0709 12:23:59.143739 (+1258350us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b540073afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:59.143934 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776395136}
I20260709 12:23:59.126458 11805 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:59.297556 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.127907 11710 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.170s
I20260709 12:23:59.298759 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.108517 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:59.248533 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.303050 10095 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.303376 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:59.305569  9946 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:23:59.199077 11794 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 12:23:59.274454 10668 rpcz_store.cc:276] 0709 12:23:58.108061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.211827 (+1103766us) service_pool.cc:224] Handling call
0709 12:23:59.273749 (+ 61922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.308455 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:59.309258 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.309808 10095 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:59.308486 10676 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.130373 11795 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.153s
I20260709 12:23:59.176080 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.284544 10022 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:23:59.212880 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.019s	sys 0.007s
I20260709 12:23:59.293829 11760 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.121423 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 478) took 1243 ms. Trace:
I20260709 12:23:59.296303 11779 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:59.184240 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.311450 10672 rpcz_store.cc:276] 0709 12:23:57.877973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:57.878116 (+   143us) service_pool.cc:224] Handling call
0709 12:23:57.886112 (+  7996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:58.961930 (+1075818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.121107 (+159177us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400610dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:23:59.121349 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2459686144}
I20260709 12:23:59.299077  9940 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:59.299199 11705 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:59.312021 11779 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.312645 11705 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:59.312798 11760 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:23:59.311471 10673 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.313258 11779 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:23:59.313512 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.019s	sys 0.007s
I20260709 12:23:59.313701 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 557) took 1068 ms. Trace:
I20260709 12:23:59.313751 11705 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.314307 10673 rpcz_store.cc:276] 0709 12:23:58.245779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.175767 (+929988us) service_pool.cc:224] Handling call
0709 12:23:59.313578 (+137811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.314872 10022 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.315728 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:59.299753 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 551) took 1150 ms. Trace:
I20260709 12:23:59.319110 10670 rpcz_store.cc:276] 0709 12:23:58.149253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.289101 (+1139848us) service_pool.cc:224] Handling call
0709 12:23:59.299673 (+ 10572us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.297626  9949 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:59.300469 11774 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:59.301009 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:59.336241 10092 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:23:59.336628 10019 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.338750 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.302201 11807 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:59.135387 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 421) took 1025 ms. Trace:
I20260709 12:23:59.282140 10675 rpcz_store.cc:276] 0709 12:23:58.036675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.070919 (+ 34244us) service_pool.cc:224] Handling call
0709 12:23:59.182716 (+1111797us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400528fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:59.182924 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":708240256}
I20260709 12:23:59.115888 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:23:59.070974  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 542072us
I20260709 12:23:59.297355 10677 rpcz_store.cc:276] 0709 12:23:58.200272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.295055 (+1094783us) service_pool.cc:224] Handling call
0709 12:23:59.296108 (+  1053us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.291530 10024 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.141784  9839 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:23:59.312737 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.274133 11758 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.189236 11743 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.243644 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:23:59.313905 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 747666us
I20260709 12:23:59.283430 11686 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:23:59.347203 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.348042 10670 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.318434 10018 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.364338 10017 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.364542 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.365381 10673 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.316025 10022 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.289990 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.367834 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 554) took 1186 ms. Trace:
I20260709 12:23:59.368223 10674 rpcz_store.cc:276] 0709 12:23:58.181691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.289743 (+1108052us) service_pool.cc:224] Handling call
0709 12:23:59.367776 (+ 78033us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.339507 10676 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.340373 10020 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:59.311874 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 558) took 1025 ms. Trace:
I20260709 12:23:59.111831 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:59.370054 10016 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.373790 10019 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:23:59.128957 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.115567 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:23:59.343081  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:23:59.348250 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 66374.5us
I20260709 12:23:59.349682 11787 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:23:59.369765 10671 rpcz_store.cc:276] 0709 12:23:58.286956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.183887 (+896931us) service_pool.cc:224] Handling call
0709 12:23:59.311813 (+127926us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.341223 11807 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:59.375944 11787 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.376478 11807 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.377303 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.377883 11807 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.378104 10672 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.378206 11787 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:59.349881 11686 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:23:59.351220 11749 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:23:59.379093 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.351977 11710 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:23:59.379908 10677 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.339772 11786 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:23:59.352648 11760 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:23:59.382093 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.358997 10091 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.358712 11814 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:23:59.359660 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.402570 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.402868  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:59.403296 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.403766 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.404109 10675 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.404847 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.382583 11792 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:59.340245 11794 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.339741 10669 rpcz_store.cc:276] 0709 12:23:58.109875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.174904 (+ 65029us) service_pool.cc:224] Handling call
0709 12:23:59.135113 (+960209us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54007f2e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:59.135308 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":454574976}
W20260709 12:23:59.387915  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44926.6us
I20260709 12:23:59.417773  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:23:59.389413  9943 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:23:59.383898 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.391992 10676 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:23:59.390501 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 261614us
I20260709 12:23:59.343381 11743 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:59.383607 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.393283 10093 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:23:59.405845 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.424335 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.310690 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:23:59.407037  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 65150.2us
W20260709 12:23:59.411296 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64436.3us
I20260709 12:23:59.425139 10672 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.423538 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.405004 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:59.435664 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.437430 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.399662 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.443338 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:59.583861  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:59.585392  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:23:59.463464 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62914.9us
I20260709 12:23:59.587903 11793 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:23:59.505362  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.509008 10024 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:23:59.588438 11793 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.588703 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:59.545619 10668 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.400976 11795 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:23:59.554199  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:23:59.590544 11793 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:59.509761 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:59.475006 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:59.468840 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:59.467557 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:23:59.593114 10673 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.593164 10025 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:59.591612 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.594266 11793 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:23:59.594306 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0659924259998661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
I20260709 12:23:59.469139 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.595989 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.596240  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:59.597731 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.598546  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:59.438488 11794 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.600584 10020 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:23:59.601291 11794 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:23:59.602159 10018 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.471200 10021 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:23:59.602769  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:59.603371 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.603747  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:59.604034 10672 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:23:59.591786 10023 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:23:59.464161  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:23:59.592352 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 168133us
I20260709 12:23:59.606963 10065 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.592141 10676 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.392261 11760 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:59.590642 10024 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
W20260709 12:23:59.609067  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 145029us
I20260709 12:23:59.614290 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.609725  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1345) took 1007 ms. Trace:
I20260709 12:23:59.616029  9943 rpcz_store.cc:276] 0709 12:23:58.601999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.602330 (+   331us) service_pool.cc:224] Handling call
0709 12:23:59.609578 (+1007248us) spinlock_profiling.cc:243] Waited 664 ms on lock 0x7b54003a0554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:59.609675 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1528022272}
I20260709 12:23:59.620312 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.621608 10096 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.622563 11787 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:23:59.623101 11787 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:59.609548 10019 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.623507 11787 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.625481 11686 ts_tablet_manager.cc:1434] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.347s	user 0.003s	sys 0.006s
I20260709 12:23:59.626901 11802 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:59.613149  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:23:59.443368 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54178.5us
I20260709 12:23:59.610994 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.611656 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.631374 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:59.623180  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:23:59.628795  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:23:59.610108  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56028.2us
I20260709 12:23:59.467088 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.617034  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:23:59.408046 11764 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.259s
W20260709 12:23:59.713330  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.670079  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:23:59.632887 10016 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:59.661721 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:23:59.434885 11710 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.638764 10677 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.716614 10674 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.715662 10672 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:59.637863 10670 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:59.608891 11761 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:59.468897 11792 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:59.719524 10017 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:59.711972 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 97759.8us
I20260709 12:23:59.466609 11805 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:59.721848  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.727216 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 547) took 1660 ms. Trace:
I20260709 12:23:59.728229 10097 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.728768 10672 rpcz_store.cc:276] 0709 12:23:58.066698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.715115 (+1648417us) service_pool.cc:224] Handling call
0709 12:23:59.715502 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.726908 (+ 11406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.727147 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.392307 11804 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.321s
I20260709 12:23:59.731256 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.607991 11819 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.717918 11758 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.739120 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.739934 10674 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:23:59.741369 11805 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:59.612507 11797 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:59.735419 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 411) took 1708 ms. Trace:
I20260709 12:23:59.655251 11790 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:59.739355 11814 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:59.744208 10676 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:59.745069 10670 rpcz_store.cc:276] 0709 12:23:58.020009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.637343 (+1617334us) service_pool.cc:224] Handling call
0709 12:23:59.637680 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.727727 (+ 90047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.728311 (+   584us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:23:59.738489 11681 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
W20260709 12:23:59.753010  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 140155us
I20260709 12:23:59.746570 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 420) took 1637 ms. Trace:
I20260709 12:23:59.468930 11725 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.258s
I20260709 12:23:59.750571 10670 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
W20260709 12:23:59.737607  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 120653us
I20260709 12:23:59.748487 11764 consensus_queue.cc:1243] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:23:59.751235 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:23:59.749152  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 163865us
I20260709 12:23:59.799861  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:59.801682  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:59.732028 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 483) took 1758 ms. Trace:
I20260709 12:23:59.805838 10677 rpcz_store.cc:276] 0709 12:23:57.964786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.630201 (+1665415us) service_pool.cc:224] Handling call
0709 12:23:59.630584 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.722473 (+ 91889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.722685 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.748437  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:23:59.728116 11760 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:23:59.753729 10671 rpcz_store.cc:276] 0709 12:23:58.109532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.686522 (+1576990us) service_pool.cc:224] Handling call
0709 12:23:59.736714 (+ 50192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.746234 (+  9520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.746262 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:23:59.746317 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:23:59.746347 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.746403 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:23:59.746515 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.752677 10669 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:23:59.773905  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:59.776903 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.762913 10668 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:59.747203 11765 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:23:59.762753 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 131471us
I20260709 12:23:59.846750 11727 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:59.776371 11814 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:23:59.776170 11756 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:23:59.789469 10675 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:59.790711 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.791417 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.860368 10090 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.792354 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.761806 11819 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.794330 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 552) took 1629 ms. Trace:
I20260709 12:23:59.864583 10670 rpcz_store.cc:276] 0709 12:23:58.165293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.750094 (+1584801us) service_pool.cc:224] Handling call
0709 12:23:59.750390 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.784890 (+ 34500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.794280 (+  9390us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.752204 11762 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:59.872532 11819 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:59.873386 11819 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:23:59.794561 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:59.786846 11805 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:23:59.815572 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.830883  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:59.832154 11782 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:23:59.833257 11738 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:23:59.833451  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:23:59.833739 11777 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:23:59.833884 11802 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:23:59.835810  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62020.6us
I20260709 12:23:59.836531 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.724542  9941 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.843426 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:23:59.843568  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 95245.8us
I20260709 12:23:59.663139 11816 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.880872 11782 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.655670 11743 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.886344 11782 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:59.886968 11743 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.887298 10673 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.883092 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.888005 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 422) took 1777 ms. Trace:
I20260709 12:23:59.888403 10673 rpcz_store.cc:276] 0709 12:23:58.110217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.886758 (+1776541us) service_pool.cc:224] Handling call
0709 12:23:59.887100 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.887735 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.887942 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.889096 10670 raft_consensus.cc:1217] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:23:59.885941 10677 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:59.881769  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:59.890235 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 563) took 1517 ms. Trace:
I20260709 12:23:59.890537 10677 rpcz_store.cc:276] 0709 12:23:58.372963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.885466 (+1512503us) service_pool.cc:224] Handling call
0709 12:23:59.885767 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.889989 (+  4222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.890163 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.874711 11805 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.876983 11764 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:59.892903 11805 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:23:59.893410 11822 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:23:59.894008 11822 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:23:59.894891 11822 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.847234 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.898109 10092 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.849467 10672 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:23:59.751240 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 548) took 1674 ms. Trace:
I20260709 12:23:59.898757 10676 rpcz_store.cc:276] 0709 12:23:58.076735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.716167 (+1639432us) service_pool.cc:224] Handling call
0709 12:23:59.744032 (+ 27865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.751008 (+  6976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.751187 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.778995 11793 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:23:59.907022 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.907608 10097 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.848938 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:23:59.855274 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.836695 11761 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:23:59.875322 10093 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:35180
W20260709 12:23:59.876192 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60681.1us
I20260709 12:23:59.910213 10093 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:23:59.878753 11792 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.879742 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 544) took 1860 ms. Trace:
I20260709 12:23:59.850669 11705 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:23:59.876924  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46149.3us
I20260709 12:23:59.910692 10093 raft_consensus.cc:614] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:23:59.911788 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62921.9us
I20260709 12:23:59.883925 11802 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:59.932955 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:23:59.933236  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:59.937333 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 561) took 1569 ms. Trace:
I20260709 12:23:59.937856 10672 rpcz_store.cc:276] 0709 12:23:58.367556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.788601 (+1421045us) service_pool.cc:224] Handling call
0709 12:23:59.845616 (+ 57015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.906480 (+ 60864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.937094 (+ 30614us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b5400545bd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:23:59.937263 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130205568}
I20260709 12:23:59.940395 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:23:59.886296 11816 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:23:59.946141  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:59.946571  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:59.947332 11727 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:59.948531 10096 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:23:59.896977  9941 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:23:59.913036 11768 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:59.934711 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 553) took 1750 ms. Trace:
I20260709 12:23:59.955782  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:23:59.981218  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:59.981407 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.982002 10672 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:23:59.982363  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:23:59.957234  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:23:59.931638 11764 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:23:59.953178 10093 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:23:59.874585 11819 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:23:59.893417 11805 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:23:59.945904 11766 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:23:59.934861 11802 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:59.986884  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:23:59.947975 11826 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.150s	user 0.002s	sys 0.001s
I20260709 12:23:59.862097 11760 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.947736 11793 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:59.948187 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:23:59.992336  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:00.003953  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:00.005134  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:23:59.906222 11777 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:23:59.978056 11768 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:23:59.911835 10668 rpcz_store.cc:276] 0709 12:23:58.019743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.655539 (+1635796us) service_pool.cc:224] Handling call
0709 12:23:59.729822 (+ 74283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.848456 (+118634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.867937 (+ 19481us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:23:59.980726 11730 tablet_copy_client.cc:538] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:23:59.887323 11782 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.975126 11749 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:59.968199 11789 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:23:59.974918 11738 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:23:59.994552 11681 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:59.953918 11705 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:00.009929 10091 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.000s
I20260709 12:23:59.912062 11807 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.078s
I20260709 12:23:59.989256 11797 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.989949 11805 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:23:59.991032 11756 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:23:59.996233 11727 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:23:59.989435 11814 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:00.115054 11829 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.320s	user 0.002s	sys 0.000s
I20260709 12:23:59.991454 11766 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:00.120042 11766 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.121486 11766 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.121636 11777 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:00.125053 11787 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
I20260709 12:24:00.125546 11787 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.128430 11802 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.129648 10095 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:00.129880 11802 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.133327 11743 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:00.141480 11760 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:00.050355 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.675s (SENT) [suppressed 38 similar messages]
I20260709 12:24:00.008241 11794 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:00.118167 11782 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.033654 11762 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.072480 10091 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:00.098973 10017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:00.100539 11765 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:00.101495 11793 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:23:59.980310 10675 rpcz_store.cc:276] 0709 12:23:58.181299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.784589 (+1603290us) service_pool.cc:224] Handling call
0709 12:23:59.789287 (+  4698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:23:59.878920 (+ 89633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:23:59.931879 (+ 52959us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:00.102716 11768 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.161154 11768 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.103804 11749 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:00.143378  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:00.163568  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 181266us
I20260709 12:24:00.030536 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:00.144552  9943 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:00.147815 10090 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
W20260709 12:24:00.149108 11795 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.557s	user 0.002s	sys 0.007s
I20260709 12:24:00.148365 10097 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
W20260709 12:24:00.088016 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 356809us
I20260709 12:24:00.155015  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:00.141482 11738 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.156081 11790 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:00.157409 10017 raft_consensus.cc:606] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:00.169564 10017 raft_consensus.cc:614] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:00.167261  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:00.134107 11743 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:00.165526 10098 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:00.147699 10092 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
W20260709 12:24:00.165643  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 264964us
I20260709 12:24:00.168318 10096 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:00.168774 10065 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:00.168393  9947 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:00.172904 10093 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:00.183277  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:00.184556 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:00.173144 11795 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:00.170852 11790 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:00.189278 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 333175us
W20260709 12:24:00.218818  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 212848us
I20260709 12:24:00.186458 10673 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:00.184113 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 269892us
W20260709 12:24:00.215183  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47984.4us
I20260709 12:24:00.223158  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:00.233803  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:00.230206  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:00.230247 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:00.247062 11764 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:00.247818 11764 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:00.172322 11738 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:00.219168 11727 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.221094 11761 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:24:00.270519 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 101726us
I20260709 12:24:00.271719 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:00.224844 11762 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.265053 11835 ts_tablet_manager.cc:933] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
I20260709 12:24:00.291836 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:00.293762 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 71088.8us
I20260709 12:24:00.250077 11836 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.249907 11795 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.223402 11789 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:24:00.249010 11764 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.288451 11822 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:00.326817  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 93127.2us
I20260709 12:24:00.311818 11787 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:00.316643 11749 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.334721 10093 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:00.333272 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:00.336306 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:00.319334 11807 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:00.336870  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 106748us
I20260709 12:24:00.242935 11705 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.340409  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:00.342445  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:00.328804 11681 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:00.321616 11804 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:24:00.318027 11848 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:24:00.352082 11747 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:00.318861 11836 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:00.351078 11821 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:00.311224 11790 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.354702 11840 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:00.372509 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.357345 11822 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:00.399506 10098 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:00.388015 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 116381us
I20260709 12:24:00.401149 10094 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:00.405169 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:00.397460 11840 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:00.397733 11747 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:00.404913 11816 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.371747 11787 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:24:00.408998 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:00.413013 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45227.5us
I20260709 12:24:00.407723 11738 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:00.429448  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.409786 10098 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:00.367013 11681 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:00.365513 11842 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:24:00.399567 11822 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:00.403929 11819 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.369807 11807 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:24:00.401674 11743 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:00.406453 11840 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:00.410284 11710 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.000s
I20260709 12:24:00.373481 11777 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:24:00.455062 10676 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:00.449445 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:00.367233 11837 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 12:24:00.449280 11822 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:00.404881 11789 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
W20260709 12:24:00.467541 11844 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:24:00.468369 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:00.468859 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.481823 10035 tablet_copy_service.cc:342] P 9aa345bcd43b44088114b7b6f2830b17: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-a7196ffc873e4e7e84c76068d5b95904 received from {username='slave'} at 127.0.0.1:36302
I20260709 12:24:00.495775 10035 tablet_copy_service.cc:434] P 9aa345bcd43b44088114b7b6f2830b17: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-a7196ffc873e4e7e84c76068d5b95904 on tablet a7196ffc873e4e7e84c76068d5b95904 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:00.497571 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89019.7us
I20260709 12:24:00.498373 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:00.461938 11832 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:24:00.499822  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:00.457911 11710 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.454562 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:00.454612 11819 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.514165 11743 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:00.518237 11760 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:00.541991 11760 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:00.543256 11760 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.544286 11761 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:00.544888 11761 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:00.551985 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:00.558792 11834 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.254s	user 0.002s	sys 0.000s
I20260709 12:24:00.537566 11743 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:00.614919 11743 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:00.619153 11819 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:00.526953 11842 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.046s
I20260709 12:24:00.502182 11747 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.635943 11835 tablet_copy_client.cc:323] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.193:34909
W20260709 12:24:00.642030 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.549328 11805 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.654438 11851 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:00.654877 11730 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:24:00.655136 11851 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:00.627385  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1361) took 1278 ms. Trace:
I20260709 12:24:00.545936 11761 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.665573 11851 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.717155 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1353) took 1435 ms. Trace:
I20260709 12:24:00.720368 11836 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:00.717778 10094 rpcz_store.cc:276] 0709 12:23:59.251308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.251504 (+   196us) service_pool.cc:224] Handling call
0709 12:23:59.251949 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:00.401550 (+1149601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.670502 (+268952us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002fbf54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:00.686442 (+ 15940us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2636233088}
I20260709 12:24:00.724344 11761 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.726616 11805 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.686498  9843 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 3 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:00.673985 11782 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.000s
I20260709 12:24:00.674237 11738 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:00.737510 11738 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:00.738727 11738 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.740370 11738 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.675218 11681 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:00.680397  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1366) took 1085 ms. Trace:
I20260709 12:24:00.682250 11725 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:00.684000 11841 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.260s	user 0.002s	sys 0.000s
I20260709 12:24:00.743691 11725 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:00.684448  9949 rpcz_store.cc:276] 0709 12:23:59.349008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.366573 (+ 17565us) service_pool.cc:224] Handling call
0709 12:23:59.399474 (+ 32901us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:59.472002 (+ 72528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.472047 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:59.662606 (+190559us) log.cc:844] Serialized 26 byte log entry
0709 12:24:00.009433 (+346827us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:00.032855 (+ 23422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.041578 (+  8723us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:00.627079 (+585501us) raft_consensus.cc:1685] finished
0709 12:24:00.627115 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.627303 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8894,"raft.queue_time_us":34168,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"thread_start_us":330396,"threads_started":1,"wal-append.queue_time_us":507560}
I20260709 12:24:00.572693 11849 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:00.705355 11864 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:24:00.706930 11813 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:00.707122 11762 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:00.559144 11857 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:24:00.735718 11797 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.000s
I20260709 12:24:00.744207 11725 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:00.752818 11813 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:00.758004 11813 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.759688 11813 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.742702 11851 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:00.752056 11756 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.001s
I20260709 12:24:00.742911  9940 rpcz_store.cc:276] 0709 12:23:59.594939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.595087 (+   148us) service_pool.cc:224] Handling call
0709 12:23:59.595414 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:23:59.618462 (+ 23048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:23:59.618493 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:23:59.619021 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 12:24:00.042708 (+423687us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:00.059193 (+ 16485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:00.091788 (+ 32595us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:00.680135 (+588347us) raft_consensus.cc:1685] finished
0709 12:24:00.680171 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:00.680325 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13342,"raft.queue_time_us":29107,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":411189,"threads_started":1,"wal-append.queue_time_us":606292}
I20260709 12:24:00.755275 11762 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:00.756326 11857 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:00.771466  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 90762.9us
W20260709 12:24:00.771648 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 431484us
I20260709 12:24:00.775561 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:00.702348 11840 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.782191 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:00.792100  9987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.013s	sys 0.034s
I20260709 12:24:00.764556 11797 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:00.810322 10642 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.026s	sys 0.022s
I20260709 12:24:00.773267 11756 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:00.810708 10642 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.026s	sys 0.022s
W20260709 12:24:00.810889  9987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.014s	sys 0.034s
W20260709 12:24:00.834865 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 784511us
W20260709 12:24:00.835440  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 615325us
I20260709 12:24:00.836366  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:00.785017 11749 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.790666 11836 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:00.792763 11782 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:00.840287 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:00.844187 11866 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.778852 11710 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:24:00.810169 11849 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:00.841956 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47023.2us
I20260709 12:24:00.848671 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:00.799095 11848 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:00.790477 11725 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:00.859584 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.833542 11860 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:00.885282 11860 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:00.792945 11758 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:24:00.891642  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:00.886445 11752 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:00.890832 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44491.6us
I20260709 12:24:00.893440 11849 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:00.839279 11862 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.898150 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:00.900233 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:00.902256 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:00.906224 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.846927 11836 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:00.909116 11849 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:00.909544 11836 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:00.909718 11849 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:00.910107 11836 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.845914 11795 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:24:00.913796 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:00.950398 10094 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:00.928692 11816 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:00.928669  9987 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:24:00.848552 11819 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:24:00.950637 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:00.883502 11837 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:00.975126 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:00.979017 11838 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:24:00.911327 11795 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:00.982637 10676 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:00.889248 11860 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:00.894253 11790 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.883695 11866 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:00.927762 11681 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.889349 11743 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.890364 11749 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.094s
W20260709 12:24:00.996518 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.963367 11725 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:00.849798 11782 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:00.981539 11822 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.000s
I20260709 12:24:00.999274 11838 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:00.927755 11802 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:24:01.000340 11749 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:24:01.009262 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:00.850301 11762 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.983238 11837 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:01.021118 11838 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:01.035900 11749 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:01.065501 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.065869 11838 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:01.067771 11838 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.082652 11782 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:01.044871 11837 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:00.997188 11866 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:01.031387 11840 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:01.032195 11743 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:01.032378 11863 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:01.033011 11822 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:00.990244 11795 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:00.990473 11792 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:00.987686 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:00.961424  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:01.101284  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 139962us
I20260709 12:24:01.102550  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:00.895138 11848 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:01.089381 11816 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:01.091357 11853 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.091900 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.122965 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:01.124110 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 136499us
I20260709 12:24:01.125075 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:01.132261 11839 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:01.119837 11786 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:01.133517 11786 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:01.133988 11786 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:01.119808 11797 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 12:24:01.118885 11814 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:00.981069 11857 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:01.141109 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:01.147962  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.005s	user 0.043s	sys 0.078s
W20260709 12:24:01.148844  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.006s	user 0.043s	sys 0.079s
W20260709 12:24:01.149173  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.00666e+06us
I20260709 12:24:01.118983 11816 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:01.127614 11863 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:01.170749 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:01.177021 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:01.125290 11853 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:01.183446 11853 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:01.163448 11713 tablet_copy_client.cc:538] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:01.188724 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:01.200197  9913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.021s	sys 0.057s
W20260709 12:24:01.200826  9913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.022s	sys 0.057s
W20260709 12:24:01.202817  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.203325  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.205241  9940 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.184242  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:24:01.162609 11862 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:01.164224 11797 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:01.168533 11737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.027s	sys 0.011s
I20260709 12:24:01.301227  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.218123  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:01.255970  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:01.258013  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.259302  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.271308  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.260564  9947 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:01.306540  9943 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:01.185110 11710 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.315371  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:01.315217  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.316817  9946 raft_consensus.cc:2410] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:24:01.303304 11737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.027s	sys 0.012s
I20260709 12:24:01.228691 11771 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:24:01.324994 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.335151 11865 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:24:01.228185 11768 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:01.308203 11857 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:01.363709  9946 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:01.169052 11795 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:01.228550 11766 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:01.176239 11749 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:01.418470 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.371264 11789 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.011s	sys 0.039s
I20260709 12:24:01.342856  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.338526 11792 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:01.425727  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.024s	sys 0.012s
I20260709 12:24:01.406526  9940 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:01.341363 11862 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:24:01.445355 10098 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.907s	user 0.022s	sys 0.046s
I20260709 12:24:01.327044 11710 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:01.374420 11821 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.022s	sys 0.104s
W20260709 12:24:01.441361 11789 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.909s	user 0.011s	sys 0.039s
I20260709 12:24:01.440420 11794 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:01.365053 11797 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:01.453959 11792 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:01.452014 11822 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:01.468238 10092 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.896s	user 0.000s	sys 0.028s
W20260709 12:24:01.498343 10092 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.000s	sys 0.029s
W20260709 12:24:01.475903 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.481324 10096 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.074s	user 0.027s	sys 0.040s
W20260709 12:24:01.500641 10096 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.093s	user 0.027s	sys 0.040s
W20260709 12:24:01.500782 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.501196 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.471284  9915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.021s	sys 0.046s
W20260709 12:24:01.502121  9915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.022s	sys 0.047s
W20260709 12:24:01.504058  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.504580  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:01.505213  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.510352  9944 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 12:24:01.439771 11764 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.012s	sys 0.015s
W20260709 12:24:01.513429 11764 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.013s	sys 0.015s
I20260709 12:24:01.416585 11768 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:01.496119  9959 tablet_copy_service.cc:140] P cd983415389444e292b2330ef630850a: Received BeginTabletCopySession request for tablet 7e7b0b17a4d646809d4dd46a0268f2eb from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:38784)
I20260709 12:24:01.517058  9959 tablet_copy_service.cc:161] P cd983415389444e292b2330ef630850a: Beginning new tablet copy session on tablet 7e7b0b17a4d646809d4dd46a0268f2eb from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:38784: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-7e7b0b17a4d646809d4dd46a0268f2eb
I20260709 12:24:01.517410 11768 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:01.518523 11790 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
W20260709 12:24:01.454205 10097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.092s	user 0.005s	sys 0.024s
I20260709 12:24:01.371289 11816 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:01.451998 10098 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.022s	sys 0.046s
I20260709 12:24:01.371167 11771 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:01.450469  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.024s	sys 0.012s
W20260709 12:24:01.526784  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.00766e+06us
W20260709 12:24:01.520699 10097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.159s	user 0.005s	sys 0.024s
I20260709 12:24:01.483246  9949 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:01.462172 11821 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.058s	user 0.022s	sys 0.105s
W20260709 12:24:01.483630 11841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.004s	sys 0.000s
W20260709 12:24:01.540683  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.097s	user 0.026s	sys 0.064s
W20260709 12:24:01.568830  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.125s	user 0.026s	sys 0.064s
W20260709 12:24:01.569169  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 1.14507e+06us
I20260709 12:24:01.482216 11749 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:01.506681 11765 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.007s	sys 0.029s
I20260709 12:24:01.478420 11797 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:01.523382 11705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.011s	sys 0.016s
I20260709 12:24:01.473383 11794 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:01.572144  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:01.595598  9959 tablet_copy_source_session.cc:215] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:24:01.595598 11765 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.190s	user 0.007s	sys 0.030s
W20260709 12:24:01.521750 11793 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.247s	user 0.032s	sys 0.043s
W20260709 12:24:01.560539  9843 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.010s	sys 0.022s
W20260709 12:24:01.665539  9843 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.011s	sys 0.022s
W20260709 12:24:01.590030 11841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.004s	sys 0.000s
I20260709 12:24:01.588922 11848 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:01.555297 11821 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:01.589452 11790 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.600983 11705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.011s	sys 0.016s
I20260709 12:24:01.671880 11863 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.666746 11793 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.399s	user 0.032s	sys 0.043s
W20260709 12:24:01.580770 11760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.030s	sys 0.030s
W20260709 12:24:01.690155 11856 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.953s	user 0.023s	sys 0.081s
I20260709 12:24:01.671592 11764 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.667364 11859 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:01.702319 11747 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.992s	user 0.006s	sys 0.024s
W20260709 12:24:01.703600 11866 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.604s	user 0.006s	sys 0.006s
I20260709 12:24:01.673316 11819 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:01.714709 11813 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.951s	user 0.010s	sys 0.004s
I20260709 12:24:01.701890 11794 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:01.718745 11760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.099s	user 0.030s	sys 0.030s
I20260709 12:24:01.694636 11873 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:01.710310 11789 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:01.825310 10669 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:01.749136 10091 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.011s	sys 0.021s
W20260709 12:24:01.756958 11747 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.006s	sys 0.024s
W20260709 12:24:01.776126 11761 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.051s	user 0.008s	sys 0.005s
I20260709 12:24:01.697543 11863 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:01.700260 11790 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:01.784511 11856 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.322s	user 0.025s	sys 0.081s
W20260709 12:24:01.844736 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.032s	sys 0.030s
W20260709 12:24:01.846904 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.031s	sys 0.056s
I20260709 12:24:01.826143  9949 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:01.844591 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.025s	sys 0.048s
I20260709 12:24:01.869848  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:01.871769 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.025s	sys 0.048s
W20260709 12:24:01.853250 10093 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.001s	sys 0.045s
I20260709 12:24:01.764016 11822 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
W20260709 12:24:01.866670 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.103s	user 0.032s	sys 0.056s
I20260709 12:24:01.844987 11859 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:01.873569  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.012s	sys 0.026s
W20260709 12:24:01.953293  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.024s	sys 0.038s
W20260709 12:24:01.953754  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 735726us
I20260709 12:24:01.891652  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:01.934621 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:01.753153 11771 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:01.777056 11764 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:01.890931  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.960592  9945 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:01.961804 11764 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:01.962693 11859 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:01.963402 11859 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.963653 11822 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.964370 11822 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:01.964993  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.965860  9949 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:01.965404 11822 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:01.967023  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.970932  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:01.971557 11777 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:01.971963 11777 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:01.972842 11777 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:01.866416 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.032s	sys 0.030s
I20260709 12:24:01.784278 11866 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:01.846572 11880 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.127s
W20260709 12:24:01.852209 10091 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.863s	user 0.011s	sys 0.021s
I20260709 12:24:01.862504 11884 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.784505 11819 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:01.886270 10093 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.046s	user 0.002s	sys 0.046s
I20260709 12:24:01.952630 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 492) took 1046 ms. Trace:
I20260709 12:24:01.846696 11752 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.859354 11789 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:01.851733 11761 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:02.011020 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:01.952379 11862 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:02.012176  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:01.982434 10094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.891s	user 0.006s	sys 0.022s
I20260709 12:24:01.870963 11710 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:01.979429 11838 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:01.998528 11764 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:01.853278 11873 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:02.036489  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:01.963979 11859 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:02.087141  9987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.033s	sys 0.063s
W20260709 12:24:02.239913  9987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.036s	sys 0.071s
I20260709 12:24:01.870236 11821 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:02.064953 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.014s	sys 0.005s
W20260709 12:24:02.282015 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.014s	sys 0.005s
W20260709 12:24:02.134384  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 122309us
I20260709 12:24:02.037065 11761 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:02.301198 10097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.008s	sys 0.029s
W20260709 12:24:02.261332  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.165748 11862 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:02.193940 11851 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.422s	user 0.005s	sys 0.010s
I20260709 12:24:02.125697 11878 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:02.051803 10094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.960s	user 0.007s	sys 0.022s
W20260709 12:24:02.198885 11877 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.508s	user 0.000s	sys 0.003s
W20260709 12:24:02.302904  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.031498 11884 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:02.302950 10097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.008s	sys 0.029s
I20260709 12:24:02.044538 11867 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:02.320576  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.016s	sys 0.014s
W20260709 12:24:02.320968  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.015s	sys 0.028s
W20260709 12:24:02.284996 11857 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.863s	user 0.005s	sys 0.007s
I20260709 12:24:02.048810 11752 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:02.193483 11838 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:02.199473 11764 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:02.148072  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.491408  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.020s	sys 0.049s
W20260709 12:24:02.682960  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.109s	user 0.026s	sys 0.052s
W20260709 12:24:02.683161  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.683571  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.436398  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.000s	sys 0.041s
W20260709 12:24:02.690977  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.000s	sys 0.054s
W20260709 12:24:02.691426  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.11928e+06us
I20260709 12:24:02.684168 11752 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:02.421646 11758 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.087s	user 0.021s	sys 0.016s
W20260709 12:24:02.684803  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.344s	user 0.017s	sys 0.023s
I20260709 12:24:02.602233 11790 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:02.686914  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.374s	user 0.017s	sys 0.037s
W20260709 12:24:02.687513  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.671295 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 645147us
W20260709 12:24:02.139014  9946 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.012s	sys 0.036s
W20260709 12:24:02.484033 11885 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.631s	user 0.000s	sys 0.001s
W20260709 12:24:02.736979  9946 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.222s	user 0.012s	sys 0.038s
W20260709 12:24:02.566552 11794 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.706s	user 0.009s	sys 0.000s
W20260709 12:24:02.574703 11681 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.576s	user 0.010s	sys 0.003s
I20260709 12:24:02.685237 11867 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:02.158102 11766 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.015s	sys 0.015s
W20260709 12:24:02.711069 11888 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.470s	user 0.001s	sys 0.001s
I20260709 12:24:02.775920 11865 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.484426 11851 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:02.022483 10675 rpcz_store.cc:276] 0709 12:24:00.905821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.912509 (+  6688us) service_pool.cc:224] Handling call
0709 12:24:01.714107 (+801598us) spinlock_profiling.cc:243] Waited 8.22 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:01.775280 (+ 61173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.843069 (+ 67789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.854574 (+ 11505us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:01.855990 (+  1416us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:01.866063 (+ 10073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.873678 (+  7615us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:01.909128 (+ 35450us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18913536}
I20260709 12:24:02.782160  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:02.783272 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1379) took 1781 ms. Trace:
W20260709 12:24:02.804906 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.012s	sys 0.011s
W20260709 12:24:02.807476 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.856s	user 0.012s	sys 0.011s
I20260709 12:24:02.687384 11813 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:02.587977 11876 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.630s	user 0.000s	sys 0.001s
W20260709 12:24:02.789172  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 111132us
I20260709 12:24:02.650265  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:02.658533 11891 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.359s	user 0.000s	sys 0.001s
I20260709 12:24:02.732721 11805 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:02.404990 11881 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.711s	user 0.000s	sys 0.001s
I20260709 12:24:02.678197 11863 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:02.778337 11884 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:02.671029 11822 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:02.800724 10090 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.045s	sys 0.086s
W20260709 12:24:02.858839 10090 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.045s	sys 0.086s
W20260709 12:24:02.801016 10063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.022s	sys 0.034s
W20260709 12:24:02.859891 10063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.022s	sys 0.034s
W20260709 12:24:02.860459 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.861003 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.861378 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:02.861927 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.748680 11790 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:02.733870 11758 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.416s	user 0.021s	sys 0.031s
I20260709 12:24:02.804162 10094 rpcz_store.cc:276] 0709 12:24:01.002405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.002615 (+   210us) service_pool.cc:224] Handling call
0709 12:24:01.003037 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.090877 (+ 87840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.090911 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.091383 (+   472us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.687949 (+1596566us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:02.688004 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.688150 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:02.782990 (+ 94840us) raft_consensus.cc:1685] finished
0709 12:24:02.783017 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:02.783192 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87328,"raft.queue_time_us":87611,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":98,"thread_start_us":1596187,"threads_started":1,"wal-append.queue_time_us":1444271}
I20260709 12:24:02.809471  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:02.788159 11766 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.342s	user 0.019s	sys 0.026s
I20260709 12:24:02.811749  9942 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:02.314801 11761 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:02.796049 11681 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:24:02.829262 10093 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.010s	sys 0.026s
W20260709 12:24:02.867846 10093 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.851s	user 0.011s	sys 0.027s
W20260709 12:24:02.830905  9944 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.008s	sys 0.040s
I20260709 12:24:02.793725 11765 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:02.843003 10062 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.010s	sys 0.022s
W20260709 12:24:02.868814 10062 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.010s	sys 0.022s
W20260709 12:24:02.869215 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 705464us
I20260709 12:24:02.809593 11866 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:02.789491 11794 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:02.868521  9944 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.008s	sys 0.040s
W20260709 12:24:02.872833 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.826436 11819 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:02.731658 11878 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:02.841023 11789 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:02.874557  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:02.828205 11853 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:02.880679  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71299.2us
I20260709 12:24:02.820538 11813 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:02.847944 11859 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:02.845470 11887 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.670s	user 0.000s	sys 0.002s
I20260709 12:24:02.838878  9949 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:02.845341 11873 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:02.834151 11705 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:02.828459 11856 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:24:02.823869 11851 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:02.806339 11865 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:02.867278 11766 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:02.935148  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:02.875962 11910 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:02.950176  9948 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:02.887573 11765 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:02.957803 11902 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.360s	user 0.001s	sys 0.000s
I20260709 12:24:02.957888 11766 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:02.960702  9941 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:02.961673  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1195) took 1660 ms. Trace:
I20260709 12:24:02.962123  9941 rpcz_store.cc:276] 0709 12:24:01.301020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.305920 (+  4900us) service_pool.cc:224] Handling call
0709 12:24:02.940607 (+1634687us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:02.945476 (+  4869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.961310 (+ 15834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.961586 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16742656}
I20260709 12:24:02.965022 11856 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:02.922994 11819 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:02.895097 11899 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:02.874886 11790 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:02.924256 11914 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:02.878826 11897 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:24:02.934149 11862 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:02.932230 11866 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:02.945698 11705 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:02.965778 11856 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:02.932114 11853 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.332835 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1380) took 2227 ms. Trace:
W20260709 12:24:03.317173  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.010s	sys 0.008s
W20260709 12:24:03.365873  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.011s	sys 0.008s
I20260709 12:24:03.365654 10090 rpcz_store.cc:276] 0709 12:24:01.105425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.106073 (+   648us) service_pool.cc:224] Handling call
0709 12:24:01.126235 (+ 20162us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.355354 (+229119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.355392 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.367352 (+ 11960us) log.cc:844] Serialized 26 byte log entry
0709 12:24:01.825193 (+457841us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:02.859353 (+1034160us) log.cc:844] Serialized 14 byte log entry
0709 12:24:02.859570 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.178577 (+319007us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b54002fbf54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:03.178665 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.178733 (+    68us) raft_consensus.cc:1685] finished
0709 12:24:03.332525 (+153792us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.332741 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169457,"prepare.queue_time_us":740329,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":256966,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":14835,"spinlock_wait_cycles":135969536,"thread_start_us":1365696,"threads_started":2,"wal-append.queue_time_us":1024566}
W20260709 12:24:03.358757  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 475611us
I20260709 12:24:02.963902  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:03.378252  9941 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 1.
I20260709 12:24:03.380445  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 338) took 1365 ms. Trace:
I20260709 12:24:03.384514  9941 rpcz_store.cc:276] 0709 12:24:02.015415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.963547 (+948132us) service_pool.cc:224] Handling call
0709 12:24:03.380356 (+416809us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:02.997862 11819 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.033474  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:02.968796 11835 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e7b0b17a4d646809d4dd46a0268f2eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:24:03.040576 11913 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:02.999336 11758 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:03.102033  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1374) took 1586 ms. Trace:
I20260709 12:24:02.987936 11681 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:03.073602 10091 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.875s	user 0.037s	sys 0.097s
I20260709 12:24:02.952004 11873 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:02.968328 11909 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:02.956492 11910 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.026350 11838 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:03.025292 11859 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:03.397615 11862 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:03.068449 10098 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.899s	user 0.021s	sys 0.035s
I20260709 12:24:03.375313 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2438) took 2977 ms. Trace:
I20260709 12:24:03.018615 11899 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:03.358502  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:03.382314 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:02.941578 11900 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.092979 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1374) took 2282 ms. Trace:
I20260709 12:24:03.403669 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:03.404228 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:03.106184 10022 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:03.137264 11749 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.527s	user 0.014s	sys 0.000s
I20260709 12:24:03.086572 11901 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:03.125283 11786 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.991s	user 0.014s	sys 0.001s
I20260709 12:24:03.406401 11764 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:03.022399 11885 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:24:03.403839 11837 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 2.316s	user 0.012s	sys 0.004s
I20260709 12:24:03.501660  9941 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:03.513129 10024 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
W20260709 12:24:03.440398 10098 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.313s	user 0.026s	sys 0.041s
I20260709 12:24:03.539991 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1372) took 3082 ms. Trace:
I20260709 12:24:03.540187 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:03.426025  9944 rpcz_store.cc:276] 0709 12:24:01.515699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.521244 (+  5545us) service_pool.cc:224] Handling call
0709 12:24:01.884674 (+363430us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:01.884935 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.885203 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:02.880524 (+995321us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.101552 (+221028us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.101620 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:03.101695 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.101791 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.101958 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1155807,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"spinlock_wait_cycles":10764544,"thread_start_us":1145914,"threads_started":2}
W20260709 12:24:03.506043 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.542483 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.542902 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:03.543251 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.405279 11795 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.504245 10093 rpcz_store.cc:276] 0709 12:24:00.780175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.780350 (+   175us) service_pool.cc:224] Handling call
0709 12:24:00.780735 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:00.780986 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:02.016524 (+1235538us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.052600 (+1036076us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.052675 (+    75us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:03.061709 (+  9034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.061841 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.061987 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14259,"prepare.queue_time_us":1288775,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"thread_start_us":2211555,"threads_started":3,"wal-append.queue_time_us":1107196}
I20260709 12:24:03.430056 11794 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:03.518647 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:03.520080  9942 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:03.440506 10096 rpcz_store.cc:276] 0709 12:24:00.398320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.398794 (+   474us) service_pool.cc:224] Handling call
0709 12:24:00.406614 (+  7820us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.406919 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.406949 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.407419 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 12:24:01.501016 (+1093597us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:01.501054 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.501144 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.375011 (+1873867us) raft_consensus.cc:1685] finished
0709 12:24:03.375045 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.375226 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10093,"raft.queue_time_us":75152,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":18459,"thread_start_us":1093217,"threads_started":1,"wal-append.queue_time_us":1112124}
W20260709 12:24:03.434580 10091 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.275s	user 0.042s	sys 0.101s
I20260709 12:24:03.438560 11838 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:03.425884 11758 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:03.430799 11762 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:03.572063  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1203) took 1678 ms. Trace:
W20260709 12:24:03.572319 11916 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:24:03.434834 11909 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:03.521786 11813 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:03.548972  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1204) took 1654 ms. Trace:
I20260709 12:24:03.581027  9942 rpcz_store.cc:276] 0709 12:24:01.894144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.945636 (+1051492us) service_pool.cc:224] Handling call
0709 12:24:03.548908 (+603272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.434527 11681 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.573232 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:03.405732 11866 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:03.438843 11859 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.540369 10098 rpcz_store.cc:276] 0709 12:24:00.457216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.458228 (+  1012us) service_pool.cc:224] Handling call
0709 12:24:00.460204 (+  1976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.498252 (+ 38048us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.498279 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.498724 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 12:24:01.523212 (+1024488us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:02.064560 (+541348us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.539708 (+1475148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.539783 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.539824 (+    41us) raft_consensus.cc:1685] finished
0709 12:24:03.539834 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.539940 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":287203,"prepare.queue_time_us":882674,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":805463,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":22969,"thread_start_us":2887836,"threads_started":3,"wal-append.queue_time_us":1180304}
I20260709 12:24:03.436914 11910 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:03.434335 11873 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:03.556367 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:03.538820 11749 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.098124 11849 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:03.537593 11901 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.583227  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 337) took 1568 ms. Trace:
I20260709 12:24:03.601466  9942 rpcz_store.cc:276] 0709 12:24:02.015090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.581838 (+1566748us) service_pool.cc:224] Handling call
0709 12:24:03.582759 (+   921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.582922 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:03.582950 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.582990 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:03.583023 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.583069 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.583176 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:03.583474 11908 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.599s	user 0.001s	sys 0.001s
I20260709 12:24:03.548884 11851 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:03.572541  9941 rpcz_store.cc:276] 0709 12:24:01.893448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.387290 (+1493842us) service_pool.cc:224] Handling call
0709 12:24:03.420877 (+ 33587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.548703 (+127826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.571993 (+ 23290us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:03.548607 11878 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:03.573783 11840 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:03.584779 10022 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:03.541393 11786 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.019019 11914 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:03.581516 11765 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:03.546003 11884 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:03.578982 11867 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:03.549394 11837 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.544283 11900 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.434010 11913 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.665098 10024 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
W20260709 12:24:03.758533 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.012s	sys 0.034s
W20260709 12:24:03.759078 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.012s	sys 0.034s
I20260709 12:24:03.580731 11909 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.804177 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:03.584659 11794 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.735265 11713 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
W20260709 12:24:03.599907 11725 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 2.575s	user 0.016s	sys 0.000s
I20260709 12:24:03.813033 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:03.615155 11865 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:03.807076 11790 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:03.588128 11866 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:03.665288 11851 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:03.639259 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1378) took 2717 ms. Trace:
I20260709 12:24:03.639539 11840 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.590044 11859 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:03.807610 11765 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:03.814368 11908 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:24:03.803589 11838 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:03.830425 11921 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:24:03.830657 11913 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:03.606427 11749 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.812390 11867 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.815693 11705 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:03.837189 11906 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.607071 11849 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.687531 11899 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:03.845315 11914 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:03.831700 11819 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:03.870947 11792 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:03.729708 11758 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.871613 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 451) took 1873 ms. Trace:
I20260709 12:24:03.871796 11758 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:03.872567 11749 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:03.872123 10670 rpcz_store.cc:276] 0709 12:24:02.000075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.011111 (+ 11036us) service_pool.cc:224] Handling call
0709 12:24:02.016763 (+  5652us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:02.017131 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.017160 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:02.038785 (+ 21625us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.875421 (+836636us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:02.875466 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:02.875531 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.871305 (+995774us) raft_consensus.cc:1685] finished
0709 12:24:03.871334 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.871521 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":714725,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":11966,"thread_start_us":196410,"threads_started":1,"wal-append.queue_time_us":720539}
I20260709 12:24:03.874243 11786 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:03.874891 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1371) took 3534 ms. Trace:
I20260709 12:24:03.875685 10020 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:24:03.807955 11884 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:03.851075 11840 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:03.855520 11761 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.971s	user 0.010s	sys 0.000s
I20260709 12:24:03.858732 11856 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:03.836390 11790 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:03.864024 11883 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.332s	user 0.003s	sys 0.009s
I20260709 12:24:03.696465 11894 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:03.864833 11906 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.836730 11866 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:03.875263 10097 rpcz_store.cc:276] 0709 12:24:00.340144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.340357 (+   213us) service_pool.cc:224] Handling call
0709 12:24:00.361738 (+ 21381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:01.528609 (+1166871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:01.528639 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:01.573042 (+ 44403us) log.cc:844] Serialized 26 byte log entry
0709 12:24:02.686635 (+1113593us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:02.816867 (+130232us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.091629 (+274762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.091738 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:03.874670 (+782932us) raft_consensus.cc:1685] finished
0709 12:24:03.874692 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.874833 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":171977,"prepare.queue_time_us":138266,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":213178,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":11058,"spinlock_wait_cycles":8546688,"thread_start_us":2458728,"threads_started":3,"wal-append.queue_time_us":1301889}
I20260709 12:24:03.825099 11794 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.881688 11914 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:03.881090 11819 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:03.883594 11761 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:03.890857 11822 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.019s	user 0.006s	sys 0.006s
I20260709 12:24:03.892903 11805 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:03.837847 11851 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:03.911383  9946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.130s	user 0.000s	sys 0.002s
W20260709 12:24:03.911720 11890 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.246s	user 0.016s	sys 0.051s
W20260709 12:24:04.013717  9946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.256s	user 0.000s	sys 0.003s
W20260709 12:24:03.912572 10064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.599s	user 0.027s	sys 0.046s
W20260709 12:24:03.915843 11771 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.234s	user 0.013s	sys 0.001s
I20260709 12:24:03.868860 11853 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:03.957927 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.168s	user 0.025s	sys 0.038s
W20260709 12:24:04.024231 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.235s	user 0.026s	sys 0.038s
W20260709 12:24:04.024616 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.23501e+06us
W20260709 12:24:03.958331  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.022s	sys 0.030s
W20260709 12:24:03.958630  9915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.209s	user 0.010s	sys 0.022s
I20260709 12:24:03.995497 11901 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:03.846222 11765 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:04.016050 10064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.702s	user 0.027s	sys 0.046s
W20260709 12:24:04.017338  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.322s	user 0.000s	sys 0.029s
I20260709 12:24:03.879045 11859 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:04.010890 11797 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.184s	user 0.034s	sys 0.062s
I20260709 12:24:03.867882 11867 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:03.906668 11790 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:04.065743  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.007s	sys 0.014s
W20260709 12:24:04.146070 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 116875us
W20260709 12:24:04.312486  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.009s	sys 0.025s
W20260709 12:24:04.311378 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.082741  9913 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.003s	sys 0.012s
W20260709 12:24:04.313460  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.619s	user 0.004s	sys 0.045s
W20260709 12:24:04.313606  9913 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.110s	user 0.008s	sys 0.018s
I20260709 12:24:04.162380  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1373) took 2648 ms. Trace:
W20260709 12:24:04.314563  9915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.565s	user 0.017s	sys 0.037s
W20260709 12:24:04.222796  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.856s	user 0.002s	sys 0.009s
W20260709 12:24:04.308956  9991 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.005s	sys 0.009s
W20260709 12:24:04.290248 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.605s	user 0.002s	sys 0.030s
I20260709 12:24:04.179948 10022 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:04.316224 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.631s	user 0.003s	sys 0.031s
W20260709 12:24:04.316020  9991 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.005s	sys 0.009s
W20260709 12:24:04.316774 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.009s	sys 0.015s
W20260709 12:24:04.316854  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 534769us
W20260709 12:24:04.317207 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.009s	sys 0.015s
I20260709 12:24:04.028575 11795 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:04.314304  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.210s	user 0.027s	sys 0.047s
W20260709 12:24:04.315512  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.002s	sys 0.013s
I20260709 12:24:04.202824 11838 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:04.020931 11771 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.340s	user 0.013s	sys 0.001s
W20260709 12:24:04.026247 11890 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.367s	user 0.017s	sys 0.051s
I20260709 12:24:04.119710 11865 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:04.321686  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.015185 11805 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:04.316538 11797 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.495s	user 0.054s	sys 0.092s
W20260709 12:24:04.336524  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.015827 11821 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.016s	sys 0.003s
I20260709 12:24:04.314574  9946 rpcz_store.cc:276] 0709 12:24:01.506629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.513888 (+  7259us) service_pool.cc:224] Handling call
0709 12:24:01.514330 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.514606 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:02.757783 (+1243177us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.154552 (+1396769us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:04.154649 (+    97us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:04.154735 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.154827 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:04.154996 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138461,"prepare.queue_time_us":1476814,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":145825,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":7734,"thread_start_us":2499801,"threads_started":2}
I20260709 12:24:04.368964 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.026017 11851 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.001406 11866 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.359400 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.362000 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:04.364171  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.381778 10094 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:24:04.384063  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:04.384958  9942 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.009701 11819 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.339568 11840 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.119993 11752 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:04.348106 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:04.011107 11914 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.325122 11822 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:04.401101 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.359722 11929 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.369660 10096 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:04.343207 11913 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.330940 11901 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.330564 11923 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:04.343523 11725 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.392037 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.421262 10095 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:24:04.332693 11838 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:04.402027 10090 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:24:04.339682 11737 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:04.321098 11922 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:04.331640 11878 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:04.352897 11758 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:04.368778 11821 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.306s	user 0.019s	sys 0.015s
I20260709 12:24:04.318467 11795 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:04.416708 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:04.417995 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.362969 11837 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:04.369417 11730 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.021s	sys 0.018s
I20260709 12:24:04.405288 11760 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.440418 10098 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:04.401755 11927 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.320986 11859 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.409010 11705 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.486256 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:04.499778 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.408874 11899 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:04.408696 11797 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:04.513325 10097 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:04.010056 11894 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.509927 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.517590 10090 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:04.504806 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.519035 10095 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:04.420836 11761 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.498863 11737 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.403484 11851 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:04.505141 11730 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.021s	sys 0.019s
I20260709 12:24:04.415997 11822 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:04.561910 11935 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:24:04.561722 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:04.562680 11934 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:24:04.571015 10095 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
W20260709 12:24:04.516601 11762 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.945s	user 0.008s	sys 0.005s
I20260709 12:24:04.572412 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:04.580317  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0577580169997418s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 12:24:04.580421 10090 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:04.581380 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.582702 10097 raft_consensus.cc:2410] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:04.585189 11930 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:04.585803 11930 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.373314 11865 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.590077 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:04.560873 11766 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.041s	sys 0.077s
I20260709 12:24:04.428174 11913 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.420258 11929 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.440968 11923 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.428431 11901 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:03.835841 10091 rpcz_store.cc:276] 0709 12:24:00.912730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.927585 (+ 14855us) service_pool.cc:224] Handling call
0709 12:24:00.988249 (+ 60664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:00.988490 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:02.044227 (+1055737us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.578878 (+1534651us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.581188 (+  2310us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:03.592459 (+ 11271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:03.600096 (+  7637us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:03.629541 (+ 29445us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1324955,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":84,"raft.queue_time_us":270677,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2357,"thread_start_us":2447983,"threads_started":2}
I20260709 12:24:04.607378 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.602248  9949 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:04.572197 11853 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.563506 10673 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:04.620369 10670 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:04.471019 11838 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.508333 11758 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.498646 11878 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.419193 11752 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.614418 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:04.637010 10090 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:24:04.521996 11899 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.639602 11929 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.498921 11922 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.614418 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:04.619822 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:04.611779  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 185217us
I20260709 12:24:04.643050 10095 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:04.440968 11725 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.415100 11840 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.605822 11761 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:04.563027 11917 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.237s
I20260709 12:24:04.510326 11760 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.522979 11797 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.522449 11705 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.623697 11794 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:04.598326 11766 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.041s	sys 0.077s
I20260709 12:24:04.615939 11857 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.626420 10097 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:04.628029 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:04.644019  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:04.748503  9942 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
W20260709 12:24:04.750278  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 106371us
I20260709 12:24:04.753401  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:04.753296  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:04.684212 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:04.516814 11927 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.761067  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:04.766476  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:24:04.766903  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.645495 11790 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.753903  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:04.769662  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:04.692045 11758 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.748024 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 493) took 3734 ms. Trace:
I20260709 12:24:04.645685 11932 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.652662 11765 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.782812 11948 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:04.783978 11901 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.785763 11913 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.786572 11856 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:04.787886 11752 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.754683 11762 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.762919 11857 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.853729 11789 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.852949 11795 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.684525 11867 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.690721 11859 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.627413 11853 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.795473 11928 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:04.799170 11737 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.804569 11725 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.806178 11840 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.807067 11937 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:04.810603 11958 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:04.810726 11930 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:04.811203 11940 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:04.833329 11947 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:24:04.834928 11760 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.889156 11937 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.884552 11725 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.884145 11928 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:04.900913 11956 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.901404 11873 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.898029 11853 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.691216 11926 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.789202 11866 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.847239 11878 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.847805 11762 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.796082 11758 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:04.950803 11758 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.951378 11758 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.952941 11819 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:04.814373 11909 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.002s	user 0.011s	sys 0.001s
I20260709 12:24:04.955065 11909 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.959547 11909 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.960347 11730 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:04.844480 11856 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:04.963330 11856 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:04.852515 11822 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:04.853780 11882 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 2.864s	user 0.006s	sys 0.015s
I20260709 12:24:04.972836 11851 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:04.973013 11849 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:04.975577 11923 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.976226 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:04.976502 11926 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.684459 11761 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.852645 11948 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.850889 11913 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:04.979750 11913 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.981487 11948 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.981585 11762 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.981856 11913 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.865636 11901 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:04.864892 11789 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.857429 11932 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.982898 11901 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:04.983407 11901 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:04.983867 11889 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 2.306s	user 0.006s	sys 0.006s
I20260709 12:24:04.984413 11789 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.985061 11932 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.985689 11894 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.988430 11857 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.888919 11797 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.988802 11761 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.991091 11906 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.887473 11840 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:04.991845 11840 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.992023 11758 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.992856 11730 tablet_bootstrap.cc:492] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:24:04.993960 11840 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.992849 11926 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.898578 11958 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:04.899223 11928 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:04.889626 11865 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.904764 11956 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.905875 11838 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:04.907825 11805 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.002s
I20260709 12:24:04.914913 11900 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:04.894944 11940 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:04.999661 11940 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:04.898505 11705 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:04.917930 11886 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 2.944s	user 0.009s	sys 0.012s
I20260709 12:24:04.905079 11899 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.935726 11922 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.937230 11938 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:05.005618 11938 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:04.940533 11862 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.942056 11835 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:42028 (request call id 549) took 6830 ms. Trace:
I20260709 12:24:04.945134 11816 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:05.013795 11901 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.946838 11927 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.961370 11884 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:05.006685 11938 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:05.008446 11835 rpcz_store.cc:276] 0709 12:23:58.111809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:59.740740 (+1628931us) service_pool.cc:224] Handling call
0709 12:24:00.307216 (+566476us) ts_tablet_manager.cc:934] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
0709 12:24:02.780780 (+2473564us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:24:04.437886 (+1657106us) tablet_metadata.cc:665] Metadata flushed
0709 12:24:04.506741 (+ 68855us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:24:04.940782 (+434041us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:24:04.941167 (+   385us) tablet_replica.cc:206] Creating consensus instance
0709 12:24:04.941605 (+   438us) ts_tablet_manager.cc:989] Replica registered.
0709 12:24:04.941697 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":67702,"tablet-copy.queue_time_us":439962,"thread_start_us":543472,"threads_started":2}
I20260709 12:24:05.029614 11835 tablet_copy_client.cc:806] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:24:05.041837 11928 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:04.998132 11958 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:05.002238 11956 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:05.003819 11914 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:04.999017 11933 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:05.047340 11933 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:05.048172 11928 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:05.048400 11933 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:05.049315 11933 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:05.090157  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.091071  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.095408  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:05.098155  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:05.098498  9839 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 327) took 6469 ms. Trace:
I20260709 12:24:05.107430  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:05.106246  9839 rpcz_store.cc:276] 0709 12:23:58.629250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:58.629382 (+   132us) service_pool.cc:224] Handling call
0709 12:24:05.098242 (+6468860us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:23:59.786485 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:23:59.832417 (+ 45932us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:23:59.832486 (+    69us) write_op.cc:435] Acquired schema lock
0709 12:23:59.832532 (+    46us) tablet.cc:662] Decoding operations
0709 12:23:59.833076 (+   544us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:23:59.833176 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:23:59.833205 (+    29us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:23:59.833494 (+   289us) tablet.cc:701] Row locks acquired
0709 12:23:59.833528 (+    34us) write_op.cc:260] PREPARE: finished
0709 12:23:59.833688 (+   160us) write_op.cc:270] Start()
0709 12:23:59.833847 (+   159us) write_op.cc:276] Timestamp: P: 1783599839833627 usec, L: 0
0709 12:23:59.833885 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:23:59.846427 (+ 12542us) log.cc:844] Serialized 541 byte log entry
0709 12:24:02.235491 (+2389064us) op_driver.cc:464] REPLICATION: finished
0709 12:24:04.563425 (+2327934us) write_op.cc:301] APPLY: starting
0709 12:24:04.588214 (+ 24789us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:04.617740 (+ 29526us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:04.747821 (+130081us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:04.788429 (+ 40608us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:04.788674 (+   245us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:04.788731 (+    57us) write_op.cc:312] APPLY: finished
0709 12:24:04.844624 (+ 55893us) log.cc:844] Serialized 23 byte log entry
0709 12:24:04.853978 (+  9354us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:04.854081 (+   103us) write_op.cc:454] Released schema lock
0709 12:24:04.854151 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:04.854578 (+   427us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:04.854621 (+    43us) write_op.cc:454] Released schema lock
0709 12:24:04.920692 (+ 66071us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955332a 00007f4e094d8f14 00007f4e094d8ec3 00007f4e094d8e38 00007f4e094d8df3 00007f4e094d80dc 00007f4e09501e16 00007f4e094e9cbb 00007f4e094d439d 00007f4e094d7d75
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":2060181,"apply.run_cpu_time_us":3541,"apply.run_wall_time_us":337916,"num_ops":1,"prepare.queue_time_us":406006,"prepare.run_cpu_time_us":21429,"prepare.run_wall_time_us":429813,"raft.queue_time_us":1238052,"raft.run_cpu_time_us":102323,"raft.run_wall_time_us":2914531,"replication_time_us":2204912,"spinlock_wait_cycles":8347520,"thread_start_us":2915240,"threads_started":4,"wal-append.queue_time_us":489717,"wal-append.run_cpu_time_us":8811,"wal-append.run_wall_time_us":2354682}]]}
I20260709 12:24:05.109167  9842 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 4, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 4 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:05.113559  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:05.116642  9940 raft_consensus.cc:2368] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:05.117764  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:05.115077  9842 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:24:05.119536  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:04.845948 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.793349 10668 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:04.789613 10669 rpcz_store.cc:276] 0709 12:24:00.950790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.959214 (+  8424us) service_pool.cc:224] Handling call
0709 12:24:01.019745 (+ 60531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:01.849555 (+829810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:04.635843 (+2786288us) spinlock_profiling.cc:243] Waited 734 ms on lock 0x7b54006d01d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:04.659476 (+ 23633us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1689064320}
I20260709 12:24:04.647316 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:04.931859 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:04.995833 11681 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:05.147730 10677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 495) took 3137 ms (client timeout 2025 ms). Trace:
W20260709 12:24:05.148242 10677 rpcz_store.cc:269] 0709 12:24:02.010152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.037387 (+ 27235us) service_pool.cc:224] Handling call
0709 12:24:04.931625 (+2894238us) spinlock_profiling.cc:243] Waited 3.07 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:05.147490 (+215865us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7053184}
I20260709 12:24:04.898957 11725 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:05.151197 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.151939 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.154019 11725 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:05.003383 11777 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:05.000409 11805 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:05.016219 11856 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:24:05.017194 11919 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.445s	user 0.000s	sys 0.003s
I20260709 12:24:05.122654  9942 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:05.018436 11937 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:05.045269 11940 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:05.212616 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:05.128988 10062 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.024s (SENT) [suppressed 25 similar messages]
I20260709 12:24:05.030350 11835 tablet_copy_client.cc:670] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:24:05.140714 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1373) took 4682 ms. Trace:
W20260709 12:24:05.226990 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1314) took 2078 ms (client timeout 1825 ms). Trace:
I20260709 12:24:05.225647 10092 rpcz_store.cc:276] 0709 12:24:00.457797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.458819 (+  1022us) service_pool.cc:224] Handling call
0709 12:24:00.459206 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:00.497809 (+ 38603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:00.497838 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:00.514524 (+ 16686us) log.cc:844] Serialized 26 byte log entry
0709 12:24:01.499487 (+984963us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:01.499543 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:01.499636 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:04.915534 (+3415898us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:05.138980 (+223446us) raft_consensus.cc:1685] finished
0709 12:24:05.139036 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.140368 (+  1332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71235,"raft.queue_time_us":61873,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":10603,"spinlock_wait_cycles":5552384,"thread_start_us":927610,"threads_started":1,"wal-append.queue_time_us":1341781}
I20260709 12:24:05.140952  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:05.142033 11821 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:05.144441 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:04.994393 11758 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:05.159358 11901 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:05.190157 11805 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:04.993832 11730 ts_tablet_manager.cc:1403] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 4.339s	user 0.055s	sys 0.039s
I20260709 12:24:05.224924 10019 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:05.204640 10018 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:05.253360 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.227564 10020 rpcz_store.cc:269] 0709 12:24:03.148050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.148180 (+   130us) service_pool.cc:224] Handling call
0709 12:24:05.226330 (+2078150us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54001c6454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:05.226636 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":829748352}
I20260709 12:24:05.228645  9842 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:05.403424 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 447) took 3417 ms. Trace:
W20260709 12:24:05.438676  9949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 339) took 3422 ms (client timeout 2553 ms). Trace:
W20260709 12:24:05.438428 10019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1309) took 2304 ms (client timeout 1945 ms). Trace:
I20260709 12:24:05.285797 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:05.280208 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:05.299458 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:05.288950  9945 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
W20260709 12:24:05.858305  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.858987  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1395) took 1409 ms. Trace:
I20260709 12:24:05.401738 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 252) took 4210 ms. Trace:
I20260709 12:24:05.859620  9945 rpcz_store.cc:276] 0709 12:24:04.449434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.130635 (+681201us) service_pool.cc:224] Handling call
0709 12:24:05.858878 (+728243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.859881 10099 rpcz_store.cc:276] 0709 12:24:01.191464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.191657 (+   193us) service_pool.cc:224] Handling call
0709 12:24:01.192129 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.399881 (+4207752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.400039 (+   158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.400126 (+    87us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:05.400166 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.401184 (+  1018us) spinlock_profiling.cc:243] Waited 4.09 s on lock 0x7b540015bdd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:05.401249 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.401487 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3598,"raft.queue_time_us":119841,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":265,"spinlock_wait_cycles":9398907136}
W20260709 12:24:05.552896 10095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.000s	sys 0.039s
I20260709 12:24:05.861538  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:05.862246 10095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.014s	sys 0.051s
W20260709 12:24:05.553350  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.030s	sys 0.031s
I20260709 12:24:05.400519 11936 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:05.887969  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.285s	user 0.038s	sys 0.049s
I20260709 12:24:05.888197 11936 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:05.895226 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:05.896159 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:05.896283 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:05.896842 10094 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:05.897763 11959 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:05.898732 11777 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:05.899436 11777 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:05.901530 11777 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:05.902011 11777 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:05.902724  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.054s	user 0.010s	sys 0.018s
W20260709 12:24:05.903119  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.010s	sys 0.018s
I20260709 12:24:05.906441 11758 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:05.898518 11959 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:05.907478 11758 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:05.909637 11959 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:05.862735 10019 rpcz_store.cc:269] 0709 12:24:03.134092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.134239 (+   147us) service_pool.cc:224] Handling call
0709 12:24:05.212083 (+2077844us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:05.438318 (+226235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9259776}
I20260709 12:24:05.399326 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 450) took 3403 ms. Trace:
W20260709 12:24:05.283865 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2384) took 1736 ms (client timeout 1722 ms). Trace:
I20260709 12:24:05.439297 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1313) took 2294 ms. Trace:
W20260709 12:24:05.296025 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 167213us
I20260709 12:24:05.493055 11940 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:05.440955  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:05.404822  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1380) took 3519 ms. Trace:
I20260709 12:24:05.911907 10671 rpcz_store.cc:276] 0709 12:24:01.995423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.005393 (+  9970us) service_pool.cc:224] Handling call
0709 12:24:05.397668 (+3392275us) spinlock_profiling.cc:243] Waited 4.75 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:05.398170 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.398713 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.398739 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.398799 (+    60us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:05.398869 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.398924 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.399058 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10914560}
I20260709 12:24:05.438990 10674 rpcz_store.cc:276] 0709 12:24:01.986282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.989765 (+  3483us) service_pool.cc:224] Handling call
0709 12:24:05.401821 (+3412056us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:05.402245 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.402805 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:05.402838 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:05.402912 (+    74us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:05.402972 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.403046 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.403214 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26871680}
I20260709 12:24:05.442349 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:05.497318 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.401165 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 491) took 4495 ms. Trace:
I20260709 12:24:05.859634 10023 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:05.288476  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:05.494226 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1318) took 2344 ms (client timeout 1975 ms). Trace:
I20260709 12:24:05.455581  9842 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:24:05.859025  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.859287 10017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1307) took 2747 ms (client timeout 1862 ms). Trace:
W20260709 12:24:05.439184  9949 rpcz_store.cc:269] 0709 12:24:02.015630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.988797 (+973167us) service_pool.cc:224] Handling call
0709 12:24:05.438272 (+2449475us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540009c9d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:05.438430 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2725013888}
W20260709 12:24:05.578904  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.010s	sys 0.031s
I20260709 12:24:05.283255 11821 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:05.910742 11758 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:05.864166  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.888720 11856 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:05.492885  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1377) took 3781 ms. Trace:
I20260709 12:24:05.484702  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1375) took 3883 ms. Trace:
W20260709 12:24:05.484755 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1316) took 2336 ms (client timeout 2212 ms). Trace:
W20260709 12:24:05.898737 10090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.013s	sys 0.026s
W20260709 12:24:05.916930 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.621673  9912 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.032s	sys 0.051s
I20260709 12:24:05.439352 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:05.920298 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1311) took 2776 ms (client timeout 1676 ms). Trace:
W20260709 12:24:05.912364 10022 rpcz_store.cc:269] 0709 12:24:03.547545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.810456 (+262911us) service_pool.cc:224] Handling call
0709 12:24:05.283427 (+1472971us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400265cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:05.283672 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":435896064}
W20260709 12:24:05.858096 11892 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.240s	user 0.017s	sys 0.032s
W20260709 12:24:05.760949 11768 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.004s	sys 0.019s
I20260709 12:24:05.916002  9948 rpcz_store.cc:276] 0709 12:24:01.885236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:02.095707 (+210471us) service_pool.cc:224] Handling call
0709 12:24:02.192538 (+ 96831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.950724 (+758186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.404152 (+2453428us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b54003a0554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:05.404557 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1725391744}
W20260709 12:24:05.927983  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.019s	sys 0.044s
W20260709 12:24:05.930163 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:05.929167 10020 rpcz_store.cc:269] 0709 12:24:03.148680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.484462 (+2335782us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:05.484659 (+   197us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:24:05.958909 10019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1322) took 2378 ms (client timeout 1862 ms). Trace:
W20260709 12:24:05.942284 10023 rpcz_store.cc:269] 0709 12:24:03.144161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.144387 (+   226us) service_pool.cc:224] Handling call
0709 12:24:05.279594 (+2135207us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:05.920223 (+640629us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10395520}
W20260709 12:24:05.921195 10017 rpcz_store.cc:269] 0709 12:24:03.112329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.113618 (+  1289us) service_pool.cc:224] Handling call
0709 12:24:05.286423 (+2172805us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:05.859177 (+572754us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14156032}
I20260709 12:24:05.944193 10025 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:05.944598 10021 raft_consensus.cc:2368] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3.
I20260709 12:24:05.928457  9947 rpcz_store.cc:276] 0709 12:24:01.711262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.711563 (+   301us) service_pool.cc:224] Handling call
0709 12:24:01.847595 (+136032us) spinlock_profiling.cc:243] Waited 39.8 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:01.847777 (+   182us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.127705 (+279928us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:02.127751 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:02.127823 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:02.127874 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.492267 (+3364393us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54000163d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:05.492387 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.492661 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734741632}
W20260709 12:24:05.920358 10024 rpcz_store.cc:269] 0709 12:24:03.149233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.572953 (+423720us) service_pool.cc:224] Handling call
0709 12:24:05.493796 (+1920843us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:05.494014 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190028544}
W20260709 12:24:05.945232 11710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.187s	user 0.022s	sys 0.047s
I20260709 12:24:05.928786  9943 rpcz_store.cc:276] 0709 12:24:01.601548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.712137 (+110589us) service_pool.cc:224] Handling call
0709 12:24:01.765073 (+ 52936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:02.301562 (+536489us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:02.885083 (+583521us) log.cc:844] Serialized 14 byte log entry
0709 12:24:03.140828 (+255745us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:03.140902 (+    74us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:03.140972 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.483985 (+2343013us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b5400189cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:05.484054 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:05.484423 (+   369us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10248,"prepare.queue_time_us":742777,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":15343,"raft.queue_time_us":381389,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":10073,"spinlock_wait_cycles":1233158400,"thread_start_us":611655,"threads_started":2,"wal-append.queue_time_us":1440377,"wal-append.run_cpu_time_us":2238,"wal-append.run_wall_time_us":1274201}
W20260709 12:24:05.944280 10090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.014s	sys 0.026s
I20260709 12:24:05.946790 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:05.946159  9842 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 4, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:24:05.947611 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.631s	user 0.016s	sys 0.056s
I20260709 12:24:05.927202 11936 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:05.951198  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:05.952353  9945 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:05.928773  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1207) took 1181 ms. Trace:
W20260709 12:24:05.957406  9912 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.376s	user 0.040s	sys 0.059s
W20260709 12:24:05.959481 10019 rpcz_store.cc:269] 0709 12:24:03.575762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.925142 (+2349380us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:05.953969 (+ 28827us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:24:05.966698 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.650s	user 0.017s	sys 0.056s
I20260709 12:24:05.902511 11777 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:05.961652 11768 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.013s	user 0.006s	sys 0.021s
I20260709 12:24:05.964916 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:05.969707  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1397) took 1421 ms. Trace:
I20260709 12:24:05.970223  9945 rpcz_store.cc:276] 0709 12:24:04.544365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.861138 (+1316773us) service_pool.cc:224] Handling call
0709 12:24:05.965907 (+104769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:05.970645 10094 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:05.961442 11892 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.017s	sys 0.033s
I20260709 12:24:05.966050  9942 rpcz_store.cc:276] 0709 12:24:04.746974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.440363 (+693389us) service_pool.cc:224] Handling call
0709 12:24:05.928714 (+488351us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:05.959816  9941 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.997s	user 0.006s	sys 0.022s
I20260709 12:24:05.921772 10673 rpcz_store.cc:276] 0709 12:24:00.905491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.907402 (+  1911us) service_pool.cc:224] Handling call
0709 12:24:00.907840 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.628699 (+3720859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.400443 (+771744us) spinlock_profiling.cc:243] Waited 3.6 s on lock 0x7b54007602d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:05.400897 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8284404096}
W20260709 12:24:05.969131 10025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1315) took 2820 ms (client timeout 1875 ms). Trace:
W20260709 12:24:05.971071 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1320) took 2529 ms (client timeout 2024 ms). Trace:
W20260709 12:24:05.984447 10022 rpcz_store.cc:269] 0709 12:24:03.441451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.970828 (+2529377us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:05.970990 (+   162us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:05.958612  9940 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:05.913389 10018 rpcz_store.cc:276] 0709 12:24:03.144912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.177746 (+ 32834us) service_pool.cc:224] Handling call
0709 12:24:05.199335 (+2021589us) spinlock_profiling.cc:243] Waited 5.99 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:05.204143 (+  4808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.437138 (+232995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.439194 (+  2056us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13782784}
W20260709 12:24:05.971863 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1312) took 2827 ms (client timeout 1774 ms). Trace:
W20260709 12:24:05.968588  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 1.38697e+06us
W20260709 12:24:05.983685 10025 rpcz_store.cc:269] 0709 12:24:03.148361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.148470 (+   109us) service_pool.cc:224] Handling call
0709 12:24:05.285262 (+2136792us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:05.969061 (+683799us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7737088}
W20260709 12:24:05.974485 11710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.287s	user 0.022s	sys 0.047s
W20260709 12:24:05.986790 10021 rpcz_store.cc:269] 0709 12:24:03.144566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.144729 (+   163us) service_pool.cc:224] Handling call
0709 12:24:05.441717 (+2296988us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:05.971787 (+530070us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10491264}
W20260709 12:24:05.997850 10016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1310) took 2863 ms (client timeout 2226 ms). Trace:
W20260709 12:24:06.003952 11929 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.362s	user 0.025s	sys 0.072s
W20260709 12:24:06.004667 11929 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.363s	user 0.025s	sys 0.073s
W20260709 12:24:06.002511 10016 rpcz_store.cc:269] 0709 12:24:03.134417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.134573 (+   156us) service_pool.cc:224] Handling call
0709 12:24:05.438985 (+2304412us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:05.997749 (+558764us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9503616}
W20260709 12:24:05.996292  9941 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.386s	user 0.006s	sys 0.023s
W20260709 12:24:06.010695  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.185s	user 0.012s	sys 0.020s
W20260709 12:24:06.011063 10019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2385) took 2463 ms (client timeout 2038 ms). Trace:
W20260709 12:24:06.011497  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.25758e+06us
I20260709 12:24:06.008240 10099 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:06.016271 10019 rpcz_store.cc:269] 0709 12:24:03.547866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.010820 (+2462954us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:06.010991 (+   171us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:24:06.027732 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:06.031358  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.012s	sys 0.021s
I20260709 12:24:05.993691  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1206) took 1301 ms. Trace:
I20260709 12:24:06.031235  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:06.032752 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:05.988269  9842 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 1 to 2, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:24:06.006884  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1399) took 1412 ms. Trace:
I20260709 12:24:06.035898 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.042757 11959 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.043218 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.043943 10023 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:06.035974 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1325) took 2429 ms (client timeout 2025 ms). Trace:
I20260709 12:24:06.046265 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.046442 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:06.047254 10025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1323) took 2448 ms (client timeout 2828 ms). Trace:
I20260709 12:24:06.047178  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:06.047851 10025 rpcz_store.cc:269] 0709 12:24:03.598959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.026431 (+2427472us) service_pool.cc:224] Handling call
0709 12:24:06.047194 (+ 20763us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.048914 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:06.049436 10025 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.049259  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:06.044883 10024 rpcz_store.cc:269] 0709 12:24:03.606802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.032041 (+2425239us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:06.035910 (+  3869us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:06.050978 11959 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.052237 10024 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:06.052947 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2375) took 2540 ms. Trace:
I20260709 12:24:06.053261 10024 rpcz_store.cc:276] 0709 12:24:03.512145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.051412 (+2539267us) service_pool.cc:224] Handling call
0709 12:24:06.051941 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.052645 (+   704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.052878 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:06.054883 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1324) took 2451 ms (client timeout 2249 ms). Trace:
W20260709 12:24:06.055323 10022 rpcz_store.cc:269] 0709 12:24:03.603497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.041012 (+2437515us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:06.054814 (+ 13802us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:06.056726 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.056977 10022 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 12:24:06.057518 10016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 255) took 1664 ms (client timeout 2124 ms). Trace:
W20260709 12:24:06.058009 10016 rpcz_store.cc:269] 0709 12:24:04.393069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.033469 (+1640400us) service_pool.cc:224] Handling call
0709 12:24:06.057447 (+ 23978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.057559 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2377) took 2534 ms. Trace:
I20260709 12:24:06.038439 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:06.042333 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1321) took 2462 ms (client timeout 2058 ms). Trace:
W20260709 12:24:06.060585 10020 rpcz_store.cc:269] 0709 12:24:03.575486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.033938 (+2458452us) service_pool.cc:211] Skipping call since client already timed out
0709 12:24:06.037914 (+  3976us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:24:06.059576 10022 rpcz_store.cc:276] 0709 12:24:03.523237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.056129 (+2532892us) service_pool.cc:224] Handling call
0709 12:24:06.056629 (+   500us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.057346 (+   717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.057514 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.062271 10024 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:06.044708 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2381) took 2514 ms (client timeout 2823 ms). Trace:
W20260709 12:24:06.064056 10023 rpcz_store.cc:269] 0709 12:24:03.532827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.038684 (+2505857us) service_pool.cc:224] Handling call
0709 12:24:06.044638 (+  5954us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.067016 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.071668 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2376) took 2551 ms. Trace:
I20260709 12:24:06.063194  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.072331 10024 rpcz_store.cc:276] 0709 12:24:03.517182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.054009 (+2536827us) service_pool.cc:224] Handling call
0709 12:24:06.054436 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.062861 (+  8425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.068000 (+  5139us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.075760 10016 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.076500 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2378) took 2550 ms. Trace:
I20260709 12:24:06.076835 10016 rpcz_store.cc:276] 0709 12:24:03.525780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.074953 (+2549173us) service_pool.cc:224] Handling call
0709 12:24:06.075473 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.076235 (+   762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.076439 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.078476 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.079273 10093 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.079842 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:06.080986 10017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2391) took 1482 ms (client timeout 1716 ms). Trace:
W20260709 12:24:06.081509 10017 rpcz_store.cc:269] 0709 12:24:04.598310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.031966 (+1433656us) service_pool.cc:224] Handling call
0709 12:24:06.080906 (+ 48940us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.082788  9842 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:24:06.086767 10094 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:06.086869  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.31721e+06us
W20260709 12:24:06.087072 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2389) took 1498 ms (client timeout 1907 ms). Trace:
W20260709 12:24:06.087795 10021 rpcz_store.cc:269] 0709 12:24:04.588410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.016078 (+1427668us) service_pool.cc:224] Handling call
0709 12:24:06.086967 (+ 70889us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.088002  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.089509  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.074129  9948 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.090448 11940 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.079058 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.091060  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1212) took 1011 ms. Trace:
I20260709 12:24:06.091566  9948 rpcz_store.cc:276] 0709 12:24:05.079456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.036350 (+956894us) service_pool.cc:224] Handling call
0709 12:24:06.091005 (+ 54655us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.093427  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.085795  9842 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:06.090324 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 254) took 1697 ms. Trace:
I20260709 12:24:06.050417  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1398) took 1490 ms. Trace:
I20260709 12:24:06.033612  9940 rpcz_store.cc:276] 0709 12:24:04.692235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.287948 (+595713us) service_pool.cc:224] Handling call
0709 12:24:05.991889 (+703941us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.099825  9947 rpcz_store.cc:276] 0709 12:24:04.560472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.048823 (+1488351us) service_pool.cc:224] Handling call
0709 12:24:06.050307 (+  1484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.091017 11940 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:06.058784 11713 thread.cc:641] log-alloc [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.006s	sys 0.025s
I20260709 12:24:06.104668  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.041905  9949 rpcz_store.cc:276] 0709 12:24:04.583794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:05.950771 (+1366977us) service_pool.cc:224] Handling call
0709 12:24:05.995940 (+ 45169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.101310 10096 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:06.047274 10018 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:06.098918 10021 rpcz_store.cc:276] 0709 12:24:04.392798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.088874 (+1696076us) service_pool.cc:224] Handling call
0709 12:24:06.089516 (+   642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.089921 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.089959 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.090020 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:06.090070 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.090147 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.090254 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.110123 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2388) took 2530 ms. Trace:
I20260709 12:24:06.111250 10016 rpcz_store.cc:276] 0709 12:24:03.575158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.077632 (+2502474us) service_pool.cc:224] Handling call
0709 12:24:06.100767 (+ 23135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.104582 (+  3815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.104623 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.104704 (+    81us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:06.104796 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.104864 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.104991 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.112944  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.112468 10019 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:06.112468  9948 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.110805  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.114494  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1213) took 1034 ms. Trace:
I20260709 12:24:06.114987  9948 rpcz_store.cc:276] 0709 12:24:05.079915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.100909 (+1020994us) service_pool.cc:224] Handling call
0709 12:24:06.114439 (+ 13530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.114355 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.067529 11959 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:06.116523 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:06.114419 10019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2379) took 2588 ms (client timeout 3073 ms). Trace:
I20260709 12:24:06.121402 10093 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.103837 11940 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:06.109901 11713 thread.cc:608] log-alloc [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.007s	sys 0.026s
W20260709 12:24:06.121675 10019 rpcz_store.cc:269] 0709 12:24:03.526102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.059516 (+2533414us) service_pool.cc:224] Handling call
0709 12:24:06.114358 (+ 54842us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.121781  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.124760 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2387) took 2550 ms. Trace:
I20260709 12:24:06.126511 10024 rpcz_store.cc:276] 0709 12:24:03.571511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.091562 (+2520051us) service_pool.cc:224] Handling call
0709 12:24:06.116776 (+ 25214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.121166 (+  4390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.121207 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.121261 (+    54us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:06.121306 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.121375 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.121488 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.130088 10022 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:06.129081 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.121133 11710 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.117097 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 257) took 1714 ms. Trace:
I20260709 12:24:06.135237 10098 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:06.130558  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1209) took 1050 ms. Trace:
I20260709 12:24:06.137065  9942 rpcz_store.cc:276] 0709 12:24:05.077844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.105896 (+1028052us) service_pool.cc:224] Handling call
0709 12:24:06.127958 (+ 22062us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.134037  9945 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.140383  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1218) took 1058 ms. Trace:
I20260709 12:24:06.140784  9945 rpcz_store.cc:276] 0709 12:24:05.081904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.100951 (+1019047us) service_pool.cc:224] Handling call
0709 12:24:06.140305 (+ 39354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.135819 10023 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:06.139315 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2382) took 2599 ms. Trace:
I20260709 12:24:06.146822 10022 rpcz_store.cc:276] 0709 12:24:03.533465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.071602 (+2538137us) service_pool.cc:224] Handling call
0709 12:24:06.111857 (+ 40255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.131750 (+ 19893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.132806 (+  1056us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.128559 11901 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.127465 11959 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.144979 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2383) took 2599 ms. Trace:
I20260709 12:24:06.152495 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.153505 10023 rpcz_store.cc:276] 0709 12:24:03.545326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.069712 (+2524386us) service_pool.cc:224] Handling call
0709 12:24:06.118046 (+ 48334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.144656 (+ 26610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.144909 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.155189 10092 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.136472 11710 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.160168  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.160727 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 256) took 1767 ms. Trace:
I20260709 12:24:06.161037  9940 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:06.161196 10016 rpcz_store.cc:276] 0709 12:24:04.393347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.112552 (+1719205us) service_pool.cc:224] Handling call
0709 12:24:06.145383 (+ 32831us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.160372 (+ 14989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.160405 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.160448 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:06.160489 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.160551 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.160671 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.161844  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 345) took 1516 ms. Trace:
I20260709 12:24:06.151451  9943 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.143918 11777 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:06.162896  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1210) took 1084 ms. Trace:
I20260709 12:24:06.163489  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.164609  9943 rpcz_store.cc:276] 0709 12:24:05.078497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.101772 (+1023275us) service_pool.cc:224] Handling call
0709 12:24:06.162832 (+ 61060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.149372  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:06.153226 11901 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:06.166728 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 253) took 1774 ms. Trace:
I20260709 12:24:06.172633 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1391) took 1762 ms. Trace:
I20260709 12:24:06.139219 10018 rpcz_store.cc:276] 0709 12:24:04.400890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.046048 (+1645158us) service_pool.cc:224] Handling call
0709 12:24:06.114884 (+ 68836us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:06.153431 11938 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.016s	user 0.022s	sys 0.073s
I20260709 12:24:06.175738  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.162410  9940 rpcz_store.cc:276] 0709 12:24:04.645962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.126830 (+1480868us) service_pool.cc:224] Handling call
0709 12:24:06.161753 (+ 34923us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.178997  9943 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.175618  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.172525 10093 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.175827  9942 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.173918 10017 rpcz_store.cc:276] 0709 12:24:04.392496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.082379 (+1689883us) service_pool.cc:224] Handling call
0709 12:24:06.112559 (+ 30180us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.156501 (+ 43942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.156543 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.162771 (+  6228us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:06.166470 (+  3699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.166533 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.166669 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.163211 11710 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.180162  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1211) took 1101 ms. Trace:
I20260709 12:24:06.183207  9942 rpcz_store.cc:276] 0709 12:24:05.079025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.143930 (+1064905us) service_pool.cc:224] Handling call
0709 12:24:06.180106 (+ 36176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:06.183801  9946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.848s	user 0.044s	sys 0.051s
W20260709 12:24:06.186533  9946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.044s	sys 0.051s
I20260709 12:24:06.184779  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 340) took 1752 ms. Trace:
I20260709 12:24:06.188656  9945 rpcz_store.cc:276] 0709 12:24:04.432503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.153485 (+1720982us) service_pool.cc:224] Handling call
0709 12:24:06.184692 (+ 31207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.178076 11901 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.174033 10090 rpcz_store.cc:276] 0709 12:24:04.406367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.645049 (+238682us) service_pool.cc:224] Handling call
0709 12:24:04.646321 (+  1272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:04.646649 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.646693 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:04.746720 (+100027us) log.cc:844] Serialized 26 byte log entry
0709 12:24:05.970038 (+1223318us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:05.970111 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.970251 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.162518 (+192267us) raft_consensus.cc:1685] finished
0709 12:24:06.162537 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.168413 (+  5876us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120576,"thread_start_us":1219425,"threads_started":1,"wal-append.queue_time_us":1413469}
W20260709 12:24:06.183562 11938 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.156s	user 0.022s	sys 0.074s
I20260709 12:24:06.200151  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:06.196805 10643 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.061s	sys 0.070s
W20260709 12:24:06.201908 10643 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.061s	sys 0.071s
I20260709 12:24:06.201632  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.208704  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1386) took 1818 ms. Trace:
W20260709 12:24:06.202383 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.56627e+06us
I20260709 12:24:06.191942 11821 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:06.226714 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.225752  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.229739 11965 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.230437 11965 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.231575 11978 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:06.232142 11978 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:06.232677 11978 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.229583  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.180713 11959 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.246481  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.221513  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:06.243705  9949 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:06.250442  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.232973 11965 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:06.249446 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.254812 10093 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:06.246580  9942 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:06.221232  9946 rpcz_store.cc:276] 0709 12:24:04.382434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.382577 (+   143us) service_pool.cc:224] Handling call
0709 12:24:05.161245 (+778668us) spinlock_profiling.cc:243] Waited 8.85 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:05.161712 (+   467us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:05.162316 (+   604us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:05.203479 (+ 41163us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.193479 (+990000us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.193543 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:06.193614 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.193703 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.200371 (+  6668us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26697,"prepare.run_cpu_time_us":18801,"prepare.run_wall_time_us":234796,"raft.queue_time_us":7310,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":9945,"spinlock_wait_cycles":20359552,"thread_start_us":927529,"threads_started":1}
I20260709 12:24:06.267797 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.272289  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.271303 10092 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:06.273140  9943 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:06.270493 11971 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:06.270758  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1217) took 1184 ms. Trace:
I20260709 12:24:06.299858  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.307173 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.310048  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.311252  9942 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:06.311797 10065 leader_election.cc:400] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:06.312163 10065 leader_election.cc:403] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:06.244976 11936 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 12:24:06.298275  9940 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:06.322506  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.324188 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.327044  9948 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:06.327848  9945 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:06.328521  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 343) took 1764 ms. Trace:
I20260709 12:24:06.328904  9945 rpcz_store.cc:276] 0709 12:24:04.563749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.201490 (+1637741us) service_pool.cc:224] Handling call
0709 12:24:06.328454 (+126964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.343580 11959 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:06.345144 11758 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:06.346923 11977 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.347412 11977 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.347785 11977 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.350122  9946 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:06.359995 10025 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:06.303956  9949 rpcz_store.cc:276] 0709 12:24:05.081510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.117402 (+1035892us) service_pool.cc:224] Handling call
0709 12:24:06.266115 (+148713us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.366557 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2393) took 1757 ms. Trace:
W20260709 12:24:06.368328  9944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.112s	sys 0.106s
I20260709 12:24:06.367597  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:06.370990  9944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.113s	sys 0.106s
I20260709 12:24:06.370827 10025 rpcz_store.cc:276] 0709 12:24:04.603248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.048588 (+1445340us) service_pool.cc:224] Handling call
0709 12:24:06.360670 (+312082us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.365458 10090 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.373301 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.370177 10092 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.390038 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.383045  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.398617  9948 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.410482 10020 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:06.413292 10093 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:24:06.415970  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.416150 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2380) took 2881 ms. Trace:
I20260709 12:24:06.416905 10020 rpcz_store.cc:276] 0709 12:24:03.530419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.069672 (+2539253us) service_pool.cc:224] Handling call
0709 12:24:06.371069 (+301397us) spinlock_profiling.cc:243] Waited 2.31 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431eee 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:06.371240 (+   171us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.410966 (+ 39726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.411162 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5307648}
I20260709 12:24:06.416671  9947 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:06.418982  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1401) took 1762 ms. Trace:
I20260709 12:24:06.419309  9947 rpcz_store.cc:276] 0709 12:24:04.656788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.116138 (+1459350us) service_pool.cc:224] Handling call
0709 12:24:06.415829 (+299691us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:06.418926 (+  3097us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12021120}
I20260709 12:24:06.420310  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.420884  9947 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:06.421701  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.422194  9947 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:06.422868 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.423029  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.423534  9947 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:06.424027  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.424417  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.424671  9949 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:06.425024  9947 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:06.425598  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.425846  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.426535  9947 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:06.426916 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.427611  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.430691 11969 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:24:06.431130  9947 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.432301  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.428839 10099 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:24:06.435421  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.436136  9946 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.437006  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.437669 10090 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.429986 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.352591 11971 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.440078 10094 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:06.412441  9943 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:06.441431  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1214) took 1361 ms. Trace:
I20260709 12:24:06.441864  9943 rpcz_store.cc:276] 0709 12:24:05.080401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.322073 (+1241672us) service_pool.cc:224] Handling call
0709 12:24:06.441375 (+119302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.437639  9946 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.427891 10096 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.446213  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.447496  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.444473  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.451746 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.452425 10096 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:24:06.460669 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.464387 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.465291 10094 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:24:06.467186  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.460539  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.469851 10096 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.472055 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:06.473572  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.471099 10097 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
W20260709 12:24:06.480230 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:06.481873  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.485476 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:06.487373 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:06.460877  9991 connection.cc:256] RPC call timeout handler was delayed by -1.1606707800001459s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16910000000000003s for a response
W20260709 12:24:06.489161  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2632761720001326s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
I20260709 12:24:06.490873 11727 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:06.491762  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1396) took 2015 ms. Trace:
I20260709 12:24:06.491575 11727 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:06.492168  9945 rpcz_store.cc:276] 0709 12:24:04.476220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.458378 (+1982158us) service_pool.cc:224] Handling call
0709 12:24:06.491221 (+ 32843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.491400 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.491426 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.491475 (+    49us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:06.491508 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.491565 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.491678 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.493592  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:06.494215  9945 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.496212 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:06.489074  9991 connection.cc:256] RPC call timeout handler was delayed by -1.039044209000167s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1738s for a response
W20260709 12:24:06.508083 11970 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.237s	user 0.001s	sys 0.001s
I20260709 12:24:06.510406 10099 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.470394  9949 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:06.518273  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:06.520047  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:06.520737  9949 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.486881  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1381) took 4531 ms. Trace:
I20260709 12:24:06.507879  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:06.527215 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.527714 10094 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.523499  9942 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:06.532022 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2447) took 2181 ms. Trace:
I20260709 12:24:06.506428  9943 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:06.538185 10096 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:06.546003 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.546381 10097 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.547005 10096 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:06.547178 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.547473  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:06.548466 10090 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
W20260709 12:24:06.551329  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:06.551882  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.554797  9942 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:06.555356 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.532464 10095 rpcz_store.cc:276] 0709 12:24:04.350758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.643690 (+292932us) service_pool.cc:224] Handling call
0709 12:24:04.689673 (+ 45983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:04.692859 (+  3186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:04.692893 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:04.751313 (+ 58420us) log.cc:844] Serialized 26 byte log entry
0709 12:24:05.972173 (+1220860us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:05.977654 (+  5481us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:05.989037 (+ 11383us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.523551 (+534514us) raft_consensus.cc:1685] finished
0709 12:24:06.523570 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.531951 (+  8381us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":68935,"raft.run_cpu_time_us":192,"raft.run_wall_time_us":21628,"thread_start_us":1092354,"threads_started":1,"wal-append.queue_time_us":1440534}
I20260709 12:24:06.557153 10099 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.558413  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:06.559459 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.560593  9947 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:06.561870  9946 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:06.557163 11978 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:06.566825  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59047.1us
I20260709 12:24:06.568456  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1390) took 2167 ms. Trace:
I20260709 12:24:06.568923  9946 rpcz_store.cc:276] 0709 12:24:04.397304 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.450759 (+2053455us) service_pool.cc:224] Handling call
0709 12:24:06.532140 (+ 81381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.564728 (+ 32588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.564953 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.567926  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 341) took 2128 ms. Trace:
I20260709 12:24:06.573735  9947 rpcz_store.cc:276] 0709 12:24:04.432873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.523369 (+2090496us) service_pool.cc:224] Handling call
0709 12:24:06.553165 (+ 29796us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.561052 (+  7887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.561325 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.523176  9944 rpcz_store.cc:276] 0709 12:24:01.955402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:03.549615 (+1594213us) service_pool.cc:224] Handling call
0709 12:24:03.582350 (+ 32735us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:03.582540 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:03.608101 (+ 25561us) log.cc:844] Serialized 14 byte log entry
0709 12:24:04.851631 (+1243530us) spinlock_profiling.cc:243] Waited 38.3 ms on lock 0x7b18003673c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:05.423327 (+571696us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b18003673c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:24:06.444517 (+1021190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.452045 (+  7528us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:06.452129 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.452234 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.486796 (+ 34562us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24187,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":5262,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":24302,"spinlock_wait_cycles":102821632,"thread_start_us":948056,"threads_started":1}
I20260709 12:24:06.572680 10093 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.536202 11940 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:06.579212 10062 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.020s	sys 0.040s
W20260709 12:24:06.583092 10062 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.020s	sys 0.040s
W20260709 12:24:06.583425 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 632461us
I20260709 12:24:06.590104  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:06.595359 10092 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:06.598464  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.011s	sys 0.029s
W20260709 12:24:06.599015  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.012s	sys 0.029s
W20260709 12:24:06.600178 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.028s	sys 0.038s
W20260709 12:24:06.600548 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.029s	sys 0.038s
I20260709 12:24:06.584753 11710 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:06.599449  9914 connection.cc:256] RPC call timeout handler was delayed by -1.2474832900002184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
I20260709 12:24:06.603289 10021 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:06.603495 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.025s	sys 0.036s
W20260709 12:24:06.603878 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.025s	sys 0.036s
W20260709 12:24:06.581871 11929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.027s	sys 0.033s
I20260709 12:24:06.607103 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:06.607923 10091 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.613541 11985 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:06.614217 11985 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.614718 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:06.615257 10677 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.615818  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 346) took 1533 ms. Trace:
I20260709 12:24:06.616179  9944 rpcz_store.cc:276] 0709 12:24:05.082310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.591548 (+1509238us) service_pool.cc:224] Handling call
0709 12:24:06.611484 (+ 19936us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.615457 (+  3973us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.615483 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.615526 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:06.615566 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.615626 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.615736 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.617708  9944 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:06.617799 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.619457  9945 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.612289 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:06.623242 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:06.627573 10673 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:06.628839 10677 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:06.630409 11710 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:06.631398 11727 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:06.634584 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48972.1us
I20260709 12:24:06.635748 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:06.637038 11983 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.637462 11983 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.637845 11983 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.639353  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 347) took 1556 ms. Trace:
I20260709 12:24:06.639868  9942 rpcz_store.cc:276] 0709 12:24:05.082691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.593831 (+1511140us) service_pool.cc:224] Handling call
0709 12:24:06.606752 (+ 12921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.638865 (+ 32113us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.638910 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.638968 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:06.639049 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.639138 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.639280 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:06.609046 11821 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:06.656700 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:06.659348  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 344) took 2042 ms. Trace:
I20260709 12:24:06.611632 11978 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:06.652299  9941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.001s	sys 0.004s
W20260709 12:24:06.663823  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 43587.8us
I20260709 12:24:06.674641 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:06.683542 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:06.664769  9941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.001s	sys 0.005s
I20260709 12:24:06.689146 10671 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:06.693933  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45845.8us
I20260709 12:24:06.647868 11821 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.672677 11768 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.699504 10099 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
W20260709 12:24:06.704124 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 460508us
I20260709 12:24:06.706022 10093 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:06.684492 11971 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:06.700753 11768 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.710135 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:06.699039  9940 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:06.711520 10670 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.712399 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.698630  9949 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.701388 11969 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:06.714663  9943 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:06.701260 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.710891 11768 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.660606 11730 raft_consensus.cc:359] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:06.727110 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.725289  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1387) took 2342 ms. Trace:
I20260709 12:24:06.729012  9941 rpcz_store.cc:276] 0709 12:24:04.382833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:04.393239 (+ 10406us) service_pool.cc:224] Handling call
0709 12:24:04.499206 (+105967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:04.519671 (+ 20465us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:06.038985 (+1519314us) log.cc:844] Serialized 14 byte log entry
0709 12:24:06.711968 (+672983us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.712058 (+    90us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:06.712157 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.712253 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.724080 (+ 11827us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14750,"prepare.queue_time_us":296341,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":21978,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":8693,"thread_start_us":1988992,"threads_started":2,"wal-append.queue_time_us":572952}
I20260709 12:24:06.737643 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2390) took 2148 ms. Trace:
I20260709 12:24:06.739548 11985 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:06.740804  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:06.659708  9947 rpcz_store.cc:276] 0709 12:24:04.617364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.583355 (+1965991us) service_pool.cc:224] Handling call
0709 12:24:06.602720 (+ 19365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.642577 (+ 39857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.642651 (+    74us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:06.642716 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:06.659069 (+ 16353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.659152 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.659294 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":38655}
I20260709 12:24:06.741616 11938 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.742340 11938 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:06.742786 11938 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.745504 11821 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:06.741807  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:06.748445 11978 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.748896 11978 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:06.749306 11978 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:06.751272 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.752166 10091 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.753599 11969 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:06.755138 11977 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:06.757329 11977 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:06.758457 11713 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:24:06.759642 11713 tablet_bootstrap.cc:492] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:24:06.760394 11713 ts_tablet_manager.cc:1403] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 3.025s	user 0.041s	sys 0.038s
I20260709 12:24:06.715366 11971 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.738274 10021 rpcz_store.cc:276] 0709 12:24:04.588799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.122751 (+1533952us) service_pool.cc:224] Handling call
0709 12:24:06.167698 (+ 44947us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:06.605358 (+437660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.737258 (+131900us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b540011ba54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:06.737515 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997552896}
W20260709 12:24:06.762360  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 423714us
I20260709 12:24:06.765457 10097 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:06.769852 11992 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:06.770295 11992 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:06.770869 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.771414 10095 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.772281 11993 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:06.773288 11991 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.773764 11991 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.774123 11991 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:06.782160 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61760.6us
I20260709 12:24:06.813220 11768 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.814759 11986 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:24:06.816002 11768 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:06.816205 11986 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:06.816558 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.767560 11971 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.817987 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.774150 11938 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.818702 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.819315 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.819854 11978 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.820111 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.820654 10670 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:06.821183 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.822118 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.823128 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.722657 11730 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:24:06.775872 11940 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:06.835568 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51845.6us
W20260709 12:24:06.842931  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 88163.2us
I20260709 12:24:06.846073 10020 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:06.847503 10021 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:06.850879  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 102931us
I20260709 12:24:06.853340 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.781286 11965 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:24:06.850416 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.857332 10092 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:06.850533 10024 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:06.861620 10016 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:06.835845 11971 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.826033 11986 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:06.881145 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:06.882535 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 223266us
I20260709 12:24:06.882833 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.883876 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.884905 10095 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:06.887285 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:06.887529 10022 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.888178 11993 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:06.888583 11993 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:06.888744 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:06.889004 11993 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:06.889737 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:06.891777  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.895279  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.896653 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.897272  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 342) took 2389 ms. Trace:
I20260709 12:24:06.897342 10023 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.884526 10017 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
W20260709 12:24:06.860671 11901 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.000s
I20260709 12:24:06.899673 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.902966 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.896682  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.905519 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:06.906406 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2392) took 2307 ms. Trace:
I20260709 12:24:06.909250 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.910141 10025 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:06.911047 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.906908 10019 rpcz_store.cc:276] 0709 12:24:04.598630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.129897 (+1531267us) service_pool.cc:224] Handling call
0709 12:24:06.168666 (+ 38769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.182665 (+ 13999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.182718 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.242565 (+ 59847us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.699405 (+456840us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:06.699455 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.699572 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.906175 (+206603us) raft_consensus.cc:1685] finished
0709 12:24:06.906193 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.906325 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":421235,"threads_started":1,"wal-append.queue_time_us":448426}
I20260709 12:24:06.911639 10017 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.897810  9948 rpcz_store.cc:276] 0709 12:24:04.507584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:06.478393 (+1970809us) service_pool.cc:224] Handling call
0709 12:24:06.515011 (+ 36618us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:06.515439 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:06.515464 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:06.515982 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 12:24:06.803907 (+287925us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:06.803966 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:06.804072 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:06.897051 (+ 92979us) raft_consensus.cc:1685] finished
0709 12:24:06.897069 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:06.897193 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"thread_start_us":287457,"threads_started":1,"wal-append.queue_time_us":351354}
I20260709 12:24:06.920814 11758 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:06.908282 11710 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:06.930389 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.931313 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.933524 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:06.934548  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.882946 11983 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:06.934204 10017 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:06.936899 10670 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:06.910845 11835 tablet_copy_client.cc:538] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:24:06.912043 11901 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:06.966029  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:06.936067 11710 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:06.975391 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:06.975971 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 86314.2us
W20260709 12:24:06.977172  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 98339.4us
W20260709 12:24:06.977970  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48817.5us
I20260709 12:24:06.924204 11992 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:06.973479 11710 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:06.938481 11940 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:06.911479 11730 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:06.991755 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:06.938380 12003 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:24:06.983668 11977 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:07.008982  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 126442us
I20260709 12:24:06.983490 11938 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:06.984891 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:07.012697 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:06.938818 11986 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.015511 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.017360 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.019116 11940 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.019527 11940 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.019858 11940 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.021699 11713 raft_consensus.cc:359] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:07.018256  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.010001 11977 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:07.032737 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:07.035753 10062 connection.cc:256] RPC call timeout handler was delayed by -1.4639636360002442s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18620000000000003s for a response
W20260709 12:24:07.036639 10062 connection.cc:256] RPC call timeout handler was delayed by -1.3657567170002949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:24:07.036962 10062 connection.cc:256] RPC call timeout handler was delayed by -1.3499563140003374s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18100000000000002s for a response
I20260709 12:24:07.022650 11713 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
W20260709 12:24:07.043248 10062 connection.cc:256] RPC call timeout handler was delayed by -1.2253223389998311s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
W20260709 12:24:07.043730 10062 connection.cc:256] RPC call timeout handler was delayed by -1.1956223390002378s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
W20260709 12:24:07.043979 10062 connection.cc:256] RPC call timeout handler was delayed by -1.02372233899996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
W20260709 12:24:07.044203 10062 connection.cc:256] RPC call timeout handler was delayed by -1.02372233899996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22490000000000002s for a response
I20260709 12:24:07.023043 11938 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:07.047289 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.026973 11768 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:24:07.048300 10669 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:07.041637 11977 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.050662 11758 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.047742 11986 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:07.063165  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 99620.9us
I20260709 12:24:07.049145 11856 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.052286 10674 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 1.
I20260709 12:24:07.066052 12017 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.055877 11938 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:07.029119 11993 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.077440  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.068902 11758 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.083797 11727 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:24:07.087687 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:07.090785 12006 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:24:07.091346 12008 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.091818 12008 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.092185 12008 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.094234 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.095197 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.095788 10676 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.081566 11856 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.095660 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:07.101656 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.067732 12010 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.079593 11901 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:07.107474  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.086460 11727 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:07.081369 12017 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:07.080832 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.110369 11727 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:07.119246 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.119626 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.120030 11727 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.120483 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.120754 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.121284 10673 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:07.121450 10677 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:07.121582 11727 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:07.122176 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.122340 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.112366 11856 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:07.112964 12007 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:24:07.113046 11938 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.151201 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.151664 11938 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.152400 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.153017 10677 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:07.090791 11983 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:07.142057 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.111763 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.155247 10669 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:07.123517 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.127292 11929 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.155889 11929 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.156234 11929 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.128835 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:07.116043 11758 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.159317 11977 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.129528 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:07.131266 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.131561 11985 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:07.132948 11978 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.133713 10091 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:07.140534 11991 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.143268 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.147269 10676 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:07.145828  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.148406 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:07.149546 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 154205us
I20260709 12:24:07.155462 10672 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:07.160527 11758 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:07.144515 12017 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.163728 11971 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:07.175890  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69666.1us
W20260709 12:24:07.164494  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57985.6us
I20260709 12:24:07.129294 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:07.124372 11940 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.165792 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.105794 12010 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:07.071321 11713 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:07.204768 12013 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.121s
I20260709 12:24:07.206386  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.207206  9842 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 308) took 5539 ms. Trace:
I20260709 12:24:07.207499 11936 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:07.208456  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:07.171028 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:07.173682 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:07.170305  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:07.178380 11991 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:07.181723 11971 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:07.182176 10094 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:07.182468 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.182842 10671 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:07.185197 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.175349 11993 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.189524 11959 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:07.208336 11936 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.213459 11959 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:07.214232 10096 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:07.190171 11992 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.190492 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.215987 11936 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.217423 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:07.218097 10099 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:07.210191 12010 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.218999 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:07.214489 11959 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.219585 10099 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.220104 11959 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:07.220585 10062 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:24:07.192516 11940 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.193063  9843 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 210) took 6747 ms. Trace:
I20260709 12:24:07.190172 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.194463 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.231581 10020 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:07.195595 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:07.195907 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:07.232791 10097 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:07.195459 10098 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:07.196436 10092 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:07.196563 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:07.170327 11985 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:07.232653 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:07.199852 11977 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.201282 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:07.166571 12000 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.207227 11938 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.180510 11730 ts_tablet_manager.cc:1434] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 1.698s	user 0.016s	sys 0.032s
I20260709 12:24:07.249243 10092 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:07.211546 10025 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:07.212445 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:07.191732 11710 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.222409 11986 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:07.223351 11835 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
I20260709 12:24:07.224509 12008 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.225992  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:07.260694 10062 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:24:07.226966 12013 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:07.261185 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:07.262392 12008 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.262681 11936 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:07.263167 10093 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:07.262987 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.229682 12017 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:07.266544 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 95590.8us
I20260709 12:24:07.266906 12017 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:07.236528 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64725.3us
I20260709 12:24:07.269033 11768 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.269424 11768 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.269794 11768 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.247767 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:07.272657 11985 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.274025 11985 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:07.274986 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.275774 10022 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:07.276289  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:07.276677 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.277211  9941 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:07.278093 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.278824  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:07.279407 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.279713  9947 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:07.281497  9949 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:07.253620 11977 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:24:07.254043  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108579us
I20260709 12:24:07.255220 10095 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:07.257066 10091 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:07.257393 11997 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.257274 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.257900 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.258114 10090 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:07.260224 11992 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:07.251060 11938 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.277549 10021 raft_consensus.cc:2368] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3.
I20260709 12:24:07.231204  9843 rpcz_store.cc:276] 0709 12:24:00.445568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:00.450081 (+  4513us) service_pool.cc:224] Handling call
0709 12:24:07.192859 (+6742778us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:01.951450 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:01.974327 (+ 22877us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:01.979211 (+  4884us) write_op.cc:435] Acquired schema lock
0709 12:24:02.000106 (+ 20895us) tablet.cc:662] Decoding operations
0709 12:24:02.193341 (+193235us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:02.198232 (+  4891us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:02.241266 (+ 43034us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:24:02.440488 (+199222us) tablet.cc:701] Row locks acquired
0709 12:24:02.500283 (+ 59795us) write_op.cc:260] PREPARE: finished
0709 12:24:02.552396 (+ 52113us) write_op.cc:270] Start()
0709 12:24:02.585204 (+ 32808us) write_op.cc:276] Timestamp: P: 1783599842549868 usec, L: 0
0709 12:24:02.586666 (+  1462us) op_driver.cc:348] REPLICATION: starting
0709 12:24:04.564117 (+1977451us) log.cc:844] Serialized 536 byte log entry
0709 12:24:06.433397 (+1869280us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09551e28 00007f4e0cf8c581 00007f4e0cf8bd58 00007f4e0cf909f1 00007f4e0cf909a9 00007f4e0cf90939 00007f4e0cf90901 00007f4e0cf8fb2d 0000000000388634 0000000000388479
0709 12:24:07.009352 (+575955us) op_driver.cc:464] REPLICATION: finished
0709 12:24:07.182643 (+173291us) write_op.cc:301] APPLY: starting
0709 12:24:07.182773 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:07.182900 (+   127us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:07.182943 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:07.183272 (+   329us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:07.183484 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:24:07.183527 (+    43us) write_op.cc:312] APPLY: finished
0709 12:24:07.183847 (+   320us) log.cc:844] Serialized 23 byte log entry
0709 12:24:07.184040 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:07.184141 (+   101us) write_op.cc:454] Released schema lock
0709 12:24:07.184177 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:07.184595 (+   418us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:07.184643 (+    48us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":156557,"apply.run_cpu_time_us":1964,"apply.run_wall_time_us":13436,"num_ops":1,"prepare.queue_time_us":1103888,"prepare.run_cpu_time_us":67584,"prepare.run_wall_time_us":4560140,"raft.queue_time_us":798871,"raft.run_cpu_time_us":18929,"raft.run_wall_time_us":109960,"replication_time_us":4395776,"spinlock_wait_cycles":4062779776,"thread_start_us":2622984,"threads_started":4,"wal-append.queue_time_us":1582071}]]}
I20260709 12:24:07.251343 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:07.243101  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 258813us
I20260709 12:24:07.207808  9842 rpcz_store.cc:276] 0709 12:24:01.694535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:01.753488 (+ 58953us) service_pool.cc:224] Handling call
0709 12:24:07.207139 (+5453651us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:06.433936 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:06.434655 (+   719us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:06.434702 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:24:06.434716 (+    14us) tablet.cc:662] Decoding operations
0709 12:24:06.435694 (+   978us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:06.435765 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:06.435791 (+    26us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:24:06.436570 (+   779us) tablet.cc:701] Row locks acquired
0709 12:24:06.436632 (+    62us) write_op.cc:260] PREPARE: finished
0709 12:24:06.436794 (+   162us) write_op.cc:270] Start()
0709 12:24:06.436937 (+   143us) write_op.cc:276] Timestamp: P: 1783599846436726 usec, L: 0
0709 12:24:06.436952 (+    15us) op_driver.cc:348] REPLICATION: starting
0709 12:24:06.774742 (+337790us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b5400329ea0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094d3520 00007f4e0955cb42 00007f4e09551f67 00007f4e09551da4 00007f4e0cf8c581 00007f4e0cf8bd58 00007f4e0cf909f1 00007f4e0cf909a9 00007f4e0cf90939 00007f4e0cf90901
0709 12:24:06.780386 (+  5644us) log.cc:844] Serialized 3307 byte log entry
0709 12:24:07.149551 (+369165us) op_driver.cc:464] REPLICATION: finished
0709 12:24:07.185017 (+ 35466us) write_op.cc:301] APPLY: starting
0709 12:24:07.185085 (+    68us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:07.185307 (+   222us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:07.185329 (+    22us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:07.186537 (+  1208us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:07.198961 (+ 12424us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:24:07.199016 (+    55us) write_op.cc:312] APPLY: finished
0709 12:24:07.199496 (+   480us) log.cc:844] Serialized 65 byte log entry
0709 12:24:07.199684 (+   188us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:07.199810 (+   126us) write_op.cc:454] Released schema lock
0709 12:24:07.199866 (+    56us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":11232,"reported_tablets":12,"spinlock_wait_cycles":3968,"child_traces":[["op",{"apply.queue_time_us":19061,"apply.run_cpu_time_us":2910,"apply.run_wall_time_us":15219,"mutex_wait_us":4136,"num_ops":8,"prepare.queue_time_us":208439,"prepare.run_cpu_time_us":7577,"prepare.run_wall_time_us":346778,"replication_time_us":674040,"spinlock_wait_cycles":47188864,"threads_started":1}]]}
I20260709 12:24:07.485033 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:07.375614  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.297161 10020 raft_consensus.cc:2368] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:07.488154 10023 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:07.421514 10099 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:07.310714  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:07.381422 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:07.393404 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.350014 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.490427 10025 raft_consensus.cc:2368] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3.
W20260709 12:24:07.378764  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 152890us
I20260709 12:24:07.488390  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:07.491163  9945 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:07.491858  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 283448us
I20260709 12:24:07.492483 10024 raft_consensus.cc:2368] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:07.275799  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:07.379245 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 369276us
I20260709 12:24:07.487339 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.249626 12000 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.424757 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:07.516400 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.497277  9942 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:07.254467 11730 ts_tablet_manager.cc:1467] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Tablet startup took 1790ms
I20260709 12:24:07.523010  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:07.523473 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.272575 11713 ts_tablet_manager.cc:1434] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.512s	user 0.015s	sys 0.005s
I20260709 12:24:07.523360  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:07.523665 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.523895 10065 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.524612  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.530038  9947 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.529505 10674 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:24:07.464742 12012 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:07.496454 10016 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:07.527007  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.532980 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.437610 11965 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.464864 12024 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:07.536270 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.533313 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:07.532985 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:07.278828 11710 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:07.424125 12018 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:07.539089 10023 raft_consensus.cc:2368] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:07.490983 11938 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.516120 11997 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.540918 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.249960 11929 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.542925 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:07.531469 11713 ts_tablet_manager.cc:1467] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Tablet startup took 770ms
I20260709 12:24:07.544613 10670 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:07.541589 12019 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:07.545481 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:07.545831  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58071.4us
I20260709 12:24:07.546108 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.552096 12009 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.552245 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.552556 12009 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:07.552784  9941 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:07.553011 12009 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.551483 10674 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:07.546659 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.547611  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.547787  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:07.522745 12000 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.511885 11977 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:07.549928 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.554783  9948 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.557210 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.603263 10023 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:07.534503 12012 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:07.557868  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.536711 11965 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.540748 11992 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.608232 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.608881 10671 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.539068  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.609968  9949 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.540170 12024 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:07.560992 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.560462 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.560642 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.614070 10022 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:07.562045 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:07.560240 12013 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:24:07.577603 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.578234  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:07.578210 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 198790us
W20260709 12:24:07.553959 12020 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:07.579634  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.579838 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.559299  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.548413 11997 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.550508 11940 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:07.550729 11993 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:07.584570  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:07.592739 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.595625 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.554643 12018 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:07.581622 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:07.601090  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.605798 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:07.606046 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 73169.8us
I20260709 12:24:07.578401 11986 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:07.650405 11986 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.650472 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:07.650902 11986 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:07.651705 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:07.560127 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:07.652575 10018 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:07.598515 12021 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:07.622222 10670 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:07.655428  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 77277.3us
I20260709 12:24:07.656172  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.656306 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:07.656994  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.657573 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:07.658972 11936 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:07.659622 11936 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:07.659652 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.625341 10021 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:07.660533 10677 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:07.661372 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:07.630945  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51391.6us
I20260709 12:24:07.664027  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.661379 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.665697 10677 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:07.631547 11710 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.666550 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.666774 11758 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.612720 11965 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.667315 11758 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.667263 10677 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:07.667860 11758 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:07.639475  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84923.5us
I20260709 12:24:07.640161 12017 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.668710 11940 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:07.669098 12017 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.669564 12017 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.670615  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.671243 12018 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.673307 11978 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:07.668653  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:07.673906 11978 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:07.632447 11977 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.646045 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:07.646157  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 68982.4us
I20260709 12:24:07.648440 11999 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.649214 12024 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.621888 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.661872 12010 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:07.662856 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 126805us
I20260709 12:24:07.608234 11992 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.663033 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.699173 12001 raft_consensus.cc:2764] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:07.613284  9940 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.699864 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.700168 11992 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.700999 10673 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:07.701938 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.663096 12022 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.702826 12022 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:07.703081 12024 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.703243 12022 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.638371  9948 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:07.668820 11936 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:07.641741 11768 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.630762 10024 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
W20260709 12:24:07.643265 12025 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:24:07.648263 10020 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:07.677592  9947 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:07.668666 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:07.669914 11940 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.680763 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.684893 10671 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:07.691123 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.695664 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:07.699923 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.666771 11710 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:07.690722 12021 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:07.667282 11929 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:07.715996  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45469.9us
I20260709 12:24:07.684084 11999 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:07.724272 12015 raft_consensus.cc:2764] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:24:07.702497 10673 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.707314 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:07.681588 12003 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
W20260709 12:24:07.735608 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.725067 10096 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:07.751544 12013 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.751762  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.752800  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:07.753770 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85197.8us
I20260709 12:24:07.732239 11710 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.736493 11999 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.758914  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.720961 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:07.759292 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 107661us
I20260709 12:24:07.756379 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.759496  9948 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:07.760262  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 201048us
W20260709 12:24:07.766121  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 85696.4us
W20260709 12:24:07.527643 11730 ts_tablet_manager.cc:1469] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Trace:
0709 12:23:55.183212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.187641 (+  4429us) service_pool.cc:224] Handling call
0709 12:23:56.124971 (+937330us) ts_tablet_manager.cc:934] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer cd983415389444e292b2330ef630850a (127.8.137.193:34909)
0709 12:23:57.791857 (+1666886us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:23:58.291115 (+499258us) tablet_metadata.cc:665] Metadata flushed
0709 12:23:58.291243 (+   128us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:23:58.533525 (+242282us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:23:58.533966 (+   441us) tablet_replica.cc:206] Creating consensus instance
0709 12:23:58.534344 (+   378us) ts_tablet_manager.cc:989] Replica registered.
0709 12:23:58.534415 (+    71us) inbound_call.cc:177] Queueing success response
0709 12:24:00.654326 (+2119911us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:05.482554 (+4828228us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:05.484720 (+  2166us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:06.037442 (+552722us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":54807,"mutex_wait_us":2395,"raft.queue_time_us":71237,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":20514,"tablet-copy.queue_time_us":909241,"tablet-download-pool.queue_time_us":42538,"tablet-download-pool.run_cpu_time_us":1655,"tablet-download-pool.run_wall_time_us":1005678,"thread_start_us":2234568,"threads_started":3}
I20260709 12:24:07.777750 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:07.767405  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 110476us
I20260709 12:24:07.774320 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:07.778901  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:07.780210  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.756376 11993 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:07.781337 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81510.6us
I20260709 12:24:07.788357  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.776162 11971 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.793067 11768 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:07.793340 11971 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:07.793860 11971 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.790252 10669 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 2: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:07.794862 11997 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.782035 11821 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:24:07.791512  9940 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:07.792119 12000 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.792850 11727 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:07.775725  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.797986 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.725004 12012 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:07.787385 11936 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.845254  9948 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
W20260709 12:24:07.847250 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59597.7us
I20260709 12:24:07.796465 12001 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:07.851471  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.852892 10092 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:07.853595  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.858037 10673 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:07.858551 11992 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:07.861784  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 88289.5us
I20260709 12:24:07.837225 11993 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:07.772781 11978 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.865898  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:07.838732 11727 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:07.847307 11936 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:07.868783 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78830us
W20260709 12:24:07.869128  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94353.9us
I20260709 12:24:07.799593 11983 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 12:24:07.870178  9943 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
W20260709 12:24:07.549595 11713 ts_tablet_manager.cc:1469] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Trace:
0709 12:23:55.120205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:23:55.123086 (+  2881us) service_pool.cc:224] Handling call
0709 12:23:55.155528 (+ 32442us) ts_tablet_manager.cc:934] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
0709 12:23:58.017712 (+2862184us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:23:58.780695 (+762983us) tablet_metadata.cc:665] Metadata flushed
0709 12:23:58.780823 (+   128us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:23:59.103610 (+322787us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:23:59.119672 (+ 16062us) tablet_replica.cc:206] Creating consensus instance
0709 12:23:59.120379 (+   707us) ts_tablet_manager.cc:989] Replica registered.
0709 12:23:59.120458 (+    79us) inbound_call.cc:177] Queueing success response
0709 12:24:03.734874 (+4614416us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:24:06.760706 (+3025832us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:24:06.897180 (+136474us) tablet_replica.cc:261] Starting instrumentation
0709 12:24:06.898005 (+   825us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":41410,"mutex_wait_us":74591,"raft.queue_time_us":67268,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"tablet-copy.queue_time_us":28709,"tablet-download-pool.queue_time_us":140,"tablet-download-pool.run_cpu_time_us":1123,"tablet-download-pool.run_wall_time_us":1849759,"thread_start_us":44338,"threads_started":2}
I20260709 12:24:07.847124  9949 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:07.869813  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:07.871520 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:07.873850 11936 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.876686  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.892109  9942 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:07.895035  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 116200us
I20260709 12:24:07.896129  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:07.897981  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60726.4us
W20260709 12:24:07.918220 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.859416 12003 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.019s
I20260709 12:24:07.926359  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.925209 11727 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.877374 11959 raft_consensus.cc:2764] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260709 12:24:07.932044  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56863.7us
W20260709 12:24:07.931082 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55510.5us
W20260709 12:24:07.932830 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 158597us
I20260709 12:24:07.933198 12040 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:07.937283  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.940047 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:07.942121  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46126.9us
I20260709 12:24:07.943272  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.943931 12032 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:07.944458 12032 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:07.944864 12032 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:07.943334 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71862.9us
I20260709 12:24:07.931349 11993 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:07.948925  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.949723  9943 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:07.950773  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:07.931345 12010 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.934676 12018 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:07.956545 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:07.957407 12036 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.955647 12018 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:07.957917 12036 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:07.958132 11992 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.958252 12018 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:07.958371 12036 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.958890 11992 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:07.959319 11992 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:07.959703 11992 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:07.946249 12012 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:07.963146 12027 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:24:07.964887 11938 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:07.965683 11965 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:07.939585 12031 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:07.970844 12031 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:07.971259 12031 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.973204 11938 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:07.962090 12015 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:24:07.957078 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 156839us
I20260709 12:24:07.933761 11997 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:07.975122 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:07.948312 11985 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.674s	user 0.011s	sys 0.001s
I20260709 12:24:07.985554 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:07.954916  9941 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:07.952620 11993 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:08.002153  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:07.998553 11727 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:08.015650 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 58958.1us
I20260709 12:24:08.016736 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:08.018955 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:07.995630 12015 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:08.022549 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:08.023342  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 153601us
I20260709 12:24:08.023924  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:08.025343  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:24:08.007091 12044 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
W20260709 12:24:08.025403  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 97475.1us
I20260709 12:24:07.971277 11768 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:24:08.015314 12038 raft_consensus.cc:1064] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:24:08.036785  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:08.014386 11993 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.001499 12009 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:08.003331 12039 raft_consensus.cc:2764] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:24:08.022893 11985 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.046262 12047 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:08.046933 11758 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.048844 12046 raft_consensus.cc:2764] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:08.049923 12039 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.050371 12039 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.050756 12039 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.052861  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.055084 12021 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:08.055608 12021 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.056010  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:08.028645 12003 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.056557 12021 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:08.060075  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.060487  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.061276  9949 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:08.067077  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.067993  9943 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:08.025003 12015 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:08.040732 12017 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.074676 12000 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.075085 12037 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:08.075287 12017 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.075723 12037 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.075942 12017 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.076328 12017 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.076690 12017 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.076761 12037 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.040560 12040 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.057758 11929 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.078966 12040 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.079170 11929 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.079461 12040 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.080323 11929 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.081832 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.083191 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.084254 10065 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:08.089205 11758 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.089996 12013 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:08.093433  9947 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:08.113974  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.115521 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 74149.3us
W20260709 12:24:08.105826  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78781.5us
I20260709 12:24:08.116557 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.123297  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.124117  9943 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:08.125982 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 55672.2us
W20260709 12:24:08.133844  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 131786us
I20260709 12:24:08.134984  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:08.121698 12013 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.121665 12000 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.127166 12047 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.121788 12010 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.117163 12003 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:08.141083 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.126323 11959 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:08.146463 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 211371us
W20260709 12:24:08.155220  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 215432us
I20260709 12:24:08.127288 12001 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.127009 11985 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:08.150960  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94983.9us
I20260709 12:24:08.153050 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:08.171505  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.128507 11971 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:08.156263 11986 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.002s
I20260709 12:24:08.152066 12047 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:08.152802 12010 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:08.147800 12000 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:08.155448 11959 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:08.186249 12010 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.188377 12000 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.188205 11821 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.183952 11965 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:08.226577  9943 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:08.209767 11983 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:08.190948 11959 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.183951 12047 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.214608 12003 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.215863 12035 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.216508  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 received from {username='slave'} at 127.0.0.1:38784
I20260709 12:24:08.216604 11971 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:08.216957 12054 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:08.218415  9949 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:08.219018  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.188897  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:08.266055  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100665us
I20260709 12:24:08.156454 12015 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:08.246789 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 113727us
I20260709 12:24:08.247731  9942 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:08.264405  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.183696 11986 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.192291 11710 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.311165 12039 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.289361 11983 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:08.249827 11821 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:08.339721 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:08.343463 12050 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:24:08.364711 12019 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:08.348574 12036 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:08.351989 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 101005us
W20260709 12:24:08.352600  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80403.6us
W20260709 12:24:08.367590  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 137122us
I20260709 12:24:08.404069  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:08.365106  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 176496us
W20260709 12:24:08.372677  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 201251us
I20260709 12:24:08.250284 11768 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.105s
W20260709 12:24:08.352794 12022 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.004s
I20260709 12:24:08.256472 11985 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.391408  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:08.341940  9941 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:08.246447 12060 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.167s
I20260709 12:24:08.344573 12018 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.397221 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:08.400302  9946 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:08.394797  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 on tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:08.367815 11965 raft_consensus.cc:2764] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:24:08.366729 12054 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:08.406808  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.364642 12064 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.320432 11936 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.407907  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:08.408893  9948 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:08.388309 12035 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.392602 12009 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:08.440900  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49580.1us
I20260709 12:24:08.402995 12019 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:08.364898 12032 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.429273 11978 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:24:08.464195  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56348.8us
I20260709 12:24:08.480196  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.428966 12001 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:08.481593 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53878.2us
I20260709 12:24:08.484230 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.485908 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:08.486258  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 82595.5us
I20260709 12:24:08.489993 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:08.461925 12024 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.758s	user 0.013s	sys 0.000s
I20260709 12:24:08.448653 12054 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.499509 10096 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.455119 12045 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:08.454963 12053 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:08.512671 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.454320 12046 raft_consensus.cc:2764] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:24:08.443228 11727 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.501070  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:08.448684 12064 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:08.414384 12031 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.459475 11929 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:08.456015 11936 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.510710 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.513310 10093 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.432102 12022 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.515594 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.518195 11758 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:08.545962  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45004.8us
I20260709 12:24:08.518853 12065 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:08.519281 12062 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:08.554728  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:08.534318 11999 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.758s	user 0.008s	sys 0.003s
I20260709 12:24:08.523756  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:08.528165  9943 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:08.528968  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.527329 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:08.560228  9949 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:08.471470 11992 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:08.564158  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 83067.6us
I20260709 12:24:08.565303  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:08.533159 10094 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.535526  9942 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.544389 10096 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
W20260709 12:24:08.552940 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 164552us
W20260709 12:24:08.553154 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 63256us
I20260709 12:24:08.454797 12063 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.143s
I20260709 12:24:08.521052 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.523765 11727 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:08.562156 10090 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:08.527135 12045 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.527983 12064 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.572649 12024 raft_consensus.cc:2764] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:24:08.576056  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:08.576645  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.552348 12065 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:08.554015 11758 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.594990  9943 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:08.595067 12065 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:08.578310  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.515954 11993 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.594800 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.597090 10095 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.560885 11936 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:08.577127  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.559218 12062 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:08.587875  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.533910 12040 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.557194 11978 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.609549 10097 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:08.612118  9942 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 4. Ops: []
I20260709 12:24:08.580020 12046 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:24:08.590652  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.613862 12048 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:08.614485 12084 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.614995 12058 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:08.615175 12081 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.615821 11936 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:08.659595 12039 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.581300 11727 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.617141 12018 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.530045 12001 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.512229 11821 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.567639 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:08.586241 10091 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:08.659710 11936 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.557875 12035 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.658488 12078 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:08.702131 12078 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:08.703178 12078 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.703405 11983 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.704453 11983 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.705066 10093 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:08.661024 10099 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:08.706414 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-230693f05eb74e8095e27d669f16c8ee received from {username='slave'} at 127.0.0.1:57396
I20260709 12:24:08.707005 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-230693f05eb74e8095e27d669f16c8ee on tablet 230693f05eb74e8095e27d669f16c8ee with peer 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:08.708523 12071 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:24:08.661581  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.646394 12048 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:08.661468  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:08.665314  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 109601us
I20260709 12:24:08.701542 12032 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.618193 11986 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:08.563444 11999 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.646466 12046 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:08.654656 10098 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:08.607957 12034 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.243s
I20260709 12:24:08.701424 11758 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:08.704536 12078 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.717378 10095 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:08.662861  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.717368 10094 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:08.617939 12062 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:08.671722 12084 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.729014  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.697206 12036 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.734193  9943 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:08.749156  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:08.750553 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:08.752831 10093 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:08.568028 12019 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.753281  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:08.753888  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:08.755687 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.756623 10094 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:08.698346 11929 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.762420  9947 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:08.758888  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.764788 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:08.735994 12046 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.766547  9945 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.765573 10095 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:08.783002  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:08.785392  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:08.768046 10020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "062c203e13aa4527a094ec24339001e5"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:08.787739 12082 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:08.792332  9944 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:08.811843 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.768247 12088 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:08.780051 12019 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:08.816430  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.816058  9949 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:08.815376 11992 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.781075 12081 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:08.817229 12066 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:08.794289 12036 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:24:08.818720 12000 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.000s
I20260709 12:24:08.793855 12058 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:08.711828 12045 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:08.832307 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.759670 11978 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.833127 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.820165 12031 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:08.818917 12022 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:08.834966 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:08.835677 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44684.8us
I20260709 12:24:08.836321 10024 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:08.823952 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:08.823441 12048 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.840757 12048 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:08.834883 12017 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.758s	user 0.012s	sys 0.000s
W20260709 12:24:08.876361 12047 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.003s
I20260709 12:24:08.876879 12047 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:08.870409 11993 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:08.879400 10096 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.887902 10016 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.888643 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.887948 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.889200 10022 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:08.836726 12058 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:08.890661 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.827718 12081 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:08.903641 12085 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:24:08.909161 10024 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.833250 12000 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.829211 12066 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.924727  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:08.925576  9943 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:08.927615 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:08.912391 11959 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.000s
I20260709 12:24:08.903184 12083 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:08.931208 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.933645 10024 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:08.930790 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:08.838394 11929 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.885187 12017 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:08.926851 12013 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.769s	user 0.011s	sys 0.000s
I20260709 12:24:08.944272 12013 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.942371 10016 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.882548 12054 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:08.944741 12094 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:24:08.944411 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.943153 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.955487 10019 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:08.837499 12084 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:08.937073 11999 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:08.961975 10020 raft_consensus.cc:606] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:08.962507 10020 raft_consensus.cc:614] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:08.963505  9946 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:08.964411 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:08.951754 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:08.969451 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:08.932073 11959 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.991734 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.970783 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:08.974222 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.974524 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:08.975000 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:08.975656 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:08.974236 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:08.970003 10019 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:08.989337 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:09.048262  9947 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:09.040572 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:08.982232  9912 connection.cc:256] RPC call timeout handler was delayed by -2.1787303090000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
I20260709 12:24:09.041359 10025 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:24:08.991541 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:08.989042 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:09.039829 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:08.939803 12083 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:08.961951 12058 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:09.814845 10021 raft_consensus.cc:2368] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4.
W20260709 12:24:09.124934  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.816952  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1447) took 1087 ms. Trace:
W20260709 12:24:09.005030 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.827780 10677 raft_consensus.cc:2376] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 3. Re-sending same reply.
I20260709 12:24:09.034345 10024 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:09.336973 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:09.052577 10674 raft_consensus.cc:2376] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 4. Re-sending same reply.
I20260709 12:24:09.827607  9946 rpcz_store.cc:276] 0709 12:24:08.729736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.760555 (+ 30819us) service_pool.cc:224] Handling call
0709 12:24:08.778893 (+ 18338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.968499 (+189606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.816652 (+848153us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400206f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:09.816884 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":424157952}
I20260709 12:24:09.229192 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:09.830263 10098 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:09.120187 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:09.115680 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:09.834311 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:09.723256 10062 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.014s	sys 0.009s
W20260709 12:24:09.842978 10062 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.921s	user 0.017s	sys 0.010s
I20260709 12:24:09.840112 10023 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:09.843475 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 922149us
W20260709 12:24:09.844378 10062 leader_election.cc:336] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.704s (SENT)
I20260709 12:24:09.849978 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2475) took 1252 ms. Trace:
I20260709 12:24:09.052301 10016 raft_consensus.cc:2376] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 4. Re-sending same reply.
W20260709 12:24:09.231501 12072 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.571s	user 0.005s	sys 0.000s
W20260709 12:24:09.451236 12106 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
W20260709 12:24:09.485190 12065 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.889s	user 0.005s	sys 0.006s
W20260709 12:24:08.990834 12012 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.028s	user 0.001s	sys 0.012s
W20260709 12:24:09.227913 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:09.489032 10091 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.780s	user 0.010s	sys 0.000s
I20260709 12:24:09.865229 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:09.365387 12092 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.209s	user 0.000s	sys 0.001s
I20260709 12:24:09.866915 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:09.776170 12078 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.028s	user 0.007s	sys 0.005s
W20260709 12:24:09.450091 12105 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
I20260709 12:24:09.867769 10017 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:09.868863 10670 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:09.868793 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:09.869637  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1450) took 1033 ms. Trace:
I20260709 12:24:09.153695 12013 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:09.758060 12081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.021s	sys 0.058s
I20260709 12:24:09.110394 10675 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
W20260709 12:24:09.870914 12081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.896s	user 0.026s	sys 0.059s
I20260709 12:24:09.870721 12013 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.862517  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:09.695962 11993 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:09.871811  9946 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:09.872069 11993 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:09.872817  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:09.873944  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:09.472795 12019 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.001s
W20260709 12:24:09.773741 12048 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.932s	user 0.010s	sys 0.000s
W20260709 12:24:09.119289 12097 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 12:24:09.874986 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:09.009254 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:09.842123 10092 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:09.876428 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2471) took 1301 ms. Trace:
I20260709 12:24:09.876760 10092 rpcz_store.cc:276] 0709 12:24:08.575316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.110323 (+535007us) service_pool.cc:224] Handling call
0709 12:24:09.876372 (+766049us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:09.289381 11835 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:24:09.877892 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:08.986667 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:08.977607 12047 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:09.044224 11959 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:09.845647 12046 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.063s	user 0.011s	sys 0.003s
I20260709 12:24:09.881152 12046 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:09.882030 10673 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:09.825225 12066 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:09.856050 11985 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.100231 11727 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:09.883196 11985 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:09.883618 12046 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:09.865361 10091 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:09.884689 11727 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:09.867044  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 375) took 1650 ms. Trace:
I20260709 12:24:09.042820 12017 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:09.170076 12103 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:09.883829  9945 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:09.884636 10091 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1440) took 1389 ms (client timeout 1682 ms). Trace:
W20260709 12:24:09.475114 12062 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.720s	user 0.007s	sys 0.004s
I20260709 12:24:09.885278 11985 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:09.044724 12010 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.819s	user 0.006s	sys 0.003s
I20260709 12:24:09.870034  9949 rpcz_store.cc:276] 0709 12:24:08.836678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.836820 (+   142us) service_pool.cc:224] Handling call
0709 12:24:08.837151 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.837685 (+   534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:08.837716 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:08.869551 (+ 31835us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.969258 (+ 99707us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:08.969299 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.969363 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.869419 (+900056us) raft_consensus.cc:1685] finished
0709 12:24:09.869440 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.869576 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":521,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":60048,"threads_started":1,"wal-append.queue_time_us":253565}
I20260709 12:24:09.856527 10098 rpcz_store.cc:276] 0709 12:24:08.578078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.228814 (+650736us) service_pool.cc:224] Handling call
0709 12:24:09.830855 (+602041us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:09.885394  9940 rpcz_store.cc:276] 0709 12:24:08.216236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.247442 (+ 31206us) service_pool.cc:224] Handling call
0709 12:24:08.390316 (+142874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.430792 (+ 40476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:08.433423 (+  2631us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:08.548046 (+114623us) log.cc:844] Serialized 26 byte log entry
0709 12:24:08.793315 (+245269us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:08.793889 (+   574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:08.816907 (+ 23018us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:09.866793 (+1049886us) raft_consensus.cc:1685] finished
0709 12:24:09.866823 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.866985 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10212,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":36152,"thread_start_us":115010,"threads_started":1,"wal-append.queue_time_us":304428}
I20260709 12:24:08.987850 12000 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:09.833740 12054 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:09.874455  9946 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:09.900856 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2476) took 1300 ms. Trace:
W20260709 12:24:09.901906 10062 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.774s (SENT)
I20260709 12:24:09.904608 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:09.912981 10092 rpcz_store.cc:276] 0709 12:24:08.578469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.877640 (+1299171us) service_pool.cc:224] Handling call
0709 12:24:09.878551 (+   911us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:09.906859  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:08.977574 12064 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:09.912320 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1441) took 1396 ms. Trace:
I20260709 12:24:09.913251 10025 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:09.907222  9942 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:09.914649 10098 rpcz_store.cc:276] 0709 12:24:08.512634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.907645 (+1395011us) service_pool.cc:224] Handling call
0709 12:24:09.908064 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.908574 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.908623 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:09.908691 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:09.908734 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:09.908845 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:09.909016 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:09.914047 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:09.918283 11993 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:09.906937 12010 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:09.930124 10093 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:09.899822 12065 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:09.936688 10062 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.875s (SENT)
I20260709 12:24:09.940227 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:09.900070 10091 rpcz_store.cc:269] 0709 12:24:08.495011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.520787 (+ 25776us) service_pool.cc:224] Handling call
0709 12:24:09.884527 (+1363740us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:09.905696 12062 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:09.934943 11993 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:09.941663  9940 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:09.951913  9949 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:09.871493 12013 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:09.948684 10097 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:09.908222 12103 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:09.964756  9943 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:09.970202  9949 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:09.963375 11993 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:09.968516 12010 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:09.969297  9940 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:09.970839 10025 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:09.966879 10062 leader_election.cc:336] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.975s (SENT)
I20260709 12:24:09.984269 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:09.994997  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:09.996161 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59744 (request call id 13) took 1248 ms. Trace:
I20260709 12:24:09.996811 10020 rpcz_store.cc:276] 0709 12:24:08.749647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.755888 (+  6241us) service_pool.cc:224] Handling call
0709 12:24:09.983217 (+1227329us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400265cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09550853 00007f4e0e51e60a 00007f4e07aa47d4 00007f4e07aa4762 00007f4e07aa4691 00007f4e07aa460c 00007f4e07aa38c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:09.983384 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":399985024}
I20260709 12:24:09.999810 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:10.002111 10062 leader_election.cc:336] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.024s (SENT)
I20260709 12:24:10.003508 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.005787 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:10.006335 10020 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:09.992728 12047 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:10.043404 10062 leader_election.cc:336] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.212s (SENT)
I20260709 12:24:10.043990 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:10.041585 12010 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:10.036722 12012 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.041285 10091 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:10.052964 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.055357 10025 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:10.034668 12054 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.046456 11993 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:10.060025 10062 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.226s (SENT)
W20260709 12:24:10.065534 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.071806 12065 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.072208 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2477) took 1493 ms. Trace:
I20260709 12:24:10.072714 10091 rpcz_store.cc:276] 0709 12:24:08.578758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.965314 (+1386556us) service_pool.cc:224] Handling call
0709 12:24:09.993493 (+ 28179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.057189 (+ 63696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.072135 (+ 14946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.080405 10025 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:10.081718 10025 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.083056 12051 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.083359  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:10.074594  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.083624 12051 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.084213  9949 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:10.060950 12054 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:10.084908  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1451) took 1162 ms. Trace:
I20260709 12:24:10.085517  9947 rpcz_store.cc:276] 0709 12:24:08.922876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.923035 (+   159us) service_pool.cc:224] Handling call
0709 12:24:08.923432 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.052678 (+129246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.084709 (+1032031us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400222ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:10.084855 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286484352}
W20260709 12:24:10.083673 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2479) took 1414 ms (client timeout 1868 ms). Trace:
I20260709 12:24:10.087248 12083 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:10.087790 10093 rpcz_store.cc:269] 0709 12:24:08.669712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.781372 (+111660us) service_pool.cc:224] Handling call
0709 12:24:10.083496 (+1302124us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400164c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:10.083616 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2569162624}
I20260709 12:24:10.089288 10093 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:10.089375 12103 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:10.091279 10021 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.091748 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:10.092520 12047 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.093935 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:10.051867 12000 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.074448 10091 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:10.095180 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 280) took 1429 ms. Trace:
I20260709 12:24:10.095227 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 523) took 1590 ms. Trace:
I20260709 12:24:10.095453 12013 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:10.094368 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 606) took 1123 ms. Trace:
I20260709 12:24:10.077929 12038 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.096217 10670 rpcz_store.cc:276] 0709 12:24:08.971244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.974774 (+  3530us) service_pool.cc:224] Handling call
0709 12:24:10.094136 (+1119362us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b54007b0cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:10.094268 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877711104}
I20260709 12:24:10.058473 12012 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.097520 11710 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.095716 10673 rpcz_store.cc:276] 0709 12:24:08.504681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.510163 (+  5482us) service_pool.cc:224] Handling call
0709 12:24:08.964234 (+454071us) spinlock_profiling.cc:243] Waited 4.48 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:10.095064 (+1130830us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b54006dfe54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:10.095173 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2103748224}
I20260709 12:24:10.098812 11993 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.099076 12012 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:10.048444 12010 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.101786 12010 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:10.089488 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.103338 10025 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:10.104995 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2420) took 1114 ms. Trace:
I20260709 12:24:10.105279 11985 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.105417 10019 rpcz_store.cc:276] 0709 12:24:08.990430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.855750 (+865320us) service_pool.cc:224] Handling call
0709 12:24:10.089348 (+233598us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:10.104927 (+ 15579us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41134464}
I20260709 12:24:10.105809 11985 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.084828  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:10.107827 11985 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:10.108167  9940 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:10.111032 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1432) took 1625 ms (client timeout 1798 ms). Trace:
W20260709 12:24:10.111143  9944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1422) took 2359 ms (client timeout 1819 ms). Trace:
W20260709 12:24:10.111449 10099 rpcz_store.cc:269] 0709 12:24:08.485206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.485676 (+   470us) service_pool.cc:224] Handling call
0709 12:24:10.110834 (+1625158us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400305554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:10.110977 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393502080}
W20260709 12:24:10.111577  9944 rpcz_store.cc:269] 0709 12:24:07.751310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.751475 (+   165us) service_pool.cc:224] Handling call
0709 12:24:10.110983 (+2359508us) spinlock_profiling.cc:243] Waited 756 ms on lock 0x7b540042e354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:10.111077 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1739828992}
I20260709 12:24:10.112563  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.113466  9947 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:10.090318 12103 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.114512 12103 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:10.110002  9946 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:10.116863  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1452) took 1062 ms. Trace:
I20260709 12:24:10.116719 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.117631  9946 rpcz_store.cc:276] 0709 12:24:09.054843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.873704 (+818861us) service_pool.cc:224] Handling call
0709 12:24:10.116807 (+243103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.122979 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 281) took 1456 ms. Trace:
I20260709 12:24:10.123425 10093 rpcz_store.cc:276] 0709 12:24:08.666267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.088702 (+1422435us) service_pool.cc:224] Handling call
0709 12:24:10.089057 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.122715 (+ 33658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.122922 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:10.063066  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4209303090001413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2723s for a response
W20260709 12:24:10.125519  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0113666909996937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 12:24:10.129930 10670 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:10.095631 10091 rpcz_store.cc:276] 0709 12:24:08.665913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.073715 (+1407802us) service_pool.cc:224] Handling call
0709 12:24:10.074189 (+   474us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.094884 (+ 20695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.095106 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:10.138674  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.139423  9941 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:10.139798 11835 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 12:24:10.149297 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.131233 11985 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.130162 12083 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:10.179632  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 381) took 1192 ms. Trace:
I20260709 12:24:10.180703  9943 rpcz_store.cc:276] 0709 12:24:08.986896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.991008 (+  4112us) service_pool.cc:224] Handling call
0709 12:24:09.041060 (+ 50052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.970573 (+929513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.179269 (+208696us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b540018a1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:10.179546 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2123671808}
I20260709 12:24:10.186638 12083 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.168306 12054 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:10.188335  9940 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:10.190583 12040 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.190810 12019 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.191883 12019 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.194686 12019 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:10.196651  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64914.2us
I20260709 12:24:10.196870 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.197507  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:10.153523 12000 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:10.198359 12000 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.199633  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:10.200553 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 106694us
I20260709 12:24:10.205830  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:10.213150 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:10.213768  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:10.215337  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:10.219521 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:10.200851 11835 tablet_bootstrap.cc:492] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:24:10.227921 12047 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:10.230882 12040 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:10.231436 12040 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:10.232318 12040 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.234916 12000 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.234683 12010 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:24:10.218988 12102 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.049s	user 0.009s	sys 0.002s
I20260709 12:24:10.213500 12083 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:10.213241 12051 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.238057 12062 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.184262 11985 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.239302 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1439) took 1744 ms. Trace:
I20260709 12:24:10.239805 10096 rpcz_store.cc:276] 0709 12:24:08.494708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.581975 (+ 87267us) service_pool.cc:224] Handling call
0709 12:24:08.582302 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:08.879817 (+297515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.239005 (+1359188us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b5400341054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:10.239226 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":682826496}
I20260709 12:24:10.240985 11985 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:10.243356 12103 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:10.243886 12115 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:10.218458 11710 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:10.212580 11997 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:10.245412 11997 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.247677 11997 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:10.228922 12021 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:10.248853 12021 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.249876 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.250686 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2423) took 1203 ms. Trace:
I20260709 12:24:10.226814 11991 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:24:10.251148 10017 rpcz_store.cc:276] 0709 12:24:09.047256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.869881 (+822625us) service_pool.cc:224] Handling call
0709 12:24:10.249787 (+379906us) spinlock_profiling.cc:243] Waited 1.17 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431f8c 00007f4e06431ef6 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:10.250624 (+   837us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2700160}
I20260709 12:24:10.229117 12054 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:10.252748 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.253400 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.255698 11835 ts_tablet_manager.cc:1403] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 3.033s	user 0.059s	sys 0.038s
I20260709 12:24:10.256947 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.257666 10016 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:10.258077 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2421) took 1267 ms. Trace:
I20260709 12:24:10.259289 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 277) took 2493 ms. Trace:
I20260709 12:24:10.259639 10090 rpcz_store.cc:276] 0709 12:24:07.766499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:07.766812 (+   313us) service_pool.cc:224] Handling call
0709 12:24:07.779701 (+ 12889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:08.580168 (+800467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.258968 (+1678800us) spinlock_profiling.cc:243] Waited 782 ms on lock 0x7b5400304654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:10.259207 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1798912384}
I20260709 12:24:10.260146 11993 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:10.250052 12021 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:10.260867 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.261410 10090 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:10.263062 10093 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.265137 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.266204 10096 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:10.268164 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.268813 10096 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.262343 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.269634 10090 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:10.253491 10024 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:10.270232 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 272) took 1165 ms (client timeout 1522 ms). Trace:
W20260709 12:24:10.270978 10024 rpcz_store.cc:269] 0709 12:24:09.105089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.855382 (+750293us) service_pool.cc:224] Handling call
0709 12:24:10.252609 (+397227us) spinlock_profiling.cc:243] Waited 33.4 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:10.270178 (+ 17569us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76842752}
I20260709 12:24:10.258363 10016 rpcz_store.cc:276] 0709 12:24:08.996008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.858772 (+862764us) service_pool.cc:224] Handling call
0709 12:24:10.256804 (+398032us) spinlock_profiling.cc:243] Waited 28.1 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:10.258021 (+  1217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":64667392}
W20260709 12:24:10.271751 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 58608.8us
I20260709 12:24:10.273509  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:10.279667  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64412us
I20260709 12:24:10.279974 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:10.281217  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:10.273790 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:10.293491  9943 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:10.279460 12010 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.295367 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:10.297022  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:10.295786  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.299710 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.301432 10090 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.300494 10096 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:10.295048 11993 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:10.306294 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.295925 12010 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.389679  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1242) took 1495 ms. Trace:
I20260709 12:24:10.298578 11710 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:10.307271 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.299535 11997 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:10.389173  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 92260us
I20260709 12:24:10.304369 12019 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:10.300493 12059 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:10.299254 12103 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:10.446322  9949 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:10.401903  9945 rpcz_store.cc:276] 0709 12:24:08.815969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.816190 (+   221us) service_pool.cc:224] Handling call
0709 12:24:10.311239 (+1495049us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540003fa54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:10.311422 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2445384448}
I20260709 12:24:10.433522  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:10.396826 11938 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:10.433614 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.406461  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:10.436455 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81733.8us
I20260709 12:24:10.419763 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.484541 10096 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:10.438120  9947 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:10.310864 12115 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.308064 12048 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:10.449859  9946 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.492821 10090 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:10.452085  9914 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.879s (SENT) [suppressed 31 similar messages]
I20260709 12:24:10.454630  9944 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:24:10.399430 11993 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.454385  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:10.399322 12054 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.439728 12019 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:10.455659 12012 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.456595 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:10.502578 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46110.2us
I20260709 12:24:10.307435 12062 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:10.458443 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2474) took 1880 ms. Trace:
I20260709 12:24:10.466068 10094 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:10.399565  9940 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:10.471140 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.513230 10671 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:10.472676 10097 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2469) took 1900 ms (client timeout 2224 ms). Trace:
I20260709 12:24:10.472574 12010 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:10.474493 12015 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.475196 10099 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:10.478098 12109 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.653s	user 0.000s	sys 0.002s
I20260709 12:24:10.445994 11710 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.478472 11835 raft_consensus.cc:359] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:10.534714 11993 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:10.534878 12111 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:10.534735 11835 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:24:10.536067 12111 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:10.536188 11835 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:10.537134 12111 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:10.537526 12111 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.445323 11997 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.445796 12103 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.498811 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.505225  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.507517 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2419) took 1517 ms. Trace:
I20260709 12:24:10.534322 11710 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.539937  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.548269 10671 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
W20260709 12:24:10.513811 10097 rpcz_store.cc:269] 0709 12:24:08.572331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.119887 (+547556us) service_pool.cc:224] Handling call
0709 12:24:10.472454 (+1352567us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b54003aded4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:10.472613 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248116608}
I20260709 12:24:10.457267 12059 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.556000 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 278) took 1987 ms. Trace:
I20260709 12:24:10.522375  9842 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 2 to 4, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:10.527654  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.570681  9842 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 3 to 4, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 4 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:10.500927 12048 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.530213 11940 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:24:10.530347 12082 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:24:10.569651 10092 rpcz_store.cc:276] 0709 12:24:08.568892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.914177 (+1345285us) service_pool.cc:224] Handling call
0709 12:24:09.928648 (+ 14471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:09.937938 (+  9290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:09.937971 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:09.938506 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:24:10.133499 (+194993us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:10.133556 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.133646 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:10.555605 (+421959us) raft_consensus.cc:1685] finished
0709 12:24:10.555708 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.555925 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3204,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":10391,"thread_start_us":181785,"threads_started":1,"wal-append.queue_time_us":353268}
I20260709 12:24:10.532876 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.573454 10094 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:10.465895 11938 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.538434 11835 ts_tablet_manager.cc:1434] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.282s	user 0.006s	sys 0.002s
I20260709 12:24:10.580884 12019 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.582232 12019 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.583093 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:10.583986 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 270) took 1589 ms. Trace:
I20260709 12:24:10.584332 10018 rpcz_store.cc:276] 0709 12:24:08.994286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.876496 (+882210us) service_pool.cc:224] Handling call
0709 12:24:10.582937 (+706441us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:10.583926 (+   989us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65421568}
I20260709 12:24:10.585623 12017 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.586539 12037 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.587085 12037 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.587450 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.588712 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.589176 12037 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.589742 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.591034  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1444) took 1862 ms. Trace:
I20260709 12:24:10.591391  9942 rpcz_store.cc:276] 0709 12:24:08.728794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.736906 (+  8112us) service_pool.cc:224] Handling call
0709 12:24:08.761700 (+ 24794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.914957 (+1153257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.590827 (+675870us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001ba154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:10.590979 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2629920128}
I20260709 12:24:10.514355 12078 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:24:10.562103  9946 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:10.562318 10671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 527) took 1581 ms (client timeout 2018 ms). Trace:
I20260709 12:24:10.561549 10020 rpcz_store.cc:276] 0709 12:24:08.990193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.042029 (+1051836us) service_pool.cc:224] Handling call
0709 12:24:10.042427 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.042698 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:10.400495 (+357797us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.507123 (+106628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.507183 (+    60us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.507240 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.507320 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.507452 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":272341,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":8079,"raft.queue_time_us":86,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":14129,"thread_start_us":365872,"threads_started":2}
W20260709 12:24:10.596083 10671 rpcz_store.cc:269] 0709 12:24:08.980381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.980497 (+   116us) service_pool.cc:224] Handling call
0709 12:24:10.470960 (+1490463us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:10.562243 (+ 91283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24465152}
I20260709 12:24:10.529120 12062 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:10.516026  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53871.9us
I20260709 12:24:10.557668 12124 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.498449 12051 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.539331  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:10.562757  9949 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:10.515195 12054 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.529980 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 283) took 1863 ms. Trace:
I20260709 12:24:10.609995 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:10.610255  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71018.3us
I20260709 12:24:10.610632 10092 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.601521 12021 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:10.545526  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:10.586122 12017 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.539695 12113 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:10.565459 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.552908 12064 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:10.507790 12013 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:10.569841 12103 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.578688 12048 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:10.603716 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:10.616803  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 71351.1us
I20260709 12:24:10.612318 10097 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.508423 12040 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:10.569877 11985 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:24:10.540869 12038 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260709 12:24:10.543360 12111 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.623900 12115 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:10.646514 10096 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:10.633903 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.615780 12021 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.632115  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:10.647300 10090 raft_consensus.cc:2376] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4. Re-sending same reply.
I20260709 12:24:10.624111 12113 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.647338 10092 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:10.648994 10097 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:10.609597 10095 rpcz_store.cc:276] 0709 12:24:08.666951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.113488 (+1446537us) service_pool.cc:224] Handling call
0709 12:24:10.113768 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.113966 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:10.290891 (+176925us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.529387 (+238496us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.529473 (+    86us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:10.529569 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.529658 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.529861 (+   203us) inbound_call.cc:177] Queueing success response
0709 12:24:10.558398 (+ 28537us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b180046b820. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
Metrics: {"mutex_wait_us":510,"prepare.queue_time_us":420363,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":2116,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"spinlock_wait_cycles":49945472,"thread_start_us":413987,"threads_started":2}
I20260709 12:24:10.617184 11940 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:10.613674 12124 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.623447 12060 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:24:10.616954 12058 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:24:10.660188  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:10.643857 12116 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:10.625837 12037 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:24:10.615840 12051 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.667724 10090 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:24:10.677230 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 164147us
I20260709 12:24:10.632999 12045 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 12:24:10.674288  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.648483 12115 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:10.678280  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:10.629792 12112 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:24:10.691941 12100 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:10.679562  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55934.4us
I20260709 12:24:10.675303 11997 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:10.629369 11938 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.633137 12123 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:24:10.668510 11940 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.670112 12013 raft_consensus.cc:2764] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 12:24:10.703267  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:10.676167 12116 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.687928 12021 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:10.689041 12137 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:24:10.689426 12051 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.689514 12147 raft_consensus.cc:2764] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260709 12:24:10.697168 12078 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:24:10.690505 12115 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:10.705528 12147 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:10.705731 12112 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:10.705991 12147 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:10.706218 12112 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.706459 12147 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.707187 12112 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.707752 12123 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:10.708139 12123 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.709339 12058 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:10.709654  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:10.711127 12139 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:24:10.692335 12100 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.633133 12059 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.712833 12100 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.714130 12100 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.714818 12051 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:10.702057 10097 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:10.704520 12116 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.652407 12082 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:24:10.708985 12123 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.704473 11938 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.693102 12125 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:24:10.717730  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 265661us
I20260709 12:24:10.718683 10092 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:10.731033  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:10.733500  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 112858us
I20260709 12:24:10.735675 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:10.716187 12066 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.736868  9942 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:10.705551 12078 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:10.741586 10097 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:10.739017  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:10.675902 12062 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.742602  9959 tablet_copy_service.cc:342] P cd983415389444e292b2330ef630850a: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-7e7b0b17a4d646809d4dd46a0268f2eb received from {username='slave'} at 127.0.0.1:38784
I20260709 12:24:10.748157  9959 tablet_copy_service.cc:434] P cd983415389444e292b2330ef630850a: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-7e7b0b17a4d646809d4dd46a0268f2eb on tablet 7e7b0b17a4d646809d4dd46a0268f2eb with peer 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:10.744194 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.727872 12125 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:10.737635 12021 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:10.760254 10020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:10.760324  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:10.761224  9946 raft_consensus.cc:2376] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4. Re-sending same reply.
I20260709 12:24:10.727725 12118 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:24:10.757470 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.758441 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:10.768306  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.768572 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.768872  9942 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:10.754372 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.747771 11985 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:10.744632 12078 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:10.767242  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.774320 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:10.802363 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.803061 12017 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:10.794713  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.804162 10677 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.757110 12013 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:24:10.767784 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.764416  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:10.806137  9941 raft_consensus.cc:2376] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4. Re-sending same reply.
I20260709 12:24:10.806872 10097 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:24:10.807502  9987 leader_election.cc:336] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.518s (SENT)
W20260709 12:24:10.808353  9987 leader_election.cc:336] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.571s (SENT)
I20260709 12:24:10.808954 10097 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260709 12:24:10.809964  9987 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.738s (SENT)
I20260709 12:24:10.811374 12012 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.811678 10676 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:10.812799 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.812903 10677 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:10.813727 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:10.814400 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.770573 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:10.815407  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:10.817004 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 284) took 2149 ms. Trace:
I20260709 12:24:10.817378 10093 rpcz_store.cc:276] 0709 12:24:08.667283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.124189 (+1456906us) service_pool.cc:224] Handling call
0709 12:24:10.134306 (+ 10117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.263471 (+129165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.816793 (+553322us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400340654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:10.816948 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295683328}
I20260709 12:24:10.817847 10675 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:10.752470 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2478) took 2081 ms. Trace:
I20260709 12:24:10.818758 10093 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:10.818915 12054 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.820031 10093 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:10.821125 10093 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:10.772197 12104 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.614s	user 0.002s	sys 0.030s
I20260709 12:24:10.772154 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:10.810015 10097 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:10.814535 10677 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:10.769598 10670 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:10.824254  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 146046us
I20260709 12:24:10.825054  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:10.826116 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.826670 10674 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.813997 10676 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Advancing to term 3
I20260709 12:24:10.769634 10092 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:10.753688 12123 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:24:10.835342  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:10.858911  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48959.8us
W20260709 12:24:10.859715  9987 leader_election.cc:336] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.766s (SENT)
I20260709 12:24:10.810153 11940 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:10.860584  9987 leader_election.cc:336] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.767s (SENT)
I20260709 12:24:10.831557 11985 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.862124  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:10.860112 12125 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.869704 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:10.800545 12151 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:24:10.859748 10670 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:24:10.744221 12066 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.871341  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:10.872210 10640 connection.cc:256] RPC call timeout handler was delayed by -1.9286022080000293s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
I20260709 12:24:10.836264 10677 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:24:10.874447 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.872936 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.886529 10677 pending_rounds.cc:85] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 4
W20260709 12:24:10.860155 12142 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:24:10.729697 12113 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.892264 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.914129 10670 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:10.910902 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.831390 12013 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:10.912336 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.911613 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.912142 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:10.868919 12064 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.888187 10677 pending_rounds.cc:107] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:10.822391 12147 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:10.882145 11985 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:10.893743 12059 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.948674 12147 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.948264 12048 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:10.920378  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49203.8us
I20260709 12:24:10.923957 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:10.921648 10677 raft_consensus.cc:2889] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:10.920631 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:10.923544 10640 connection.cc:256] RPC call timeout handler was delayed by -1.8071022079998329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18830000000000002s for a response
W20260709 12:24:10.953694 10640 connection.cc:256] RPC call timeout handler was delayed by -1.70630220799967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
I20260709 12:24:10.936445 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:10.955654 10640 connection.cc:256] RPC call timeout handler was delayed by -1.5110022079998089s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
W20260709 12:24:10.956027 10640 connection.cc:256] RPC call timeout handler was delayed by -1.191502207999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25670000000000004s for a response
W20260709 12:24:10.915932  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.923933 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:10.925243  9940 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:10.943270 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:10.915323 12153 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:10.915867 12013 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:10.962276  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 223317us
I20260709 12:24:10.922786 12054 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.965354 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:10.967927 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:10.963649 10671 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
W20260709 12:24:10.965606  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.967000 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:10.967064  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:10.955471 12124 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.959003 12143 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:10.934983 12064 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.932993 12115 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:10.963738 12035 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.802s	user 0.015s	sys 0.021s
I20260709 12:24:10.965358 12013 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:10.981338 12115 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:10.510341 10098 rpcz_store.cc:276] 0709 12:24:08.577801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:09.920309 (+1342508us) service_pool.cc:224] Handling call
0709 12:24:09.920822 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:09.921139 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:10.221124 (+299985us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.457667 (+236543us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.457788 (+   121us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.457908 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.458053 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.458320 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":586348,"raft.queue_time_us":165,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":117,"thread_start_us":414625,"threads_started":2}
I20260709 12:24:10.981968 12115 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:10.972321 11985 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:24:10.990631 12019 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:10.982455 12115 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:10.990707 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 282) took 2324 ms. Trace:
I20260709 12:24:10.993366 12115 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:10.993670 10099 rpcz_store.cc:276] 0709 12:24:08.666582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.112500 (+1445918us) service_pool.cc:224] Handling call
0709 12:24:10.475020 (+362520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.528216 (+ 53196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.990495 (+462279us) spinlock_profiling.cc:243] Waited 362 ms on lock 0x7b54003aded0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:10.990649 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":832976256}
I20260709 12:24:10.996170 12111 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:10.996400 10022 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:10.997869 10022 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:24:10.999251 12035 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.842s	user 0.016s	sys 0.022s
I20260709 12:24:10.978219 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:11.002146 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:11.002432 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:11.003336 10018 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:11.037920 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:10.971220 10675 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:11.054379 10096 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:11.054227 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:10.941200 12078 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.084702 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.084980 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.088740 10093 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:11.055823 10674 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:11.057215 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:11.061652 12170 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.061827 12166 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.075484 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 325365us
I20260709 12:24:11.064546 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.122845 10092 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
W20260709 12:24:11.076337 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74279.1us
I20260709 12:24:11.125957 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:10.818771 10091 rpcz_store.cc:276] 0709 12:24:08.669398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.169596 (+1500198us) service_pool.cc:224] Handling call
0709 12:24:10.252303 (+ 82707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.252527 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:10.453237 (+200710us) log.cc:844] Serialized 14 byte log entry
0709 12:24:10.725861 (+272624us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.729808 (+  3947us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:10.731214 (+  1406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:10.740392 (+  9178us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:10.750309 (+  9917us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":327471,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":1044,"raft.queue_time_us":1689,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":379771,"threads_started":2,"wal-append.queue_time_us":302844}
I20260709 12:24:11.091143 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:11.110168  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.085752  9912 leader_election.cc:400] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4.
I20260709 12:24:11.130496 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:11.063480 12164 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.130918  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45248.5us
I20260709 12:24:11.130115 12117 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:11.064325 12153 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.128652 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:11.137363  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 170349us
I20260709 12:24:11.142426  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:11.140092  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:11.139709 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 134125us
I20260709 12:24:11.143395 12161 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:11.143481  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:11.143862  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:11.143867 12161 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.144343  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:11.144585 12161 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:10.978940 12143 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.074054 12048 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:11.082868 12040 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.065058 11985 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.118145 12010 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.155910 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:11.140964 12047 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.117825 12147 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:24:11.064196 12162 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.105s
I20260709 12:24:11.182318 12171 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.064935 12165 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.107s
I20260709 12:24:11.190869 12171 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.191776 12171 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:11.190966 12157 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:24:11.175740 11993 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.175427 12048 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.207950  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:11.232851  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 88584.9us
I20260709 12:24:11.233781  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:11.131179 12164 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.161845 12170 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
W20260709 12:24:11.255530  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47651.1us
I20260709 12:24:11.152851 12153 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.257642  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:11.251188  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 107400us
I20260709 12:24:11.169811 12040 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.138394 12166 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.259271  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:11.259922 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 116466us
W20260709 12:24:11.265689 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 135269us
W20260709 12:24:11.268910 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 134621us
I20260709 12:24:11.271421 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:11.280269  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:11.281760 12158 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 12:24:11.282112  9949 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:24:11.257452 12147 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:11.261658 12021 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:11.285166  9949 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:11.278988 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:11.193178 12171 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.283341 12174 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.296105 12178 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:11.296691 12178 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:11.297086 12178 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.293108 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:11.290581  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:11.290001 12100 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:24:11.300712 12161 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:11.301865 11993 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.302531 11993 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:11.303284  9948 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:24:11.303655 11993 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.309962 12010 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:24:11.315207  9949 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:11.270618 12035 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:11.319821 12177 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:11.292670 12116 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.002s
I20260709 12:24:11.323989 12040 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:11.325038 12143 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:11.276526 12165 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:24:11.326138 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:11.317173 12013 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:11.336123  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.340512  9944 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.011s	sys 0.045s
W20260709 12:24:11.359181  9944 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.011s	sys 0.045s
I20260709 12:24:11.341841  9947 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:11.359936 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49709.5us
I20260709 12:24:11.360342  9947 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:11.344738  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.361340 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:11.361913 12179 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:11.362378 12163 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.304s	user 0.003s	sys 0.000s
I20260709 12:24:11.321312 12147 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:11.362838 12147 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:11.362957 12179 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.363648 12179 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.318157 12021 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:11.369352  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78924.7us
I20260709 12:24:11.347043 12035 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:11.372315  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.369096  9947 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:11.373441  9946 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:11.372154  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:11.367568 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 239004us
I20260709 12:24:11.364213 12010 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:11.420295  9947 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.422943 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:11.369913 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 100078us
I20260709 12:24:11.363811 12147 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:11.394748  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 161073us
I20260709 12:24:11.426476 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.428269  9943 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:11.432168  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.433610 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:11.434659 10099 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:11.435381 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:11.436815 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.437517 10096 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:11.438802 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.439657 10097 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:11.430991  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:11.367779 12182 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.442399  9943 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 2: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:11.352337 12116 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.420887 12035 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:11.447925 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:11.453959 10093 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:11.443275  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71189.2us
I20260709 12:24:11.373745 12143 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:24:11.417212 11938 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:24:11.369228 12048 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.075s
W20260709 12:24:11.453153 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 91871us
I20260709 12:24:11.425527 12114 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:11.363847 12175 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:24:11.426055 12165 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:11.459180  9947 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:11.364233 12179 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.456459  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:11.454963 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:11.353010 12177 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:11.728966 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 269061us
I20260709 12:24:11.491428  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:11.587801 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.731196  9947 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:11.460460 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.732494 10099 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.732520 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:11.732928 10098 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:11.459573  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:11.729974 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:11.771078 10099 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:11.773378  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 316964us
I20260709 12:24:11.742141 10677 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:24:11.780099 12173 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.780197 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:11.780714 10672 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:11.780882 12173 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.781584 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:11.781947 10668 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:11.484292 12021 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.784379 11938 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:11.784899 11938 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:11.782977 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:11.448985 12161 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:11.729372 12111 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.733s	user 0.005s	sys 0.006s
W20260709 12:24:11.482654 12180 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:24:11.482966 12169 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:11.782541 10669 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:11.483413 12190 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:11.832350 12172 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:24:11.832763 12190 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.732174 12147 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:11.839063 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 417476us
I20260709 12:24:11.849869 12190 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:11.773064 12174 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:11.831768 12186 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:11.854424 12186 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:11.854282 12174 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.855850 12186 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.857167 12174 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:11.858712 12125 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:11.481381 12184 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:11.834107  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1468) took 1087 ms. Trace:
W20260709 12:24:11.837100  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.888613  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.889106  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.889402  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.888273  9942 rpcz_store.cc:276] 0709 12:24:10.747161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.005057 (+257896us) service_pool.cc:224] Handling call
0709 12:24:11.005462 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.008191 (+  2729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.008227 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.008688 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.260518 (+251830us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:11.260556 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.260624 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:11.833889 (+573265us) raft_consensus.cc:1685] finished
0709 12:24:11.833908 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:11.834046 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11670,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4764,"thread_start_us":142172,"threads_started":1,"wal-append.queue_time_us":375865}
W20260709 12:24:11.889674  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.887130 12184 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.892697 12184 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:11.893301 12190 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.899427 12190 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.901973 12190 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:11.820828 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:11.853307 12019 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:24:11.922209 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81600.9us
I20260709 12:24:11.885697 12066 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:11.792222 12189 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.561818 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.927800 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:11.796578 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:11.933633 10099 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:11.935439 10093 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:11.939597 12015 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:11.940092 12162 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:11.833770 12179 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:11.941445 12162 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:11.941448 12193 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:11.942148 12162 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:11.945327 10098 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:11.949644  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.950043 12037 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:11.950564 12037 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.952853 12037 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:11.719336 12010 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.830111 12169 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:11.963694 10016 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:11.964713  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 314122us
W20260709 12:24:11.966755  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:11.972940 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 243037us
I20260709 12:24:11.469666 12114 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:11.922901 12019 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:11.973697 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:11.975728  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:11.925503 12143 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:11.930914 12013 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:11.785851 11938 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:11.983682 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:11.984689 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:11.971484 12123 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.108s	user 0.006s	sys 0.007s
W20260709 12:24:11.977331 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 235281us
I20260709 12:24:11.994081 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:11.988065 10093 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:11.997466 10018 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:11.998097 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:11.998695 10095 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:12.001538 12187 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:24:11.978461 12015 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:24:12.007195 12175 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:11.976815 12193 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.008966 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.009469 12193 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.009925 12193 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.010149 10019 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:12.010253 12193 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:12.011060 12054 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.039s	user 0.011s	sys 0.000s
I20260709 12:24:12.011512 12054 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.017997 10095 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:11.930629 12048 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:24:11.985733 12064 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:11.947860 12066 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
W20260709 12:24:12.027186 10641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.026s	sys 0.031s
I20260709 12:24:11.943028 12162 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.019431 12173 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:12.043596 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 232515us
I20260709 12:24:12.053011 10098 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:12.073422 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.043167 11938 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:12.081746 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 501) took 1314 ms. Trace:
I20260709 12:24:12.044075 12116 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:12.082249 10673 rpcz_store.cc:276] 0709 12:24:10.764590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.823189 (+ 58599us) service_pool.cc:224] Handling call
0709 12:24:10.823476 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:10.863151 (+ 39675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.863181 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:10.863221 (+    40us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:10.863250 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.079117 (+1215867us) spinlock_profiling.cc:243] Waited 39.5 ms on lock 0x7b54007396d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:12.079159 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.079326 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90795904}
W20260709 12:24:12.088303 10062 leader_election.cc:336] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.686s (SENT)
W20260709 12:24:12.086479  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 295510us
I20260709 12:24:12.092095 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.074738 12100 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:12.089735 12083 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.123s	user 0.009s	sys 0.034s
I20260709 12:24:12.085918 12162 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.105567 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:12.111586 10673 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:12.106837  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.116065  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.117633  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:12.113248 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:12.118417 10098 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:12.119554  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:12.131407  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:12.102128 12116 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.098165 12165 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.132346  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:11.137073 11991 consensus_queue.cc:1175] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" responder_term: 3 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: [2.5-2.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:12.135667  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.105708 12066 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.109596 12200 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:12.114120 12103 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.469s	user 0.007s	sys 0.008s
I20260709 12:24:12.137421  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:12.136058 12183 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.168308 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:12.169152 10094 raft_consensus.cc:2376] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 4. Re-sending same reply.
I20260709 12:24:12.139284 12179 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.072409 12169 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:12.136729 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48465.7us
I20260709 12:24:12.156075 10673 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:12.155202  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 206420us
I20260709 12:24:12.138078  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.115676 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:12.178323 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.133056 12125 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:12.099298 12201 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:24:12.134505 12149 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.130687 12190 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:12.106822 12083 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.142s	user 0.010s	sys 0.034s
I20260709 12:24:12.086669 12019 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.105633 12100 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:12.172569 10062 leader_election.cc:336] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.731s (SENT)
I20260709 12:24:12.203843 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:12.165787  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:12.135639 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.133203 12013 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.167608 12184 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.140072 12171 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.136373 12177 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:12.205509  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:12.191844 10674 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:12.167410 12183 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.225430  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:12.225489 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 231182us
W20260709 12:24:12.213452 10062 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.738s (SENT)
I20260709 12:24:12.227049 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:12.230053 10098 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:12.178474 12179 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.178126 12012 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.178962 12064 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.200567 12125 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:12.142400 12037 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 12:24:12.154795 11977 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 12:24:12.222558 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.232161 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:12.246199 10095 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:12.156823 12116 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:12.232540 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:12.179162 12148 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:24:12.179092 12103 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.204510 12021 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:24:12.177196 12066 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.250025  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.246109  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:12.164489 12200 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:12.327919  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81894.1us
I20260709 12:24:12.217574 12174 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.234130 12013 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.314306 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.315733  9947 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:12.365458 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:12.235163 12183 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.235129 12175 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.239727 12054 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:12.378059 12054 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:12.237300 12084 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.235270 12179 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.237541 11821 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.226347 12019 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.330816  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:12.331089 10090 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:12.250870 12184 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.334997 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:12.202891 12149 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:12.364136 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.372108 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:12.320184 12199 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.137s
W20260709 12:24:12.374893 12204 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:24:12.330148  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.449292  9941 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:12.451144  9941 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:12.336124 12012 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:12.452790 12012 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:12.453929 12012 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.454412 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.456037 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.246032 12064 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.326562 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:12.321774 11993 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:12.336369 12201 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.366051 10672 raft_consensus.cc:2376] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4. Re-sending same reply.
W20260709 12:24:12.467533 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 132655us
W20260709 12:24:12.435175  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 104440us
I20260709 12:24:12.433863 12084 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:12.439553 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.438555  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 280745us
W20260709 12:24:12.430857 12035 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.961s	user 0.008s	sys 0.004s
I20260709 12:24:12.446889 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:12.346482 12190 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260709 12:24:12.448197 11985 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.450579 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.394999 11821 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:12.464533  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.465364  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.468827  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:12.484367  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.444233 10091 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:12.476392 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.473755 12201 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.469591 11993 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:12.479977 12197 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.263s	user 0.006s	sys 0.003s
I20260709 12:24:12.480635 10668 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:12.480840 12185 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.433022 12200 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.508591 12185 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:12.509043 12201 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:12.509162 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.509581 12185 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.504766 12190 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.509200 12200 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.510030 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.513430 12149 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:12.513581 12207 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:24:12.515244 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.516441 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.516834  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1465) took 1771 ms. Trace:
I20260709 12:24:12.517387 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.518584 12084 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:12.517198  9946 rpcz_store.cc:276] 0709 12:24:10.745489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.920330 (+174841us) service_pool.cc:224] Handling call
0709 12:24:10.944135 (+ 23805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.419791 (+475656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.516572 (+1096781us) spinlock_profiling.cc:243] Waited 429 ms on lock 0x7b5400207e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:12.516776 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":986767104}
I20260709 12:24:12.518505 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:12.520131 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.483744 10677 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
W20260709 12:24:12.439706  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.441440 12054 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:12.462136 12147 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:24:12.469678 12178 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.172s	user 0.014s	sys 0.000s
I20260709 12:24:12.495005 12100 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:12.495836 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.495469 11821 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:12.526965  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.536197  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.537240  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.537662  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.520884 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.477324 12143 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.483314 11985 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:12.528502 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52187.9us
I20260709 12:24:12.539006 10677 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 LEARNER]: Advancing to term 2
I20260709 12:24:12.541517 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:12.528272 12164 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:12.546891 10092 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:12.544751 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:12.546315 10062 leader_election.cc:336] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.802s (SENT)
I20260709 12:24:12.443521 12125 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.541990 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.548334  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.551380  9946 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
W20260709 12:24:12.551745  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81477.3us
W20260709 12:24:12.552357  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.552627 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.554047 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:12.548934 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.521552 12189 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:12.482722 12062 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:24:12.477659 12035 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.573248  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:12.582566  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:12.585963 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.506299 12083 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:24:12.539650 12143 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.588498 10017 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:12.596426 10677 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:24:12.598817 10672 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:12.607056 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.608112 10093 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
W20260709 12:24:12.584384 12115 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.587s	user 0.007s	sys 0.004s
W20260709 12:24:12.609184 12115 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.612s	user 0.007s	sys 0.004s
I20260709 12:24:12.611198 10677 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Advancing to term 4
I20260709 12:24:12.580582 11993 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:12.624732 10677 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:24:12.623368 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.626355 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.626868 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.589397 11821 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.538336 12178 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.589733 12103 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:12.591830 12116 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:12.603565 12149 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.587052 12189 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:12.639113 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 92818.8us
W20260709 12:24:12.668916  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44489.5us
W20260709 12:24:12.669889 10062 leader_election.cc:336] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.906s (SENT)
I20260709 12:24:12.603839 12112 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.589762 12215 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.634917  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.679665  9946 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:12.585876 11940 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:24:12.638545 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.627439 12047 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:12.677243 10062 leader_election.cc:336] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.921s (SENT)
I20260709 12:24:12.685174 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.677830  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:12.592597 12161 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:24:12.678393 10672 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:24:12.680614  9949 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:24:12.686939 12224 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.687314  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:12.691864 10062 leader_election.cc:336] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.922s (SENT)
I20260709 12:24:12.692317 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.690793 12115 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:12.696834  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:12.631569 11978 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:24:12.679442 12215 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.703742 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.704387 10017 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:12.700500  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.682861  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.674819 12220 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.706933 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.706305  9947 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:12.694543 12208 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.709753  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:12.687500 12224 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.694944 12103 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.694859 12047 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.635073 12217 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 12:24:12.683332 12148 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:24:12.663806 11985 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.719472 12196 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.694037 12216 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:24:12.699851 12115 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:12.705438 12215 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:12.732370 12219 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:24:12.731976  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.733116 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:12.735562 12211 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:24:12.719028 12208 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.736662  9947 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:12.724220 12178 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:12.714886 12225 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.720359  9941 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:12.731215 12196 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.750957  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.753515 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.733050 12064 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:12.755488  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.756173  9946 raft_consensus.cc:2410] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:12.752645 12196 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:12.758206 12182 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.298s	user 0.006s	sys 0.005s
I20260709 12:24:12.724776 12238 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:24:12.759322  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.761198 12115 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.745337 12208 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.767058 10674 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:12.770807 12182 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.775673  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:12.779192 12229 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
W20260709 12:24:12.780498 12222 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:24:12.781520  9942 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:12.783316 12227 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:12.754923 11985 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.787343 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.731741 12189 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.797142  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 398) took 1370 ms. Trace:
I20260709 12:24:12.797415 12224 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:12.797592  9944 rpcz_store.cc:276] 0709 12:24:11.427186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.427314 (+   128us) service_pool.cc:224] Handling call
0709 12:24:11.447243 (+ 19929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.134262 (+687019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:12.134319 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:12.177890 (+ 43571us) log.cc:844] Serialized 26 byte log entry
0709 12:24:12.458061 (+280171us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:12.458107 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.796781 (+338674us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b540018a6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:12.796825 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.796868 (+    43us) raft_consensus.cc:1685] finished
0709 12:24:12.796881 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:12.797080 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41224,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":16472,"spinlock_wait_cycles":799154048,"thread_start_us":471428,"threads_started":2,"wal-append.queue_time_us":285310}
I20260709 12:24:12.805004 10669 raft_consensus.cc:1217] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4]   Dedup: 3.4->[]
I20260709 12:24:12.805600 10669 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260709 12:24:12.806543 12183 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.807654  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1472) took 1598 ms. Trace:
I20260709 12:24:12.808012  9948 rpcz_store.cc:276] 0709 12:24:11.209509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.209774 (+   265us) service_pool.cc:224] Handling call
0709 12:24:11.210111 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.303664 (+ 93553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:12.807408 (+1503744us) spinlock_profiling.cc:243] Waited 92.9 ms on lock 0x7b54001ba154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:12.807596 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213731584}
I20260709 12:24:12.808521 10020 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:12.808750 12183 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:12.808976 10020 raft_consensus.cc:614] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:12.809330 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.809500 12112 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:12.809706 12013 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.810006 12112 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:12.810963 11938 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.725s	user 0.008s	sys 0.003s
I20260709 12:24:12.823314 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:12.829073 10021 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
W20260709 12:24:12.819700 12193 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.809s	user 0.012s	sys 0.000s
W20260709 12:24:12.836678 12159 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.841s	user 0.008s	sys 0.022s
I20260709 12:24:12.841248 12078 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.841954 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:12.851032 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.851366 10095 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:12.842800 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:12.851843 10017 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:12.852115 10099 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:12.842867 10016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2425) took 1975 ms (client timeout 1993 ms). Trace:
I20260709 12:24:12.812394 12084 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:12.822808 12162 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.704s	user 0.012s	sys 0.000s
W20260709 12:24:12.847121  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:12.855821 10016 rpcz_store.cc:269] 0709 12:24:10.867578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.867724 (+   146us) service_pool.cc:224] Handling call
0709 12:24:12.842706 (+1974982us) spinlock_profiling.cc:243] Waited 952 ms on lock 0x7b54001c5054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:12.842807 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2189809920}
I20260709 12:24:12.864434 10099 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:12.838181  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:12.830422  9941 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
W20260709 12:24:12.880726 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 187432us
W20260709 12:24:12.882040  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43974.8us
I20260709 12:24:12.882428 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:12.882747 10062 leader_election.cc:336] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.000s (SENT)
I20260709 12:24:12.883257 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.883750  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:12.883989 12083 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.882059 11938 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
W20260709 12:24:12.885842 12167 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.822s	user 0.004s	sys 0.009s
I20260709 12:24:12.887069 12038 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.887424 12148 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.887516 12038 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.889568 12148 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:12.889923 12038 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.891865 12189 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.892237 12125 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:12.892382 12189 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.880817 12084 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.844434 10668 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:12.895943 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:12.897446 10062 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.026s (SENT)
I20260709 12:24:12.897935 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:12.897614 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:12.895606  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:12.899379  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:12.900868 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.901732 10017 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:24:12.904367  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:12.908003 10016 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
W20260709 12:24:12.908269 10062 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.114s (SENT)
I20260709 12:24:12.908768 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:12.909401 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:12.909610 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:12.831719 12143 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.903811 12177 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:12.904134 11938 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.908917 11758 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:12.918627 12216 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:24:12.919988 10062 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.191s (SENT)
I20260709 12:24:12.920308 11993 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:12.920683 12235 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:12.850106 12193 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.934406  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:12.890458 12208 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.916844 12196 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.905175 12083 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:12.949079 12234 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:12.950217 10675 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:12.950112 12234 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:12.950780 12234 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:12.951645 12234 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.952788 12234 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:12.955127 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:12.955461  9944 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:12.955917 12200 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:12.956298 12200 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.956616 12200 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:12.957346 12181 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.032s	user 0.002s	sys 0.000s
I20260709 12:24:12.931327 10025 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:12.904258 12189 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:12.896137 12179 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.852885 12215 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:12.909085 12064 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.935310  9948 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
W20260709 12:24:12.937604 12188 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.926s	user 0.003s	sys 0.000s
W20260709 12:24:12.939054 12214 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:24:12.939576 12153 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:12.940959 10668 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:12.941004 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:12.942543 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:12.943934 10093 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:12.921851 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:12.923967 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.946769 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:12.954792 12225 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:12.958498 10097 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:12.985163 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.932971 10670 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:12.986443 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:12.987139 10097 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:12.988142 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:12.988690 10097 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:12.989565 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:12.989711  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:12.990132 10097 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:12.932405 12177 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:12.990480 11758 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.934612 11993 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:12.990919 11758 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:12.971276  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48882.1us
I20260709 12:24:12.979241 12176 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:12.991434 11758 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.991940  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:12.991745 12176 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:12.993299 12245 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:12.993734 12245 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:12.984752 10016 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:12.984972 10096 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:12.994052 12245 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:12.994544 10093 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:12.995649 10093 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:12.995671 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.999116 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.999765 11758 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.000245 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.000465 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:13.001382 10016 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:24:13.002447 11758 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.002291 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:12.991425 12177 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.003770 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:13.004783 11758 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.004853 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:13.005082 12236 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:12.988317 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:13.007097 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.008261 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.008921 10016 raft_consensus.cc:2368] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
W20260709 12:24:13.009564 12243 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:13.009853 11821 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:13.005761 12236 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.013603 12236 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:13.011240 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:13.017288 10671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 612) took 2151 ms (client timeout 1652 ms). Trace:
W20260709 12:24:13.017742 10671 rpcz_store.cc:269] 0709 12:24:10.865913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.912051 (+ 46138us) service_pool.cc:224] Handling call
0709 12:24:13.017094 (+2105043us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b540073a354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:13.017218 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10215808}
I20260709 12:24:13.019150  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:12.969338 12064 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.022675 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.023012  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1469) took 2275 ms. Trace:
I20260709 12:24:13.029795 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:12.947712 12078 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:24:13.033499  9945 rpcz_store.cc:276] 0709 12:24:10.747708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.059911 (+312203us) service_pool.cc:224] Handling call
0709 12:24:11.060356 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:11.064748 (+  4392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:11.064788 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:11.065385 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.425105 (+359720us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:11.425151 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.425238 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.936691 (+1511453us) spinlock_profiling.cc:243] Waited 6.64 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:13.014580 (+ 77889us) raft_consensus.cc:1685] finished
0709 12:24:13.014632 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.022951 (+  8319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3886,"raft.queue_time_us":4007,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":162,"spinlock_wait_cycles":15262592,"thread_start_us":270874,"threads_started":1,"wal-append.queue_time_us":802902}
I20260709 12:24:13.032101 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:13.033720 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:13.033998 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:13.036222 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:12.985823 11991 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.039022 10672 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:13.005585 11758 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.043439 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:13.044242 10672 raft_consensus.cc:2376] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4. Re-sending same reply.
I20260709 12:24:13.045730 10672 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:12.979035 12217 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:24:13.056217  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:13.052603 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:13.022306 12174 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:13.035210 12181 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:13.059046 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:13.061118 10673 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:13.018432 10675 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:13.035614 12153 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:24:13.024330 12066 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.064589 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:13.089030 10093 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.039424 12162 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.079668 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.040155 11821 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:13.020803 12236 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:24:13.038846 12225 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:24:13.053527 12186 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.004s	sys 0.006s
W20260709 12:24:13.086440 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.033600 12112 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.048759 12062 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:13.059106 12240 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:13.095909 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 491) took 2336 ms. Trace:
I20260709 12:24:13.112673 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.096601 11821 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.056715 11991 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.112927 11758 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:13.114262 12248 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
W20260709 12:24:13.106580 12168 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 2.049s	user 0.004s	sys 0.000s
I20260709 12:24:13.108168 12062 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:13.159991 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.114475 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:13.102022 12038 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.132273 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:13.110971 12241 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.096s
I20260709 12:24:13.092226 12015 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:13.063259 12083 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:24:13.109081 10676 rpcz_store.cc:276] 0709 12:24:10.759677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.813571 (+ 53894us) service_pool.cc:224] Handling call
0709 12:24:10.813844 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.955921 (+142077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.955951 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.956390 (+   439us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.337972 (+381582us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:11.454439 (+116467us) log.cc:844] Serialized 14 byte log entry
0709 12:24:11.457730 (+  3291us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.457998 (+   268us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:12.956610 (+1498612us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:13.090670 (+134060us) raft_consensus.cc:1685] finished
0709 12:24:13.095609 (+  4939us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.095830 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3055,"prepare.queue_time_us":691568,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2270,"raft.queue_time_us":370708,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":14317,"spinlock_wait_cycles":42046336,"thread_start_us":454524,"threads_started":2,"wal-append.queue_time_us":778535}
W20260709 12:24:13.105088 12186 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.244s	user 0.005s	sys 0.007s
I20260709 12:24:13.159046 10093 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:13.161264  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:13.080054 12174 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.163955 10019 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:13.165989 10023 raft_consensus.cc:2368] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
I20260709 12:24:13.165740 10099 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:13.151173 12177 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.153131 11821 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.227612 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:13.228262 10099 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.168455 12208 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:13.188148 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.162281 12143 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.217602 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:13.218701 12258 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:24:13.219368 12245 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.168462 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:13.231973  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:13.233708  9945 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:13.253854  9948 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.237924  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:13.238760 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:13.240644 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.255025 10017 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:13.256184 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.215773 10676 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:24:13.257299 12245 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.164462 12062 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.174989 12215 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.252121 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:13.257364 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:13.265585 10094 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
W20260709 12:24:13.258394 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 90053.5us
I20260709 12:24:13.266806 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:13.263288  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:13.267771 10099 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:13.267764 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:13.244948 12177 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.268575 10098 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:24:13.268898 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:13.265913 10017 raft_consensus.cc:2376] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5. Re-sending same reply.
I20260709 12:24:13.245455 12193 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.278381  9948 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:13.289355 12195 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:13.248514 12179 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.270730 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:13.291428 10099 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.170807 12015 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.264911 12236 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:13.296633 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:13.289925 12195 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:13.274981 12233 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.272449 11758 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:13.268982  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1467) took 2518 ms. Trace:
I20260709 12:24:13.303535 10099 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:13.307024 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:13.298285 12236 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:13.304246 12195 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:13.287263 12225 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:13.278978 12084 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.273082 12153 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:13.327363 10099 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:13.309337 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:13.328343 10017 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 3.
I20260709 12:24:13.315392 11758 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:13.318133 12015 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.328729 12233 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:13.295131 12196 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.314852  9940 rpcz_store.cc:276] 0709 12:24:10.746619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.991309 (+244690us) service_pool.cc:224] Handling call
0709 12:24:10.991697 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:10.992070 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:10.992109 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:10.992623 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 12:24:11.083754 (+ 91131us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:11.083814 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:11.083894 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:13.028882 (+1944988us) spinlock_profiling.cc:243] Waited 60.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:13.227021 (+198139us) raft_consensus.cc:1685] finished
0709 12:24:13.228111 (+  1090us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:13.264838 (+ 36727us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":184,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":139640192,"thread_start_us":72580,"threads_started":1,"wal-append.queue_time_us":64179}
I20260709 12:24:13.323385 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:13.272508 12083 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:24:13.325093 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:13.307081  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:13.308022 12234 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:13.307888 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:13.346097 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:13.330777 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.331938 12153 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:13.318532  9945 raft_consensus.cc:2410] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:13.343443  9943 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:13.346125 10092 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:13.331152 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:13.351467 12153 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.351668 12038 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:13.334385 12064 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:13.334482 12233 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.361508 12220 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:13.354499 12225 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:13.358533 12083 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:13.355633 12234 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:13.368709 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.361181 12233 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.371380 10092 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 12:24:13.372120  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.365664 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:13.334468 11758 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.381938 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.368997 12083 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:13.388100 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57050us
I20260709 12:24:13.388228 12234 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.375437 12174 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:13.391093 12251 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:24:13.391189 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.391963 10017 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 1.
I20260709 12:24:13.359872 12236 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:13.396504 12261 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.073s	user 0.002s	sys 0.000s
I20260709 12:24:13.397112 10669 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:13.397466 12177 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.404731 12062 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.406441 10093 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:13.387405 12083 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.420753 12185 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:13.424878 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.407928 12062 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:13.430104 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 84136.4us
I20260709 12:24:13.431897 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:13.436678 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.437472 11821 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.441126 12147 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:13.441684 12147 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.445142 12195 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.457646 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:13.421738 12233 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.462410 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:13.466500 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:13.467502 12220 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:13.467921 12225 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:13.469199 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:13.470033 12220 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:13.465807 12010 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:13.459892 12174 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:13.470685 12010 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:13.471132 12010 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.471405 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:13.472787 12174 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:13.473157 12174 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:13.473479 12174 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:13.456087 12200 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.476696 12200 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.477514 10099 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:24:13.481841 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:13.482311 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.486454 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:13.487337 10677 raft_consensus.cc:2368] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5.
I20260709 12:24:13.487363 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:13.488121  9943 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:13.490507  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:13.490756 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:13.491428 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:13.489090 12179 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:13.492318 12179 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.495322 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:13.500984  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.514883  9948 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:13.516431  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:13.517285  9946 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:13.483945 12174 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:13.522222  9948 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:13.504153 12220 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:24:13.524641  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:13.539330  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.536458 12245 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:24:13.543043 12169 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.544910 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 510) took 1217 ms. Trace:
I20260709 12:24:13.545320 10674 rpcz_store.cc:276] 0709 12:24:12.327763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.455724 (+127961us) service_pool.cc:224] Handling call
0709 12:24:13.544733 (+1089009us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:13.544846 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713463552}
I20260709 12:24:13.546903 12166 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:13.536453 12215 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.011s
I20260709 12:24:13.547938 11758 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:13.548296 11758 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:13.548864 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:13.549501 10099 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:13.550459 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:13.555536 10099 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.559839  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:13.521559 12176 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.521199 12186 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.616286  9944 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:13.621909  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:13.622545  9948 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:13.521214 12240 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:13.521023 12208 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:13.619232  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:13.625195 10099 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:13.626151 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:13.549103 11758 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:13.636384 12186 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:13.639788 12270 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 12:24:13.640509 12265 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:24:13.641048 11821 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.604108 12179 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:13.641567  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:13.535647 12100 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.095s
I20260709 12:24:13.630239 12125 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.284s
I20260709 12:24:13.643378 12174 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:13.645081  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:13.658629 12272 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:13.664000 12208 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.715052 12215 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:24:13.718005 12271 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:24:13.721899 10096 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:13.814961 10099 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:13.622201  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1261) took 1016 ms. Trace:
I20260709 12:24:13.540114 12153 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:13.646641 12245 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.378s
W20260709 12:24:14.220089 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1379) took 1846 ms (client timeout 2191 ms). Trace:
W20260709 12:24:14.438032  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.002s	sys 0.000s
W20260709 12:24:13.862804 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:14.488590  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.004s	sys 0.000s
W20260709 12:24:14.179142  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.002s	sys 0.007s
W20260709 12:24:14.489329  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.637466 12148 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:14.466012  9943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.000s	sys 0.019s
W20260709 12:24:14.463274  9948 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.840s	user 0.006s	sys 0.006s
I20260709 12:24:14.490880  9944 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:14.489363  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.003s	sys 0.007s
I20260709 12:24:14.494717 10099 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:14.455565 12283 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.510672 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 309) took 1025 ms. Trace:
I20260709 12:24:14.511052 10099 rpcz_store.cc:276] 0709 12:24:13.485142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.625915 (+140773us) service_pool.cc:224] Handling call
0709 12:24:14.510582 (+884667us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:14.488328 10023 rpcz_store.cc:269] 0709 12:24:12.373616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.940758 (+567142us) service_pool.cc:224] Handling call
0709 12:24:14.219765 (+1279007us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400021054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:14.219995 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421613824}
W20260709 12:24:14.514501  9943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.873s	user 0.001s	sys 0.020s
W20260709 12:24:14.361419 12169 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.818s	user 0.004s	sys 0.005s
W20260709 12:24:13.974972 12010 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.002s
W20260709 12:24:14.231950 12147 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.764s	user 0.008s	sys 0.001s
W20260709 12:24:14.196623 12045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.012s	sys 0.008s
W20260709 12:24:14.367969 12083 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.953s	user 0.011s	sys 0.000s
I20260709 12:24:14.497079 12177 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:14.519273  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 892993us
I20260709 12:24:14.519886  9948 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:14.527810 10099 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:13.637812 12240 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.172865 12166 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:14.518321 11758 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:13.722810 12233 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:14.522034 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:13.727521 12062 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:14.523451  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.850736 12220 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.525925  9945 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:14.542438 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 298) took 1055 ms. Trace:
I20260709 12:24:14.542888 10023 rpcz_store.cc:276] 0709 12:24:13.487385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.512280 (+1024895us) service_pool.cc:224] Handling call
0709 12:24:14.542376 (+ 30096us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:14.088572 12111 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.014s
I20260709 12:24:14.520542  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1480) took 1006 ms. Trace:
I20260709 12:24:14.513242 12125 consensus_queue.cc:1243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:24:14.524113 12045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.979s	user 0.017s	sys 0.008s
I20260709 12:24:14.458545 12054 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.545310  9943 rpcz_store.cc:276] 0709 12:24:13.514033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.514155 (+   122us) service_pool.cc:224] Handling call
0709 12:24:13.514587 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.514885 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.514934 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.537761 (+ 22827us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.514977 (+977216us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:14.515031 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.515125 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.520304 (+  5179us) raft_consensus.cc:1685] finished
0709 12:24:14.520324 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.520475 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":873733,"threads_started":1,"wal-append.queue_time_us":896005}
W20260709 12:24:14.548367  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:13.853984 12195 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:14.518236 12283 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:14.555488 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 304) took 1401 ms. Trace:
I20260709 12:24:14.549358 12220 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.560783 10095 rpcz_store.cc:276] 0709 12:24:13.153709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.414970 (+261261us) service_pool.cc:224] Handling call
0709 12:24:13.415274 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.415604 (+   330us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.415627 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.416091 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.464439 (+ 48348us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.464477 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.464657 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.555278 (+1090621us) raft_consensus.cc:1685] finished
0709 12:24:14.555298 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.555427 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2121,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"thread_start_us":47929,"threads_started":1,"wal-append.queue_time_us":114587}
I20260709 12:24:14.556310 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:14.556694 12286 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:14.547276 12111 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.000s	sys 0.019s
I20260709 12:24:14.557220 12233 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:14.537609 12288 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.525796 10675 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:14.523639 12148 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:14.492334  9941 rpcz_store.cc:276] 0709 12:24:12.599351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.760438 (+161087us) service_pool.cc:224] Handling call
0709 12:24:12.760746 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.885019 (+124273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.546516 (+661497us) spinlock_profiling.cc:243] Waited 54 ms on lock 0x7b540042e854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:13.615801 (+ 69285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124178176}
I20260709 12:24:14.563468 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 297) took 1077 ms. Trace:
I20260709 12:24:14.583815 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 526) took 1335 ms. Trace:
I20260709 12:24:14.584031 10023 rpcz_store.cc:276] 0709 12:24:13.485512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.556029 (+1070517us) service_pool.cc:224] Handling call
0709 12:24:14.563399 (+  7370us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.526418 12147 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.523808 12010 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.568464 12153 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.569118  9946 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:14.589171  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1485) took 1068 ms. Trace:
I20260709 12:24:14.537228 12177 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.579443 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 521) took 1571 ms. Trace:
W20260709 12:24:14.592638 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.10526e+06us
I20260709 12:24:14.586566 12148 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:14.594439 11985 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.594964 11985 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.592994 10672 rpcz_store.cc:276] 0709 12:24:13.008483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.048004 (+ 39521us) service_pool.cc:224] Handling call
0709 12:24:13.053254 (+  5250us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.064491 (+ 11237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.064524 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.066485 (+  1961us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.267657 (+201172us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:13.289950 (+ 22293us) log.cc:844] Serialized 14 byte log entry
0709 12:24:14.578663 (+1288713us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b18003b3140. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:24:14.578863 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.578972 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.579034 (+    62us) raft_consensus.cc:1685] finished
0709 12:24:14.579147 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.579356 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15442,"prepare.queue_time_us":22162,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":1303913,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":14516,"spinlock_wait_cycles":2774784,"thread_start_us":104031,"threads_started":1,"wal-append.queue_time_us":172391,"wal-append.run_cpu_time_us":3548,"wal-append.run_wall_time_us":1075399}
W20260709 12:24:14.582798 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.27098e+06us
I20260709 12:24:14.527503 12287 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.596968 11985 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:14.596719 12287 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.597522 11985 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:14.528420 12083 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:14.599578 12010 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.600147 12083 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:14.600961 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:14.601783 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 296) took 1116 ms. Trace:
I20260709 12:24:14.584231 10673 rpcz_store.cc:276] 0709 12:24:13.248014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.253578 (+  5564us) service_pool.cc:224] Handling call
0709 12:24:13.253903 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.254238 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.254266 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.290870 (+ 36604us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.390231 (+ 99361us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:13.390272 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.390348 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.583563 (+1193215us) raft_consensus.cc:1685] finished
0709 12:24:14.583587 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.583743 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1348,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":10394,"thread_start_us":81183,"threads_started":1,"wal-append.queue_time_us":187488}
I20260709 12:24:14.602278 12176 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.570371 12220 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.589526  9946 rpcz_store.cc:276] 0709 12:24:13.521134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.521535 (+   401us) service_pool.cc:224] Handling call
0709 12:24:14.568766 (+1047231us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:14.568963 (+   197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.588920 (+ 19957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.589107 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6863744}
I20260709 12:24:14.602073 10023 rpcz_store.cc:276] 0709 12:24:13.485360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.585147 (+1099787us) service_pool.cc:224] Handling call
0709 12:24:14.601725 (+ 16578us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.592136  9940 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:14.593581 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:14.539486 12166 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.579993 12179 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.555712 12054 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:14.606930 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:14.605691  9944 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:14.585458 12100 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.608629 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 295) took 1123 ms. Trace:
I20260709 12:24:14.609889 10023 rpcz_store.cc:276] 0709 12:24:13.485184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.603638 (+1118454us) service_pool.cc:224] Handling call
0709 12:24:14.608561 (+  4923us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.608436 12179 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:14.610980 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:14.611438 12277 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:24:14.611670 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1391) took 1603 ms. Trace:
I20260709 12:24:14.612027 10023 rpcz_store.cc:276] 0709 12:24:13.008131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.610686 (+1602555us) service_pool.cc:224] Handling call
0709 12:24:14.611606 (+   920us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.612747 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1480) took 1645 ms. Trace:
I20260709 12:24:14.613878 12196 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:14.614259 12196 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:14.603710 11985 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:14.615095 12196 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.616214 11985 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.617127 12186 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.618219 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:14.618474 12209 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.618865 12147 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.613133 10097 rpcz_store.cc:276] 0709 12:24:12.967081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:12.998654 (+ 31573us) service_pool.cc:224] Handling call
0709 12:24:12.998924 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.020027 (+ 21103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:13.020054 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:13.257670 (+237616us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:24:13.488664 (+230994us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b2c00115cd8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:24:13.496777 (+  8113us) log.cc:844] Serialized 26 byte log entry
0709 12:24:13.702959 (+206182us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:13.702997 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:13.703071 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:14.612535 (+909464us) raft_consensus.cc:1685] finished
0709 12:24:14.612552 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:14.612690 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20609,"raft.queue_time_us":24068,"raft.run_cpu_time_us":2324,"raft.run_wall_time_us":224845,"spinlock_wait_cycles":50261120,"thread_start_us":192419,"threads_started":1,"wal-append.queue_time_us":1016972}
I20260709 12:24:14.623459  9912 leader_election.cc:400] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5.
I20260709 12:24:14.623860  9912 leader_election.cc:403] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:14.624154 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:14.624455 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:14.625017 10097 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:14.628345 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:14.603003 12220 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:14.640868 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:14.630527 10673 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:14.631927 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:14.642385  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:14.644462 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:14.645587 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1400) took 1602 ms. Trace:
I20260709 12:24:14.646178  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:14.582614 12233 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.637353 12147 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.637513 12292 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:14.637990 12186 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.639608 12166 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.602810 12176 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.602816 12283 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:14.601042 12083 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:14.629520 10095 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:14.646039 10023 rpcz_store.cc:276] 0709 12:24:13.043601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.619561 (+1575960us) service_pool.cc:224] Handling call
0709 12:24:14.645511 (+ 25950us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.636667 10097 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:14.700908 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:14.656840 12282 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:14.656994 12293 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:14.657770 12240 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.658484 10672 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:14.673542 12233 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.659396 12111 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.660231 12015 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.644692 12153 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5.
I20260709 12:24:14.619096 12209 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.644524 12169 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.612766 12179 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:14.588186 12200 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:24:14.584954 12288 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.582270 12286 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.666795 12289 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:14.667104 12208 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.667461 11978 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:14.669149 12177 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.669430 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:14.683228 12186 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.670909 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:14.672144 12147 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.673318 12296 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:14.673635 12297 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.671586 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:14.697815 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:14.626202 12078 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:14.878424 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1402) took 1488 ms. Trace:
I20260709 12:24:14.878923 10016 rpcz_store.cc:276] 0709 12:24:13.389892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.671361 (+1281469us) service_pool.cc:224] Handling call
0709 12:24:14.878362 (+207001us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:14.659062  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.647375 12195 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:14.648267 12185 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:14.673884 12220 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:14.678066 11985 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:14.702064 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:14.907219 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 205226us
I20260709 12:24:14.907970 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:14.812347 10023 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 4. Ops: []
I20260709 12:24:14.626291 12100 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:14.866900  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 158396us
I20260709 12:24:14.809659 12282 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:14.871537 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1401) took 1827 ms. Trace:
W20260709 12:24:14.930114  9987 connection.cc:256] RPC call timeout handler was delayed by -1.1752808600003846s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30960000000000004s for a response
I20260709 12:24:14.698706 12292 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:14.704104 12083 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.866442 12300 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:14.863391 12233 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:14.884958 12287 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.880707 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:14.881129 10016 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:14.978431 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2434) took 2375 ms. Trace:
I20260709 12:24:14.979104 10016 rpcz_store.cc:276] 0709 12:24:12.603490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.879668 (+2276178us) service_pool.cc:224] Handling call
0709 12:24:14.879982 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.978061 (+ 98079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.978348 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.809696 12111 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:14.813391 12015 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:14.871014 11978 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:14.865211 12240 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:24:14.863642 12169 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:14.904717 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 264991us
I20260709 12:24:15.071743 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:14.905589 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:14.916177 10677 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:14.952843 10022 rpcz_store.cc:276] 0709 12:24:13.043865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.669174 (+1625309us) service_pool.cc:224] Handling call
0709 12:24:14.871470 (+202296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:14.670027 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:14.954353  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.081679 11758 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:15.081725  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:14.955753 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 291) took 2925 ms. Trace:
I20260709 12:24:15.082726 12125 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.082935 12285 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:15.083284 12125 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.083877 12045 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:14.957647 12147 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.877300 12193 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.085064  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:15.084961 12193 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.085453 12125 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.952042 12289 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.086026 12045 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:14.886420 12296 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.086479 12045 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.086246 12289 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.086948 12045 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:14.968336 12176 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.087467 12193 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:14.969283 12100 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.886054 12297 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.088145 12296 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.088498 12289 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:14.887193 12078 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.090196 12297 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:14.970402 12288 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.970686 10668 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:15.090976 12078 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:14.886727 12195 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:15.091691 12195 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:14.957356 12303 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:14.987344 10025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2428) took 4118 ms (client timeout 2157 ms). Trace:
W20260709 12:24:15.006983 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 126337us
I20260709 12:24:15.046902 12304 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:15.050051 12118 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:15.053035 12281 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:15.056598 12035 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:14.990196 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.109871 10016 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.062877  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.111200  9948 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:15.112401  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:14.956300 11985 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.067404  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.068099  9943 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:15.068413  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:15.075277 10673 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:15.076227 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 545) took 1809 ms. Trace:
I20260709 12:24:15.076354 12015 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:14.981158 12287 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:15.078156 12292 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:14.979835 12233 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:14.966910 10018 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:14.955515 12282 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:14.903762 12021 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:14.980747 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54007.3us
W20260709 12:24:15.095176 10025 rpcz_store.cc:269] 0709 12:24:10.868447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.868708 (+   261us) service_pool.cc:224] Handling call
0709 12:24:14.987184 (+4118476us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54001f7f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:14.987276 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4557026944}
I20260709 12:24:15.096361  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:15.105499 12281 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:15.082737 10023 rpcz_store.cc:276] 0709 12:24:11.983494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.704206 (+2720712us) service_pool.cc:224] Handling call
0709 12:24:14.812108 (+107902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.908740 (+ 96632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.908944 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.099428 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:15.174011 10022 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:14.989565 12294 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.106644 10669 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:15.110343 12305 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:15.113112  9948 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:15.065475  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81907us
I20260709 12:24:15.066980 12309 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.162636 12292 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.147162 10676 rpcz_store.cc:276] 0709 12:24:13.267187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.267410 (+   223us) service_pool.cc:224] Handling call
0709 12:24:13.296061 (+ 28651us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:14.580411 (+1284350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.580459 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:14.581045 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:24:14.751031 (+169986us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:14.751072 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:14.751138 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:15.075990 (+324852us) raft_consensus.cc:1685] finished
0709 12:24:15.076011 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.076157 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1302093,"raft.queue_time_us":1286883,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":6644,"thread_start_us":147917,"threads_started":1,"wal-append.queue_time_us":312861}
I20260709 12:24:15.151199  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.153301  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.098930 12304 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.157202 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1451) took 4690 ms. Trace:
W20260709 12:24:15.163915 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 256011us
I20260709 12:24:15.148097 12185 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:15.168165 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.198251 10090 rpcz_store.cc:276] 0709 12:24:10.466872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.691140 (+224268us) service_pool.cc:224] Handling call
0709 12:24:10.691439 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:12.434906 (+1743467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.156867 (+2721961us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400304b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:15.157112 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3770503936}
W20260709 12:24:15.168228  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 71967.2us
I20260709 12:24:15.094911 12303 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.166664  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:15.154891  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.170696 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:15.105705  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:15.145709 12118 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:15.146111  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:15.128417 12078 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:15.108130 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:15.183410 10016 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:15.236954 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.237572 10016 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:15.082413 11758 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.239390 12169 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.161336 12193 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.240075 12303 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.190414  9944 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:15.197669 12113 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.240661 12193 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.241353 12114 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.241437  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.242789  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.242911 12193 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.244071  9944 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:15.190660 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 95210.2us
I20260709 12:24:15.246810 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:15.160270 12035 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:24:15.195683 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.190982 12289 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.196648 12209 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.167984 12054 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.106469 12288 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:24:15.175853 12245 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:15.200271  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:15.067672  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:15.202611  9948 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:15.205848  9946 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.205282  9945 raft_consensus.cc:2410] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:15.180079 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1360) took 2930 ms. Trace:
I20260709 12:24:15.158608 12287 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.177017 12021 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.200987 12304 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.189396 12305 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:15.218696 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.230414 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59822.4us
I20260709 12:24:15.240149 12169 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:15.249264 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.196053 12286 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:15.255560 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 147484us
W20260709 12:24:15.185771  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.181809 12294 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.266278  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:15.268641  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.276979 12245 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.217459 12300 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:15.158882 11978 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.206916 12215 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:15.222294 11985 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.204691 12045 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.221170 12111 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.223802 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 525) took 2109 ms. Trace:
I20260709 12:24:15.235426 12078 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.270510 12289 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.283727  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:15.218393 12314 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:15.222810 12083 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.220532 12220 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.219393 12233 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.269071 12103 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:24:15.267524 12235 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.198478 12292 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.290771 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:15.190876 12295 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
W20260709 12:24:15.298259 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.382196 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.268653 10016 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
W20260709 12:24:15.403021 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 112347us
I20260709 12:24:15.403347  9943 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:15.403852 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:15.404138 10016 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:15.404855 10668 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:15.405009 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2435) took 2736 ms. Trace:
I20260709 12:24:15.405532 10016 rpcz_store.cc:276] 0709 12:24:12.668891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.403515 (+2734624us) service_pool.cc:224] Handling call
0709 12:24:15.403909 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.404697 (+   788us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.404943 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.405908 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:15.406246 12282 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.243602 12281 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.407124  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1478) took 4062 ms. Trace:
I20260709 12:24:15.407423 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.270289 12315 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:15.362221 10025 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:15.244837 12118 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.363536 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59744 (request call id 14) took 4545 ms. Trace:
I20260709 12:24:15.420513 10020 rpcz_store.cc:276] 0709 12:24:10.754818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.760120 (+  5302us) service_pool.cc:224] Handling call
0709 12:24:15.298834 (+4538714us) spinlock_profiling.cc:243] Waited 2.05 s on lock 0x7b54001f7f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09550853 00007f4e0e51e60a 00007f4e07aa47d4 00007f4e07aa4762 00007f4e07aa4691 00007f4e07aa460c 00007f4e07aa38c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:15.298989 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4708898560}
I20260709 12:24:15.285969 11758 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:15.372215  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 75470.8us
I20260709 12:24:15.374951  9946 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:15.280717  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:15.220021 12185 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.378473 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:15.378515  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 178331us
I20260709 12:24:15.189214 12309 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.364048 12245 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.222262 11991 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.385257  9948 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.376972 12240 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.440548 12240 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.441044 12195 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.442153 10672 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:15.379717 12123 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:15.281261 10018 rpcz_store.cc:276] 0709 12:24:12.249240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.871971 (+2622731us) service_pool.cc:224] Handling call
0709 12:24:14.958843 (+ 86872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.163615 (+204772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.179997 (+ 16382us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.384598 12215 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:15.379452 12286 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.381726 11978 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.299005 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.360234 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:15.409300 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.387749 12078 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.376199 12317 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:15.407527  9949 rpcz_store.cc:276] 0709 12:24:11.344879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.351196 (+  6317us) service_pool.cc:224] Handling call
0709 12:24:11.352414 (+  1218us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:12.686448 (+1334034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.406891 (+2720443us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54001b9754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:15.407065 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3053870848}
I20260709 12:24:15.389437 12314 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.378054 12153 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:24:15.475209  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:15.426215 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:15.399070 12235 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.429317 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2443) took 2593 ms. Trace:
I20260709 12:24:15.389797 12111 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:15.438006 12045 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:15.440083 12312 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:24:15.394955 12220 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:15.407894 12281 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.443145  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.464823  9945 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:15.411264 12083 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:15.425088 11985 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:15.476011 12316 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.430797 12295 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:15.433809 12315 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:15.383745 10675 rpcz_store.cc:276] 0709 12:24:13.114002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.114178 (+   176us) service_pool.cc:224] Handling call
0709 12:24:15.223497 (+2109319us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400490a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:15.223675 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3123371776}
I20260709 12:24:15.480221 12111 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.389989 12103 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:15.466040 12123 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.508625 12220 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.517979 12088 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.465763 12215 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.481556 12288 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:24:15.430526 12300 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.467083 10677 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:15.527053 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.468878 12286 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.512799 12316 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:15.479821 12287 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.518505 12103 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:15.513077 12327 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:15.442531 12240 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.481842 12285 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:24:15.434743 12305 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.522290 12294 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.522459 12215 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.520973 12123 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.522971 12021 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.520354 12111 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.475490 12233 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.508756 12045 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:24:15.482965 12035 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:15.618196 12035 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:15.427783  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:15.473367  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:15.607486  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:15.476939 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50801.4us
W20260709 12:24:15.609695  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.482126  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 215939us
I20260709 12:24:15.478794 10016 rpcz_store.cc:276] 0709 12:24:12.835649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.428679 (+2593030us) service_pool.cc:224] Handling call
0709 12:24:15.428937 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.429068 (+   131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.429085 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.429134 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:15.429161 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.429194 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.429273 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.611419 10018 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:24:15.611299  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 543543us
I20260709 12:24:15.522855 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.509316 11985 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.739338 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2445) took 2903 ms. Trace:
I20260709 12:24:15.740011 10668 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:15.740514  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:15.740801 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:15.531318 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.742620 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.618929 12035 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.621248 12045 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:15.547062 12062 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:15.529927 11993 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.518546 12088 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:15.554378 12148 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.959s	user 0.011s	sys 0.000s
I20260709 12:24:15.626019  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.626360 12186 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.557839 12195 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.541792 12220 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.555024 12233 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.628530 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.550971 12285 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:15.559243 12286 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.756650 12062 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.756937 12285 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:15.758771 12285 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.759488 11977 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:15.566231 12245 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:15.631915 10670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 614) took 4765 ms (client timeout 2183 ms). Trace:
I20260709 12:24:15.561148 12316 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.545970 12315 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:15.761412 10670 rpcz_store.cc:269] 0709 12:24:10.866443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.961679 (+ 95236us) service_pool.cc:224] Handling call
0709 12:24:15.631328 (+4669649us) spinlock_profiling.cc:243] Waited 1.95 s on lock 0x7b540073afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:15.631585 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4489353984}
I20260709 12:24:15.763113 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.763808 12245 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.567390 12314 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.568189 12309 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.630139 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1492) took 2521 ms. Trace:
I20260709 12:24:15.478375 12317 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.769001 10092 rpcz_store.cc:276] 0709 12:24:13.108916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.356247 (+247331us) service_pool.cc:224] Handling call
0709 12:24:13.356562 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:13.371989 (+ 15427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.629194 (+2257205us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b5400341f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:15.629789 (+   595us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33277824}
I20260709 12:24:15.770731 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:15.770903 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:15.626394  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.771890 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:15.772271  9946 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:15.773576  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.773808 10092 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:15.651281 12236 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.658131 12034 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:15.774967 12034 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:15.659083  9948 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:15.659591 12327 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:15.659861 12235 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.661731 12083 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:15.570331 12304 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.670940  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.692895 12123 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.701786 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.704202 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:15.704613 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:15.704905 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.706790 10091 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:15.706787 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:15.793406  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:15.709227 12322 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.271s	user 0.004s	sys 0.000s
W20260709 12:24:15.716806  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 288939us
I20260709 12:24:15.716279 12153 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:15.714952 12292 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.694207 12103 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.729957  9949 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.733914  9945 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.709697 12078 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.738818 10096 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:24:15.743013 10676 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:15.744446 12045 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:15.739930 10018 rpcz_store.cc:276] 0709 12:24:12.836310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.466837 (+2630527us) service_pool.cc:224] Handling call
0709 12:24:15.504660 (+ 37823us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.738883 (+234223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.739190 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:15.750120 10675 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:15.742224 11985 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.547025 12289 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.760401 12288 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:15.815894  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1487) took 2264 ms (client timeout 1803 ms). Trace:
W20260709 12:24:15.816424  9940 rpcz_store.cc:269] 0709 12:24:13.551645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.558757 (+  7112us) service_pool.cc:224] Handling call
0709 12:24:15.815438 (+2256681us) spinlock_profiling.cc:243] Waited 99.2 ms on lock 0x7b54000163d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:15.815643 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228105600}
W20260709 12:24:15.817184  9912 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.844s (SENT)
I20260709 12:24:15.818279 12147 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:15.819159 12147 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.819628 12220 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.819764  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.820245 12220 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.820943  9944 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:15.751612  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:15.774199 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:15.772326 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:15.822255 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48386.9us
I20260709 12:24:15.794759 10092 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
W20260709 12:24:15.823087 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.823561  9942 raft_consensus.cc:2376] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5. Re-sending same reply.
I20260709 12:24:15.823745 10090 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.824132 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:15.807566 10017 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:15.792310 12317 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.808321 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:15.807371 10096 pending_rounds.cc:85] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 4
I20260709 12:24:15.815420 12148 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.822822 10098 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:15.814436  9948 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 2.
I20260709 12:24:15.765893 11977 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:15.822757 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77906.2us
I20260709 12:24:15.774464  9946 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:15.829164 12327 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.829627 12083 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.829672 10095 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:15.760723 12285 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.804219 12153 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:15.825424  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.857s (SENT) [suppressed 40 similar messages]
W20260709 12:24:15.824198  9942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1490) took 2206 ms (client timeout 2092 ms). Trace:
I20260709 12:24:15.823001 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.823550 10671 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:15.792482 12034 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.840665 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:15.841212 12034 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:15.841596 10671 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.827744 10096 pending_rounds.cc:107] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:15.842842 10096 raft_consensus.cc:2889] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:15.844326 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:15.844480  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.845628  9944 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
W20260709 12:24:15.837010  9942 rpcz_store.cc:269] 0709 12:24:13.617539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.617715 (+   176us) service_pool.cc:224] Handling call
0709 12:24:15.644799 (+2027084us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:15.824117 (+179318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16726784}
I20260709 12:24:15.860826 12289 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:15.861879 12289 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:15.863821 11977 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.865309 12309 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:15.866374 12282 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:15.867323 12282 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:15.867194 12045 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.874567 10677 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:15.874807 10094 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:15.860873 10670 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:15.888669 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:15.877133 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 129101us
W20260709 12:24:15.876803  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:15.878397 10016 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:24:15.922822  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 180264us
I20260709 12:24:15.888135  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1489) took 2283 ms. Trace:
I20260709 12:24:15.926488 10017 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:15.881556  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.874433 10019 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:15.836306 12220 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.869541  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.926007  9947 rpcz_store.cc:276] 0709 12:24:13.604603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.604755 (+   152us) service_pool.cc:224] Handling call
0709 12:24:15.886822 (+2282067us) spinlock_profiling.cc:243] Waited 7.3 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:15.887210 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.887552 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.887581 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:15.887642 (+    61us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:15.887684 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.887746 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:15.887922 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16795008}
I20260709 12:24:15.939383  9945 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:15.838374 12153 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:15.941916  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:15.919728 12329 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
I20260709 12:24:15.890156 12315 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:15.868249 12078 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.870471  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.879236 12148 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.952163 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1408) took 1038 ms. Trace:
I20260709 12:24:15.952971 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:15.952696 10025 rpcz_store.cc:276] 0709 12:24:14.914045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.167924 (+253879us) service_pool.cc:224] Handling call
0709 12:24:15.921173 (+753249us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:15.951962 (+ 30789us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374846208}
I20260709 12:24:15.955242 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:15.955708 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:15.957031 10025 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:15.958448 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:15.919363 12317 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:15.873749 12281 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:15.878542 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:15.988178  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:15.988507 10091 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
W20260709 12:24:15.989236  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46031.3us
I20260709 12:24:15.989694 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:15.990729 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:15.991835 12282 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.993317 12103 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.995241 12083 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.967048 12289 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:15.959715 12045 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:15.882880 12240 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:16.002118 12240 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.969869 12111 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.004920 12111 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.971422  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.972311 12220 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:15.934535 12062 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:15.969398 12215 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:15.976858  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79412.3us
I20260709 12:24:16.007833 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:15.978480 12233 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:15.978619 12217 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:15.983114  9942 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:15.984395  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:15.985356 10675 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:15.950968 12123 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:15.986547 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:15.986474 12317 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:15.987421  9948 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:15.873693 12304 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:16.002532 12148 raft_consensus.cc:2764] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:16.098855  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:16.098973 12148 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:16.100159 12324 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.100497  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:16.101565 12010 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.102048 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 632) took 1403 ms. Trace:
I20260709 12:24:16.102275 12323 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:15.977489 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:15.978080 12147 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:16.005966 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.017908 10092 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:16.104970  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.113124  9949 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:16.008831 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:16.022727 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1491) took 2997 ms (client timeout 2562 ms). Trace:
I20260709 12:24:16.014420 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:16.022941 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:16.006047  9947 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
W20260709 12:24:16.013582 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 286) took 4036 ms (client timeout 2951 ms). Trace:
I20260709 12:24:16.025493 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 531) took 1100 ms. Trace:
I20260709 12:24:16.013347 10024 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:16.088037 12062 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:16.092443 12125 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:16.093585 10097 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:16.095432  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72813.3us
I20260709 12:24:16.097393  9940 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:16.098085 12083 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.016624 10096 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:24:16.013348 10023 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:16.124203 10671 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:16.009977 10091 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:16.102550 10673 rpcz_store.cc:276] 0709 12:24:14.698638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.700668 (+  2030us) service_pool.cc:224] Handling call
0709 12:24:16.101648 (+1400980us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:16.101828 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":859399040}
W20260709 12:24:16.116389 10021 rpcz_store.cc:269] 0709 12:24:11.977045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:11.977190 (+   145us) service_pool.cc:224] Handling call
0709 12:24:16.013140 (+4035950us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54001c5554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:16.013317 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922687872}
I20260709 12:24:16.128384 10090 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:24:16.114995 10093 rpcz_store.cc:269] 0709 12:24:13.025278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:13.238398 (+213120us) service_pool.cc:224] Handling call
0709 12:24:16.022139 (+2783741us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400093154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:16.022435 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321353472}
I20260709 12:24:16.014333 12215 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:16.117050 10672 rpcz_store.cc:276] 0709 12:24:14.925331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.925530 (+   199us) service_pool.cc:224] Handling call
0709 12:24:14.925966 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.442552 (+516586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.024896 (+582344us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b5400615b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:16.025231 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1186724736}
I20260709 12:24:16.108654 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.152316 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:16.102640 12148 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:16.153630 12125 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:24:16.010921 12315 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:16.162469  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:16.023252 12220 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:16.111312 12217 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:16.118427 12186 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.107018 12309 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:16.160075 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 209619us
I20260709 12:24:16.160382 10017 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:16.165019  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.164994 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.165359  9944 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:16.008658 12088 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:16.166754  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.102427 12010 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.164206 10670 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:16.176457 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.169356 10676 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:16.156893 12148 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:16.158555 12323 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:16.182700 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:16.183288 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 160469us
I20260709 12:24:16.129515  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:16.183723 10090 raft_consensus.cc:2410] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:16.184974 10671 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:16.169795 12309 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.186766 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.178045 12301 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:16.188812  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.191326 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:16.170756 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:16.186717 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.219239  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.219540 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:16.221756 12295 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.218297 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:16.222172 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.222605 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:16.223855 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:16.220002  9944 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:16.225916 10671 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:16.165850 12083 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:16.225871 10676 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Advancing to term 4
I20260709 12:24:16.226714 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:16.231328 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50046us
I20260709 12:24:16.218173  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:16.171787 12035 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.239431  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.241739  9945 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.218338 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.243088 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:16.236588 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:16.236811  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 418235us
I20260709 12:24:16.192031 12301 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.233568 10096 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:16.245615 10670 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:16.243237  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54991.5us
I20260709 12:24:16.244088 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:16.192433 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.244563 10673 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 1.
I20260709 12:24:16.228201 12215 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 12:24:16.255254 12019 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:16.255808 12019 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.256295 12019 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.258736 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.258993 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:16.260044 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:16.260586 10674 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.260825 10017 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:16.262702 12328 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.609s	user 0.005s	sys 0.000s
I20260709 12:24:16.264205 12268 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.264886 12268 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.261673 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:16.272131  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.279364 10674 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:16.254074  9948 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:16.278242 10093 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:16.178754 12186 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.319602 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:16.283042 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:16.247581 12301 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:16.281752  9912 leader_election.cc:336] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.993s (SENT)
W20260709 12:24:16.322286 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 71356.6us
W20260709 12:24:16.331321 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64935.1us
W20260709 12:24:16.324787  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71847.7us
I20260709 12:24:16.278476 12062 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.331842  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.331447  9944 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
W20260709 12:24:16.361245  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82402.2us
I20260709 12:24:16.266667 12010 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:16.364396 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:16.360767  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.370200  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1286) took 1067 ms. Trace:
I20260709 12:24:16.370276 10017 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:16.332726 10091 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:16.370702  9949 rpcz_store.cc:276] 0709 12:24:15.302951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.266943 (+963992us) service_pool.cc:224] Handling call
0709 12:24:16.370122 (+103179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.222178 12323 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.334743 12047 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:16.391495  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.278371 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:16.397861  9944 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:16.398339  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1278) took 1139 ms. Trace:
I20260709 12:24:16.334941 11938 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:16.403352 10021 raft_consensus.cc:2368] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:16.269673 10018 raft_consensus.cc:2376] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 3. Re-sending same reply.
I20260709 12:24:16.398684  9944 rpcz_store.cc:276] 0709 12:24:15.258841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.391110 (+1132269us) service_pool.cc:224] Handling call
0709 12:24:16.398278 (+  7168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.427772  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:16.435842  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 165720us
I20260709 12:24:16.436014 12083 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:16.437361 12035 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:16.437599  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.407925 11938 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:16.438973 11938 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:16.443141  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51358.1us
I20260709 12:24:16.445616 10670 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:16.434813  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.270507 12123 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.126s
I20260709 12:24:16.448084 12268 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:16.443354  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73504.3us
I20260709 12:24:16.492164 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.387283 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 637) took 1019 ms. Trace:
I20260709 12:24:16.492913 10093 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:16.499063 10673 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:16.536260 10676 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:16.332180 12315 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:16.574446  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.569649  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:16.570796 10019 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.000s
I20260709 12:24:16.579262 10019 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
W20260709 12:24:16.573997 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.580384 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:16.493471 12164 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.825s	user 0.004s	sys 0.009s
I20260709 12:24:16.590903  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:16.591568  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:16.591598 12047 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:16.592820 12078 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:16.598793 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.598654 12185 raft_consensus.cc:2764] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:24:16.599344 12153 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.637s	user 0.004s	sys 0.008s
I20260709 12:24:16.611260 11978 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.611927 11977 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.404165 12062 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:24:16.612254  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 489876us
W20260709 12:24:16.615370  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 151707us
W20260709 12:24:16.567727 12336 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
I20260709 12:24:16.653295 10021 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:16.499394 12317 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:16.620038 12164 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.205s	user 0.004s	sys 0.010s
I20260709 12:24:16.387408 12148 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:16.692438 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 293) took 3780 ms. Trace:
I20260709 12:24:16.693005 10021 rpcz_store.cc:276] 0709 12:24:12.908633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.433519 (+3524886us) service_pool.cc:224] Handling call
0709 12:24:16.647910 (+214391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:16.688284 (+ 40374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.688535 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.694893 10096 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:16.695544 12295 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:16.697114 12062 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:16.585754 12268 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:16.499172 12111 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.171s
W20260709 12:24:16.677233 12034 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.835s	user 0.014s	sys 0.000s
I20260709 12:24:16.394922 12217 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.713035 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.708053 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.450004 12220 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.713918 10096 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:16.698071  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:16.724951 10021 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:24:16.704537 12305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.015s	sys 0.046s
I20260709 12:24:16.732050 10025 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:16.704766 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.711848  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:16.738301  9945 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:16.743223 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1407) took 1818 ms. Trace:
I20260709 12:24:16.676872 12153 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.682415 12078 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.751134 10019 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:24:16.750731 10021 rpcz_store.cc:276] 0709 12:24:14.913741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.694106 (+1780365us) service_pool.cc:224] Handling call
0709 12:24:16.694534 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.725427 (+ 30893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.732475 (+  7048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.711016 11978 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:16.503153 10668 rpcz_store.cc:276] 0709 12:24:15.359124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.741205 (+382081us) service_pool.cc:224] Handling call
0709 12:24:15.741827 (+   622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.742524 (+   697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.742570 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.799094 (+ 56524us) log.cc:844] Serialized 26 byte log entry
0709 12:24:16.140151 (+341057us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:16.156615 (+ 16464us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.161904 (+  5289us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.335318 (+173414us) raft_consensus.cc:1685] finished
0709 12:24:16.358438 (+ 23120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.370040 (+ 11602us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7181,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":13758,"thread_start_us":196981,"threads_started":1,"wal-append.queue_time_us":389810}
I20260709 12:24:16.760967 12047 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:16.761580 11978 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:16.761940 10093 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
W20260709 12:24:16.762161 12353 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:16.754491 12034 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.763214 12338 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.110s
I20260709 12:24:16.763711 12034 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:16.754645  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56672.7us
I20260709 12:24:16.688318 11977 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:16.755203 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 294) took 3270 ms. Trace:
I20260709 12:24:16.756486 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:16.735206 12305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.176s	user 0.015s	sys 0.047s
I20260709 12:24:16.767760 10019 rpcz_store.cc:276] 0709 12:24:13.484924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.579999 (+3095075us) service_pool.cc:224] Handling call
0709 12:24:16.703182 (+123183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:16.754889 (+ 51707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.755127 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.774832  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:16.775640  9945 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.786872  9943 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:16.787128  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59481.2us
W20260709 12:24:16.797786 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 362736us
W20260709 12:24:16.798427 12341 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:24:16.798926 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.799609 10096 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:16.800168 11977 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:16.736843 12268 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.768759 10091 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:16.782977 12149 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:16.784142 10018 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:16.805583 12153 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:16.807505 12019 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.003s
I20260709 12:24:16.808034 12019 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.810684 12054 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.811218 12054 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.811628 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2449) took 1305 ms. Trace:
I20260709 12:24:16.812007 10019 rpcz_store.cc:276] 0709 12:24:15.506136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.776163 (+1270027us) service_pool.cc:224] Handling call
0709 12:24:16.810931 (+ 34768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.811304 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:16.811336 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:16.811381 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:16.811418 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.811471 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.811577 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":346,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":97}
I20260709 12:24:16.787652 12034 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:16.827057 12174 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.827793 12174 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:16.812214 12309 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.580s	user 0.007s	sys 0.004s
I20260709 12:24:16.831096 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.833586 10096 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:16.835338  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 136905us
I20260709 12:24:16.811231 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1388) took 3803 ms. Trace:
I20260709 12:24:16.841271 10018 rpcz_store.cc:276] 0709 12:24:13.007404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.586998 (+3579594us) service_pool.cc:224] Handling call
0709 12:24:16.734941 (+147943us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.810965 (+ 76024us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.811176 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.842967 10018 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:24:16.837109 12153 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:24:16.843330 10018 pending_rounds.cc:85] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 3
I20260709 12:24:16.843616 10018 pending_rounds.cc:107] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:24:16.843822 12153 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:16.843880 10018 raft_consensus.cc:2889] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:16.839373 12078 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:16.840584  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:16.812014 12019 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:16.841735 12301 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.852025 12268 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:16.852253 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:16.853200  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 424) took 1234 ms. Trace:
I20260709 12:24:16.853614  9943 rpcz_store.cc:276] 0709 12:24:15.618904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.155888 (+536984us) service_pool.cc:224] Handling call
0709 12:24:16.853012 (+697124us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b540003f554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:16.853133 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398018816}
W20260709 12:24:16.855336 10669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 629) took 2223 ms (client timeout 2051 ms). Trace:
W20260709 12:24:16.855950 10669 rpcz_store.cc:269] 0709 12:24:14.631789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.641991 (+ 10202us) service_pool.cc:224] Handling call
0709 12:24:16.854936 (+2212945us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:16.855119 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37682048}
I20260709 12:24:16.856855 12174 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:16.857506  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:16.858076 10016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2468) took 1163 ms (client timeout 1511 ms). Trace:
W20260709 12:24:16.858471 10016 rpcz_store.cc:269] 0709 12:24:15.694849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.742319 (+ 47470us) service_pool.cc:224] Handling call
0709 12:24:16.857860 (+1115541us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002652d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:16.858002 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309743360}
I20260709 12:24:16.859819 10016 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:16.859743  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.860571 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2458) took 1206 ms. Trace:
I20260709 12:24:16.861275  9943 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:16.861215 10016 rpcz_store.cc:276] 0709 12:24:15.671971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.859301 (+1187330us) service_pool.cc:224] Handling call
0709 12:24:16.859595 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.860346 (+   751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.860502 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.862901  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:16.863770 12149 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:16.864260 12149 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:16.864638 12343 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:24:16.865397 12186 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:16.865505 10675 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.863638 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:16.855247 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:16.867887 10016 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:16.871467 10016 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:16.864655 12149 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:16.879593 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.846649 12217 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.895303 10672 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:16.869484  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:16.884811  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1501) took 1809 ms. Trace:
I20260709 12:24:16.910888  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.907223 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:16.913213  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:16.895515 12186 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:16.915014 10096 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:16.915328 10093 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:16.873071 12153 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:24:16.916368 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:16.919456  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50078.2us
I20260709 12:24:16.921274 12327 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:16.919481  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79014.7us
I20260709 12:24:16.836112 12309 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.884372 10091 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:16.928242 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 301) took 1861 ms. Trace:
I20260709 12:24:16.946187 10021 rpcz_store.cc:276] 0709 12:24:15.066719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.786312 (+1719593us) service_pool.cc:224] Handling call
0709 12:24:16.849685 (+ 63373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.927752 (+ 78067us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.927789 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:16.927844 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:16.927906 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.927992 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.928163 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.901937  9949 rpcz_store.cc:276] 0709 12:24:15.063660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.603468 (+1539808us) service_pool.cc:224] Handling call
0709 12:24:16.603981 (+   513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.604337 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:16.604778 (+   441us) log.cc:844] Serialized 14 byte log entry
0709 12:24:16.872908 (+268130us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:16.873026 (+   118us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:16.873110 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.873233 (+   123us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.873385 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":812,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":57596,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":12295,"thread_start_us":253771,"threads_started":2}
I20260709 12:24:16.949877  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:16.961835 10093 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:16.930465 12186 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:16.937280 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2457) took 1304 ms. Trace:
I20260709 12:24:16.967147 10090 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:16.968220 10018 rpcz_store.cc:276] 0709 12:24:15.618580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.842184 (+1223604us) service_pool.cc:224] Handling call
0709 12:24:16.842556 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.883816 (+ 41260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.922998 (+ 39182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:16.941386 12164 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:16.971107  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:16.962073 10016 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:16.906096 12289 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:24:16.981174 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 313) took 2350 ms. Trace:
I20260709 12:24:16.981509 10016 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:16.983788 12347 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:24:16.986306 12354 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:16.986711 12084 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:16.986909 11993 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:16.987504 12084 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:16.988629 12354 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:16.989081 12354 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:16.989465 12354 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:16.989943 12084 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:16.991031 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:16.993105 10640 leader_election.cc:336] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.572s (SENT)
I20260709 12:24:16.993255 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 535) took 1072 ms. Trace:
I20260709 12:24:16.994848 12308 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:24:16.993706 10671 rpcz_store.cc:276] 0709 12:24:15.920908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.259774 (+338866us) service_pool.cc:224] Handling call
0709 12:24:16.260101 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:16.334708 (+ 74607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.334747 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:16.358185 (+ 23438us) log.cc:844] Serialized 26 byte log entry
0709 12:24:16.651264 (+293079us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:16.651315 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.651390 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:16.993029 (+341639us) raft_consensus.cc:1685] finished
0709 12:24:16.993057 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.993194 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17962,"raft.queue_time_us":60073,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":39927,"thread_start_us":198276,"threads_started":1,"wal-append.queue_time_us":608546}
I20260709 12:24:16.996032 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.997458 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:16.998059 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2518) took 1296 ms. Trace:
I20260709 12:24:16.998852 12360 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.000254 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 541) took 1004 ms. Trace:
I20260709 12:24:17.001413 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 547) took 1600 ms. Trace:
I20260709 12:24:17.000661 10676 rpcz_store.cc:276] 0709 12:24:15.995817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.682365 (+686548us) service_pool.cc:224] Handling call
0709 12:24:16.696534 (+ 14169us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:16.697083 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.697116 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:16.697682 (+   566us) log.cc:844] Serialized 26 byte log entry
0709 12:24:16.949136 (+251454us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:16.949202 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.949304 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:17.000027 (+ 50723us) raft_consensus.cc:1685] finished
0709 12:24:17.000050 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.000193 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1751,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":10463,"thread_start_us":245081,"threads_started":1,"wal-append.queue_time_us":300229}
I20260709 12:24:17.001776 10677 rpcz_store.cc:276] 0709 12:24:15.401115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.526809 (+125694us) service_pool.cc:224] Handling call
0709 12:24:17.001190 (+1474381us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400490a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.001355 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":715276544}
I20260709 12:24:16.981829 10099 rpcz_store.cc:276] 0709 12:24:14.630540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:14.630932 (+   392us) service_pool.cc:224] Handling call
0709 12:24:14.631348 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:14.674632 (+ 43284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:14.674662 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:14.674722 (+    60us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:14.674774 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.969154 (+2294380us) spinlock_profiling.cc:243] Waited 43 ms on lock 0x7b54002fc454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:16.975009 (+  5855us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:16.980977 (+  5968us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98912000}
I20260709 12:24:17.003429 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:17.003585 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 558) took 1125 ms. Trace:
I20260709 12:24:17.004014 10099 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.003607 12164 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:17.004626 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2519) took 1302 ms. Trace:
I20260709 12:24:17.005009 10097 rpcz_store.cc:276] 0709 12:24:15.702009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.705446 (+  3437us) service_pool.cc:224] Handling call
0709 12:24:16.025735 (+320289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.133322 (+107587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.004425 (+871103us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b5400093150. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.004561 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":735311616}
I20260709 12:24:17.005913  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:17.006546 10097 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:17.004037 10673 rpcz_store.cc:276] 0709 12:24:15.877970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.259513 (+381543us) service_pool.cc:224] Handling call
0709 12:24:16.452519 (+193006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.567958 (+115439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.003397 (+435439us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b5400408dd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.003527 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":430473472}
W20260709 12:24:17.023694  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72334.1us
I20260709 12:24:17.006683 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:17.011358  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.048875  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:16.998505 10094 rpcz_store.cc:276] 0709 12:24:15.701529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.702813 (+  1284us) service_pool.cc:224] Handling call
0709 12:24:15.703392 (+   579us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:15.931423 (+228031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:16.997795 (+1066372us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:16.997978 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285058432}
I20260709 12:24:17.008255  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:16.994156 10640 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.590s (SENT)
I20260709 12:24:17.053731 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:17.054641  9945 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:17.056011  9942 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:17.055596 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:17.057421 10096 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
W20260709 12:24:17.058925 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.059635 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:17.058576 10016 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:24:17.061316 10016 pending_rounds.cc:85] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 1
I20260709 12:24:17.061622 10016 pending_rounds.cc:107] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:24:17.060186  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.007335 12164 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:17.079516  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71376.6us
I20260709 12:24:17.080024  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1502) took 2016 ms. Trace:
I20260709 12:24:17.080374  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.061926 10016 raft_consensus.cc:2889] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.064088  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.065467  9945 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:24:17.065692 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71563.7us
I20260709 12:24:17.067160  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.081705  9942 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.068920  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.007486 12301 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:17.008737 12309 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:17.070520 10095 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:17.071391 12326 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.516s	user 0.016s	sys 0.020s
I20260709 12:24:17.086298 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:17.071522 12282 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.072412 12355 raft_consensus.cc:2764] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
W20260709 12:24:17.060973  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 422) took 1438 ms (client timeout 1908 ms). Trace:
I20260709 12:24:17.063370 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:17.082583  9943 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.086776 10671 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:17.087708 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2459) took 1415 ms. Trace:
I20260709 12:24:17.099455 10016 rpcz_store.cc:276] 0709 12:24:15.672378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.862211 (+1189833us) service_pool.cc:224] Handling call
0709 12:24:16.862510 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.087442 (+224932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.087644 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:17.093752 10094 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:17.080588  9944 rpcz_store.cc:276] 0709 12:24:15.063900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.676624 (+1612724us) service_pool.cc:224] Handling call
0709 12:24:16.752562 (+ 75938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.805875 (+ 53313us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:24:16.835229 (+ 29354us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.079631 (+244402us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.079691 (+    60us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:17.079760 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.079829 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.079967 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17271,"prepare.queue_time_us":17042,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":20090,"thread_start_us":153669,"threads_started":1}
I20260709 12:24:17.086117 12103 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.100847 12103 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.071429 12308 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.101145  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.105324 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.086329 12153 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.106874  9944 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
W20260709 12:24:17.110127  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.115609 12224 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:24:17.117424 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:17.117908 12112 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:17.118364 12112 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:17.118760 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:17.119634 10094 raft_consensus.cc:2410] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:17.119300 12112 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.120189 12338 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:17.120591 12338 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:17.120672 12112 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.121600 12338 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.120554 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:17.095247 12282 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:17.134747 10065 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.109589 12153 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:17.099596  9940 rpcz_store.cc:269] 0709 12:24:15.615521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.838450 (+222929us) service_pool.cc:224] Handling call
0709 12:24:17.013083 (+1174633us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b540003e654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.053970 (+ 40887us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288875776}
I20260709 12:24:17.131258 12282 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.155025  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.163180 10094 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:17.120052 10091 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:17.162000 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 542) took 1165 ms. Trace:
W20260709 12:24:17.165397  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.165969  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1500) took 2102 ms. Trace:
I20260709 12:24:17.167164 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:17.168888 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:17.169323 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.170662 12327 raft_consensus.cc:2764] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:17.170771 12220 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.172784 10093 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:17.173087 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:17.138537 12338 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.176223 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.177081 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.180045 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.180549  9945 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:17.165199 10668 rpcz_store.cc:276] 0709 12:24:15.996124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.761037 (+764913us) service_pool.cc:224] Handling call
0709 12:24:16.761419 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.761779 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:16.806277 (+ 44498us) log.cc:844] Serialized 14 byte log entry
0709 12:24:17.123981 (+317704us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.125421 (+  1440us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:17.135489 (+ 10068us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.138146 (+  2657us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.153461 (+ 15315us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":239847,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":11420,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"thread_start_us":242077,"threads_started":2}
I20260709 12:24:17.184212  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.152726 12153 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.184803  9945 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.186939 10094 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:24:17.189168 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 145181us
I20260709 12:24:17.171463 12327 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.170773 12103 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:17.171862 11993 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:17.165230  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.193907 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.194831 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:17.198899 10018 raft_consensus.cc:2410] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:24:17.202752  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 122464us
I20260709 12:24:17.203315 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:17.205576  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.136349 12186 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.206162  9944 raft_consensus.cc:2376] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4. Re-sending same reply.
W20260709 12:24:17.208997  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 145021us
I20260709 12:24:17.213927  9945 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:17.191039 12308 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.191291 12327 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:17.221331 12372 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:17.214679 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:17.226950 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56392.5us
I20260709 12:24:17.227028 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.197160 11993 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:17.222297  9940 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:17.228670 12287 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:17.246850 10062 leader_election.cc:336] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.554s (SENT)
I20260709 12:24:17.247428 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:17.248821 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.233116 12300 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.250532 12300 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:17.250938 12300 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.250784 12062 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:24:17.251334 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52090.3us
I20260709 12:24:17.251628 12185 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.252033 12185 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.252372 12185 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.252312 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.253986 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.254750 12307 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:17.255323 12307 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.257560 12307 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:17.258183 12307 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.166527  9941 rpcz_store.cc:276] 0709 12:24:15.063391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.246202 (+182811us) service_pool.cc:224] Handling call
0709 12:24:15.246628 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:15.246950 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:15.247007 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:15.247619 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 12:24:15.598651 (+351032us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:15.599701 (+  1050us) log.cc:844] Serialized 14 byte log entry
0709 12:24:15.600421 (+   720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:15.600617 (+   196us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:17.070784 (+1470167us) spinlock_profiling.cc:243] Waited 85.3 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:17.165371 (+ 94587us) raft_consensus.cc:1685] finished
0709 12:24:17.165518 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.165734 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":42630,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":5763,"raft.queue_time_us":22331,"raft.run_cpu_time_us":339,"raft.run_wall_time_us":340,"spinlock_wait_cycles":196220672,"thread_start_us":232251,"threads_started":1,"wal-append.queue_time_us":290875}
I20260709 12:24:17.258836 12078 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.259317 12078 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:17.259717 12078 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.259768  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.260591  9941 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:17.261685  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:17.263197 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43817.6us
I20260709 12:24:17.209928 12289 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.269265 12289 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.268973 12149 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.269703 12289 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.219570 12308 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.270267 12308 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.272125 12149 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:17.196508 12366 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:17.274829 12366 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.275235 12366 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:17.274933 12185 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.275847 12300 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.277209 12185 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:17.223793 12327 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:17.278918 11938 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.839s	user 0.011s	sys 0.000s
I20260709 12:24:17.268920 12078 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.279376 11938 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.278935 12300 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:17.281344 11938 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:17.228816 11993 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.285481 12153 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.209807 12301 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260709 12:24:17.286955  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.289006 12078 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:17.230420 12354 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.292023 12354 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:17.303201 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.304400 10021 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:17.307188 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2533) took 1328 ms (client timeout 1724 ms). Trace:
W20260709 12:24:17.307776 10098 rpcz_store.cc:269] 0709 12:24:15.978942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.998559 (+ 19617us) service_pool.cc:224] Handling call
0709 12:24:17.306855 (+1308296us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.307086 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081856768}
I20260709 12:24:17.311543 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.312340 12078 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:17.312453 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.312876 12078 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:17.313242 10019 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:17.231433 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.313879 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2456) took 1695 ms. Trace:
I20260709 12:24:17.233072  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.233392 12290 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.234001 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:17.239171 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.239224 12186 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.242865 12309 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.248734 12103 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.263115 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:17.232447 12169 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.049s
I20260709 12:24:17.274873 12315 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:24:17.314570 10018 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.322650 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 540) took 1327 ms. Trace:
I20260709 12:24:17.314793 12290 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.323047 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:17.324121 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.325126 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.325428 12148 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.326105 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:17.320027 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57044.2us
I20260709 12:24:17.327996 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:17.329031 10095 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2532) took 1391 ms (client timeout 1644 ms). Trace:
W20260709 12:24:17.329456 10095 rpcz_store.cc:269] 0709 12:24:15.938022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.938168 (+   146us) service_pool.cc:224] Handling call
0709 12:24:17.328887 (+1390719us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400163854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.328976 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2567804160}
I20260709 12:24:17.319001 12309 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.331044 12309 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:17.331076 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 79969us
I20260709 12:24:17.332270 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:17.333144 12287 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.333575 12287 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.333698 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:17.269809 12164 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.335273 12019 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.335582 12305 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:17.335716 12019 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:17.336169 12019 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.336458  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.337708  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.337917 10018 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.338795 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:17.339062 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2529) took 1540 ms (client timeout 1524 ms). Trace:
W20260709 12:24:17.339448 10092 rpcz_store.cc:269] 0709 12:24:15.802223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.823841 (+ 21618us) service_pool.cc:224] Handling call
0709 12:24:17.338887 (+1515046us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b5400094054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.339000 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421304576}
I20260709 12:24:17.340430  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:17.341176 10025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2450) took 1834 ms (client timeout 2007 ms). Trace:
I20260709 12:24:17.321026 12062 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:17.341734 10025 rpcz_store.cc:269] 0709 12:24:15.506338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.013929 (+507591us) service_pool.cc:224] Handling call
0709 12:24:17.340938 (+1327009us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540011ba54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.341109 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1402267904}
I20260709 12:24:17.342120 12062 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:17.335992 12305 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:17.343143 12305 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.343099 12062 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.343660 12100 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.344208 12100 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.321673 12103 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:17.344765 12100 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.344830 12103 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:17.347137 12062 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:17.320706 12169 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:17.315425  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.349032 12169 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:17.349531  9940 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.350143 12169 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.351696 12169 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.315024 10022 raft_consensus.cc:2368] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 2. Current term is 3.
I20260709 12:24:17.353596 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.323055 10675 rpcz_store.cc:276] 0709 12:24:15.995503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.682776 (+687273us) service_pool.cc:224] Handling call
0709 12:24:16.865294 (+182518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:16.879539 (+ 14245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.322451 (+442912us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b5400610150. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.322573 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":353691008}
I20260709 12:24:17.357193 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:17.320621 12185 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.358317 12185 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:17.358742 12185 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.358868 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.359611 10025 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:17.360484 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:17.333988 12287 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.362262  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.362923  9946 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.362993  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.363852  9949 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:17.313809 12078 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.362270 12301 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
W20260709 12:24:17.373045 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2426) took 6504 ms (client timeout 1844 ms). Trace:
W20260709 12:24:17.373664 10024 rpcz_store.cc:269] 0709 12:24:10.867895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:10.868159 (+   264us) service_pool.cc:224] Handling call
0709 12:24:17.371513 (+6503354us) spinlock_profiling.cc:243] Waited 4.88 s on lock 0x7b54002161d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.372038 (+   525us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11226902912}
W20260709 12:24:17.375533 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1403) took 2463 ms (client timeout 3167 ms). Trace:
W20260709 12:24:17.376029 10023 rpcz_store.cc:269] 0709 12:24:14.912433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.359866 (+447433us) service_pool.cc:224] Handling call
0709 12:24:17.375287 (+2015421us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54001f6654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.375466 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1243462016}
I20260709 12:24:17.232036 10016 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.377802  9942 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:17.379067  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.380117  9942 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:17.381036  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.382021  9942 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:17.367435 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.314342 10019 rpcz_store.cc:276] 0709 12:24:15.618136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.813449 (+1195313us) service_pool.cc:224] Handling call
0709 12:24:17.312798 (+499349us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:17.313028 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.313646 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.313830 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9879808}
I20260709 12:24:17.385119 10024 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:17.385481 10024 pending_rounds.cc:85] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 5
I20260709 12:24:17.385730 10024 pending_rounds.cc:107] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:17.385941 10024 raft_consensus.cc:2889] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.386260  9941 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:24:17.386993  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1504) took 2322 ms. Trace:
I20260709 12:24:17.387323  9941 rpcz_store.cc:276] 0709 12:24:15.064424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.385616 (+2321192us) service_pool.cc:224] Handling call
0709 12:24:17.385991 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.386690 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.386916 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:17.388428 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2444) took 4552 ms. Trace:
I20260709 12:24:17.389209  9941 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:24:17.389701  9941 pending_rounds.cc:85] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 3
I20260709 12:24:17.390038  9941 pending_rounds.cc:107] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:24:17.404021 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.405622 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.407049 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.361771 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.406908 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.346011 12103 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.388788 10020 rpcz_store.cc:276] 0709 12:24:12.835976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:15.844912 (+3008936us) service_pool.cc:224] Handling call
0709 12:24:17.372156 (+1527244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.372559 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.372635 (+    76us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.372702 (+    67us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:17.372782 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.372847 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.388136 (+ 15289us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54002161d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.388236 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":448,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"spinlock_wait_cycles":3511273216}
I20260709 12:24:17.364681  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.231369 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:17.443282 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 211989us
I20260709 12:24:17.444440  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:17.446538 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:17.450747 10018 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.451143 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.390363  9941 raft_consensus.cc:2889] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:17.453828  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.455037 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.438802 12103 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.457245  9943 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:17.458431 12220 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:17.459424 10025 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.460041  9949 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:17.460403 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.458905 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.462908 10019 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:17.462707 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.474210 10020 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.458892 12220 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.481554 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:17.459324 12295 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.453229 12300 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:17.490293  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:17.493481  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 261076us
I20260709 12:24:17.488554 12220 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.499178 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.502336  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.504244 10022 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.508656  9949 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:17.476158 12035 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:17.475270 12342 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.501858 12289 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.519794 12100 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.534483 12315 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:17.534971 12315 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:17.535302 12315 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.536091 12100 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:17.537835 12147 raft_consensus.cc:2764] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:17.538064  9944 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:17.521289 12287 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.531481 12342 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:17.538795 12342 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.540055 12287 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:17.543181 12054 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:17.543874 12111 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.544441 12111 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:17.555164  9946 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:17.526999  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:17.556330  9945 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:17.531876  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.558409 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.550755  9843 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 4, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 4 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:17.488950 12295 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:17.567018 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.549554 12315 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:17.591037  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64129.9us
I20260709 12:24:17.613327  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1508) took 2502 ms. Trace:
I20260709 12:24:17.618449  9949 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:17.618827  9943 rpcz_store.cc:276] 0709 12:24:15.065520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.464061 (+2398541us) service_pool.cc:224] Handling call
0709 12:24:17.530138 (+ 66077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.530710 (+   572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.530741 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:17.567999 (+ 37258us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:17.568083 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.568172 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:17.568341 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:17.625177  9946 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:17.623850  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1507) took 2558 ms. Trace:
I20260709 12:24:17.638646  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1511) took 2525 ms. Trace:
I20260709 12:24:17.644973 10022 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.463047  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:24:17.531888 12233 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:17.571471  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60230.4us
I20260709 12:24:17.559650 12035 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:17.559024 12084 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.638767  9945 rpcz_store.cc:276] 0709 12:24:15.065265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.396791 (+2331526us) service_pool.cc:224] Handling call
0709 12:24:17.556112 (+159321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.556795 (+   683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.623598 (+ 66803us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b540003e650. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:17.623782 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365401984}
I20260709 12:24:17.678990  9942 raft_consensus.cc:606] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:17.694896  9942 raft_consensus.cc:614] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:17.695590  9945 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:17.659384 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.675987  9843 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 3 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:17.639114  9946 rpcz_store.cc:276] 0709 12:24:15.113697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.560429 (+2446732us) service_pool.cc:224] Handling call
0709 12:24:17.624871 (+ 64442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.638249 (+ 13378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.638549 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:17.680310  9948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.009s	sys 0.004s
W20260709 12:24:17.701342  9948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.009s	sys 0.004s
W20260709 12:24:17.705896 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:17.688840  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.585479  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1505) took 2501 ms. Trace:
I20260709 12:24:17.587800 12054 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:17.585711 12295 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:17.710636  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:17.703512  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 423) took 2087 ms. Trace:
W20260709 12:24:17.715523  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 286504us
I20260709 12:24:17.596498 12290 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:17.570966 12111 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:18.037667 12185 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.678s	user 0.003s	sys 0.006s
W20260709 12:24:18.128031 12148 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.802s	user 0.013s	sys 0.000s
I20260709 12:24:17.778957 12327 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:17.822928 12308 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:24:18.526844 10098 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.526748 12308 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.803936 12220 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:18.023798  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:17.869451 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:17.976469 12373 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:24:18.533504  9942 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:18.534301  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 429) took 2913 ms. Trace:
I20260709 12:24:18.534713  9942 rpcz_store.cc:276] 0709 12:24:15.642218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.533018 (+2890800us) service_pool.cc:224] Handling call
0709 12:24:18.533336 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.534061 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.534246 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:17.871675 12300 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:18.186798 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 705285us
I20260709 12:24:17.802990 12118 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:17.846483  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:17.720142 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:17.969923 12153 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:17.743764 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:17.751580  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 62823.4us
I20260709 12:24:18.529469 12185 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:18.196799 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:17.839154 12282 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.653s	user 0.007s	sys 0.006s
W20260709 12:24:18.472342 12377 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:24:18.548305 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.540565 12315 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:18.592836 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.548393 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:18.037091  9940 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.687s	user 0.012s	sys 0.000s
I20260709 12:24:18.549115 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:18.125303 12338 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.943s	user 0.011s	sys 0.000s
W20260709 12:24:17.833287 12366 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.558s	user 0.011s	sys 0.000s
W20260709 12:24:17.790783 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.549998 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:17.974098 12305 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.630s	user 0.002s	sys 0.009s
I20260709 12:24:18.551697 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:18.063576 12035 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.580027 12111 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:18.611044 10640 leader_election.cc:336] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.951s (SENT)
I20260709 12:24:18.613497 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.618508 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:18.619969 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:18.620391 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:18.619590 10018 raft_consensus.cc:2410] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:24:18.129508 11993 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.847s	user 0.010s	sys 0.000s
I20260709 12:24:18.553064 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:18.592228 12301 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.626343 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.576368 10025 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:17.687634 12196 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:18.482456 12342 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.943s	user 0.007s	sys 0.003s
W20260709 12:24:18.521063 12019 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.184s	user 0.006s	sys 0.006s
I20260709 12:24:17.806321  9843 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 3 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:17.708993 12084 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:18.633198 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:18.634197 12084 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.635624 12308 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:18.637010 12379 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:24:18.629185 12111 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:18.638310  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1506) took 3573 ms. Trace:
I20260709 12:24:18.639513 12185 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:18.589406 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 845676us
I20260709 12:24:18.639757 10021 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:18.640288 10021 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:18.641192 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.641038 10021 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:17.751492 12267 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.642175 10021 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.642527 12267 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.643230 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:18.643579 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.644181 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:18.644850 12267 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:18.645149 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:18.638792  9947 rpcz_store.cc:276] 0709 12:24:15.064993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.394427 (+2329434us) service_pool.cc:224] Handling call
0709 12:24:17.397736 (+  3309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.398265 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.398303 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.398933 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.695526 (+296593us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:17.695564 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.695645 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.638044 (+942399us) raft_consensus.cc:1685] finished
0709 12:24:18.638080 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.638241 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1404,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":296230,"threads_started":1,"wal-append.queue_time_us":893401}
I20260709 12:24:18.647444  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:18.648200 10672 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:18.631947 12301 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.650784 12378 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.651154 10674 raft_consensus.cc:2368] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5.
I20260709 12:24:18.654834 12327 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:18.655354 12301 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:18.655618 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.658703  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:18.659480  9947 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:17.976912  9842 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 319) took 9174 ms. Trace:
W20260709 12:24:18.590466  9949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.036s	sys 0.072s
I20260709 12:24:17.777154 12186 raft_consensus.cc:2764] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:18.523475  9945 rpcz_store.cc:276] 0709 12:24:15.618405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.695066 (+2076661us) service_pool.cc:224] Handling call
0709 12:24:17.695437 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.696108 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.703417 (+  7309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:17.718432 12307 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:18.576721 12148 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.797493 12240 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:18.598130 10024 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:18.535820 12300 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.595798 10020 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:18.679137 10021 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 12:24:18.597270  9912 connection.cc:256] RPC call timeout handler was delayed by -2.6640485869997974s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.275s for a response
I20260709 12:24:18.588552 12078 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:17.705302 12233 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.623287 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:18.592590 12147 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.575457 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1516) took 1524 ms. Trace:
I20260709 12:24:18.625334 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:18.625654  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1503) took 3561 ms. Trace:
I20260709 12:24:18.627550 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1416) took 1159 ms. Trace:
I20260709 12:24:18.068325 12371 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:18.628429 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:18.595247 12290 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:18.620648 12309 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.289s	user 0.010s	sys 0.003s
I20260709 12:24:18.691952 12309 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.591378 12118 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:18.523867  9941 rpcz_store.cc:276] 0709 12:24:15.064697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.388298 (+2323601us) service_pool.cc:224] Handling call
0709 12:24:17.388746 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.502818 (+114072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.559418 (+ 56600us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.638955 12111 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.628360 11993 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:18.663743  9949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.859s	user 0.036s	sys 0.072s
I20260709 12:24:17.980728 12289 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:17.819382  9943 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:18.713508  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1287) took 3327 ms. Trace:
I20260709 12:24:18.713868  9943 rpcz_store.cc:276] 0709 12:24:15.385772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.686948 (+2301176us) service_pool.cc:224] Handling call
0709 12:24:17.819125 (+132177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.713246 (+894121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.713453 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.630119 12315 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260709 12:24:18.672626  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:18.719532  9945 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
W20260709 12:24:18.661023 12169 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.309s	user 0.009s	sys 0.002s
I20260709 12:24:18.679651 10021 pending_rounds.cc:85] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 5
I20260709 12:24:18.682248 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.683074 10098 rpcz_store.cc:276] 0709 12:24:17.050668 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.310222 (+259554us) service_pool.cc:224] Handling call
0709 12:24:17.310550 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.527464 (+1216914us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.575139 (+ 47675us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540014a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:18.575370 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2796739328}
I20260709 12:24:18.683988 10025 rpcz_store.cc:276] 0709 12:24:17.468134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.825497 (+357363us) service_pool.cc:224] Handling call
0709 12:24:18.627495 (+801998us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.721984 10021 pending_rounds.cc:107] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:18.684566  9843 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 3, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:18.722381 10021 raft_consensus.cc:2889] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:18.683247  9948 rpcz_store.cc:276] 0709 12:24:15.064126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.689941 (+1625815us) service_pool.cc:224] Handling call
0709 12:24:16.769710 (+ 79769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:17.053682 (+283972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:17.053702 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.065984 (+ 12282us) log.cc:844] Serialized 26 byte log entry
0709 12:24:17.701756 (+635772us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:17.701796 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:17.701858 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.625444 (+923586us) raft_consensus.cc:1685] finished
0709 12:24:18.625462 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.625583 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14280,"raft.queue_time_us":31532,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":21452,"thread_start_us":812260,"threads_started":2,"wal-append.queue_time_us":1410014}
I20260709 12:24:18.593300 12282 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.688941 10670 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:18.691334 12233 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:18.614204 12366 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:18.695272  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.700721  9947 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:18.701246 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:18.622746 12305 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.709175 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:18.688793 12062 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.341s	user 0.006s	sys 0.008s
I20260709 12:24:18.676380 12220 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:18.712299 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 118507us
I20260709 12:24:18.708930 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:18.597922  9940 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:18.715921  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1284) took 3455 ms. Trace:
I20260709 12:24:18.750730 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:18.722868  9912 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.157s (SENT)
I20260709 12:24:18.751889  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:18.688061 12290 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.754534  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:18.683830 12384 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.668278 12019 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.694169 12309 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:18.687089 12371 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:18.700218 12164 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.733220 10024 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:18.708745 12297 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:18.749375  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 225281us
I20260709 12:24:18.678556 12148 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:18.753177  9942 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:18.753772 10097 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:18.754340 10020 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:18.665469 12342 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.755148  9948 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.716854 10675 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 LEARNER]: Advancing to term 3
I20260709 12:24:18.755584  9843 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 3 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:18.758836  9944 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:18.688513 12147 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:18.706128 12301 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:24:18.761121  9946 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:18.758534 12307 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.760596 10091 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:18.774138  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.06366e+06us
I20260709 12:24:18.827759  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:18.662750  9842 rpcz_store.cc:276] 0709 12:24:08.570820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:08.570988 (+   168us) service_pool.cc:224] Handling call
0709 12:24:17.707412 (+9136424us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:10.745349 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:10.745907 (+   558us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:10.745938 (+    31us) write_op.cc:435] Acquired schema lock
0709 12:24:10.745956 (+    18us) tablet.cc:662] Decoding operations
0709 12:24:10.746368 (+   412us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:10.746440 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:10.746470 (+    30us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:24:10.746800 (+   330us) tablet.cc:701] Row locks acquired
0709 12:24:10.746818 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:24:10.746946 (+   128us) write_op.cc:270] Start()
0709 12:24:10.747055 (+   109us) write_op.cc:276] Timestamp: P: 1783599850746904 usec, L: 0
0709 12:24:10.747077 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:24:10.765861 (+ 18784us) log.cc:844] Serialized 919 byte log entry
0709 12:24:11.353536 (+587675us) spinlock_profiling.cc:243] Waited 62.6 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:12.008416 (+654880us) op_driver.cc:464] REPLICATION: finished
0709 12:24:12.674343 (+665927us) write_op.cc:301] APPLY: starting
0709 12:24:12.682090 (+  7747us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:12.695185 (+ 13095us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:12.698230 (+  3045us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:12.738785 (+ 40555us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:12.757802 (+ 19017us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:24:12.757852 (+    50us) write_op.cc:312] APPLY: finished
0709 12:24:12.758076 (+   224us) log.cc:844] Serialized 29 byte log entry
0709 12:24:12.787639 (+ 29563us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:12.787760 (+   121us) write_op.cc:454] Released schema lock
0709 12:24:12.787809 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:12.788325 (+   516us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:12.788367 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":511,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":615703,"apply.run_cpu_time_us":14320,"apply.run_wall_time_us":158239,"num_ops":2,"prepare.queue_time_us":152153,"prepare.run_cpu_time_us":15372,"prepare.run_wall_time_us":312786,"raft.queue_time_us":629816,"raft.run_cpu_time_us":31046,"raft.run_wall_time_us":216977,"replication_time_us":1261112,"spinlock_wait_cycles":144080128,"thread_start_us":668734,"threads_started":5,"wal-append.queue_time_us":317431,"wal-append.run_cpu_time_us":18976,"wal-append.run_wall_time_us":2471649}]]}
W20260709 12:24:18.784569 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 156954us
I20260709 12:24:18.688342 12078 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.826869 12389 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.831130 12389 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:18.831565 12327 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:18.831984 12327 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:18.832204 12342 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:18.832342 12078 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:18.833164 12389 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:18.832844 12327 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.834170 12327 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:18.835011 12366 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:18.835857 12385 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:24:18.750950  9949 rpcz_store.cc:276] 0709 12:24:15.260324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.655570 (+2395246us) service_pool.cc:224] Handling call
0709 12:24:17.675509 (+ 19939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:17.675753 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:17.752576 (+ 76823us) log.cc:844] Serialized 14 byte log entry
0709 12:24:18.715443 (+962867us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:18.715520 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:18.715597 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.715685 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.715853 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":944173,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":7167,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":6807,"spinlock_wait_cycles":866176,"thread_start_us":980785,"threads_started":2,"wal-append.queue_time_us":901396}
I20260709 12:24:18.837527 12147 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:18.797475 10640 leader_election.cc:336] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.673s (SENT)
I20260709 12:24:18.814087 12084 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.758534 12290 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:18.838317 12301 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:24:18.806089  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:18.810372  9843 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 4, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 4 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:24:18.758936  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1529) took 1470 ms (client timeout 1752 ms). Trace:
I20260709 12:24:18.809389 12297 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.822089 12309 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:18.835772  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 122209us
I20260709 12:24:18.802928 12186 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.838140  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:18.849475  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:18.800149  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:18.850230  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1288) took 3153 ms. Trace:
I20260709 12:24:18.875046  9942 rpcz_store.cc:276] 0709 12:24:15.696927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.536143 (+2839216us) service_pool.cc:224] Handling call
0709 12:24:18.753025 (+216882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.806630 (+ 53605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.849930 (+ 43300us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54003a0550. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:18.850125 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498043136}
I20260709 12:24:18.850409  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 418) took 3247 ms. Trace:
I20260709 12:24:18.876420  9944 rpcz_store.cc:276] 0709 12:24:15.604000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.679341 (+2075341us) service_pool.cc:224] Handling call
0709 12:24:17.708317 (+ 28976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.811011 (+1102694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.850201 (+ 39190us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540018b0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:18.850355 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2415615872}
I20260709 12:24:18.803707 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:18.871251  9843 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 3, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 3 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:18.877866 10092 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:18.851203 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.879685 10021 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
W20260709 12:24:18.843834 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46410.9us
I20260709 12:24:18.846375 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:18.881074 10670 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:18.839359 10673 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:18.852245 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 326) took 1229 ms. Trace:
I20260709 12:24:18.881778 10019 rpcz_store.cc:276] 0709 12:24:17.622763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.622921 (+   158us) service_pool.cc:224] Handling call
0709 12:24:18.851977 (+1229056us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540038ccd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:18.852168 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2531711360}
I20260709 12:24:18.853298 10016 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:18.854476  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1307) took 1137 ms. Trace:
I20260709 12:24:18.858480  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:18.839960 12196 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 12:24:18.882849  9946 rpcz_store.cc:276] 0709 12:24:17.716844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.743794 (+ 26950us) service_pool.cc:224] Handling call
0709 12:24:18.854301 (+1110507us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54000168d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:18.854420 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406125824}
I20260709 12:24:18.843310 12078 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:24:18.860301 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:18.884428  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:18.884644 10677 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:18.885231  9946 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:18.844904 12084 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.860847 12371 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:18.862859 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:18.886777 10674 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:18.886509  9946 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:18.887378 12371 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.887760  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 434) took 1721 ms. Trace:
I20260709 12:24:18.887878 12084 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:18.888264  9946 rpcz_store.cc:276] 0709 12:24:17.166498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.885910 (+1719412us) service_pool.cc:224] Handling call
0709 12:24:18.886213 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.887455 (+  1242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.887634 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.858716 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:18.871879  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51600.8us
W20260709 12:24:18.838979 12387 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:24:18.883325  9945 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:18.884093 12078 consensus_queue.cc:1243] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 5.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:18.860736 10095 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:24:18.881415 10673 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:18.892210  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:18.893571  9949 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:18.842950 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1511) took 1796 ms. Trace:
W20260709 12:24:18.902993  9912 connection.cc:256] RPC call timeout handler was delayed by -2.5227570499996546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16590000000000002s for a response
W20260709 12:24:18.903590  9912 connection.cc:256] RPC call timeout handler was delayed by -2.1192632639999829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20510000000000003s for a response
W20260709 12:24:18.903978  9912 connection.cc:256] RPC call timeout handler was delayed by -2.0434837729999344s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1511s for a response
W20260709 12:24:18.904285  9912 connection.cc:256] RPC call timeout handler was delayed by -1.8517837730000792s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
W20260709 12:24:18.904593  9912 connection.cc:256] RPC call timeout handler was delayed by -1.7086495149997063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2528s for a response
W20260709 12:24:18.904882  9912 connection.cc:256] RPC call timeout handler was delayed by -1.5702216799995767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2599s for a response
W20260709 12:24:18.905191  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4804619859996819s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
W20260709 12:24:18.905563  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4323837729998559s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
I20260709 12:24:18.901357 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:18.908802  9843 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:18.910719 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.913501 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:18.914366  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 433) took 2950 ms. Trace:
I20260709 12:24:18.914767  9949 rpcz_store.cc:276] 0709 12:24:15.952606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.874534 (+2921928us) service_pool.cc:224] Handling call
0709 12:24:18.874883 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.902724 (+ 27841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.903004 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:18.915307 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56718.6us
I20260709 12:24:18.808315 12164 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.922865  9943 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:18.916508 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:18.917980 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:18.924677 10092 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:18.924853 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:18.925069  9946 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:18.918915  9944 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:18.921872 10095 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:18.926484  9942 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:18.870394 12309 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:18.930047  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1300) took 1849 ms. Trace:
I20260709 12:24:18.930783  9942 rpcz_store.cc:276] 0709 12:24:17.077513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.892513 (+1815000us) service_pool.cc:224] Handling call
0709 12:24:18.923750 (+ 31237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.926911 (+  3161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.927117 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.821240 12295 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:18.930001  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1516) took 2316 ms. Trace:
W20260709 12:24:18.933660  9912 leader_election.cc:336] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.511s (SENT)
I20260709 12:24:18.935778  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:18.935647  9943 rpcz_store.cc:276] 0709 12:24:16.607508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.874870 (+2267362us) service_pool.cc:224] Handling call
0709 12:24:18.922653 (+ 47783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.923257 (+   604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.923460 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.936452  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1528) took 1814 ms. Trace:
I20260709 12:24:18.937043  9944 rpcz_store.cc:276] 0709 12:24:17.115389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.892697 (+1777308us) service_pool.cc:224] Handling call
0709 12:24:18.918693 (+ 25996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.926760 (+  8067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.929678 (+  2918us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.939746  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:18.945264 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:18.946082 10672 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:18.901904 12147 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.933204 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:18.948241 10677 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:18.946841 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.926687  9941 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:18.863505  9940 rpcz_store.cc:269] 0709 12:24:17.288661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.315115 (+ 26454us) service_pool.cc:224] Handling call
0709 12:24:18.755308 (+1440193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.909212 12240 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.950969 12301 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:18.863938 12376 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 12:24:18.952461 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:18.963366 10676 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:18.952566  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 124011us
I20260709 12:24:18.954048  9947 raft_consensus.cc:2410] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:18.926900 12084 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:18.926801 12185 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:18.929980 12308 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:18.930806 12384 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:18.959404 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 96719us
I20260709 12:24:18.960784  9945 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:18.962220 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:18.967546 10094 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:18.944677 12078 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:18.921783 10669 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:18.967010 10673 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.953181 12342 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.957330 12148 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:18.863698 12118 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:18.956425 12186 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:18.986688 10673 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:18.986754 10062 connection.cc:256] RPC call timeout handler was delayed by -1.5121827470002245s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
I20260709 12:24:18.993691 10020 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2.
I20260709 12:24:18.987389  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1308) took 1270 ms. Trace:
W20260709 12:24:18.999293 10062 connection.cc:256] RPC call timeout handler was delayed by -1.481582747000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
I20260709 12:24:18.999504 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 292) took 6078 ms. Trace:
I20260709 12:24:18.961917 12054 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:18.998981  9945 rpcz_store.cc:276] 0709 12:24:17.717175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.960153 (+1242978us) service_pool.cc:224] Handling call
0709 12:24:18.960469 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.987127 (+ 26658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.987325 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:19.000309  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 60641.8us
I20260709 12:24:19.000855 10019 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:24:18.999732 10062 connection.cc:256] RPC call timeout handler was delayed by -1.4491827470001226s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1684s for a response
I20260709 12:24:19.002372 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:18.987691  9943 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:19.003603  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1310) took 1285 ms. Trace:
I20260709 12:24:19.005079 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:19.003961  9943 rpcz_store.cc:276] 0709 12:24:17.717710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.960797 (+1243087us) service_pool.cc:224] Handling call
0709 12:24:18.985730 (+ 24933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.003362 (+ 17632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.003549 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:18.971632 12185 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:18.955041 12147 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:19.007920 12392 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:24:19.007812  9940 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:18.909340 10091 rpcz_store.cc:276] 0709 12:24:17.046407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.191905 (+145498us) service_pool.cc:224] Handling call
0709 12:24:17.220116 (+ 28211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.830779 (+1610663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.842668 (+ 11889us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54002fce54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:18.842880 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3509086080}
I20260709 12:24:18.971915 12308 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:19.024039 12308 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:19.024906 12103 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.561s	user 0.011s	sys 0.001s
I20260709 12:24:19.026814 12282 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:18.985836 12394 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:18.999183 12019 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.015306 12400 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.985212 12240 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.015849 12186 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.018707 12035 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:19.020486 12396 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:24:19.021936 12147 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.022445 12164 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:19.008565 12315 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:24:19.012068  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:18.987856 12084 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:19.014806 10062 connection.cc:256] RPC call timeout handler was delayed by -1.244882747000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
I20260709 12:24:18.971307 11993 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:18.971212 12220 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:18.973026 12169 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:24:18.999926 10017 rpcz_store.cc:276] 0709 12:24:12.908367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.382138 (+3473771us) service_pool.cc:224] Handling call
0709 12:24:16.506874 (+124736us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:16.592153 (+ 85279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:16.592185 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:17.352104 (+759919us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b2c00046bc8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:24:17.352695 (+   591us) log.cc:844] Serialized 26 byte log entry
0709 12:24:18.523649 (+1170954us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:18.783846 (+260197us) log.cc:844] Serialized 14 byte log entry
0709 12:24:18.949381 (+165535us) spinlock_profiling.cc:243] Waited 6.35 ms on lock 0x7b1800454d80. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094d1733 00007f4e09556dbf 00007f4e0955583e
0709 12:24:18.978350 (+ 28969us) spinlock_profiling.cc:243] Waited 40.2 ms on lock 0x7b540024a520. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094d1733 00007f4e09556dbf 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c
0709 12:24:18.978424 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:18.978542 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:18.981181 (+  2639us) raft_consensus.cc:1685] finished
0709 12:24:18.981210 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:18.987128 (+  5918us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40385,"prepare.queue_time_us":305808,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":3302,"raft.queue_time_us":68932,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":142,"spinlock_wait_cycles":216817408,"thread_start_us":602016,"threads_started":2,"wal-append.queue_time_us":1275674}
I20260709 12:24:19.045172 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:19.231951 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.105460 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.340708 12301 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:19.340817 12084 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.341482 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.342109 12233 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:19.009012 12384 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:19.332394 12300 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.004s
I20260709 12:24:19.033975 10676 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:19.364207  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:19.209996 10669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 559) took 2301 ms (client timeout 2144 ms). Trace:
I20260709 12:24:19.044412 12309 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.211383 12338 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:24:19.343537  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1306) took 1882 ms. Trace:
W20260709 12:24:19.046376  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.197924 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.033856 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:19.339679 10062 connection.cc:256] RPC call timeout handler was delayed by -1.1899827470001583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19720000000000001s for a response
I20260709 12:24:19.346146 12240 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.047520 10673 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:19.231118 12378 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.000s
W20260709 12:24:19.228830 12404 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
W20260709 12:24:19.320842 12406 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:24:19.209146 12297 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.348227 12186 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.011482 12185 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.013168 12305 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:19.351018  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 213381us
I20260709 12:24:19.042164 12062 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.351300 12308 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.354156  9912 leader_election.cc:400] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5.
I20260709 12:24:19.466975  9912 leader_election.cc:403] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:19.224838 12295 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:19.339835 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 414968us
I20260709 12:24:19.097429 12103 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.207221  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.332583 12290 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:19.355423 10675 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.000s
I20260709 12:24:19.180747  9949 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:19.316520 10672 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:19.050187  9948 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:19.112061  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 435) took 1945 ms. Trace:
I20260709 12:24:19.077394 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.371986 10099 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:24:19.375504 10062 connection.cc:256] RPC call timeout handler was delayed by -1.1647827470001175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2s for a response
W20260709 12:24:19.472417  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.364331 12164 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.371335  9942 rpcz_store.cc:276] 0709 12:24:17.461242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.931633 (+1470391us) service_pool.cc:224] Handling call
0709 12:24:18.958772 (+ 27139us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.342711 (+383939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.342741 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:19.342791 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:19.342825 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.343318 (+   493us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b540009bad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:19.343356 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.343481 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1518,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"spinlock_wait_cycles":882180352}
W20260709 12:24:19.466642  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81492.9us
W20260709 12:24:19.468189 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 106562us
I20260709 12:24:19.468375 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:19.466616  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:19.472136 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 109684us
I20260709 12:24:19.512975  9944 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.471257  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:19.005707 12342 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.466776 12186 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:19.472709  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 118722us
W20260709 12:24:19.370747  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 358697us
I20260709 12:24:19.359647 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.513687  9947 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:19.375483 12338 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:19.359653 12233 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:19.517198  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:19.522377 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:19.506491  9946 rpcz_store.cc:276] 0709 12:24:17.166865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.889142 (+1722277us) service_pool.cc:224] Handling call
0709 12:24:18.906785 (+ 17643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:18.925519 (+ 18734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.111753 (+186234us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b540003e150. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:19.111984 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39742720}
I20260709 12:24:19.379761 12220 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.537058 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:19.527860  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:19.467104 12378 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:24:19.469960 12111 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:19.526420 10096 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.009s	sys 0.032s
W20260709 12:24:19.548079 10096 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.009s	sys 0.032s
I20260709 12:24:19.512032 12289 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.385227 12300 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.539680 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.384567 12400 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.537755 10023 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:19.558835 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.559508 10016 raft_consensus.cc:2376] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5. Re-sending same reply.
I20260709 12:24:19.550268 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.563273 10099 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.552222  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 436) took 2385 ms. Trace:
I20260709 12:24:19.563791  9949 rpcz_store.cc:276] 0709 12:24:17.167180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.915515 (+1748335us) service_pool.cc:224] Handling call
0709 12:24:19.155039 (+239524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.505312 (+350273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.552011 (+ 46699us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400207950. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:19.552157 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":510795008}
W20260709 12:24:19.566977 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 98657.1us
I20260709 12:24:19.568588 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:19.471639 12196 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:24:19.464977 12118 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.511322 12267 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:19.557931 10090 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:19.515390 12240 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5.
I20260709 12:24:19.516098 12315 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:24:19.535226 12371 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.381212 10669 rpcz_store.cc:269] 0709 12:24:16.908766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:16.915981 (+  7215us) service_pool.cc:224] Handling call
0709 12:24:19.209738 (+2293757us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b54007f2e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:19.209894 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4542031616}
W20260709 12:24:19.565846 12327 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.731s	user 0.012s	sys 0.000s
I20260709 12:24:19.504262 12035 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 12:24:19.544255  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:19.513751 12103 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.585565 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.543061 12399 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.587422 12415 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:19.587536 12233 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.533512 12389 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:24:19.588291 12267 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.591316 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.657488 10090 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:19.592662 12403 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:19.595736 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.566126 12169 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:19.554639 12054 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:19.543260 12305 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:19.556731 10675 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 3. (term mismatch)
I20260709 12:24:19.610433 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.521867 12164 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.590389 12220 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:19.601246 12118 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.611279 12400 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.534914 12307 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 12:24:19.612818 12420 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:19.612903 12342 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:19.533998 12186 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.613641 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.614104 12295 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:19.569516 12412 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:24:19.666728 10675 pending_rounds.cc:85] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 2
I20260709 12:24:19.619212 12147 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:19.598410 10091 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:19.572971 12289 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.600211 12384 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.619654 12019 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.541246 12062 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.635499 10099 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:19.763200 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.764003 10099 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:19.639185  9944 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:19.646116 10096 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:19.648269  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105047us
W20260709 12:24:19.659428 12411 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:24:19.659262 12233 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.667544 10098 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:19.670586 12267 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:19.718154 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 177745us
I20260709 12:24:19.737932 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.667186 12054 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.801846 10090 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:19.650183 12399 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:19.756975 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 234662us
I20260709 12:24:19.751267 10675 pending_rounds.cc:107] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 2.3
W20260709 12:24:19.803864 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.627293 12371 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.807703 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.811120 10099 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:19.811305  9944 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:19.812142 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.814134  9944 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:19.808024 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.815177 10090 raft_consensus.cc:2410] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:19.672133 12305 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.756179 12400 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:19.826493 10094 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:19.800364 12413 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:24:19.826905 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 258425us
I20260709 12:24:19.757568 12342 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:19.828979 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 275505us
I20260709 12:24:19.829718 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.782095 12420 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:19.831837 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:19.799425 12414 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:24:19.801554 12220 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.627095 12419 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.134s
I20260709 12:24:19.834847 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.835940 10098 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.833444 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:19.806205 12054 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.806376 12300 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.809165 12186 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:19.864107  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:19.864487 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.864771  9942 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.861845 10093 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.806694 12103 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:19.839445  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:19.839748 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.805344 12111 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:19.826148 12147 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:19.810643 10675 raft_consensus.cc:2889] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:19.868194 12424 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:19.869050 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.871023 12420 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.892820 10096 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:19.881757 12315 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:19.894094 12315 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:19.894888 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.872876 12427 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:19.895569 12427 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:19.896265 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:19.896463 12427 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.897451 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:19.897630 12427 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:19.874975 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.898844 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:19.899550 10024 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:19.875747 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:19.877892 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.875092 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.901021 10090 raft_consensus.cc:2376] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5. Re-sending same reply.
I20260709 12:24:19.836920 12290 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:19.881901 10093 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:19.916132 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2547) took 1100 ms. Trace:
I20260709 12:24:19.901214 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:19.882936 10016 raft_consensus.cc:2376] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6. Re-sending same reply.
I20260709 12:24:19.881130 12425 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.885519 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:19.886004  9949 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:19.888718 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.889416 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.890511 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.891625 12118 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:19.891990 12399 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:19.943027 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.895094 12315 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:19.943586 10017 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:19.943838 10090 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:19.899346 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:19.944666 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 644) took 1318 ms. Trace:
I20260709 12:24:19.944730 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:19.876219  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.881242 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:19.902187 10019 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:19.894447 10098 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:19.904735  9942 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:19.834920 12308 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:19.905520 10096 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:19.949213 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.880296 10094 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:19.900360 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:19.931337  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.949412 12185 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.935827 10092 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:19.937124 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:19.916553 10097 rpcz_store.cc:276] 0709 12:24:18.816103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.816242 (+   139us) service_pool.cc:224] Handling call
0709 12:24:18.861918 (+ 45676us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:18.862333 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:18.862366 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:18.912680 (+ 50314us) log.cc:844] Serialized 26 byte log entry
0709 12:24:19.005200 (+ 92520us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:19.005239 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.005303 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:19.915905 (+910602us) raft_consensus.cc:1685] finished
0709 12:24:19.915923 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:19.916068 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8107,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":21070,"thread_start_us":53266,"threads_started":1,"wal-append.queue_time_us":537129}
I20260709 12:24:19.940207 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.840240 12342 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:19.882426 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:19.940742 12425 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:19.949793 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:19.955926 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 88527.1us
I20260709 12:24:19.956066 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.945202 10675 rpcz_store.cc:276] 0709 12:24:18.626284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.629062 (+  2778us) service_pool.cc:224] Handling call
0709 12:24:18.686023 (+ 56961us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:19.944405 (+1258382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:19.944607 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:19.957127 10025 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:19.957422 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.881546 12220 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:19.957978 10099 raft_consensus.cc:2376] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4. Re-sending same reply.
I20260709 12:24:19.935292 12295 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:19.959219 12295 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:19.959604 12420 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:19.959687 12295 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:19.960495 12111 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.961833 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:19.962723  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:19.963109 10669 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:19.964799 10023 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:19.956369 10096 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:19.965150 12342 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.955096 10020 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:19.966276 10096 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:19.969234 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:19.965184 12425 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:19.887662 12147 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:19.971714 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:19.971364 12147 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.972436 12147 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.972438 10099 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:19.950459 10098 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:19.978619 10098 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:19.979429 12309 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.980034 10098 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
W20260709 12:24:19.984912  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 467748us
I20260709 12:24:19.959851 12220 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:19.986950  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:19.987680 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 179085us
I20260709 12:24:19.987905  9942 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:19.989130 12420 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:19.989778 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:19.990782 12308 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:19.992702 12308 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:19.983526 10640 connection.cc:256] RPC call timeout handler was delayed by -1.0654613100000461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
W20260709 12:24:19.993955 10640 connection.cc:256] RPC call timeout handler was delayed by -1.0123613099999602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 12:24:19.994359  9944 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:19.996467 12078 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:19.996605 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:19.996896  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:19.997011 12078 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:19.997536  9946 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:19.998795  9942 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:20.001374  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:20.002295  9946 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:20.003700 12078 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:20.005867 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:20.006726  9946 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:19.952914 12062 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:20.006170 12118 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.009773 12118 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.012262 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:20.014648 12420 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:20.015146 12420 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.015517 12420 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.015470  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:20.017577 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 116451us
I20260709 12:24:20.018810 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:20.020416 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:20.020520 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.021801 10098 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:20.022063 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:20.025382 10098 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:24:20.027143  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 482927us
I20260709 12:24:20.027609 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.028453  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:20.031030 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:20.032946 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:20.034458 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.036401 10098 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:20.004864 12308 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:20.041433  9945 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.004683 12403 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:20.014165 12185 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:20.052222  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:19.988883 12437 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:24:20.053198 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.057629  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:20.058483  9942 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:20.029851 12062 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:20.035128 12220 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:20.016443 12078 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:24:20.049783 12423 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:20.066761 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:20.067595 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:20.002547 12338 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.069471 10672 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 3: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:20.073619  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:20.054949 12419 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:20.074797  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:20.057756 12233 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:20.077409 10016 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 1. Current term is 2. Ops: []
I20260709 12:24:20.076666 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:20.078809  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:20.036731 12054 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.041641 12267 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:24:20.081938  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:20.083662 10673 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:20.093590 12403 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.094799 12054 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.096099 12019 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:20.098006  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:20.099489  9947 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:20.100205 10019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:20.062086 12433 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:20.068131 12435 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:20.077927 12422 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.081339 12389 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.077504 12419 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:20.075284 12432 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.101361  9940 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:20.083666 12425 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:20.137472 12432 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.108561 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.138249 12425 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:20.140208 12425 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.102515 12431 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:20.157420  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 633975us
I20260709 12:24:20.167816  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:20.169629 10092 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.107184 12400 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:20.111644 12338 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:24:20.107131 12429 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:20.203608  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 124902us
I20260709 12:24:20.145452 12118 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:20.099103 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:20.146646 12419 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:20.173684  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 74704.2us
W20260709 12:24:20.209524  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.155244  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:20.211683 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:20.166792 12431 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.223941 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:20.221805  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.225418 10095 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:20.227845  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:20.172152 12432 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.231850 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:20.107230 12430 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.138s
I20260709 12:24:20.206579 12429 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.156188 12389 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.206025 12400 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:20.244956 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.229192 12423 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:20.134819 12435 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.228816 12431 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.172541 12433 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:24:20.235075 12384 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:20.164528 12308 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.223104 12427 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 12:24:20.256312  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 168433us
I20260709 12:24:20.237296 12035 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:20.245411 12267 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:20.257031  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 89365.4us
I20260709 12:24:20.270803 10019 raft_consensus.cc:606] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:20.234052  9941 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:20.243894 12429 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.234397 12084 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 12:24:20.276683  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:20.275377 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43625.6us
I20260709 12:24:20.275892 10019 raft_consensus.cc:614] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:20.248679 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 645) took 1604 ms. Trace:
I20260709 12:24:20.248306 12423 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:20.245730 12432 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:24:20.304574 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:20.306959  9940 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.307664 12389 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.274241 12035 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.312739 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:20.313117  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:20.315008  9949 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:20.315193 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:20.305524 12342 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:20.311264  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:20.343271  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.347052 12384 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:20.349295 12439 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:24:20.349413 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:20.350381 12384 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:20.286583 10668 rpcz_store.cc:276] 0709 12:24:18.642817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.642947 (+   130us) service_pool.cc:224] Handling call
0709 12:24:20.238150 (+1595203us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b54006dfe54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:20.246917 (+  8767us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1995785600}
I20260709 12:24:20.337767 12062 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:20.351851 12062 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:20.353387 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:20.355257 12342 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.355752 12342 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.357369 12384 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:20.309096 12267 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:20.357777 12342 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:20.358875 12267 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.335420 12035 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.309065 12450 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.334811 12315 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:20.351128 12443 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:24:20.309446 12423 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:20.315470 12389 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:20.413467  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.416543 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.398434 12342 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:20.419539  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.421285  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105322us
I20260709 12:24:20.432489  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:20.342332 12400 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.414285 12384 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.438133  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:20.416337 12389 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:20.416167 12458 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:20.437077 12342 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:20.437330 12427 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.454192 12297 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:20.454949 12084 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:20.437738 12460 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:20.416430 12450 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.446640 10675 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:20.458387 12297 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:20.485055 12295 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.003s
W20260709 12:24:20.488662 12147 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.000s
I20260709 12:24:20.489351 12371 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.466892 12427 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.478158 12460 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.449723  9940 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:20.456804 12458 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:20.439867 12432 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
W20260709 12:24:20.493892 12309 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.000s
I20260709 12:24:20.447027 12384 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.515323 12394 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:20.514995 12435 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.529611 10669 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:20.529645 12394 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:20.515202 12460 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:20.532258 12467 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:20.540715 12467 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.538693 12297 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:20.543265  9987 connection.cc:256] RPC call timeout handler was delayed by -2.8698306870001034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
W20260709 12:24:20.544041  9987 connection.cc:256] RPC call timeout handler was delayed by -2.8527306870000757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
W20260709 12:24:20.545001  9987 connection.cc:256] RPC call timeout handler was delayed by -2.6088306870001361s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19540000000000002s for a response
W20260709 12:24:20.545686  9987 connection.cc:256] RPC call timeout handler was delayed by -2.5438270049999119s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
W20260709 12:24:20.546351  9987 connection.cc:256] RPC call timeout handler was delayed by -2.4585306870003478s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2121s for a response
W20260709 12:24:20.547107  9987 connection.cc:256] RPC call timeout handler was delayed by -2.3755270050000945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
I20260709 12:24:20.510847 12295 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:20.554270  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 206637us
W20260709 12:24:20.547839  9987 connection.cc:256] RPC call timeout handler was delayed by -2.0416270050000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22290000000000001s for a response
W20260709 12:24:20.558888  9987 connection.cc:256] RPC call timeout handler was delayed by -1.44852700499996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2888s for a response
W20260709 12:24:20.559348  9987 connection.cc:256] RPC call timeout handler was delayed by -1.4377270049999424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29s for a response
W20260709 12:24:20.558110 12420 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.542s	user 0.010s	sys 0.002s
I20260709 12:24:20.541200 12467 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.540099 12394 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:20.567816  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 122348us
I20260709 12:24:20.534260 12422 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:20.548177 12435 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:20.571489  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:20.572616  9947 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:20.574139 10090 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:20.575037  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.575562  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.577267 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.578951 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:20.579222 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.580107 10022 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:20.582921 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:20.583165 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:20.508741 12389 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.538751 12458 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:20.586974  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.594792  9943 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:20.591233 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:20.595844 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.596683 10098 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:20.596812  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:20.597652 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.598279 10096 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
W20260709 12:24:20.598577  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 287389us
I20260709 12:24:20.600266 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.549269 12464 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:24:20.604943 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.610864 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.611560 10020 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
W20260709 12:24:20.613520  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.528729 12342 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.623816 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.624282 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:20.629329 10016 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:20.631086 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.635283  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:20.586207 12420 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.637122 10098 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
W20260709 12:24:20.639540  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54623.5us
I20260709 12:24:20.633072 12389 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:20.641217 12457 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:24:20.641414 12463 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 12:24:20.642819 10098 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:24:20.647095 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.647956 10096 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:20.671801 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.672582 10023 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:20.674719 12431 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.677136 12431 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:20.678573 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:20.679369 10098 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:20.688725 10098 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
W20260709 12:24:20.693600 12425 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.002s
I20260709 12:24:20.696604 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2552) took 1158 ms. Trace:
I20260709 12:24:20.697031 10093 rpcz_store.cc:276] 0709 12:24:19.538311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.595456 (+ 57145us) service_pool.cc:224] Handling call
0709 12:24:20.696403 (+1100947us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b5400341554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:20.696534 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":462738816}
W20260709 12:24:20.699119 10640 leader_election.cc:336] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.782s (SENT)
W20260709 12:24:20.700037 10640 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.826s (SENT)
I20260709 12:24:20.700433 12474 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:20.701540 12474 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:20.702960 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.703603 10017 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
I20260709 12:24:20.704068 12423 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.704543 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:20.704844  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 69651.1us
I20260709 12:24:20.731810 10018 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:20.736630 10097 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:20.694160 12425 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.753957 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.754979 10020 raft_consensus.cc:2410] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:24:20.750440  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:20.752559 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44258.1us
W20260709 12:24:20.755352  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49823.1us
I20260709 12:24:20.756973  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:20.750173 12465 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 12:24:20.773298  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94003.6us
W20260709 12:24:20.774811  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 134524us
I20260709 12:24:20.736194 12450 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.756635 12432 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:20.736389 12427 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.799683  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:20.798424  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:20.796926  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.669478 12400 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.103s
I20260709 12:24:20.790194 12422 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.835893 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:20.774760 12338 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.795783 12456 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:20.839059  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105495us
I20260709 12:24:20.763247 12384 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.848258  9949 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:20.850134  9990 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.002s (SENT) [suppressed 61 similar messages]
I20260709 12:24:20.848707  9944 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:20.774500 12078 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:24:20.850286 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:20.854370 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:20.855782  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:20.856460  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:20.858927  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 102110us
I20260709 12:24:20.839639 12295 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.849294 12186 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.853591 12405 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:20.854260 12315 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.869270 12378 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:20.894667 12378 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:20.895601 12378 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.840914 12371 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:24:20.842185 12447 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:24:20.853453 12456 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.901477 12456 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:20.853713 12422 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:20.871294 12425 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:20.873338 12464 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:24:20.797442 12429 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.875257  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:20.838657 12480 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:20.841881 12473 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 12:24:20.876739 10094 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1527) took 1671 ms (client timeout 1619 ms). Trace:
I20260709 12:24:20.853929 12196 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:20.891139 12420 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:20.856892 12308 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:20.905879 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:20.878279 12035 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:20.914223  9949 raft_consensus.cc:2410] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:20.862375  9941 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:20.855309 12338 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.022549 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 343) took 1495 ms. Trace:
I20260709 12:24:20.918229 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:21.025003 10097 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.026268 10093 raft_consensus.cc:2410] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:21.026726 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.027407 10098 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
W20260709 12:24:21.027482 10094 rpcz_store.cc:269] 0709 12:24:19.205523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.341203 (+135680us) service_pool.cc:224] Handling call
0709 12:24:20.876474 (+1535271us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b5400341f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:20.876619 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051213184}
W20260709 12:24:21.028348  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 153196us
I20260709 12:24:21.024605 10099 rpcz_store.cc:276] 0709 12:24:19.527322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.973259 (+445937us) service_pool.cc:224] Handling call
0709 12:24:19.973584 (+   325us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:19.978909 (+  5325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:19.978938 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:19.979509 (+   571us) log.cc:844] Serialized 52 byte log entry
0709 12:24:20.072992 (+ 93483us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:20.970339 (+897347us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b1800370420. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf
0709 12:24:20.970537 (+   198us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.015751 (+ 45214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.015874 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:21.015934 (+    60us) raft_consensus.cc:1685] finished
0709 12:24:21.022233 (+  6299us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.022470 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49536,"prepare.queue_time_us":176041,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":2459,"raft.queue_time_us":50073,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":243,"spinlock_wait_cycles":34762624,"thread_start_us":212839,"threads_started":2,"wal-append.queue_time_us":223392}
I20260709 12:24:20.958453 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:21.029874 10096 raft_consensus.cc:2376] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 4. Re-sending same reply.
W20260709 12:24:21.031414  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:21.025800  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 166163us
W20260709 12:24:21.033777 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 179471us
I20260709 12:24:21.036695  9945 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:21.036602  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:21.037369  9944 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.043809 10097 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:21.045486  9944 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:21.045439 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:21.022980 12444 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.860s	user 0.005s	sys 0.001s
I20260709 12:24:20.879603 12405 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:21.021235 12439 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:24:20.913616 12423 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:20.956117 12427 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.073074  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:20.903672 12467 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.071919 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:21.073503 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.079555 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.025595 12196 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:20.956067 12422 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:21.088944  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.081701 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.093099 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:21.084524 10674 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:21.093734 10672 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.093654 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.094393 10024 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:21.095211 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.959385  9940 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:21.097646 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.024384 12425 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:20.891733 12186 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:21.076552 10640 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.971s (SENT)
I20260709 12:24:21.080157 10093 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:20.955587 12429 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.029793 12308 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.081439 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:20.953181 12460 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.081921 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:20.914170 12315 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.082265 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.095255 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.096194 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:21.080250 10025 raft_consensus.cc:2376] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 2. Re-sending same reply.
I20260709 12:24:21.106137 12308 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.100666 12473 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.140307 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59744 (request call id 15) took 1061 ms. Trace:
I20260709 12:24:21.140715 10019 rpcz_store.cc:276] 0709 12:24:20.078525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.078669 (+   144us) service_pool.cc:224] Handling call
0709 12:24:21.140069 (+1061400us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54001c5054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09550853 00007f4e0e51e60a 00007f4e07aa47d4 00007f4e07aa4762 00007f4e07aa4691 00007f4e07aa460c 00007f4e07aa38c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:21.140184 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390914816}
I20260709 12:24:21.141965 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.082876 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:20.915793 12419 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.679s	user 0.007s	sys 0.004s
I20260709 12:24:21.138722  9940 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:21.142900 10018 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:21.142995 12419 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.143798 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.144407 10019 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:21.144582 10018 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:21.144676 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.145040 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 573) took 1093 ms. Trace:
I20260709 12:24:21.145210 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.145884 10019 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.081664 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.145258 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.104147  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.147557 10673 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:21.105398 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.106753 10016 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:21.148250 10020 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:20.955895 12464 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:21.107554 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.116376 12338 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:21.138666 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1525) took 1982 ms. Trace:
I20260709 12:24:21.145820 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.099872 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:21.160657 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.160878 10097 raft_consensus.cc:2376] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4. Re-sending same reply.
I20260709 12:24:21.096462 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.161319 10020 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:21.124293 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.124807 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.162112 10093 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:21.162073 10672 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:21.146914 10676 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:21.103551 12425 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.163153 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.164049 10023 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:24:21.164601 12289 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:21.164781 12425 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.160351 10092 rpcz_store.cc:276] 0709 12:24:19.156330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.963226 (+806896us) service_pool.cc:224] Handling call
0709 12:24:19.963519 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.170233 (+206714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.138418 (+968185us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54002fd354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:21.138586 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":473368960}
I20260709 12:24:21.147605 10018 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:21.126856  9945 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:21.165288 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.166640 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.167105 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:21.123631 12498 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
I20260709 12:24:21.154237 10021 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:21.154472 10099 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:21.177605 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:21.157361  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53266.2us
I20260709 12:24:21.158439 10024 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:21.084113 12196 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.165706 12464 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.167649 10093 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
I20260709 12:24:21.167600 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.167887 10018 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:21.166756 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.182219 10016 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:21.182482 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.101568 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:21.166278 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.172976  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:21.173630  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.186503 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.173709 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:21.145455 10669 rpcz_store.cc:276] 0709 12:24:20.051426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.060617 (+  9191us) service_pool.cc:224] Handling call
0709 12:24:21.144877 (+1084260us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b54004d35d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:21.144982 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":697390464}
I20260709 12:24:21.176491 12035 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.179209 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:21.178504 10094 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:21.128003 12186 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:21.176613 12084 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.202100 12084 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.204378 12084 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.179050  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.205698  9941 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:21.206195  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.206998  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:21.207813  9941 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:21.208025  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.171936 12460 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.208776 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.184963  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 461) took 1170 ms. Trace:
I20260709 12:24:21.180645 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:21.183478 10097 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:21.209746 10099 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:21.187801 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.210287 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1520) took 2252 ms. Trace:
I20260709 12:24:21.189615 12450 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.209400  9946 rpcz_store.cc:276] 0709 12:24:20.014756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.036097 (+ 21341us) service_pool.cc:224] Handling call
0709 12:24:20.036431 (+   334us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:20.042090 (+  5659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:20.042133 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:20.042634 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:24:20.043866 (+  1232us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:20.043907 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:20.043972 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:20.919047 (+875075us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:21.178516 (+259469us) raft_consensus.cc:1685] finished
0709 12:24:21.178540 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:21.184902 (+  6362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6300,"raft.queue_time_us":14983,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":4939,"spinlock_wait_cycles":23569792,"thread_start_us":825,"threads_started":1,"wal-append.queue_time_us":30596}
I20260709 12:24:21.210685 10095 rpcz_store.cc:276] 0709 12:24:18.957481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:18.963202 (+  5721us) service_pool.cc:224] Handling call
0709 12:24:18.963478 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.226692 (+1263214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.210086 (+983394us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400162954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:21.210237 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2901829504}
I20260709 12:24:21.211251 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.211755 12338 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.191191 12420 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:21.191349 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 114810us
I20260709 12:24:21.205835 12394 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.210965 12450 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.190189 12470 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.211930 10020 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:21.190732  9940 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:21.209393 10016 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.187903 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.215849 10019 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:21.160034 12422 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:21.227718 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.228659 10094 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:21.232412 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.233068 10669 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:21.233320 10668 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:21.234262 10669 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:21.234808 10668 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:21.235086 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.235380 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.235860 10020 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:21.236552 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.236630 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.237478 12450 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.237696 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.238024 12470 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.238871 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.239981 10673 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:21.240229 12470 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.241191 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.242969 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.243734 10677 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:21.244585 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.244774 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.244946 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.245342 10019 raft_consensus.cc:2368] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:21.245954 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.246130 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.246907  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:21.247573  9946 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 2.
I20260709 12:24:21.248453  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:21.248662  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:21.249068  9946 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:21.249553  9941 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:21.250193 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.250983 10017 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:21.245962 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.251518 12419 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:21.251786 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.250860 10020 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:21.252233 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 574) took 1200 ms. Trace:
I20260709 12:24:21.252724 10675 rpcz_store.cc:276] 0709 12:24:20.051745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.054409 (+  2664us) service_pool.cc:224] Handling call
0709 12:24:20.075912 (+ 21503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.508334 (+432422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.251978 (+743644us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b54007f3ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:21.252177 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":817432832}
I20260709 12:24:21.252851 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.254616 10017 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:21.254195 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.252305 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.227506 12378 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:21.218892 12425 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:21.257184 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:21.257580 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:21.235463 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.258347 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:21.258751 10024 raft_consensus.cc:606] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:21.259205 10024 raft_consensus.cc:614] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:21.226660  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.263034 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.264036 10020 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:24:21.227766 12289 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:21.266052 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.258806 12420 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.267220 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.267858  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:21.275270 12468 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:21.268529 12422 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.269009 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:21.276580 10020 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:21.261548 12378 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.276986 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.279002 12378 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:21.279589 10018 pending_rounds.cc:85] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 2
I20260709 12:24:21.279966 10018 pending_rounds.cc:107] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:24:21.280267 10018 raft_consensus.cc:2889] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:21.269703 12315 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:21.262941 12494 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:21.205770 12308 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.274986 10024 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 2.
W20260709 12:24:21.285470 10090 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2555) took 1738 ms (client timeout 2236 ms). Trace:
I20260709 12:24:21.285621 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.285799 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:21.286072 10090 rpcz_store.cc:269] 0709 12:24:19.546974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.944483 (+397509us) service_pool.cc:224] Handling call
0709 12:24:21.285222 (+1340739us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b5400163854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:21.285401 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445299456}
W20260709 12:24:21.293795 10642 connection.cc:256] RPC call timeout handler was delayed by -1.1224754699996993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1637s for a response
I20260709 12:24:21.275903 12468 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:21.267395 12196 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.267225 10016 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:21.308279 12196 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.309325 10671 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:21.310583  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.311261 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.312484  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:21.312644 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:21.313282 12422 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.313346  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:21.313678 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.314280  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:21.315505 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:21.306098 10019 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:21.317023  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:21.299405 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:21.317821 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:21.319320 10676 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:21.319260 10671 raft_consensus.cc:2368] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
W20260709 12:24:21.299400  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72869.9us
I20260709 12:24:21.320580 10019 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.323589 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:21.323951 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.314442 12186 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.325947  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:21.329355 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:21.297421 12494 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:21.321393 12425 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.323385  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:21.331526  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:21.333861  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:21.334583 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:21.335630  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:21.338868 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:21.339650 10091 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:21.325119 12422 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.340348 12450 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.340586 12422 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.340885 12450 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.341053 12422 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.341343 12450 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.331874 12456 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
I20260709 12:24:21.343797 10016 pending_rounds.cc:85] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 2
I20260709 12:24:21.344203 10016 pending_rounds.cc:107] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 2.3
I20260709 12:24:21.344461 10016 raft_consensus.cc:2889] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:21.344673 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.345218  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:21.334749 12470 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.345234 10671 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:21.350572  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.351754 12186 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:21.352202 12308 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.352993  9940 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:21.333316 12425 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.354126 12425 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.356794 12460 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:21.357242 12460 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.357626 12460 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.358789 12035 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:21.359844 12035 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:21.360116 12305 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:21.360646  9948 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.360994 12478 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:21.361374 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.361388 12478 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:21.362239 12308 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.362514 12478 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.365213 10020 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:21.326910 12419 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:21.353091 12458 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.348793  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.369113 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:21.331243 12315 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:21.389549 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55035.3us
I20260709 12:24:21.392562 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.348205 12423 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.375243 10019 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
W20260709 12:24:21.399416  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50735.1us
W20260709 12:24:21.399647  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49164.8us
I20260709 12:24:21.349450 12447 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.432691  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.405086 12400 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:21.405313 12240 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:21.396946  9948 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:21.417871 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.401728 12164 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:21.420169  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.407017 10671 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:21.398841 12458 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.401363 12147 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.427748 10669 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:21.416185 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47182.4us
I20260709 12:24:21.465086 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.428687  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1540) took 1389 ms. Trace:
W20260709 12:24:21.429497  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 93929.5us
I20260709 12:24:21.465767  9943 rpcz_store.cc:276] 0709 12:24:20.039576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.077172 (+ 37596us) service_pool.cc:224] Handling call
0709 12:24:20.094511 (+ 17339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:20.595354 (+500843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.428442 (+833088us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:21.428624 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149956352}
I20260709 12:24:21.399731 12419 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:21.393447 12186 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:21.399492 12062 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:24:21.406921 12493 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:21.306794 12468 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.445798  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:21.409456 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.409173 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.449131 10673 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:21.418540 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:21.431579 12423 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.431342 12378 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:21.484205 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.485394 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:21.450735  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.459302  9912 leader_election.cc:400] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
W20260709 12:24:21.487213  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.487361  9912 leader_election.cc:403] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:21.431224 12447 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.398764 12196 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.049s
W20260709 12:24:21.461367  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58778.7us
W20260709 12:24:21.467485  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47398.5us
I20260709 12:24:21.469008 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.444922 12464 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:21.472862 10674 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:21.493438  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:21.489945  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:21.494973  9943 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:21.475158 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:21.477203 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.478029  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:21.446445 12400 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:21.480371  9949 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:21.499523 12186 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:21.481150 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72029.7us
I20260709 12:24:21.501706 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:21.503531 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:21.503757  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44624.5us
I20260709 12:24:21.482754  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:21.483623 12295 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.486646 10673 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:21.462792 12458 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.497749 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:21.489552 12394 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:24:21.489810 12485 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:24:21.499280 12437 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.473241 12419 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:21.469641 12240 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:21.472389 12371 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:21.468904 12289 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.511142 12233 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.506021 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:21.484113 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:21.517441  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:21.485512 12423 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.522544  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:21.486706 12477 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:24:21.470149 12429 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:24:21.487108 12062 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:21.487447 12415 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:21.514509  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:21.511294 12384 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:21.552397 10677 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:21.555855 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49902.1us
I20260709 12:24:21.511024 12473 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:21.556576  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:21.502898 12447 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.556686  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:21.464046 12494 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.505992 12470 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:24:21.592280  9945 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.548813 12420 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:21.570545 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.593448 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:21.568825  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:21.529628 12147 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:21.567090  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:21.619104  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62502.5us
I20260709 12:24:21.623553  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:21.592844  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78338.5us
I20260709 12:24:21.621271 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:21.548560 12233 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:21.560954 12464 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:21.685869 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 188183us
W20260709 12:24:21.682765 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61572.9us
I20260709 12:24:21.687985 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:21.635223 12305 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:21.568188 12449 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.289s	user 0.002s	sys 0.002s
I20260709 12:24:21.690352 12305 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.691339 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:21.670953  9945 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:21.626031 12295 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.697970 12305 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.582096 12240 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.710772  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:21.620456 12384 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:21.712874 12384 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:24:21.554097 12371 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:21.626614 12289 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.568087  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:21.720584  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 90980.7us
I20260709 12:24:21.568372 12315 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.693426 12295 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:21.547515 12485 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:24:21.623143 12164 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.729182  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:21.686726 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.730281  9943 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:21.624929 12429 consensus_queue.cc:1243] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 4.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:24:21.730751 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44130.3us
I20260709 12:24:21.624657 12420 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:21.622951 12147 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.732168  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.682328 12233 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.721704 12473 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:21.635808 12415 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:21.686498 12480 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:21.742133 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:21.563325 12342 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.720082 12289 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.771694  9943 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:24:21.773464  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46874us
I20260709 12:24:21.731828 12164 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.731472 12295 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:21.734010 12378 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:21.748080 12147 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:21.784992  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 280497us
W20260709 12:24:21.786931 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44906.9us
I20260709 12:24:21.787905 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:21.791126  9943 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: [4.4-5.5]
I20260709 12:24:21.770314 12456 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:24:21.719545 12371 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.795893  9943 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:21.769797 12473 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:21.735139 12460 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.773329 12480 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:21.786988  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:21.780872 12164 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:21.804811  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 181338us
I20260709 12:24:21.765262 12450 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.781698 12289 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:21.807974  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 325283us
I20260709 12:24:21.791666 12019 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:21.807238  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:21.797046 12421 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:21.800099 12473 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:21.815486 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 127569us
I20260709 12:24:21.796583 12378 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:21.818519 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:21.819909 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:21.797303 12308 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:24:21.853981 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:21.817101 12429 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:24:21.855293 12305 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:21.855468 12429 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:21.855810 12305 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:21.856065  9940 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:21.856274 12305 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.856824 12378 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:21.859112 12508 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:24:21.860955 12504 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:24:21.861296 12419 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:21.861826 12506 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:24:21.865989 12305 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.821707 12315 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:21.868538 12305 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:21.815475 12422 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:21.791412 12062 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.876730 12460 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:21.855935 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:21.879534  9946 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:21.855939 12429 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:21.908579 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52723.8us
I20260709 12:24:21.899809 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:21.913332 10095 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:21.876819 12422 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:21.928212 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:21.814919 12420 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:21.903216 12305 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:21.944367 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 126022us
I20260709 12:24:21.942307 12019 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:22.119767 12450 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:21.942524 12464 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:22.136337 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.135462 12518 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.136445 10024 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:22.142373 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 288459us
I20260709 12:24:21.951205 12423 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.039681 12520 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:21.817531 12415 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.128475 12474 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:21.880167 12425 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:22.039749 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 147510us
I20260709 12:24:22.143615 12233 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:21.911978 12421 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
W20260709 12:24:22.030429 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.235072 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:22.120270  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:22.235823 10098 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.139732 12315 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:22.046274 10091 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:22.046317 12477 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.181005 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:22.183087 10020 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:21.878222 12480 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.247532 10095 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:22.247468  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:22.247875 10096 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.248761 10095 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:22.248934 10024 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:24:22.190384  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.196465 12464 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:22.201115 12513 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:24:22.208482 12147 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.209614 12118 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:22.210549  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 462) took 2195 ms (client timeout 1879 ms). Trace:
I20260709 12:24:22.210723  9944 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.205890 12384 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:22.220836 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 336) took 1644 ms (client timeout 1971 ms). Trace:
W20260709 12:24:22.190946 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 253482us
W20260709 12:24:22.228780 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:22.243989  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 123811us
I20260709 12:24:22.217928 12295 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.250059 10093 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:22.240258 12240 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.003s	sys 0.012s
I20260709 12:24:22.252570 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:22.229267 12474 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:22.260711 12470 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:22.149034 12524 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:22.268321 12118 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:22.275887 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:22.255191 12433 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:22.277810 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:22.258522  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.266080 10090 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:22.282049 12240 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.003s	sys 0.012s
I20260709 12:24:22.265064 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:22.313714 10097 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.278606 10098 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:22.270550 10062 leader_election.cc:400] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
I20260709 12:24:22.314721 10062 leader_election.cc:403] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:22.314967 10020 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:24:22.315575 10020 pending_rounds.cc:85] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 3
W20260709 12:24:22.315383 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45045.8us
I20260709 12:24:22.268563 10092 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:22.273191  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:22.268486  9948 rpcz_store.cc:269] 0709 12:24:20.015080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.015209 (+   129us) service_pool.cc:224] Handling call
0709 12:24:22.210297 (+2195088us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54000b6654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:22.210464 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3090729728}
I20260709 12:24:22.269011 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.275127 10095 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:22.255868 12456 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:22.264891 12342 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:22.255556 12419 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.307106 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.281941 12186 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.311054 10099 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.312726 10024 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:22.315868 10020 pending_rounds.cc:107] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:24:22.269028 10021 rpcz_store.cc:269] 0709 12:24:20.576767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.576929 (+   162us) service_pool.cc:224] Handling call
0709 12:24:22.220609 (+1643680us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:22.220746 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325229440}
I20260709 12:24:22.318022 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:22.319861 10096 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:24:22.322407 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 87953.9us
I20260709 12:24:22.341166 10016 raft_consensus.cc:2368] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
W20260709 12:24:22.323881  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44093us
I20260709 12:24:22.328435 10670 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:22.266857 12315 consensus_queue.cc:1243] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:24:22.307329 12480 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.342276 10020 raft_consensus.cc:2889] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:22.303179 12524 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:22.347616  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:24:22.330013  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56919.2us
I20260709 12:24:22.268744 12450 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 12:24:22.351613 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:22.376012 10673 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.350368 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:22.222339 12518 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.312886 12433 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:22.359300 12240 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:24:22.374977 12513 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:24:22.365712 12423 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.328533 12019 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.345911 12384 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.340229 12430 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:22.435333 12430 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:22.436484 12430 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:24:22.437436 12522 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.338s	user 0.000s	sys 0.001s
I20260709 12:24:22.375066 12524 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.340351  9941 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:22.344067 12494 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:24:22.301694 12485 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 12:24:22.369490 12164 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.555s	user 0.007s	sys 0.000s
I20260709 12:24:22.369297 12429 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.370882 12474 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.372138 12295 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:22.373497 10675 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:22.374732 10674 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:24:22.344344 12289 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.528s	user 0.006s	sys 0.006s
I20260709 12:24:22.393405 10670 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:22.374439 12473 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.542s	user 0.002s	sys 0.005s
I20260709 12:24:22.364539 12478 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.416177 10097 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:22.374807 12315 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.264741 12477 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:22.427410  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 65269.1us
W20260709 12:24:22.428067  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.391716 12470 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:22.428361 10673 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:22.356197 12186 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:22.432257 12538 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:24:22.432509 12433 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.433526 12539 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:22.522922 12539 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:22.339486 12111 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:22.445324  9949 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.524341 12539 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.445324 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2513) took 1183 ms. Trace:
I20260709 12:24:22.525668 12539 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.525646 10019 rpcz_store.cc:276] 0709 12:24:21.261995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.419803 (+157808us) service_pool.cc:224] Handling call
0709 12:24:21.420109 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.420337 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.420363 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.445034 (+ 24671us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.810147 (+365113us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:21.938745 (+128598us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.941143 (+  2398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.941224 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.445111 (+503887us) raft_consensus.cc:1685] finished
0709 12:24:22.445127 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.445251 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17490,"prepare.queue_time_us":60933,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":89519,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":7605,"thread_start_us":452659,"threads_started":2,"wal-append.queue_time_us":577723}
I20260709 12:24:22.394915 12419 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:22.528049 12419 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.530119 12419 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.439568 12147 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:22.453732 12342 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:22.475701 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 159493us
I20260709 12:24:22.420377 12525 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:22.475329  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:22.484337 10671 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:22.451360 12240 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:22.489665 10674 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:22.464524 12423 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.500320 12541 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.508076 12464 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:22.508673 12477 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.508914 12429 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:22.513432 12458 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.985s	user 0.004s	sys 0.007s
I20260709 12:24:22.549361 12419 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.515425 12470 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:22.518229 12233 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:22.491359 12289 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.597541 12419 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.520423 12186 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:22.599119 12547 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 12:24:22.599345 12507 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.808s	user 0.000s	sys 0.003s
I20260709 12:24:22.524820 12111 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:22.600585 12111 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.601881 12111 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.423815 12520 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:22.592367 12458 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.603724 12520 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.604861 12458 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:22.493976 12473 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.605749 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1469) took 1504 ms. Trace:
I20260709 12:24:22.425223 12523 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.226s
I20260709 12:24:22.606946 12473 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.606133 10018 rpcz_store.cc:276] 0709 12:24:21.101591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.278501 (+176910us) service_pool.cc:224] Handling call
0709 12:24:21.279395 (+   894us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.281017 (+  1622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.281048 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.282550 (+  1502us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.486161 (+203611us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:21.486189 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:21.486233 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:22.598724 (+1112491us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:22.605456 (+  6732us) raft_consensus.cc:1685] finished
0709 12:24:22.605479 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.605692 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10763648,"thread_start_us":152427,"threads_started":1,"wal-append.queue_time_us":404490}
I20260709 12:24:22.607291 12523 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:22.538820 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.607885 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.609154 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.609726 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:22.610136 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.551837 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:22.610929 12295 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.611171 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:22.611598 12295 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.612586 10090 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:22.536051 10673 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Advancing to term 4
I20260709 12:24:22.536432 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:22.531805 12342 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:22.576611 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.549045 12525 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:22.638046  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 142564us
I20260709 12:24:22.638917 10018 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.588667 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:22.607825 12523 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:22.551797 12384 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:22.595297 10065 connection.cc:256] RPC call timeout handler was delayed by -1.17975076599987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1713s for a response
I20260709 12:24:22.596895  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 466) took 1231 ms. Trace:
I20260709 12:24:22.662189  9945 rpcz_store.cc:276] 0709 12:24:21.365346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.692379 (+327033us) service_pool.cc:224] Handling call
0709 12:24:21.692657 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:21.692883 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:21.746844 (+ 53961us) log.cc:844] Serialized 14 byte log entry
0709 12:24:22.479861 (+733017us) spinlock_profiling.cc:243] Waited 35.5 ms on lock 0x7b18003637c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:24:22.596557 (+116696us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:22.596608 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:22.596664 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.596728 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:22.596840 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4636,"prepare.queue_time_us":53665,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":5381,"raft.queue_time_us":3851,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":25072,"spinlock_wait_cycles":81571584,"thread_start_us":112635,"threads_started":1}
I20260709 12:24:22.599720 12419 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.610421 10097 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:22.615917 10670 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
I20260709 12:24:22.667469 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:22.668188 10670 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:22.669386 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:22.477775 12164 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.619243 10094 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:24:22.633467 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:22.670710 10094 pending_rounds.cc:85] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 5
I20260709 12:24:22.671025 10094 pending_rounds.cc:107] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:24:22.671080 10096 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:22.671329 10094 raft_consensus.cc:2889] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:22.672080 12164 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:22.672482 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:22.555164  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79929.4us
I20260709 12:24:22.652267 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.673075 10094 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.673271 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:22.673903 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.632953 12295 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:22.674456 10019 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:22.643730 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:22.582525 12535 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 12:24:22.636003 10668 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:22.604521 12524 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.599143 12289 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.583508 12423 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:22.645251 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54100.2us
W20260709 12:24:22.662099 10065 connection.cc:256] RPC call timeout handler was delayed by -1.1536507660002826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17420000000000002s for a response
W20260709 12:24:22.694271 10065 connection.cc:256] RPC call timeout handler was delayed by -1.0240507660000731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 12:24:22.648873 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:22.579445 12456 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:24:22.534922 12425 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:22.638267 12525 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.672010 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:22.679955 10017 raft_consensus.cc:2368] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:22.554155  9940 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:22.680395  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:22.748848  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68539.6us
I20260709 12:24:22.752107 10019 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:22.753142  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57075.1us
I20260709 12:24:22.750258  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:22.680948 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:22.674543 10670 raft_consensus.cc:2368] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:22.641649 12541 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.649159 12433 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.691255 10673 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:22.649238 12553 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.692966 10018 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:22.647056 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:22.647509 12518 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:22.725497  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.731324 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:22.731979 12523 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.733240 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:22.733479 12550 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:24:22.678809 12384 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.734637 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:22.735589 12054 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:22.737097 10093 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
W20260709 12:24:22.738037 12447 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.053s	user 0.011s	sys 0.002s
I20260709 12:24:22.738401 12561 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:22.738963 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 566) took 1504 ms. Trace:
I20260709 12:24:22.708181 12515 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:22.716243 12470 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.739491 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:22.715294 12483 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:22.642416 12240 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:22.747920 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:22.760149 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:22.777169 10094 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:22.778281 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:22.765791 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:22.793758 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 112890us
I20260709 12:24:23.059489 12456 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.114444 12456 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:23.115460 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:23.117144 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:22.804665  9945 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:22.777801 12477 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:23.119460  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1558) took 1915 ms. Trace:
I20260709 12:24:23.119921  9944 rpcz_store.cc:276] 0709 12:24:21.204543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.210378 (+  5835us) service_pool.cc:224] Handling call
0709 12:24:22.210476 (+1000098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.268662 (+ 58186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.119259 (+850597us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b54000b6650. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:23.119401 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299289984}
I20260709 12:24:22.804665  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:23.122494  9949 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:23.124439 10670 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.806944  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:23.057322  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:22.851320 10097 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:22.843840 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:23.128579 10018 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:22.832618  9948 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:23.129118 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:23.129125 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 389718us
W20260709 12:24:23.127938 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 380104us
I20260709 12:24:23.131501 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:23.127589 10019 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:23.129762 10098 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:23.126880  9947 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:23.132743 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:22.871116  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 194323us
I20260709 12:24:23.130496  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:23.136188 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:23.136461 10018 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:23.137480 10062 leader_election.cc:400] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
I20260709 12:24:23.137631 10094 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:23.139642 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:23.144289  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.101223 12552 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.266s	user 0.000s	sys 0.000s
I20260709 12:24:23.158212  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:23.158815  9945 raft_consensus.cc:2376] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6. Re-sending same reply.
W20260709 12:24:23.058107 12557 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:24:23.156040 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:23.177959 10094 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:22.755477 12535 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:22.970208 12424 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.150646 12477 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:23.160663  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:23.171459 10098 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:23.175271 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.180661 10670 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
W20260709 12:24:23.180997  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 382842us
I20260709 12:24:23.181280 10019 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:24:22.876101 12494 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.100209 12342 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.189873  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:23.144578 12419 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.150820 12518 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:23.125561 12297 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:22.874716 12553 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:23.135684  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:23.193616  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:23.134145 12062 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:22.919682 12435 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:23.071651 12559 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.229105  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:23.135196 12054 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:23.202064 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:23.238564 12515 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:23.239144 10098 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:23.239279 12054 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.240379 10098 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:23.240710 12054 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:23.241680 10098 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 12:24:23.204022 10670 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:23.245455 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 568) took 1433 ms. Trace:
I20260709 12:24:23.245863 10676 rpcz_store.cc:276] 0709 12:24:21.811887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.812036 (+   149us) service_pool.cc:224] Handling call
0709 12:24:22.526575 (+714539us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:22.526797 (+   222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:22.582121 (+ 55324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:22.582153 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:22.595882 (+ 13729us) log.cc:844] Serialized 26 byte log entry
0709 12:24:22.747085 (+151203us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:22.747123 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.747208 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:23.245148 (+497940us) raft_consensus.cc:1685] finished
0709 12:24:23.245167 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.245396 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71572,"raft.queue_time_us":58502,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":7294,"spinlock_wait_cycles":4720512,"thread_start_us":99286,"threads_started":1,"wal-append.queue_time_us":549178}
I20260709 12:24:23.150420 12539 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 12:24:23.151759 12480 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:24:22.918730 12554 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:23.155184 12563 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:23.151027 12295 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:22.778501 12464 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.412s
I20260709 12:24:23.085444 12516 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.354s
I20260709 12:24:23.180198 12548 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.142813 12425 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:24:23.143182 12289 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.066s
I20260709 12:24:23.143838 12534 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.066s
I20260709 12:24:23.135100 12447 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:22.892531 12118 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.231549 12477 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:23.208941 12494 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.210551 12518 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.228355 12419 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:23.234318 12559 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:22.784570  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1318) took 1409 ms. Trace:
I20260709 12:24:23.252581 10094 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:23.206848 12483 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:23.123713 10668 rpcz_store.cc:276] 0709 12:24:21.234558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.491352 (+1256794us) service_pool.cc:224] Handling call
0709 12:24:22.555524 (+ 64172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.682949 (+127425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:22.738871 (+ 55922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:23.220950 12523 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:23.226918 12111 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.625s	user 0.011s	sys 0.000s
I20260709 12:24:23.213146 12524 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:23.268916  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.271358 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 133056us
I20260709 12:24:23.136540 12561 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.222715 12535 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:23.500586 12535 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:23.502972 12384 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:23.503979 12525 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.741s	user 0.009s	sys 0.000s
W20260709 12:24:23.505088 12540 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.990s	user 0.000s	sys 0.002s
I20260709 12:24:23.504458 12525 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:23.506726 12315 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.006s	user 0.011s	sys 0.000s
I20260709 12:24:23.506980 12084 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.507203 12315 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
I20260709 12:24:23.507628 12084 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.505126 12553 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:23.511266 12568 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.214s	user 0.000s	sys 0.000s
I20260709 12:24:23.515192 12535 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:23.278582 10025 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 12:24:23.263145 12557 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:24:23.206720 12019 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
W20260709 12:24:23.290995 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 151403us
W20260709 12:24:23.519582 12433 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.744s	user 0.009s	sys 0.000s
I20260709 12:24:23.521364 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:23.294993 10020 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.001s	sys 0.009s
I20260709 12:24:23.289345  9912 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:23.296192  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:23.522579 10020 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.001s	sys 0.009s
I20260709 12:24:23.522914  9945 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
W20260709 12:24:23.523113  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 233873us
I20260709 12:24:23.524384  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:23.525833  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:23.209172 12342 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:23.347908  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 221657us
W20260709 12:24:23.390163 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.201478 12424 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.532522 12419 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:23.533593 12424 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:23.533785 12419 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:23.536015 12342 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:23.538269 12566 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:24:23.276696 12563 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:23.543481 12563 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:23.545406 12561 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:23.549839 10095 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:23.295511 12548 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.538951 12534 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:23.561033 12534 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:23.568658 12447 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:23.480679  9940 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:23.478389 10675 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:23.454705  9941 rpcz_store.cc:276] 0709 12:24:21.366279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.567795 (+201516us) service_pool.cc:224] Handling call
0709 12:24:22.709320 (+1141525us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540018a6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:22.775571 (+ 66251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1241494272}
I20260709 12:24:23.573355  9945 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:23.588727 12494 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:23.480484 12541 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:23.288880 12423 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:23.289268 12555 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:23.612524 12555 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.268848 12394 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:24:23.453831  9943 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:24:23.454648 10094 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 12:24:23.616535 10094 pending_rounds.cc:85] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 4
I20260709 12:24:23.616840 10094 pending_rounds.cc:107] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 12:24:23.617115 10094 raft_consensus.cc:2889] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:23.619123 10094 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:23.455222 12480 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:23.412444 12572 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.274065 12554 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.447299 12518 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.274314 12473 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:23.527231  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:23.674198  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 147072us
I20260709 12:24:23.528518  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:23.451138 12164 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:23.520114 12111 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.529477 12295 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:23.291980 12539 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:23.460610 12118 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.513628  9839 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:23.510280 12084 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:23.506465 10062 leader_election.cc:336] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.937s (SENT)
W20260709 12:24:23.444315  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 254471us
I20260709 12:24:23.697840 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:23.699957  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 506411us
I20260709 12:24:23.552613 12019 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:23.629916 10094 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:23.633641 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2588) took 1709 ms. Trace:
W20260709 12:24:23.637066 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 442751us
I20260709 12:24:23.592621 12342 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.551651 12561 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:23.592193 12553 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:23.676522  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 148106us
I20260709 12:24:23.715931  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:23.711337  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:23.721408 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 214952us
I20260709 12:24:23.722887 10062 leader_election.cc:400] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
I20260709 12:24:23.723289 10062 leader_election.cc:403] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:23.725190 10062 leader_election.cc:400] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:23.725559 10062 leader_election.cc:403] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:24:23.631420 12315 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.630714 12480 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:23.552815 12289 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:23.710606 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2595) took 1056 ms. Trace:
I20260709 12:24:23.710932 10099 rpcz_store.cc:276] 0709 12:24:21.924150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.924328 (+   178us) service_pool.cc:224] Handling call
0709 12:24:21.924699 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.341407 (+416708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.633394 (+1291987us) spinlock_profiling.cc:243] Waited 386 ms on lock 0x7b5400163d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:23.633579 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":888053632}
I20260709 12:24:23.741300 10094 rpcz_store.cc:276] 0709 12:24:22.654565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.619808 (+965243us) service_pool.cc:224] Handling call
0709 12:24:23.629606 (+  9798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.710293 (+ 80687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.710525 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:23.712015 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:23.688275 12471 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:23.741981 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:23.745760 10099 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:23.728006  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.732410 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.733506  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:23.735227 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:23.699359 12473 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:23.693403 12539 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:23.757527 10094 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:23.764402 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1561) took 1108 ms. Trace:
I20260709 12:24:23.764925 10094 rpcz_store.cc:276] 0709 12:24:22.655937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.750182 (+1094245us) service_pool.cc:224] Handling call
0709 12:24:23.757300 (+  7118us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.764046 (+  6746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.764332 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:23.767649 10099 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:24:23.769685 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1563) took 1021 ms. Trace:
I20260709 12:24:23.770346 10094 rpcz_store.cc:276] 0709 12:24:22.748057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.765969 (+1017912us) service_pool.cc:224] Handling call
0709 12:24:23.768988 (+  3019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.769260 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.769299 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:23.769357 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:23.769399 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.769447 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:23.769582 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:23.747002  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:23.774746 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:23.775864 10676 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:23.714939 12447 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:23.730245 12423 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.716723 12561 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:23.689802 12164 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:23.748880 12525 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:23.780699  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:23.744490 12289 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.695317 12535 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 12:24:23.739416 12541 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 3. Current term is 4.
I20260709 12:24:23.785070 12574 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.782261 12433 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:23.789459 12549 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:24:23.744127 12084 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:23.790529  9947 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:24:23.694813 12111 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:23.822448 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 301158us
I20260709 12:24:23.828146 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:23.801985 10094 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:23.803038 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:23.749038 12553 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.779385 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1562) took 1031 ms. Trace:
I20260709 12:24:23.754763 12378 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:23.730506 12575 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5.
I20260709 12:24:23.758474 12425 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:23.726122 12019 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.770821 12471 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:23.815011 10674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.006s	sys 0.015s
I20260709 12:24:23.743988 12559 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:24:23.778683 12555 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:23.840651 10676 raft_consensus.cc:2410] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:23.840451 10099 rpcz_store.cc:276] 0709 12:24:22.747675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.762630 (+1014955us) service_pool.cc:224] Handling call
0709 12:24:23.763053 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.778991 (+ 15938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.779304 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:23.861614 10674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.006s	sys 0.016s
I20260709 12:24:23.848790 12582 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:23.849123 12535 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:23.849480  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68893.1us
I20260709 12:24:23.852394 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:23.803025 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:23.807899 12548 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:23.863102  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:23.814500 12289 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.816651 12433 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.803242 12447 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.915814 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:23.868276 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:23.919728  9947 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:23.819664 12549 consensus_queue.cc:1243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:24:23.942938 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 74746.8us
W20260709 12:24:23.945595 10065 connection.cc:256] RPC call timeout handler was delayed by -1.1485606789997291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32530000000000003s for a response
I20260709 12:24:23.942233 10676 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:23.904042 10094 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:23.977036 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:23.978807  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 227485us
I20260709 12:24:23.858460 12425 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:23.907063 12423 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:23.998670  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:23.865262 12535 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:23.909901 12378 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:24.006946  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:23.915498 12555 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:24.027997 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 359) took 1389 ms. Trace:
I20260709 12:24:23.998484  9942 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:23.920277 12582 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:23.868842 12525 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:23.869469 12541 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:23.933903 12433 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:23.997377 12523 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:24:24.004141 12419 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:24.034937  9947 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:23.918889 10668 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:24.040660 10094 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:23.998947  9940 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:24.030339 12054 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.789s	user 0.004s	sys 0.008s
I20260709 12:24:23.980827 12473 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:23.998080 12425 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:23.858453 12084 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:24.076812  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 478) took 1007 ms. Trace:
I20260709 12:24:24.030875 12534 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:24.101454 12164 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:23.921787 12561 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:23.815102 12480 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:24.102737 12561 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.088728 12471 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:24.112624 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.114210 10675 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.539s	user 0.010s	sys 0.000s
I20260709 12:24:24.114876 12593 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.115211 12447 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:24.115427 12593 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:24.117888 12593 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:24.120052 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:24.120889 10099 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:24.076660 12588 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:24.102885 12556 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.595s	user 0.010s	sys 0.006s
I20260709 12:24:24.085202 12580 raft_consensus.cc:1064] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:24:24.022928 12240 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:23.759865 12405 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:24.092307  9947 rpcz_store.cc:276] 0709 12:24:23.069162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.999721 (+930559us) service_pool.cc:224] Handling call
0709 12:24:24.034636 (+ 34915us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.073636 (+ 39000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.076699 (+  3063us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:23.906375 12295 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.159s
I20260709 12:24:23.866887 12553 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:24.086529 12535 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.087631 12578 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:24.099855 12019 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:24.100047 12435 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:24.155134 12435 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.120110 10096 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:24.114611 10675 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:24.073971 12419 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:24.129552 12297 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.909s	user 0.006s	sys 0.006s
I20260709 12:24:24.118409 12593 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.173730 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 577) took 1603 ms. Trace:
I20260709 12:24:24.173918 12593 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.174157 10674 rpcz_store.cc:276] 0709 12:24:22.570371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.122387 (+552016us) service_pool.cc:224] Handling call
0709 12:24:23.122743 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.138419 (+ 15676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.138456 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.138966 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 12:24:23.862078 (+723112us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:23.862117 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.862189 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.173504 (+311315us) raft_consensus.cc:1685] finished
0709 12:24:24.173525 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.173669 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2047,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":17737,"thread_start_us":660330,"threads_started":1,"wal-append.queue_time_us":985288}
I20260709 12:24:24.175649 12084 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:24.175978 12593 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:23.857947 12539 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:24.176584 12111 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:24.075639 12541 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:24.177191 12539 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:24.179138 12541 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:24.181195 12553 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:24.156599 12578 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:24.141980  9946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1328) took 2723 ms (client timeout 3253 ms). Trace:
W20260709 12:24:24.189096  9946 rpcz_store.cc:269] 0709 12:24:21.418135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.568589 (+150454us) service_pool.cc:224] Handling call
0709 12:24:24.141714 (+2573125us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54000163d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:24.141898 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591507328}
I20260709 12:24:24.145524 12518 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.145591 12289 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.148063 12582 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:24.130604 12588 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:24.093127 12554 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:24.156261  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:24.129977 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:24.163013 12405 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.158876 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:24.134420 12240 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:24.170660 12378 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.171624 12435 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.080008 12477 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:24.204082 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:24.184815 12419 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:24.195501  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.241644  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 378710us
W20260709 12:24:24.246481  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.247077  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.226914 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 280660us
I20260709 12:24:24.226965 10094 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:24.248718 12541 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:24.248905 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:24.250083 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:24.227217  9944 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:24.250900  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1339) took 1438 ms. Trace:
I20260709 12:24:24.251241  9944 rpcz_store.cc:276] 0709 12:24:22.812086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.121150 (+309064us) service_pool.cc:224] Handling call
0709 12:24:24.156096 (+1034946us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:24.250839 (+ 94743us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6390144}
I20260709 12:24:24.183768 12420 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.193495 12297 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:24.230072 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71290us
W20260709 12:24:24.080006  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73153.8us
I20260709 12:24:24.255643 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2510) took 2994 ms. Trace:
I20260709 12:24:24.190733 12578 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.200500 12535 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.249126 12494 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:24:24.319451 12588 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.249257 12541 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:24.289232 12420 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.323535 10672 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:24.324327 12468 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:24.265588 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.214221 12338 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.185475 12425 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.257180 10094 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:24.329967  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:24.220463 12019 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:24:24.250162 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:24.227018 12240 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:24.325634 10672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 557) took 3101 ms (client timeout 2094 ms). Trace:
W20260709 12:24:24.332408 10672 rpcz_store.cc:269] 0709 12:24:21.223816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.238659 (+ 14843us) service_pool.cc:224] Handling call
0709 12:24:24.325415 (+3086756us) spinlock_profiling.cc:243] Waited 3.08 s on lock 0x7b5400501ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:24.325548 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7092739968}
I20260709 12:24:24.249992 12549 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:24.326418  9945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.027s	sys 0.035s
I20260709 12:24:24.250905 12554 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:24.340207  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:24.343205  9946 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 3: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:24.336346 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:24.334204 10023 rpcz_store.cc:276] 0709 12:24:21.261184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.367435 (+106251us) service_pool.cc:224] Handling call
0709 12:24:21.367723 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:21.446631 (+ 78908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:21.446672 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:21.447146 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:24:21.792283 (+345137us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:21.820468 (+ 28185us) log.cc:844] Serialized 14 byte log entry
0709 12:24:21.880248 (+ 59780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.255254 (+2375006us) spinlock_profiling.cc:243] Waited 34.2 ms on lock 0x7b54001f6154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:24.255303 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.255343 (+    40us) raft_consensus.cc:1685] finished
0709 12:24:24.255426 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.255584 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107567,"prepare.queue_time_us":78574,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":115399,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":36292,"spinlock_wait_cycles":78606720,"thread_start_us":308070,"threads_started":1,"wal-append.queue_time_us":430187,"wal-append.run_cpu_time_us":4565,"wal-append.run_wall_time_us":1593216}
I20260709 12:24:24.352491  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 471) took 1539 ms. Trace:
I20260709 12:24:24.353432 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:24.358021 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:24.326380 12470 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.015s	sys 0.008s
I20260709 12:24:24.326097 12478 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:24:24.333479 12477 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 12:24:24.334647 12389 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:24:24.330772 12553 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:24:24.319413 12297 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.334926 12196 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
W20260709 12:24:24.518827 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.604290  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.486984 12608 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:24:24.401885 12534 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.649611  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:24.588099 12539 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.361034 12601 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:24.534389 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 284366us
I20260709 12:24:24.425700  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:24.756578  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:24.449321  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.361219  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:24.335703 12338 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.335119 12447 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.349098 12554 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:24.755807 12470 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.510s	user 0.030s	sys 0.026s
W20260709 12:24:24.359787 12118 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.000s
I20260709 12:24:24.737643 12589 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.833246  9946 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:24.835672  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:24.427487 12600 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.756937 12435 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:24.802310  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 380004us
W20260709 12:24:24.486236 12603 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:24:24.389621 12515 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.822587 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 574) took 2433 ms. Trace:
I20260709 12:24:24.853706 10671 rpcz_store.cc:276] 0709 12:24:22.368137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.373828 (+  5691us) service_pool.cc:224] Handling call
0709 12:24:22.374106 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.579655 (+205549us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.801546 (+2221891us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b5400615b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:24.801768 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251218176}
W20260709 12:24:24.351764 10091 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.008s	sys 0.047s
W20260709 12:24:24.484368 12595 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
W20260709 12:24:24.832293  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.483469 12597 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:24:24.419072 12593 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.485103 12602 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:24:24.422241 12541 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:24.842029  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:24.813102  9944 raft_consensus.cc:2410] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:24.759727 12078 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.760149 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:24.873541 10094 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 7 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:24.484609 12289 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:24.849763  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.755273 12534 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.822566 12477 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:24:24.889969 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.788944 12494 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:24:24.824785 12425 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:24.834728 12554 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.796008 12539 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.834961 12424 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.872177  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.860808 10091 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.078s	user 0.023s	sys 0.070s
I20260709 12:24:24.871060 12541 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:24.877512  9914 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.023s	sys 0.046s
W20260709 12:24:24.920162  9914 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.024s	sys 0.046s
W20260709 12:24:24.920562  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 544401us
W20260709 12:24:24.922046  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:24.922605  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:24.854031 12470 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:24.880436 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:24.876724 12078 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.902904  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:24.923750  9944 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.783234 12580 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.823235 12447 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.822794 12601 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 12:24:24.896366 12429 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:24.932855 12289 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.936695 12078 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:24.936439 12429 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:24.937033 12289 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.909440 12400 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.862946 12515 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.938817 12429 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:24.939152 12400 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:24.890405 12196 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:24.939839 12289 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:24.940160 12515 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:24.940065 12196 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.942149 12196 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:24.354714  9945 rpcz_store.cc:276] 0709 12:24:22.812975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.574370 (+761395us) service_pool.cc:224] Handling call
0709 12:24:23.593186 (+ 18816us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.593463 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:23.816328 (+222865us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.352144 (+535816us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.352198 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:24.352245 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.352316 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.352437 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11565,"prepare.queue_time_us":495052,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":14578,"thread_start_us":706893,"threads_started":2}
I20260709 12:24:24.943313  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:24.944779 12584 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.154s	user 0.002s	sys 0.000s
I20260709 12:24:24.946115 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 579) took 2112 ms. Trace:
I20260709 12:24:24.946461 10670 rpcz_store.cc:276] 0709 12:24:22.834017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.243645 (+409628us) service_pool.cc:224] Handling call
0709 12:24:23.243930 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.244233 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:23.244257 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:23.244754 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:24:23.569656 (+324902us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:23.569695 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:23.569761 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:24.944193 (+1374432us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:24.945837 (+  1644us) raft_consensus.cc:1685] finished
0709 12:24:24.945856 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.946054 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47670,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":10613,"spinlock_wait_cycles":7241472,"thread_start_us":324534,"threads_started":1,"wal-append.queue_time_us":523602}
I20260709 12:24:24.948405 12474 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:24.953337 12456 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:24.956648 10675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 586) took 2378 ms (client timeout 1813 ms). Trace:
W20260709 12:24:24.957269 10675 rpcz_store.cc:269] 0709 12:24:22.578677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.592738 (+ 14061us) service_pool.cc:224] Handling call
0709 12:24:24.956367 (+2363629us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b5400544f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:24.956552 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":796471040}
I20260709 12:24:24.958508 12612 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:24.958963 12612 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:24.912606  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:24.960722  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60174.6us
I20260709 12:24:24.855580 12600 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:24.870878 12593 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.846990 12589 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:24.905544 12594 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.361s	user 0.001s	sys 0.001s
I20260709 12:24:24.925498 10094 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:24:24.928321 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 134461us
W20260709 12:24:24.912864 12371 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.990s	user 0.013s	sys 0.013s
W20260709 12:24:24.920322 12588 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.000s	sys 0.035s
I20260709 12:24:24.920055 12424 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:24.979245  9944 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:24.982390  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:25.018005 10094 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:25.017930 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:25.018952 10676 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:24.934746 12425 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:25.014508 12371 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.105s	user 0.014s	sys 0.013s
I20260709 12:24:25.012769 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.021337 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:25.033715 10676 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.033895 12464 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:25.034201 12419 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.747s	user 0.005s	sys 0.006s
I20260709 12:24:24.977522 12622 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:25.030308 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.014423 10671 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Advancing to term 5
W20260709 12:24:25.029788 12240 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.680s	user 0.008s	sys 0.000s
I20260709 12:24:24.926296 12541 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:24.984149 12477 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:25.063977  9944 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:24:25.068866  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.039810 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.045001 10676 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:24.926982 12470 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:25.042048 12468 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.717s	user 0.005s	sys 0.005s
I20260709 12:24:24.959793 12612 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:25.089171  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 106880us
I20260709 12:24:25.104367  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:25.041015 12405 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.810s	user 0.009s	sys 0.001s
I20260709 12:24:25.037428 12589 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.062783 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.087893 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 360) took 2449 ms. Trace:
I20260709 12:24:25.026221 12297 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.043408 12447 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.036150 12593 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.109885 10021 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:25.124655  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.124852 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2586) took 3225 ms. Trace:
I20260709 12:24:25.125531 10092 rpcz_store.cc:276] 0709 12:24:21.898962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.899083 (+   121us) service_pool.cc:224] Handling call
0709 12:24:21.900735 (+  1652us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:22.330639 (+429904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.124630 (+2793991us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b5400162454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:25.124798 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845488384}
I20260709 12:24:25.097575 10671 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:25.112172 10019 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:25.126648 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.126965 12394 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.083650 12477 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.131325 12477 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:25.133790 12560 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 2.616s	user 0.009s	sys 0.007s
I20260709 12:24:25.135129 12405 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:25.136039 12405 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:25.136371 12405 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:25.137171 12405 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.138316 12405 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.102984 12621 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:25.113142 12389 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:25.034479 12464 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.117239 12620 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.127381 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 587) took 2537 ms. Trace:
I20260709 12:24:25.114820 12485 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.130365 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:25.142504 12578 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.806s	user 0.010s	sys 0.000s
I20260709 12:24:25.171605 10092 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:25.143760 12338 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.088824 10669 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:25.145186 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2603) took 1030 ms. Trace:
I20260709 12:24:25.178200 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.177870 10099 rpcz_store.cc:276] 0709 12:24:24.122724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:24.124627 (+  1903us) service_pool.cc:224] Handling call
0709 12:24:24.124932 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:24.182153 (+ 57221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:24.182183 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:24.182815 (+   632us) log.cc:844] Serialized 52 byte log entry
0709 12:24:24.655169 (+472354us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:24.898948 (+243779us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.899146 (+   198us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.144821 (+245675us) spinlock_profiling.cc:243] Waited 56.8 ms on lock 0x7b5400304b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:25.144867 (+    46us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.144914 (+    47us) raft_consensus.cc:1685] finished
0709 12:24:25.144995 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.145125 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":253505,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":40497,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":44126,"spinlock_wait_cycles":130647424,"thread_start_us":613211,"threads_started":2,"wal-append.queue_time_us":718462}
I20260709 12:24:25.180017 12582 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.180600 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.180557 12582 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.181806 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:25.182175 12535 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.862s	user 0.010s	sys 0.000s
W20260709 12:24:25.183601 10096 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1558) took 2537 ms (client timeout 1730 ms). Trace:
I20260709 12:24:25.145329 12541 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.146160 12539 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.146771 12630 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.127491 12483 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.147047 12534 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.086385 12240 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.148742 12470 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.117357 10091 rpcz_store.cc:276] 0709 12:24:22.638518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.242919 (+604401us) service_pool.cc:224] Handling call
0709 12:24:23.243213 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.243419 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:23.748285 (+504866us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.077421 (+1329136us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:25.087441 (+ 10020us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:25.087514 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.087624 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.087808 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":614627,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":106453,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":12442,"thread_start_us":1764847,"threads_started":3,"wal-append.queue_time_us":1126004}
I20260709 12:24:25.051208 12425 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.151710 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.157745 12533 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:24:25.120379 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.077092 12419 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.124693 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 360) took 2746 ms. Trace:
I20260709 12:24:25.164484 10673 rpcz_store.cc:276] 0709 12:24:22.589634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.128212 (+538578us) service_pool.cc:224] Handling call
0709 12:24:23.128504 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.690638 (+562134us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:23.690885 (+   247us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.817871 (+126986us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:23.817936 (+    65us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:23.817998 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.127169 (+1309171us) spinlock_profiling.cc:243] Waited 562 ms on lock 0x7b54004d49d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:25.127215 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.127331 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":203,"prepare.queue_time_us":170,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":2570,"raft.queue_time_us":1682,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":17668,"spinlock_wait_cycles":1292292480,"thread_start_us":126072,"threads_started":1,"wal-append.queue_time_us":296871,"wal-append.run_cpu_time_us":1469,"wal-append.run_wall_time_us":1099806}
W20260709 12:24:25.167701 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.151448 12612 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.178931 10092 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:25.185388 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:25.184020 10096 rpcz_store.cc:269] 0709 12:24:22.646399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.671758 (+ 25359us) service_pool.cc:224] Handling call
0709 12:24:25.183439 (+2511681us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400192dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:25.183539 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3157256064}
I20260709 12:24:25.105738 12622 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.182901 12582 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.172931 12592 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:25.224545 12592 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.225664 12622 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.127461 12394 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.179530 12389 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:25.228777 12389 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.203032 10671 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
W20260709 12:24:25.113953 12433 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.033s	user 0.013s	sys 0.000s
W20260709 12:24:25.208297 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.250559 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:25.215945  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:25.164075 12371 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
I20260709 12:24:25.164595 12297 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:24:25.209717  9945 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:25.223588 12425 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.220189 10099 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:25.172935 12378 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:24:25.210409 10020 rpcz_store.cc:276] 0709 12:24:22.378489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.379078 (+   589us) service_pool.cc:224] Handling call
0709 12:24:22.379488 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:22.521823 (+142335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:22.521852 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:22.522282 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 12:24:22.750878 (+228596us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:23.523092 (+772214us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.547430 (+ 24338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.121921 (+1574491us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54002d5354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:25.121970 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.122021 (+    51us) raft_consensus.cc:1685] finished
0709 12:24:25.124486 (+  2465us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.124629 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24013,"prepare.queue_time_us":736362,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":24595,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":212,"spinlock_wait_cycles":326504832,"thread_start_us":943489,"threads_started":2,"wal-append.queue_time_us":602921,"wal-append.run_cpu_time_us":4051,"wal-append.run_wall_time_us":1389966}
I20260709 12:24:25.254103 12433 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.254869 12582 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.194242 12630 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.198258 12483 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.173683 12620 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.179788 12578 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.185488 12485 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:25.218636 12554 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.541489 12458 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.323437 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.541827 12582 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.542212 12458 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:25.316193 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 349) took 2707 ms (client timeout 1946 ms). Trace:
I20260709 12:24:25.543758 12620 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.544063 12582 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.397673  9947 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:25.545365 12604 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.341478 10016 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:25.355437 10020 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:24:25.549149 12604 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.320411 10090 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:25.549894 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.314690 12315 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.550752 12315 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.385638 12592 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:25.483021  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.307068 12626 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:25.552309  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.472678 12295 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:25.339809 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1478) took 2800 ms (client timeout 1886 ms). Trace:
I20260709 12:24:25.553094 12295 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:25.553656 10021 rpcz_store.cc:269] 0709 12:24:22.539303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.539441 (+   138us) service_pool.cc:224] Handling call
0709 12:24:25.339556 (+2800115us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b54001f7554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:25.339723 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5747134208}
W20260709 12:24:25.335443  9942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1335) took 2529 ms (client timeout 1787 ms). Trace:
I20260709 12:24:25.323835 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:25.555290  9942 rpcz_store.cc:269] 0709 12:24:22.806424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.806626 (+   202us) service_pool.cc:224] Handling call
0709 12:24:25.335208 (+2528582us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b5400206f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:25.335357 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833460480}
I20260709 12:24:25.476930 12524 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.556295 10018 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:25.556547 12524 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.556931  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.326290 12518 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.545133 12342 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:25.356261  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1323) took 3957 ms. Trace:
I20260709 12:24:25.558423 12342 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.558693  9943 rpcz_store.cc:276] 0709 12:24:21.398480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.800506 (+402026us) service_pool.cc:224] Handling call
0709 12:24:21.800809 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:23.616533 (+1815724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.355883 (+1739350us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54001a40d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:25.356178 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3801168640}
I20260709 12:24:25.559336 12630 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:25.363037  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.313740 12433 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.560155  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:25.560672  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.328141  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.561496  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:25.542415 10017 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:25.551510 12604 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.454356 12111 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.370941  9945 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:24:25.543808 10093 rpcz_store.cc:269] 0709 12:24:22.609028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.610903 (+  1875us) service_pool.cc:224] Handling call
0709 12:24:25.316015 (+2705112us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b540022ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:25.316128 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93289216}
I20260709 12:24:25.563973  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.533995 12419 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.536285  9948 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [2.4-4.5]
I20260709 12:24:25.565320  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1575) took 1071 ms. Trace:
I20260709 12:24:25.565354  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.565858  9948 rpcz_store.cc:276] 0709 12:24:24.494158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.344868 (+850710us) service_pool.cc:224] Handling call
0709 12:24:25.345634 (+   766us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:25.564964 (+219330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.565261 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:25.566304  9945 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:25.376637 12062 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.186s
I20260709 12:24:25.563373  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:21.712059  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:25.636371  9947 raft_consensus.cc:2376] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5. Re-sending same reply.
I20260709 12:24:25.637461  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.638355  9947 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:25.639084 12062 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:24:25.563196 12111 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.639600 12062 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:25.641028 12062 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.560606 12342 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:24.032447 10098 rpcz_store.cc:276] 0709 12:24:22.638229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.242408 (+604179us) service_pool.cc:224] Handling call
0709 12:24:23.242686 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:23.247987 (+  5301us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:23.651715 (+403728us) log.cc:844] Serialized 14 byte log entry
0709 12:24:23.997245 (+345530us) log.cc:844] Serialized 14 byte log entry
0709 12:24:24.027668 (+ 30423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:24.027715 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:24.027759 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:24.027823 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:24.027942 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91196,"prepare.queue_time_us":747878,"prepare.run_cpu_time_us":1342,"prepare.run_wall_time_us":53562,"raft.queue_time_us":48380,"raft.run_wall_time_us":19240,"spinlock_wait_cycles":10581248,"thread_start_us":656651,"threads_started":2}
I20260709 12:24:25.643069  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:25.643689  9947 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.644687 12240 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.567138  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.646515 12458 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.647006  9948 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:25.362298 12394 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.401456 12480 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:25.340658 10669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 588) took 2723 ms (client timeout 2077 ms). Trace:
I20260709 12:24:25.336634 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.535185 12420 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.682744 10025 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:25.305338 12631 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:25.357851  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.571149  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:25.562300  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.559180 12524 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.555766 12295 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.686829  9946 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:25.686833 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.577144 10022 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:25.561960 12578 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.548563  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 475) took 2558 ms. Trace:
I20260709 12:24:25.690146  9949 rpcz_store.cc:276] 0709 12:24:22.990124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.792188 (+802064us) service_pool.cc:224] Handling call
0709 12:24:23.809480 (+ 17292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.946374 (+1136894us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b180037fcc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e0954ebfb 00007f4e0955964b
0709 12:24:24.982980 (+ 36606us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:25.025647 (+ 42667us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.548232 (+522585us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:25.548283 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:25.548337 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.548401 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.548509 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1712,"prepare.queue_time_us":48941,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":165487,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":19085,"spinlock_wait_cycles":46858496,"thread_start_us":631946,"threads_started":2,"wal-append.queue_time_us":460546}
I20260709 12:24:25.692203  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.692766  9949 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:25.554749  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 338887us
I20260709 12:24:25.584271 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.693655  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.693985 10093 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:25.577168 12315 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:25.605618 12563 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.054s	user 0.001s	sys 0.020s
I20260709 12:24:25.694262  9949 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:25.325343 12477 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:25.395258 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.321960  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 472) took 2508 ms. Trace:
I20260709 12:24:25.557520  9942 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:25.611800 12612 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.615760 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.615885 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.705353  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.618175 12554 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.578315 10670 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:25.454169 12467 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:25.627358 12196 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:25.628795 12378 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:25.629359 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 365) took 2830 ms. Trace:
I20260709 12:24:25.630627 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2607) took 1425 ms. Trace:
I20260709 12:24:25.632613 12424 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.633159 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:25.635221  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.639509 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 354) took 3029 ms. Trace:
I20260709 12:24:25.641667 12111 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.388855 12528 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:25.728042 12528 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:25.728159 12315 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.715788 12295 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.728689  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:25.639559  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1571) took 1284 ms. Trace:
W20260709 12:24:25.328321 10097 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2594) took 2674 ms (client timeout 1708 ms). Trace:
I20260709 12:24:25.419713  9839 catalog_manager.cc:5697] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:25.649504  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.729619 10097 rpcz_store.cc:269] 0709 12:24:22.654316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.666420 (+ 12104us) service_pool.cc:224] Handling call
0709 12:24:25.328109 (+2661689us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400163354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:25.328230 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240370432}
W20260709 12:24:25.651968 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.732776 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:25.733572 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:25.652840  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.653541 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.642544 12062 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:24:25.655684 12415 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.146s	user 0.008s	sys 0.021s
I20260709 12:24:25.666939 12485 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.667790 12600 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.643069 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:25.668840 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.671213  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.741042  9948 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:25.678254 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2609) took 1465 ms. Trace:
I20260709 12:24:25.742198 10092 rpcz_store.cc:276] 0709 12:24:24.213347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.673289 (+1459942us) service_pool.cc:224] Handling call
0709 12:24:25.673714 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.677910 (+  4196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.677936 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:25.677976 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:25.678010 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.678070 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.678180 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:25.683429 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:25.683749 10095 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 355) took 3067 ms (client timeout 2136 ms). Trace:
W20260709 12:24:25.684626 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45207.7us
I20260709 12:24:25.745051  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:25.746273 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.749060 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.749239 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:25.685339  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.685509 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.749930 10092 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:25.694331  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:25.750814  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56555.2us
W20260709 12:24:25.744590 10095 rpcz_store.cc:269] 0709 12:24:22.616584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.116914 (+500330us) service_pool.cc:224] Handling call
0709 12:24:25.683434 (+2566520us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54002fdd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:25.683648 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897380096}
I20260709 12:24:25.752524 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:25.698215  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:25.699828 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.753688  9949 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:25.753808 10093 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:25.696333  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:25.681499  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 340423us
I20260709 12:24:25.702044  9947 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:25.755254  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.755414 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.756047  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.756315  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.701363 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.756994 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.706411 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:25.696475 12563 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.145s	user 0.001s	sys 0.020s
I20260709 12:24:25.719180 10019 rpcz_store.cc:276] 0709 12:24:22.799405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.191046 (+391641us) service_pool.cc:224] Handling call
0709 12:24:23.217186 (+ 26140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.126786 (+1909600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.629036 (+502250us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54001f5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:25.629286 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4357361920}
I20260709 12:24:25.719237 10094 rpcz_store.cc:276] 0709 12:24:24.205624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.020971 (+815347us) service_pool.cc:224] Handling call
0709 12:24:25.021261 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:25.150647 (+129386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.150680 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:25.158499 (+  7819us) log.cc:844] Serialized 26 byte log entry
0709 12:24:25.599123 (+440624us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:25.599186 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.599290 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.630373 (+ 31083us) raft_consensus.cc:1685] finished
0709 12:24:25.630396 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.630534 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":67661,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":6841,"thread_start_us":528135,"threads_started":2,"wal-append.queue_time_us":433773}
I20260709 12:24:25.722822 10090 rpcz_store.cc:276] 0709 12:24:22.610484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.613390 (+  2906us) service_pool.cc:224] Handling call
0709 12:24:25.319933 (+2706543us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:25.550020 (+230087us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.639291 (+ 89271us) spinlock_profiling.cc:243] Waited 2.71 s on lock 0x7b540022ae50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:25.639434 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6223485184}
W20260709 12:24:25.682456 10669 rpcz_store.cc:269] 0709 12:24:22.617722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.648609 (+ 30887us) service_pool.cc:224] Handling call
0709 12:24:25.340489 (+2691880us) spinlock_profiling.cc:243] Waited 2.33 s on lock 0x7b540058e654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:25.340592 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5366857472}
I20260709 12:24:25.710632 12464 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.717588 12467 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.769364 12467 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.759833 12458 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:25.770112 12622 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.770260 12458 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:25.770776 12458 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.770721 12622 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.771207 12394 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:24:25.706405  9940 rpcz_store.cc:276] 0709 12:24:22.813260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:23.787641 (+974381us) service_pool.cc:224] Handling call
0709 12:24:23.787977 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:24.088967 (+300990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.321678 (+1232711us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540018b0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:25.321886 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329331072}
I20260709 12:24:25.772996  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.773527 12622 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.739341 12420 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:25.774413 12420 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:25.774823 12420 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.729001 12295 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:25.777078 12295 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.762821 12572 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:25.762630 12515 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 12:24:25.778640 12572 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.780213  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.780803 12572 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.781607 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:25.782197 10098 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.785000 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:25.760993 12578 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:25.729502  9943 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:25.729326  9944 rpcz_store.cc:276] 0709 12:24:24.355604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.125839 (+770235us) service_pool.cc:224] Handling call
0709 12:24:25.126167 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:25.126508 (+   341us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.126534 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:25.127095 (+   561us) log.cc:844] Serialized 52 byte log entry
0709 12:24:25.593617 (+466522us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:25.593676 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.593774 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.639361 (+ 45587us) raft_consensus.cc:1685] finished
0709 12:24:25.639380 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.639504 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8816,"raft.queue_time_us":19759,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":130,"thread_start_us":466076,"threads_started":1,"wal-append.queue_time_us":501479}
I20260709 12:24:25.687182 12631 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.756237 10097 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:24:25.793951  9987 connection.cc:256] RPC call timeout handler was delayed by -2.7124832990002687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
I20260709 12:24:25.756652  9947 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:25.705850 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:25.794387  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46302.3us
I20260709 12:24:25.794905 10673 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:25.795820 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:25.796640 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:25.797034 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.797369 10025 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:25.712393 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:25.739679 12415 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.230s	user 0.009s	sys 0.021s
I20260709 12:24:25.729280 12196 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:25.809556 12196 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:25.811007 12631 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:25.802995 12450 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:25.811496 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:25.802827 12563 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.811774 12450 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:25.812227 10020 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:25.812342 12563 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.813797 12450 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:25.814307 12450 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.763778  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.763721 10676 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEARNER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [2.4-4.5]
I20260709 12:24:25.815660  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.815958 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.816452 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.816737  9948 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:25.817548 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.728976 12528 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.817914  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.818423 12528 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:25.820324 12477 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.773620  9940 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:25.820822 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.766054 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.822158 10019 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:25.767027 10674 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:24:25.766490 12524 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:25.756536 12315 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:25.787118  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:25.780845  9945 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:25.827745 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.828853 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.792104 10098 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:25.713393 12612 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.793251  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.706382  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:25.798889 10099 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:25.801126  9949 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:25.802975 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.795984 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:25.803649 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.807330 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.787060 12578 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:25.808562 10023 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:24:25.808965 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.809072 12420 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.760860 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:25.810563 12196 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:25.816565 10021 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:25.762717 12518 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:25.763764  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:25.822376 10090 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:25.808900 12464 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.125s
W20260709 12:24:26.303257  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 514055us
I20260709 12:24:26.304728  9944 raft_consensus.cc:2410] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:26.308000  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1573) took 1951 ms. Trace:
I20260709 12:24:26.308144  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:26.309067  9941 rpcz_store.cc:276] 0709 12:24:24.356597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:25.147967 (+791370us) service_pool.cc:224] Handling call
0709 12:24:25.148455 (+   488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:25.149016 (+   561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:25.149061 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:25.149937 (+   876us) log.cc:844] Serialized 52 byte log entry
0709 12:24:25.583030 (+433093us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:25.723500 (+140470us) log.cc:844] Serialized 14 byte log entry
0709 12:24:25.748841 (+ 25341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:25.762062 (+ 13221us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:25.765751 (+  3689us) raft_consensus.cc:1685] finished
0709 12:24:25.790208 (+ 24457us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:25.824942 (+ 34734us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":567,"prepare.queue_time_us":187736,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":16666,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":31317,"thread_start_us":536822,"threads_started":2,"wal-append.queue_time_us":394826}
I20260709 12:24:25.839390  9946 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:26.311622  9942 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:26.314334  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:26.314941  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:26.316311  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:26.317201  9946 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.321557  9839 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:24:26.324326  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:26.324645  9839 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:26.329018  9942 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:26.341240 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:26.340981 12523 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:26.343340 10025 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:26.342394 10021 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:26.158699 12458 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.369904 12458 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:26.252923 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:26.373579 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:26.343358 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:25.842096 10020 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:24:26.085040 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:25.954449  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 159392us
I20260709 12:24:25.994180 12430 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:26.379506 10098 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:26.249181 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:26.382076 10676 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:26.380774 12240 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:24:25.997128 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:26.039383 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1453) took 4965 ms (client timeout 1937 ms). Trace:
I20260709 12:24:26.392971 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:26.033742 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2612) took 1107 ms. Trace:
I20260709 12:24:26.390506 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.884158 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.976321 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:25.854148 10065 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.601s (SENT) [suppressed 24 similar messages]
I20260709 12:24:25.839867 12580 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:25.995507 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:25.994114 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:26.395018  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71182.1us
I20260709 12:24:26.024427 10673 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:26.064519 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:25.884320 12524 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.399639 12518 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.033777 12378 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.149827 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:26.127132 12554 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:26.405536 10097 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:25.942497 12394 consensus_queue.cc:1243] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:26.406852 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:26.407311 10672 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:26.407208 12524 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:25.885510 12315 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:26.407537 10022 rpcz_store.cc:269] 0709 12:24:21.074255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:21.096222 (+ 21967us) service_pool.cc:224] Handling call
0709 12:24:26.039124 (+4942902us) spinlock_profiling.cc:243] Waited 4.41 s on lock 0x7b54001c5f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:26.039288 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10138018688}
I20260709 12:24:26.409356 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:26.410585 10091 raft_consensus.cc:2376] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 5. Re-sending same reply.
I20260709 12:24:26.411085 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:25.836329 12450 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.412514 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:26.413729 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:26.413997 12450 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.414752 12523 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:26.415309 12523 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.415823 12523 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.417138 12420 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.418138 10669 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:26.419116 12580 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.421411 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:26.035800 12483 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.422555  9942 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:26.421446  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.426493 10018 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:26.401590 10675 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:26.423913 10020 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:24:26.436866  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:26.423911 12111 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.451133 12111 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.452152 12516 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.452263 10672 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:26.452615 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 369) took 1284 ms. Trace:
I20260709 12:24:26.453946 12111 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.453055 10092 rpcz_store.cc:276] 0709 12:24:25.167896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.039663 (+871767us) service_pool.cc:224] Handling call
0709 12:24:26.040126 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:26.040572 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.040615 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.049657 (+  9042us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.344057 (+294400us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:26.403648 (+ 59591us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.452051 (+ 48403us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.452157 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.452198 (+    41us) raft_consensus.cc:1685] finished
0709 12:24:26.452207 (+     9us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.452431 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1499,"raft.queue_time_us":8945,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":826,"thread_start_us":329377,"threads_started":2,"wal-append.queue_time_us":134710}
I20260709 12:24:26.410372 12315 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:26.436257 10020 pending_rounds.cc:85] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 3
I20260709 12:24:25.978902 12563 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:26.404497 12518 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.462222  9946 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:26.416803 10095 rpcz_store.cc:276] 0709 12:24:24.926053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.032363 (+1106310us) service_pool.cc:224] Handling call
0709 12:24:26.032798 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.033231 (+   433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.033273 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.033363 (+    90us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:26.033425 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.033501 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.033663 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":114}
I20260709 12:24:26.463936 10021 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 3: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:26.468930  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:26.466871  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 136256us
I20260709 12:24:25.994943 12471 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.468s
I20260709 12:24:26.456898 10020 pending_rounds.cc:107] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 2.4
I20260709 12:24:26.473929 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:26.478624 10019 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:26.479101 10018 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:26.479683  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 715823us
I20260709 12:24:26.480110 10090 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:26.481088  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:26.481462 12471 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:26.481984 12471 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:26.454584 12553 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.090s
I20260709 12:24:26.485417 12111 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:26.472512 10017 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:26.483880 12471 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:26.489584 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.479156 10098 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:24:26.478390  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 624560us
I20260709 12:24:26.493932 10094 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:26.494112 12342 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:26.494695 12342 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.495108 12342 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.495793 10094 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:26.475586 12578 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.477034 10020 raft_consensus.cc:2917] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted by new leader
I20260709 12:24:26.497768 10094 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 12:24:26.498828 10021 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:26.502912 10020 raft_consensus.cc:2889] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:26.510428 10020 pending_rounds.cc:107] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 4.5
I20260709 12:24:26.511962 10020 raft_consensus.cc:2889] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:26.518293 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 681003us
I20260709 12:24:26.519029 12600 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:26.519548 12600 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:26.519686 12084 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.520360 12084 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:26.520838 12600 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.523399 10024 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:26.518361 10093 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
W20260709 12:24:26.524940  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 75244.6us
I20260709 12:24:26.522570 12554 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.539887 12563 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.024s
I20260709 12:24:26.541204 12563 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:26.545745  9942 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:26.547991  9943 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:26.519565 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:26.543416  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:26.537485 12600 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.549352 12315 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.549016 12524 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:26.573129  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:26.552639  9839 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 3, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:26.541559 12554 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.573879  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:26.574326  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:26.525813 12062 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:26.545135  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:26.543949  9940 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: [4.6-5.7]
I20260709 12:24:26.541684 12563 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:26.522763 12378 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.586315 12062 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.574697 12483 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:24:26.601003  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:26.612013  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67071.7us
I20260709 12:24:26.597532 12111 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.616539  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 481) took 1062 ms. Trace:
I20260709 12:24:26.618359  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:26.619359 12471 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.622761 12394 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:26.623800 12430 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:26.624426 12430 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:26.624939 12430 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:26.627310  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53135.6us
I20260709 12:24:26.622784 12515 raft_consensus.cc:2764] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:24:26.603833 12062 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.634375 12612 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:26.634634 10099 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
W20260709 12:24:26.635461  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61731.7us
I20260709 12:24:26.636709  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:26.637727 12563 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.639384 12563 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.640072 12196 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.603627 12315 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.641335 12378 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.642143 10674 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.643069 12315 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:26.643679 12315 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:26.635490  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:26.644126 12315 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.644601 12315 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.645388  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:26.646528  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 103333us
I20260709 12:24:26.612257 12524 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.647069  9943 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:26.649024 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1477) took 4110 ms. Trace:
I20260709 12:24:26.649307 12524 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:26.649515 12599 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.649473 10024 rpcz_store.cc:276] 0709 12:24:22.538888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:22.539049 (+   161us) service_pool.cc:224] Handling call
0709 12:24:22.539320 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.533825 (+3994505us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.648745 (+114920us) spinlock_profiling.cc:243] Waited 3.98 s on lock 0x7b540038dbd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:26.648944 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9161272704}
I20260709 12:24:26.650310 12599 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.639900  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:26.654742 12554 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.653616 12599 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.655458 12554 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.656814 12580 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:26.638365 12516 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:26.658541 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 370) took 1490 ms. Trace:
I20260709 12:24:26.659021 10099 rpcz_store.cc:276] 0709 12:24:25.168145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.367504 (+1199359us) service_pool.cc:224] Handling call
0709 12:24:26.634223 (+266719us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:26.635111 (+   888us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.658278 (+ 23167us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54003ac0d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:26.658468 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611484928}
I20260709 12:24:26.662063 12084 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.656841 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:26.655079  9940 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:26.662961 12111 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.663936 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:26.664952 12612 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:26.665385 12612 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:26.665534 12111 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:26.665834 12612 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.618332  9944 rpcz_store.cc:276] 0709 12:24:25.554540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.399533 (+844993us) service_pool.cc:224] Handling call
0709 12:24:26.400090 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.405417 (+  5327us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:26.455742 (+ 50325us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.599006 (+143264us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.606979 (+  7973us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.607028 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:26.615062 (+  8034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.615139 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.616468 (+  1329us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42813,"prepare.queue_time_us":40286,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":79,"raft.queue_time_us":13028,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":5300,"spinlock_wait_cycles":26470528,"thread_start_us":103936,"threads_started":1}
W20260709 12:24:26.671327  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 69588.3us
I20260709 12:24:26.677594 10025 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:26.667929  9942 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.679742  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:26.676223 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 678394us
W20260709 12:24:26.682678  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53752.7us
I20260709 12:24:26.678916 12622 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:26.704645  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:26.707552 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81284.2us
I20260709 12:24:26.712798 10674 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
W20260709 12:24:26.707269  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50916.4us
W20260709 12:24:26.717130 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 197683us
I20260709 12:24:26.723291  9942 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:24:26.726819 12662 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:26.727447 12662 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:26.727990 12662 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.697256 12523 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:26.730744 12599 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:26.732476 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:26.733740 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:26.733976 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:26.735379 12420 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:26.735821 12420 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.736181 12420 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.738829 12378 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:26.739421 12523 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.692027 12554 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.727355  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:26.748929  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:26.750020  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 113420us
I20260709 12:24:26.696002 12622 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:26.765781 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.778278 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 377) took 1012 ms. Trace:
I20260709 12:24:26.779762 12680 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:26.779688 10090 rpcz_store.cc:276] 0709 12:24:25.765898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.481155 (+715257us) service_pool.cc:224] Handling call
0709 12:24:26.481651 (+   496us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:26.482276 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.482310 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:26.534296 (+ 51986us) log.cc:844] Serialized 52 byte log entry
0709 12:24:26.574991 (+ 40695us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:26.591641 (+ 16650us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.665003 (+ 73362us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b18001fd500. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:24:26.773210 (+108207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.773316 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.773357 (+    41us) raft_consensus.cc:1685] finished
0709 12:24:26.775833 (+  2476us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.778214 (+  2381us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16430,"prepare.run_cpu_time_us":182,"prepare.run_wall_time_us":5620,"raft.queue_time_us":58721,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":274,"spinlock_wait_cycles":5158144,"thread_start_us":135512,"threads_started":2,"wal-append.queue_time_us":66910}
I20260709 12:24:26.774400  9941 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:26.784736  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:26.752703 12315 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.786195  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:26.767587 12661 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:26.799213 12342 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.801534 12342 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.802774  9839 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:26.804839  9839 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 3 to 4, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:26.777217 12483 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:26.806811 12483 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.807237 12483 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.778467 12599 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:26.808481 12599 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:26.809710 12599 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.779954 12572 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:26.811187 12572 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.812048  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1347) took 1261 ms. Trace:
I20260709 12:24:26.812448 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 376) took 1046 ms. Trace:
I20260709 12:24:26.813405 12518 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.812417  9945 rpcz_store.cc:276] 0709 12:24:25.550264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.326666 (+776402us) service_pool.cc:224] Handling call
0709 12:24:26.327276 (+   610us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.462975 (+135699us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:26.615380 (+152405us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.774816 (+159436us) log.cc:844] Serialized 14 byte log entry
0709 12:24:26.775073 (+   257us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:26.775131 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:26.775195 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.811667 (+ 36472us) spinlock_profiling.cc:243] Waited 99.5 ms on lock 0x7b54000b7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:26.811708 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.811879 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13586,"prepare.queue_time_us":43627,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":100,"spinlock_wait_cycles":228772608,"thread_start_us":153160,"threads_started":2}
W20260709 12:24:26.813862  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 92098us
I20260709 12:24:26.815346  9843 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 211) took 9799 ms. Trace:
I20260709 12:24:26.768134 12524 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.051s
I20260709 12:24:26.818677 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2539) took 1019 ms. Trace:
I20260709 12:24:26.819396 12622 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:26.840502  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65999.8us
W20260709 12:24:26.839737  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 118742us
I20260709 12:24:26.837611 10025 rpcz_store.cc:276] 0709 12:24:25.799615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.344613 (+544998us) service_pool.cc:224] Handling call
0709 12:24:26.656612 (+311999us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:26.818424 (+161812us) spinlock_profiling.cc:243] Waited 9.52 ms on lock 0x7b540038ccd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:26.818576 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48227840}
I20260709 12:24:26.819741 12420 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.822714  9946 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:26.824550  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:26.813308 12572 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.825260 12516 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:26.822714  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1360) took 1269 ms. Trace:
W20260709 12:24:26.827003  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 86576.5us
I20260709 12:24:26.793057 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:26.828281 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:26.828313 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 94755.2us
I20260709 12:24:26.829409 12315 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:26.830523 12661 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:26.830803 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 96914.3us
I20260709 12:24:26.812918 10095 rpcz_store.cc:276] 0709 12:24:25.765614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.475820 (+710206us) service_pool.cc:224] Handling call
0709 12:24:26.503135 (+ 27315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:26.503475 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.503530 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.504203 (+   673us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.612029 (+107826us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:26.612066 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.612134 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.812245 (+200111us) raft_consensus.cc:1685] finished
0709 12:24:26.812264 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.812392 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":103718,"threads_started":1,"wal-append.queue_time_us":243455}
W20260709 12:24:26.870416 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.841576 12578 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:26.873230 10022 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:26.853853  9839 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 3, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:26.875150 12661 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:26.882296  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57851us
I20260709 12:24:26.859706 12516 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.874311 12578 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.892793 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:26.855991  9947 rpcz_store.cc:276] 0709 12:24:25.553541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.327894 (+774353us) service_pool.cc:224] Handling call
0709 12:24:26.328455 (+   561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:26.434948 (+106493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:26.434977 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:26.492072 (+ 57095us) log.cc:844] Serialized 26 byte log entry
0709 12:24:26.697374 (+205302us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:26.697418 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:26.697513 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:26.822315 (+124802us) raft_consensus.cc:1685] finished
0709 12:24:26.822338 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:26.822513 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12537,"raft.queue_time_us":116569,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":17418,"thread_start_us":234409,"threads_started":2,"wal-append.queue_time_us":219454}
I20260709 12:24:26.842949 12430 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.888391  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:26.902521 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:26.904137 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:26.904672  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49082.4us
I20260709 12:24:26.895665  9839 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 3, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:26.935693  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.870815 12612 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.892656 12680 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:26.945688  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:26.948686 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:26.946729  9839 catalog_manager.cc:5697] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:26.891331 12516 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:26.907905 12572 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:26.974699 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 70731.8us
I20260709 12:24:26.979561 10672 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:26.835925  9843 rpcz_store.cc:276] 0709 12:24:17.016026 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:17.066293 (+ 50267us) service_pool.cc:224] Handling call
0709 12:24:26.815081 (+9748788us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:19.868946 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:19.869618 (+   672us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:19.869675 (+    57us) write_op.cc:435] Acquired schema lock
0709 12:24:19.869699 (+    24us) tablet.cc:662] Decoding operations
0709 12:24:19.870701 (+  1002us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:19.870792 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:19.870824 (+    32us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:24:19.871590 (+   766us) tablet.cc:701] Row locks acquired
0709 12:24:19.871640 (+    50us) write_op.cc:260] PREPARE: finished
0709 12:24:19.871818 (+   178us) write_op.cc:270] Start()
0709 12:24:19.871959 (+   141us) write_op.cc:276] Timestamp: P: 1783599859871745 usec, L: 0
0709 12:24:19.871981 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:24:19.889584 (+ 17603us) log.cc:844] Serialized 3218 byte log entry
0709 12:24:20.493096 (+603512us) op_driver.cc:464] REPLICATION: finished
0709 12:24:20.857406 (+364310us) write_op.cc:301] APPLY: starting
0709 12:24:20.862148 (+  4742us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:20.868761 (+  6613us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:20.868925 (+   164us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:20.870150 (+  1225us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:20.870497 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 12:24:20.870542 (+    45us) write_op.cc:312] APPLY: finished
0709 12:24:20.870885 (+   343us) log.cc:844] Serialized 65 byte log entry
0709 12:24:20.871066 (+   181us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:20.871192 (+   126us) write_op.cc:454] Released schema lock
0709 12:24:20.871259 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:20.871714 (+   455us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:20.871740 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":212233,"reported_tablets":19,"child_traces":[["op",{"apply.queue_time_us":344294,"apply.run_cpu_time_us":3497,"apply.run_wall_time_us":28970,"num_ops":8,"prepare.queue_time_us":541070,"prepare.run_cpu_time_us":59915,"prepare.run_wall_time_us":189108,"raft.queue_time_us":213261,"raft.run_cpu_time_us":23589,"raft.run_wall_time_us":105719,"replication_time_us":620955,"thread_start_us":657610,"threads_started":4,"wal-append.queue_time_us":336201,"wal-append.run_cpu_time_us":37204,"wal-append.run_wall_time_us":1657912}]]}
I20260709 12:24:26.982458 12464 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:26.982957 12464 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:26.981630 12062 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:26.983338 12464 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:26.987821 12062 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:26.979300 12572 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:26.988852 12572 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:26.990196  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.898317 12342 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:24:26.984583 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:26.995198 12518 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:26.995524  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:26.980413 12394 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 12:24:26.997643  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 86717.3us
I20260709 12:24:27.001225 10676 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:27.000454  9839 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 4 to 5, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 5 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.002894 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:27.008615 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:27.003091  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.007624 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2543) took 1206 ms. Trace:
I20260709 12:24:27.009289 10022 rpcz_store.cc:276] 0709 12:24:25.800849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.408909 (+608060us) service_pool.cc:224] Handling call
0709 12:24:27.007406 (+598497us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b540038d6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:27.007560 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064995072}
W20260709 12:24:27.018515 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 225587us
I20260709 12:24:26.947614 12563 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
W20260709 12:24:27.022429 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:26.991976 12450 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:27.024585  9839 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:27.018457  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.028115 12690 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.028407 12680 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.029189 12690 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:27.029275 12687 raft_consensus.cc:1064] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:24:27.029604 12690 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.030052 12690 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.030738 12683 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.030830 12196 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:27.031445 12683 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.033654 12683 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:27.034358 12415 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:27.000279 12524 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:27.037729  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:27.038350  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 159496us
I20260709 12:24:27.039558  9942 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:26.979300 12662 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.043816  9947 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:27.023134 12660 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:27.035377 12684 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 12:24:27.034905 12679 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:24:27.065161 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:27.058041 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.050361 12342 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:27.059371 10096 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:27.059669 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:27.060026 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.009778 12578 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:24:27.060798 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:27.064569 12660 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:27.080235 12660 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.081106 12695 raft_consensus.cc:1064] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
W20260709 12:24:27.081632 12695 raft_consensus.cc:1068] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Unable to promote non-voter 6c7bd3fa1bfa431c89fc9c9096452a76: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:27.060218 12662 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.082739 12694 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:27.071722 12062 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:27.083266 12694 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:27.083468 12062 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.083771 12694 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.084107 12062 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.086730 12599 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:27.087321 12599 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.089655 12599 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.090150 12342 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:27.091172 12342 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.092175  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:27.092967 12062 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.093592  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:27.094316 10099 raft_consensus.cc:2368] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:24:27.094907 12084 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.094779 12518 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:27.095578 12062 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:27.095727 12084 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:27.096259 12518 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:27.097069 12661 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.099015 12661 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.099598 10676 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:27.101094 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:27.065758 12196 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:24:27.073112 12612 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:27.076365 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:27.051455  9839 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.038198 12524 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:27.072103  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50421.5us
I20260709 12:24:27.105053  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:27.048089 12420 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:27.047700 12450 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:27.108107 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61754.1us
I20260709 12:24:27.034574 12483 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.102170 12660 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.041821 12378 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
W20260709 12:24:27.119055 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 84770.4us
I20260709 12:24:27.121307  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1589) took 1401 ms. Trace:
I20260709 12:24:27.108585 12524 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.121706  9946 rpcz_store.cc:276] 0709 12:24:25.716472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:26.470185 (+753713us) service_pool.cc:224] Handling call
0709 12:24:26.822486 (+352301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:26.854004 (+ 31518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:27.117569 (+263565us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54000b6b50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:27.117718 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":768939264}
I20260709 12:24:27.107357 10669 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:27.130158  9839 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 3 to 4, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.125190 12430 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.119597 12196 consensus_queue.cc:1243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:27.102473 12415 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:27.138653 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.138967 10096 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:24:27.136719  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68432.1us
I20260709 12:24:27.128801 12394 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:24:27.143181 10095 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:27.145426 12467 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:27.145862 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.147485 12450 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.152025 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.152972 12660 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:27.153368 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:27.153954 10092 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.159272 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:27.160238 10091 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:27.161229 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:27.162730 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 103181us
I20260709 12:24:27.162541 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:27.170943 10090 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:24:27.182158  9839 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.136509 12062 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:27.195575 12062 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.177357 10672 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
W20260709 12:24:27.195003 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71171us
I20260709 12:24:27.138651 12612 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:27.195669 10097 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:27.165177 12315 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:27.202147  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:27.164475 12528 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:27.203876 12572 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.205019 12415 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.143316 12429 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:24:27.206317 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.206357 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:27.215862  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.162495 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:27.223112 12342 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:27.223421  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 85782us
I20260709 12:24:27.225163  9839 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 4, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:27.231789 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.233955 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.203254 12062 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:27.218708 12315 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:27.247077 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.226688 12528 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:27.244992 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:27.248163 10097 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.250288  9991 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:27.257114 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 75255.8us
I20260709 12:24:27.258867  9839 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.239235 12553 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:24:27.270627 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:27.271641 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:27.274266 12528 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:27.278678 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64256.9us
I20260709 12:24:27.280629 12683 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:27.281237 12683 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:27.282208 12683 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.282932 12698 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.283452 12698 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.285571 12698 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:27.285807 12696 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:27.286715 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.287400 12687 raft_consensus.cc:1068] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Unable to promote non-voter 6c7bd3fa1bfa431c89fc9c9096452a76: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:27.288722 12700 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:24:27.292353 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:27.272931 10065 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:27.297664 10097 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
W20260709 12:24:27.309585 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44292.5us
W20260709 12:24:27.319741  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51988.8us
I20260709 12:24:27.311048  9839 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.331264  9839 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 4, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.365464  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:27.366959 12315 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.367538 12464 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.369671 12464 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:27.370292 12464 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.373247 12516 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.366204 12694 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.374609 10022 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:27.383951 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:27.376767  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.268684 12450 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:27.387120  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.388657  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:27.387806  9912 leader_election.cc:336] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.094s (SENT)
I20260709 12:24:27.392140  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:27.394089  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.397089  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.396735 10024 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:27.403460 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.383555  9839 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 3 to 4, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:27.409751 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:27.411612  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46281.1us
I20260709 12:24:27.412525 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.413635  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:27.414379  9942 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.407891 10025 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
W20260709 12:24:27.417403 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 159337us
I20260709 12:24:27.419925  9946 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
W20260709 12:24:27.422830 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 149970us
I20260709 12:24:27.379092 12315 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:27.345211 12700 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:24:27.388827 12450 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:27.427528 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.332264 12458 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:24:27.391611 12483 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.405191 12687 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:24:27.444188 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.469368 10022 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:27.438838 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:27.471346 10090 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:27.476182 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:27.477530 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.433516 12450 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.474107 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:27.491403 10024 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:27.419754 12554 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.470453 12572 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.493465 10677 raft_consensus.cc:2376] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6. Re-sending same reply.
W20260709 12:24:27.428931 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0744205339997279s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2513s for a response
I20260709 12:24:27.494894 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.495836 10092 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:27.497368 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:27.540256 10022 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:27.507476 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:27.541534 10674 raft_consensus.cc:2368] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6.
I20260709 12:24:27.517529 12516 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.522907 12563 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:27.542975 12563 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.545697 12563 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.523444 12600 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.483637 12415 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.547079 12415 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.549952 12415 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:27.498190 12663 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:27.551330 12663 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:27.552517 12663 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.553851 12663 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.546474 12600 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:27.525759  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 137978us
I20260709 12:24:27.465665 12240 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:24:27.559762 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0568910000001779s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2795s for a response
W20260709 12:24:27.536499 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 67421.9us
I20260709 12:24:27.560153  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.557277 12600 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:27.483829 12694 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:27.566466 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.592233 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:27.591709  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:27.593158 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67567.2us
I20260709 12:24:27.594048  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.598805 10024 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:27.597874  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:27.592590 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:27.608664 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:27.608517 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50654.6us
I20260709 12:24:27.610396  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.610281  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.610175 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:27.612686 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.616297  9940 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:27.618100  9947 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:27.619431  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.621644 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.623565 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:27.623733  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.632552 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:27.632643 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:27.634624  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.568143 12683 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:27.636672 10096 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:27.637072  9945 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:27.597092 12687 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:27.611680 12464 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.615832 12528 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:27.559477 12554 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.637125  9942 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:27.671134 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:27.673859 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.674712 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:27.562031 12062 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.643324 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:27.618968 12680 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:27.710148 10098 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:27.564684 12690 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.000s
I20260709 12:24:27.709506 10677 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:27.727922 10024 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 2. Current term is 3. Ops: []
I20260709 12:24:27.677240 12572 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.734334 10674 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:24:27.675015 12622 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:27.674221 12694 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.676884 12578 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.743436  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.635624 12698 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 12:24:27.752648 10098 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
W20260709 12:24:27.754177 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 129376us
I20260709 12:24:27.764756  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.767517 10025 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:27.768235 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:27.727779 12394 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:27.779372  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:27.794509  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:27.807143 12196 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.711390 12683 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:27.792167 10677 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:27.871037  9943 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 3.
I20260709 12:24:27.726725 12690 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.767299 12578 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:27.910632 12709 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
W20260709 12:24:27.964293 12712 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 12:24:27.964529 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:28.006465 10673 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:27.966346 12721 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
W20260709 12:24:27.973322 12714 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
I20260709 12:24:27.911891  9945 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:28.001999 10091 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:27.801373 12694 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.914013 12458 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:27.933272 12600 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:27.819566 12572 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:24:28.126005 12450 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.002s
W20260709 12:24:28.087515 12524 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.955s	user 0.007s	sys 0.007s
I20260709 12:24:27.978822 12696 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:28.122531 12554 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:28.123328 12458 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.124538  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:28.124785 12315 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.694s	user 0.003s	sys 0.007s
W20260709 12:24:28.128829 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 518672us
I20260709 12:24:28.131162 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:27.819921 12687 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:27.884387 12196 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.175669  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:28.180457 10018 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:28.148234 12600 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.180794 12523 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:28.181023 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:27.983011 12394 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:28.183727 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:28.183916 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.127177 12572 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:28.142149 12524 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.205295 12458 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:28.206959 12578 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:28.189471 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.183995 10099 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:28.195987 12694 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:28.197178 12690 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:28.203326 12683 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:28.203776  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:28.208202 12524 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:28.208961 10018 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:28.209385 12196 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:28.209949 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.204732 10022 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:28.210454 12523 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:28.211016 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79956.4us
I20260709 12:24:28.211354 12699 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.211483 12523 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:28.211956 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:28.212510 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.213110 12554 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:28.213340 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.212019 12699 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.213646 12554 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:28.213956 12699 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:28.152463 12719 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 12:24:28.207033 10097 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:24:28.188244 12600 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:28.207692 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 368823us
I20260709 12:24:28.214699 12554 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.219053 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.248073 10018 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:28.249531 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.250612 10018 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:28.219717 12394 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:28.220994 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:28.254244 12732 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:28.126564 12450 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:28.226641 12518 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:28.228931 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:28.236168 12663 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.682s	user 0.005s	sys 0.009s
I20260709 12:24:28.236828 12696 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:28.237974 12727 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:28.238809 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.240654 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:28.241902 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.251685 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:28.260039 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.257277 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:28.265623 12572 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:28.265929  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:28.254716 12732 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.254805 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:28.261021  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:28.215783  9941 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.299633 10674 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:28.300999 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.300408 10677 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 3.
I20260709 12:24:28.301653 10016 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:28.306344 10092 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:28.306931 10092 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:28.308090 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:28.309121 10095 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:28.273366 12696 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:28.311992  9940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.006s	sys 0.026s
W20260709 12:24:28.337087  9940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.006s	sys 0.038s
I20260709 12:24:28.337649 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.339846 10018 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:28.313174 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:28.341393 10099 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:28.342027 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.021s	sys 0.056s
W20260709 12:24:28.342478 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.021s	sys 0.056s
I20260709 12:24:28.345069 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:28.345245 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.345952 10023 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.347038  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:28.347649  9944 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.349624 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:28.351490 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.354137 10022 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:28.364224  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:28.365087  9948 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.366290 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:28.368479  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 102677us
I20260709 12:24:28.369478 10018 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:28.298972 12732 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:28.371389 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.338629 12600 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:28.379313  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:28.379814 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.313935 12062 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:28.350085 12696 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:28.368417 12686 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.006s	sys 0.028s
I20260709 12:24:28.375854  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:28.383186  9942 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:28.378180 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.384779 10095 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:28.373457 10016 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:28.303818 12695 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.363196 12663 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.374202 10092 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:28.298338 12727 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.393355 12429 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:28.393571 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:28.394022 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.390913 12695 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:28.395216 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:28.381397  9947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.015s	sys 0.038s
W20260709 12:24:28.425927  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46923.7us
W20260709 12:24:28.426316  9947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.016s	sys 0.038s
W20260709 12:24:28.382745 12686 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.007s	sys 0.028s
I20260709 12:24:28.400442 10022 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:28.400570  9948 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:28.427027  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:28.370584 12394 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:28.413602 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.414958 12699 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.415773 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.437119 10025 raft_consensus.cc:2410] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:28.381438 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:28.428274 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:28.428408 10024 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:28.428802 10018 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:28.434605 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.379721 12600 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.441902 10092 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.313403 12572 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:28.413621 12450 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:28.493993 10094 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:28.495244 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55340.3us
I20260709 12:24:28.428344 12695 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.495546 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.500770 10095 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:28.500684 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.509917 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.515683 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.493079 12600 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:28.521643  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:28.510953 12724 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:24:28.526813 10016 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:28.530359 10092 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:24:28.530980 10092 pending_rounds.cc:85] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 3
I20260709 12:24:28.531318 10092 pending_rounds.cc:107] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:24:28.531666 10092 raft_consensus.cc:2889] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:28.533113 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.537593 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:28.539140 10020 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:28.525631 10091 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:28.488924 12663 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:28.555151  9948 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:28.555461 10092 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:28.556860  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:28.548631 12680 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
I20260709 12:24:28.589286 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:28.554772 12450 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:28.519961 12732 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:28.595255  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:28.596868 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.598033 10018 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:24:28.595259 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:28.599143 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:28.599390 10091 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.599716 10018 raft_consensus.cc:2376] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6. Re-sending same reply.
I20260709 12:24:28.600732 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.602437 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.603020 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.603394 10018 raft_consensus.cc:2410] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:28.603892 10092 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:28.603734 12572 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:28.604323 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.604869 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:28.605453 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.606114 10092 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:28.606631 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:28.592713  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:28.616104  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:28.616670 10022 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:28.617717 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.606077 10018 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:28.619402 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.624935 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:28.631162 10098 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:28.631984  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 205042us
I20260709 12:24:28.632429 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:28.633796 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.635859  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:28.634585 10025 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:28.641651 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.643463 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:28.651662  9943 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 5 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:28.626878 12572 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:28.630072 12450 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:28.654338 10018 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:28.655426 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:28.652949 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:28.668834  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.669425 10022 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:28.651032 12429 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:28.683753  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:28.685928  9945 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: [4.6-5.7]
W20260709 12:24:28.690284  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57609.6us
I20260709 12:24:28.688546 10092 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:28.691902 10023 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:28.689613 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:28.700417  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:28.700528 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:28.700826 10673 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:28.689975  9949 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:28.712436 12737 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:24:28.658378 12578 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.714356  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:28.725629  9945 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:28.658903 12450 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:28.735478 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.722836  9990 leader_election.cc:400] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:24:28.736107  9990 leader_election.cc:403] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:24:28.671594 12622 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.736732 12622 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.671612 12688 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:28.720112  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 464544us
I20260709 12:24:28.736363 10092 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:28.738130 12688 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:28.739382 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.740127 10092 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:28.740904 12688 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:28.734481  9946 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:28.760265 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:28.700266 12420 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:28.765677  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:28.775224  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.784696  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:28.785535  9945 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:28.734422 12429 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:28.762740 12450 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:28.796443  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 104755us
I20260709 12:24:28.768293 12726 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:28.798760  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:28.799928  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:28.765430 12420 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:28.805948  9949 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:28.786841 12727 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:28.787768 12516 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.735249 12578 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:28.777573 12518 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:28.826473  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1602) took 1226 ms. Trace:
I20260709 12:24:28.874148  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:28.877964  9949 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:28.829496 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 154291us
I20260709 12:24:28.811749 12726 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:28.881976  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44856.4us
I20260709 12:24:28.902357  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1599) took 1377 ms. Trace:
I20260709 12:24:28.811089 12394 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:28.914294  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:28.932215 10062 connection.cc:256] RPC call timeout handler was delayed by -1.677814070000295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 12:24:28.932891 10062 connection.cc:256] RPC call timeout handler was delayed by -1.6012647289999222s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
I20260709 12:24:28.933882 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:28.904464  9940 rpcz_store.cc:276] 0709 12:24:27.524491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.633494 (+109003us) service_pool.cc:224] Handling call
0709 12:24:27.674864 (+ 41370us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:27.732873 (+ 58009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.732911 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:27.774834 (+ 41923us) log.cc:844] Serialized 52 byte log entry
0709 12:24:28.338087 (+563253us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:28.338131 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.338202 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:28.902060 (+563858us) raft_consensus.cc:1685] finished
0709 12:24:28.902084 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.902262 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14480,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":21534,"thread_start_us":530135,"threads_started":1,"wal-append.queue_time_us":711581}
I20260709 12:24:28.935622  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:28.936321  9940 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:28.942440  9944 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.000s
I20260709 12:24:28.942965  9944 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:28.944064  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:28.946947 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:28.947129 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:28.947597 10672 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:28.947788 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:28.907509 12726 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:28.947539  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 110117us
I20260709 12:24:28.905634 12663 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:28.968147  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:28.968925 12747 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:28.976609 12747 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:28.977841 12744 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:24:28.982056  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:28.985754 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:28.948719 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:28.984426 12572 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:28.985160  9941 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.687s	user 0.009s	sys 0.000s
I20260709 12:24:28.970803 12732 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:28.969606 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:28.883991 12518 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.938838 12342 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:24:29.001600  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:28.993613 12663 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:29.006336 10640 leader_election.cc:336] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.581s (SENT)
I20260709 12:24:29.053025 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:29.043016 12772 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:29.045399 10093 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:29.012225 10672 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:29.010221  9948 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:29.059650  9945 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:29.051481  9944 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:29.014986 10094 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.515s	user 0.008s	sys 0.002s
I20260709 12:24:29.061255 10094 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:29.060720 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.034526 12572 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:29.003201  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:29.067065  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:29.067746  9945 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
W20260709 12:24:29.068773 10099 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.001s
I20260709 12:24:29.069442 10099 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:29.075055  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:29.076197  9944 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:29.014223 12748 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:29.082082 10093 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:29.035053 12686 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:29.038177 12747 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:29.088923 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 364850us
I20260709 12:24:29.015995 12622 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:28.965610 12429 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.080854  9944 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:29.036666  9941 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:29.093519  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:29.134403  9945 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:29.129168 10098 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:29.138005 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:29.140234  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1607) took 1267 ms. Trace:
W20260709 12:24:29.146344 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 140050us
I20260709 12:24:29.146366  9944 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:29.140816  9943 rpcz_store.cc:276] 0709 12:24:27.872471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.180147 (+307676us) service_pool.cc:224] Handling call
0709 12:24:28.180517 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.658129 (+477612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.139903 (+481774us) spinlock_profiling.cc:243] Waited 471 ms on lock 0x7b54001b9754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:29.140158 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1083112320}
I20260709 12:24:29.062009 12663 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:29.185595 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47650.7us
I20260709 12:24:29.190400 10677 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:24:29.070541 12689 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:24:29.021049 12420 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.193255 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:29.194308 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:29.195269 10640 leader_election.cc:336] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.967s (SENT)
I20260709 12:24:29.134495 12687 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:29.190829 10677 pending_rounds.cc:85] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 5
I20260709 12:24:29.202092 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:29.203254  9948 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:29.207235 10677 pending_rounds.cc:107] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:24:29.207563 10677 raft_consensus.cc:2889] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:29.147149 12528 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:24:29.157699 10091 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.558s	user 0.012s	sys 0.008s
I20260709 12:24:29.205294 10098 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:29.158629 12622 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:29.087471 12748 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.209899  9946 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:29.188019 12578 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:29.209611 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:29.227687 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:29.209549 12698 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:29.210091 12612 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:29.229472 12612 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:29.230100 12663 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:29.210351 12747 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:29.230904 12747 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.231383 12747 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.210763 12680 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:29.234876 12680 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.235385 12680 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.211380 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:29.211643 12783 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:29.213301 12755 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:29.209736 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:29.162478  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:29.200466 12759 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:29.240756  9944 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:29.224344  9943 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:29.241062 12554 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:29.242245 12554 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:29.243317 12554 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:29.244518 12554 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.246153 12554 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:29.246801 12700 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:29.252297 12700 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:29.252763 12700 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:29.230511 12612 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.253937 12700 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.263902 12700 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:29.191244  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 504) took 1764 ms (client timeout 2127 ms). Trace:
W20260709 12:24:29.279268  9941 rpcz_store.cc:269] 0709 12:24:27.387386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.408503 (+ 21117us) service_pool.cc:224] Handling call
0709 12:24:28.124397 (+715894us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:29.151941 (+1027544us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6070528}
I20260709 12:24:29.221648 10091 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
W20260709 12:24:29.281107 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 53485.4us
W20260709 12:24:29.284847 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 339735us
I20260709 12:24:29.285200  9940 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:24:29.225196 12600 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.690s	user 0.008s	sys 0.004s
I20260709 12:24:29.284157  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:29.296669  9948 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:29.299401 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 139762us
W20260709 12:24:29.300341 10640 leader_election.cc:336] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.991s (SENT)
I20260709 12:24:29.300827 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:29.280472 12751 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:24:29.130069 12686 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.297299 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:29.290020  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 375769us
I20260709 12:24:29.316736  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:29.319638  9940 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:29.286820 12698 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:29.326329  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 323187us
I20260709 12:24:29.330295  9990 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:29.273785 12747 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:28.866408  9947 rpcz_store.cc:276] 0709 12:24:27.599955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.636247 (+ 36292us) service_pool.cc:224] Handling call
0709 12:24:27.636538 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:27.677345 (+ 40807us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:28.426797 (+749452us) log.cc:844] Serialized 14 byte log entry
0709 12:24:28.826044 (+399247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:28.826114 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:28.826179 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.826255 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:28.826395 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47822,"prepare.queue_time_us":91173,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":29557,"raft.run_wall_time_us":28756,"thread_start_us":926325,"threads_started":2}
I20260709 12:24:29.355335 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 395) took 1135 ms. Trace:
I20260709 12:24:29.355854  9947 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:29.355779 10024 rpcz_store.cc:276] 0709 12:24:28.219658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.521902 (+302244us) service_pool.cc:224] Handling call
0709 12:24:28.596225 (+ 74323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:28.596601 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.596629 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:28.597207 (+   578us) log.cc:844] Serialized 52 byte log entry
0709 12:24:28.806524 (+209317us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:29.077979 (+271455us) log.cc:844] Serialized 14 byte log entry
0709 12:24:29.354968 (+276989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.355053 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.355102 (+    49us) raft_consensus.cc:1685] finished
0709 12:24:29.355113 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.355274 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":406213,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":5518,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":139,"thread_start_us":654622,"threads_started":3,"wal-append.queue_time_us":406533}
W20260709 12:24:29.341403 10062 connection.cc:256] RPC call timeout handler was delayed by -1.5036830150002061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24710000000000001s for a response
W20260709 12:24:29.359558 10062 connection.cc:256] RPC call timeout handler was delayed by -1.4447140699999181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23060000000000003s for a response
I20260709 12:24:29.362955  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:29.325098 12342 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.364677  9947 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:29.318915 12727 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:29.342455  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:29.330484 12196 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.312512 12600 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:29.373927 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.374919 10672 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:29.376381 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:29.380926  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:29.340396 12698 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:29.337714 12726 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:29.392866  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50474.9us
I20260709 12:24:29.394354 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2646) took 1830 ms. Trace:
I20260709 12:24:29.405716 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.423496  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:29.397688 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.398397  9942 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 8. Ops: []
W20260709 12:24:29.382524 10062 connection.cc:256] RPC call timeout handler was delayed by -1.4159140699998716s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2338s for a response
I20260709 12:24:29.425451 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.454173 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.467666 10018 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:29.424515 12754 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.336s	user 0.001s	sys 0.000s
I20260709 12:24:29.388897 12663 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:29.447625 12783 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.455610 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2651) took 1095 ms. Trace:
W20260709 12:24:29.456110 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.371930 12772 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:29.423365 10096 rpcz_store.cc:276] 0709 12:24:27.559144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.740411 (+181267us) service_pool.cc:224] Handling call
0709 12:24:27.758043 (+ 17632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:27.796206 (+ 38163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.797488 (+  1282us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:28.252390 (+454902us) log.cc:844] Serialized 564 byte log entry
0709 12:24:28.637424 (+385034us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:28.637460 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.637531 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.388854 (+751323us) raft_consensus.cc:1685] finished
0709 12:24:29.388878 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.389005 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":384701,"threads_started":1,"wal-append.queue_time_us":782446}
W20260709 12:24:29.429245 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.388334 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.467968 10019 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:29.388768 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.400903 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:29.383329 12727 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:29.393400 12774 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:29.492955 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.495468  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 70541.3us
I20260709 12:24:29.495502 10016 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:29.490142 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:29.492492 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.380977 12689 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260709 12:24:29.492695 10097 rpcz_store.cc:276] 0709 12:24:28.360502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.985412 (+624910us) service_pool.cc:224] Handling call
0709 12:24:28.985816 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:28.986058 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:29.325617 (+339559us) log.cc:844] Serialized 14 byte log entry
0709 12:24:29.455275 (+129658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:29.455326 (+    51us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:29.455375 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.455439 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.455547 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12205,"prepare.queue_time_us":58942,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":30428,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":14094,"thread_start_us":402071,"threads_started":2}
W20260709 12:24:29.493960  9949 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.596s	user 0.008s	sys 0.004s
I20260709 12:24:29.469725  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:29.470583 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:29.511348 10062 leader_election.cc:400] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6.
I20260709 12:24:29.512177 10062 leader_election.cc:403] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:29.506914  9949 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:29.514281  9944 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:29.490630 12734 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:29.492050 12429 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:29.500864 12769 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.353s	user 0.000s	sys 0.001s
I20260709 12:24:29.405404 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.510670 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.523486 10022 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:29.521070 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1601) took 1233 ms. Trace:
I20260709 12:24:29.525012 10094 rpcz_store.cc:276] 0709 12:24:28.273804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.301960 (+ 28156us) service_pool.cc:224] Handling call
0709 12:24:29.507192 (+1205232us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b540015bdd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:29.507349 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321960192}
I20260709 12:24:29.521852 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.518435  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:29.526861 10025 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:29.493476 12772 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:29.474745 12787 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.491014 12747 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:29.493571 12782 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:29.539242 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:29.537289 12748 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:29.547756 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:29.519246 12734 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:29.563031 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.568499 10671 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:29.565232  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55384.9us
I20260709 12:24:29.533645 12062 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:29.573905 10020 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:29.576308  9990 leader_election.cc:400] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:24:29.576917  9990 leader_election.cc:403] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:29.576674 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.514257 12783 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 12:24:29.592592 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 378) took 1835 ms. Trace:
I20260709 12:24:29.519024 10669 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:29.615281 10674 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:29.595118 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:29.595180 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.643565 10018 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:29.633023  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 60562.9us
I20260709 12:24:29.641273 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.567996 12782 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:29.567608 12429 raft_consensus.cc:2764] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:24:29.589169 12773 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
I20260709 12:24:29.575078 12734 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.655673 10671 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:29.662070 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.640156 12787 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.657091  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:29.656821  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:29.669979  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:29.642117 12686 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:29.671661 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.649186 12663 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:29.567744 12578 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.082s
W20260709 12:24:29.678740 12784 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:24:29.673900  9948 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:29.647704 10021 rpcz_store.cc:276] 0709 12:24:27.756876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.763308 (+  6432us) service_pool.cc:224] Handling call
0709 12:24:28.411381 (+648073us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:28.411582 (+   201us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:28.412012 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:28.412041 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:28.412653 (+   612us) log.cc:844] Serialized 52 byte log entry
0709 12:24:28.753740 (+341087us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:29.060699 (+306959us) log.cc:844] Serialized 14 byte log entry
0709 12:24:29.222615 (+161916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.222722 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.576153 (+353431us) raft_consensus.cc:1685] finished
0709 12:24:29.592271 (+ 16118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.592479 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":451195,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":276896,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":8481,"spinlock_wait_cycles":9879808,"thread_start_us":808752,"threads_started":3,"wal-append.queue_time_us":637735}
I20260709 12:24:29.548667 12698 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:29.684470  9943 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:29.741984 12524 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:29.739928  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55798.3us
I20260709 12:24:29.649348 12782 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:29.703881 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.738839 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:29.746428 10640 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.560s (SENT)
W20260709 12:24:29.677196 12572 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.000s
W20260709 12:24:29.747246  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 90255.1us
W20260709 12:24:29.737396 12754 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:24:29.742190 12785 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:29.739766 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.800101 10018 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:29.800349  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1382) took 1125 ms. Trace:
W20260709 12:24:29.744354 12792 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:24:29.677325 12600 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:29.777760  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:29.779573 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:29.786576 10025 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:29.799113 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.800678  9945 rpcz_store.cc:276] 0709 12:24:28.674467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.151194 (+476727us) service_pool.cc:224] Handling call
0709 12:24:29.186036 (+ 34842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.186264 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:29.360148 (+173884us) log.cc:844] Serialized 14 byte log entry
0709 12:24:29.799990 (+439842us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:29.800047 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:29.800105 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.800165 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.800298 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":207934,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":117137,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":8221,"thread_start_us":592519,"threads_started":3,"wal-append.queue_time_us":382973}
I20260709 12:24:29.809265  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:29.738916 12429 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:29.588896 12727 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:29.815871  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 315732us
W20260709 12:24:29.816897 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 70499.8us
W20260709 12:24:29.820468 10640 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.020s (SENT)
I20260709 12:24:29.820996 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:29.774449 12524 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:29.832118 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:29.824892 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 285706us
I20260709 12:24:29.836073 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:29.816659 12747 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.842975 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:29.843387 12695 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.325s	user 0.009s	sys 0.004s
I20260709 12:24:29.843829 12695 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:29.845947 12732 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:29.847291 12788 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:29.848503 12797 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.191s	user 0.000s	sys 0.000s
I20260709 12:24:29.846583 12732 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.849855 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 392) took 1212 ms. Trace:
W20260709 12:24:29.849978 12622 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.571s	user 0.009s	sys 0.000s
I20260709 12:24:29.850219 10093 rpcz_store.cc:276] 0709 12:24:28.637411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.134564 (+497153us) service_pool.cc:224] Handling call
0709 12:24:29.198476 (+ 63912us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:29.286077 (+ 87601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:29.286120 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:29.315177 (+ 29057us) log.cc:844] Serialized 26 byte log entry
0709 12:24:29.454582 (+139405us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:29.454642 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.454718 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:29.849639 (+394921us) raft_consensus.cc:1685] finished
0709 12:24:29.849664 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:29.849799 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4417,"raft.queue_time_us":4172,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":53223,"thread_start_us":92599,"threads_started":1,"wal-append.queue_time_us":188954}
I20260709 12:24:29.831954 12528 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.853572 12732 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:29.854051 12795 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:29.854235 12732 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:29.855008 12732 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.856580  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 512) took 1179 ms. Trace:
I20260709 12:24:29.857112  9944 rpcz_store.cc:276] 0709 12:24:28.677038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.241440 (+564402us) service_pool.cc:224] Handling call
0709 12:24:29.514024 (+272584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.515046 (+  1022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.856314 (+341268us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540003e150. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:29.856498 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":626157312}
I20260709 12:24:29.859954  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1611) took 1453 ms. Trace:
I20260709 12:24:29.860494  9942 rpcz_store.cc:276] 0709 12:24:28.406945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.407129 (+   184us) service_pool.cc:224] Handling call
0709 12:24:28.407503 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.426951 (+1019448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:29.859616 (+432665us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54003a0554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:29.859871 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2278393088}
I20260709 12:24:29.862835 12803 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6.
W20260709 12:24:29.864001 12779 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 12:24:29.815094 12726 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:29.661319 12772 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:29.798245 12572 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:29.882021  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 104350us
I20260709 12:24:29.832618 12524 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.878783  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:29.886494  9945 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.849977 12774 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:29.870607 12622 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:29.894156  9912 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.507s (SENT)
I20260709 12:24:29.846076 12695 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:29.901787  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:29.878031 12062 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:29.903836 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:29.888242 12516 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.986s	user 0.002s	sys 0.011s
I20260709 12:24:29.902921  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:29.905779 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.838894 12429 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:24:29.912562 10025 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:29.915577 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:29.916527 12774 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:29.917263 12774 consensus_queue.cc:1243] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:24:29.918349 12612 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:29.918992 12763 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:29.919723 12763 consensus_queue.cc:1243] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:29.920071 10025 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:29.920679 12774 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:29.878539 12663 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.921205 12700 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:29.921726 12700 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:29.922640 12700 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.923751 12700 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:29.924554 12812 raft_consensus.cc:1064] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:24:29.924814  9945 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:29.910135 12578 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:29.923624 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.889051 12811 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.921030 12774 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:29.919147 12612 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:29.941461 10023 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
W20260709 12:24:29.905887 12800 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.079s	user 0.003s	sys 0.000s
I20260709 12:24:29.910986 12516 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:24:29.902614 12772 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:29.945691 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 102796us
I20260709 12:24:29.953588 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:29.950381 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.918385 12528 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.921007 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:29.956532 10021 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:29.957816 12688 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:29.958441 12689 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:29.903357 12726 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:29.966255  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 157088us
W20260709 12:24:29.966763  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72662.4us
I20260709 12:24:29.967142 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:29.968312 10025 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 3.
W20260709 12:24:29.967358 12696 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.539s	user 0.015s	sys 0.000s
I20260709 12:24:29.984239 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:29.997768 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.001395 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.002792 10025 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:30.002424 10018 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:29.971947 12695 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.007009 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:30.014858 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 96792.1us
I20260709 12:24:29.954471 12062 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.947321 12811 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:29.978143 12689 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:30.089581 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.038904 12815 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:30.039481 12789 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:30.022480 12803 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:30.000063 12726 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:24:30.128548 10062 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.601s (SENT)
I20260709 12:24:29.999979 12696 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.164259 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:30.167006 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.165990 12572 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:30.181902  9943 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:30.185976 10018 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:30.193156 12572 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:30.196854 12793 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:24:30.196967 12572 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:29.990732 12812 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 3, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:30.201236 12787 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.000s
I20260709 12:24:30.201753 12787 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
W20260709 12:24:30.185537 12450 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.366s	user 0.018s	sys 0.016s
I20260709 12:24:30.202282 12572 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.202489 12450 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.203241 12699 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:30.203706 10016 raft_consensus.cc:2368] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:30.204500 12726 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:30.205461 12518 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:30.205904 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:30.209213 12734 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.531s	user 0.001s	sys 0.008s
I20260709 12:24:30.209870 12734 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.211166 10016 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:30.212456 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.213198 10016 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:30.214583 10022 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:30.217294 12450 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.214293 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.219054 10016 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:30.220100 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.221139 10016 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:30.221946 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.044368 12774 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.222976  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:30.223773 12774 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:30.223692 12524 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.224851  9942 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:30.225342 12819 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:30.226042 12788 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:30.226001 12819 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.226577 12788 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:30.227777 12788 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.228209 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.228751 12788 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.229283 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.230043 10020 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:30.226567 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2645) took 2707 ms. Trace:
I20260709 12:24:30.230870 12782 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.230759 10090 rpcz_store.cc:276] 0709 12:24:27.519459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.727956 (+208497us) service_pool.cc:224] Handling call
0709 12:24:27.728296 (+   340us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:27.751720 (+ 23424us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:27.751765 (+    45us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:27.758583 (+  6818us) log.cc:844] Serialized 78 byte log entry
0709 12:24:28.652921 (+894338us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:28.904362 (+251441us) log.cc:844] Serialized 14 byte log entry
0709 12:24:28.923017 (+ 18655us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:28.923377 (+   360us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:30.208391 (+1285014us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b1800068a40. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:30.226263 (+ 17872us) raft_consensus.cc:1685] finished
0709 12:24:30.226362 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.226497 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6311,"prepare.queue_time_us":345277,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":391274,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":25174,"spinlock_wait_cycles":15655680,"thread_start_us":680703,"threads_started":2,"wal-append.queue_time_us":1240765}
I20260709 12:24:30.233234 12782 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:30.234208  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.234658 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.228773 12819 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.240057 12696 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:30.230906  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:30.241338 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 708) took 1052 ms. Trace:
I20260709 12:24:30.241427 10676 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:30.241771 10672 rpcz_store.cc:276] 0709 12:24:29.189205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.189349 (+   144us) service_pool.cc:224] Handling call
0709 12:24:30.241102 (+1051753us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b54007f4754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:30.241256 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308316032}
W20260709 12:24:30.245427  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.247014  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:30.247646  9944 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:30.250998  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:30.251837  9948 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:30.251878 12554 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.993s	user 0.010s	sys 0.002s
I20260709 12:24:30.129019 12612 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:24:30.255244  9941 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:30.253224  9949 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:30.256778 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:30.257007 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:30.257350  9947 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:30.258934 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:30.259596 10673 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:30.260317 12516 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.260433 12819 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:30.261040 12819 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:30.262367 12775 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.924s	user 0.000s	sys 0.001s
I20260709 12:24:30.262888 10017 raft_consensus.cc:2368] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7.
I20260709 12:24:30.263515 12819 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.263738 12394 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:30.264292 12394 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.264847 12420 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.265612  9944 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:30.265857  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:30.267097 12805 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.202s	user 0.001s	sys 0.009s
I20260709 12:24:30.268887 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.269771 12450 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:30.269963 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.259017  9942 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:24:30.271133 12742 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.444s	user 0.004s	sys 0.003s
I20260709 12:24:30.271296 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.272010 10024 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:30.223958 12817 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:24:30.272764 12747 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:30.273175 12747 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:30.190182 12789 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.275470  9945 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:30.283077 10016 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:30.282303  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:30.272800 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.315019 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.315491 10024 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:30.317332 10016 raft_consensus.cc:2376] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6. Re-sending same reply.
I20260709 12:24:30.279183 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:30.217362 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.320145 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.321415  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:30.322399 10016 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:30.322396  9947 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
W20260709 12:24:30.321627 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.313477 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.324293  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:30.329954 12524 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.330462 12755 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.330965 12755 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:30.330906 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.331804 10020 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:30.336463 10673 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
W20260709 12:24:30.337469 12823 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 12:24:30.337872 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 396) took 1543 ms. Trace:
I20260709 12:24:30.241053 12689 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.349454 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:30.344856 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 216325us
I20260709 12:24:30.348518 10092 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:30.349946 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:30.340953  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:30.360119  9945 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:30.360970  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:30.342139 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.364437 10018 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:30.368456 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56313.1us
I20260709 12:24:30.374176  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:30.392995 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:30.393991  9942 raft_consensus.cc:2376] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5. Re-sending same reply.
I20260709 12:24:30.338227 10095 rpcz_store.cc:276] 0709 12:24:28.794168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.144934 (+350766us) service_pool.cc:224] Handling call
0709 12:24:29.186346 (+ 41412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.186533 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:29.206932 (+ 20399us) log.cc:844] Serialized 14 byte log entry
0709 12:24:29.943542 (+736610us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b18003c2260. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:24:30.029988 (+ 86446us) log.cc:844] Serialized 14 byte log entry
0709 12:24:30.337387 (+307399us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b18003c2260. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:24:30.337545 (+   158us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:30.337605 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:30.337655 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.337718 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:30.337820 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":305849,"prepare.queue_time_us":20145,"prepare.run_cpu_time_us":4252,"prepare.run_wall_time_us":8058,"raft.queue_time_us":311687,"raft.run_cpu_time_us":292,"raft.run_wall_time_us":3377,"spinlock_wait_cycles":120153728,"thread_start_us":85795,"threads_started":1,"wal-append.queue_time_us":984111}
I20260709 12:24:30.395153 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:30.403237 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:30.403657 10062 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.813s (SENT)
I20260709 12:24:30.403971 10099 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:30.403327 10671 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:30.351060 12680 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.115s	user 0.011s	sys 0.002s
I20260709 12:24:30.342530 12816 raft_consensus.cc:1064] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
W20260709 12:24:30.412637  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 435424us
I20260709 12:24:30.403277 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.407858 10023 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:30.416608 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:30.260885 12516 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.349876 12815 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:30.409688 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:30.410322 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:30.410668 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.274029 12747 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.410967  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:30.276582 12394 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.408598 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:30.344069 12803 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:30.260890 12811 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:30.413322  9912 leader_election.cc:336] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.592s (SENT)
I20260709 12:24:30.409529  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:30.377229  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:30.450043  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.443560 10097 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:30.433389 10062 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.886s (SENT)
I20260709 12:24:30.450172  9945 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:30.445992 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.438974  9940 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:30.451467 10024 raft_consensus.cc:2368] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:30.341361 12699 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:30.451503 10022 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:30.340673 12734 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:30.405524 12528 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.458232 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:30.403048 12450 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:30.460206  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83067.5us
I20260709 12:24:30.339480 12788 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.110s
I20260709 12:24:30.445297 12696 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:30.392252 12732 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:24:30.412083 12700 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:24:30.405375 12612 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
W20260709 12:24:30.480648 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 75717.1us
I20260709 12:24:30.457933 12774 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:30.443674 12680 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.463160 12450 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:30.440172 12518 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:24:30.441990 12698 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:24:30.477201 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.524163 10024 raft_consensus.cc:2410] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 12:24:30.468926 12811 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:30.474189 12726 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:30.478729 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:30.478786  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 65517.8us
I20260709 12:24:30.487668 10099 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:30.461198 12699 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.469347 12394 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:30.481904 12523 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:24:30.491721  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 368133us
W20260709 12:24:30.530822  9912 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.602s (SENT)
I20260709 12:24:30.544124  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.534094 10090 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:30.494982 12450 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:30.489252 10091 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:24:30.471593 12528 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.483551 12696 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:30.486809 12776 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:24:30.552749 10016 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:24:30.557291 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.546798  9945 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:30.567776 10022 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
W20260709 12:24:30.568504  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.489792 12787 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260709 12:24:30.573737 10024 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:30.579133 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.580282 10016 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:30.541290 12394 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.543749 12811 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:30.585536 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.585845 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:30.586392 10024 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:30.551657 12612 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:30.584530 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.586573 10016 raft_consensus.cc:2376] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5. Re-sending same reply.
I20260709 12:24:30.568724 12780 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:30.588387 10022 raft_consensus.cc:2376] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6. Re-sending same reply.
I20260709 12:24:30.590457 10099 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:30.541079 12572 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
W20260709 12:24:30.593111  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62326.2us
I20260709 12:24:30.584415 12394 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:30.581493 12745 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:30.549209 12827 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.154s
W20260709 12:24:30.598729  9912 leader_election.cc:336] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.750s (SENT)
I20260709 12:24:30.605134  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.608042 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:30.584566 12790 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:30.615962 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 233365us
I20260709 12:24:30.622530 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:30.591634 12612 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:30.625218 10024 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:30.593045 12734 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:30.597257 12732 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:30.639330  9912 leader_election.cc:336] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.793s (SENT)
I20260709 12:24:30.638576 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:30.639925  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.634661 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:30.600090 12524 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.587342 12739 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.210s
I20260709 12:24:30.621333 12727 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:24:30.604123 12698 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:30.643378 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:30.649967 10673 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:30.615690 12763 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.000s	sys 0.019s
I20260709 12:24:30.641222 10677 raft_consensus.cc:2376] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 5. Re-sending same reply.
I20260709 12:24:30.618855 12700 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:24:30.649884 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 243466us
I20260709 12:24:30.646991 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:30.652271 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 235767us
I20260709 12:24:30.654860 10672 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Advancing to term 6
I20260709 12:24:30.571448 12803 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.656315 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:30.657927  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.659102  9912 leader_election.cc:336] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.804s (SENT)
I20260709 12:24:30.659731  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.649232 12516 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.638868 12689 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:30.668218 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:30.665944 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.651882 12780 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:30.688898 10674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 600) took 1336 ms (client timeout 1629 ms). Trace:
I20260709 12:24:30.689057 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 382) took 2776 ms. Trace:
I20260709 12:24:30.690681 10019 rpcz_store.cc:276] 0709 12:24:27.913001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:27.913151 (+   150us) service_pool.cc:224] Handling call
0709 12:24:28.895907 (+982756us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:28.901345 (+  5438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:29.494730 (+593385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:30.688844 (+1194114us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b5400265cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:30.689005 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1307006080}
I20260709 12:24:30.693104 10670 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:30.690729 10674 rpcz_store.cc:269] 0709 12:24:29.352619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.353737 (+  1118us) service_pool.cc:224] Handling call
0709 12:24:30.688717 (+1334980us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:30.688842 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":261110016}
W20260709 12:24:30.695183  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:30.653302 12763 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.731s	user 0.000s	sys 0.020s
I20260709 12:24:30.662312 12739 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:30.702303 10021 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:30.652980 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:30.665664 12698 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.748423 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:30.748947 10091 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:30.749068 10023 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:30.750185 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:30.642638 12734 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.702661 12739 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:30.698385 12695 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.004s
I20260709 12:24:30.652989 12524 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.746670 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:30.525090 12810 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:30.646502 12732 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:30.686092  9912 leader_election.cc:336] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.843s (SENT)
I20260709 12:24:30.783072  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.666898 12812 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:30.758908 10095 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:30.750283 12698 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:30.706161 12680 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:30.645983 12696 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.655107 12700 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:30.660602 12811 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:30.739825 12699 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:30.869131 10642 leader_election.cc:336] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.636s (SENT)
I20260709 12:24:30.919257 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:30.781620 12516 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.765875 12062 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:30.927805 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 242015us
I20260709 12:24:30.916509 10099 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:30.751204 12727 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.160s
I20260709 12:24:30.911917 10672 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:30.778661 12695 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:30.926543  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 445479us
I20260709 12:24:30.867681 12774 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:30.912667 10091 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:30.927412 12420 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.662s	user 0.011s	sys 0.003s
I20260709 12:24:30.917820 12780 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:30.955839 10092 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:30.918892 12528 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:30.951061  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 265006us
W20260709 12:24:30.961596  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.883s (SENT) [suppressed 51 similar messages]
I20260709 12:24:30.964176 12524 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:30.966090 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:30.967163 12842 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 12:24:30.967679 12848 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:30.968744 12848 consensus_queue.cc:1243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:24:30.972352 12839 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:24:30.912863 12518 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:24:30.974902 10668 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:30.976523 12518 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.977227 12518 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.933638 12739 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.737583 12612 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.980831 12612 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:30.929066 12062 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:30.983001 12062 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:30.987280 12838 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.208s	user 0.000s	sys 0.000s
I20260709 12:24:30.953312 12516 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:30.989055 12755 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.658s	user 0.011s	sys 0.000s
I20260709 12:24:30.989639 12755 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:30.978960 12745 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:30.991452 12745 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:30.951030 12834 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
W20260709 12:24:30.950635 10094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.006s	sys 0.006s
W20260709 12:24:31.018980 10094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.007s	sys 0.006s
W20260709 12:24:31.022500 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56501.3us
I20260709 12:24:30.933409 12849 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:30.915940 12844 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.093s
W20260709 12:24:31.038954 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.054054 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.004326 12528 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:31.010552 12734 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:31.034582 12747 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:31.071082 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.034765 12755 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.079597 12732 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:31.058676 12554 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.081075 12554 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.067299 12849 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:31.083518 12849 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.084117 12554 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.072790 12763 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:31.088119 12860 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.089409 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:31.092804 12789 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:31.094913 12827 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:31.095335 12827 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.095444 12518 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.095721 12827 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.100471  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.101171 12062 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.101615 12806 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:24:31.103238 12806 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:31.102836 12774 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:31.104842 10668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 614) took 2157 ms (client timeout 1889 ms). Trace:
I20260709 12:24:31.105123 12822 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:31.090535 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.111342 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:31.053491 12845 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:31.116886 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:31.117677 10672 raft_consensus.cc:2376] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6. Re-sending same reply.
I20260709 12:24:31.101975 12523 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:31.122747 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66205.4us
I20260709 12:24:31.035024 12450 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.130172  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:31.123034 12850 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:31.131243 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:31.105463 10668 rpcz_store.cc:269] 0709 12:24:28.947390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.948451 (+  1061us) service_pool.cc:224] Handling call
0709 12:24:31.104578 (+2156127us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54006d01d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:31.104725 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4454656256}
I20260709 12:24:31.094408 12420 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:24:31.157822  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:31.149295 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.159646 12689 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:31.165498 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:31.168978 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.148025 12780 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:31.171134 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:31.123418 12687 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.037s
W20260709 12:24:31.173352 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.162094 12689 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.180488 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.181041 12803 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:31.175112 12845 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:31.183907  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.191519 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.173782 12734 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.185110  9912 leader_election.cc:400] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:31.211256  9912 leader_election.cc:403] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:24:31.223073 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 353953us
I20260709 12:24:31.202703 12689 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.202860 12803 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:31.208999 12853 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:31.231479  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 58522us
I20260709 12:24:31.235467  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.242645 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:31.226578 12062 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:31.231217 12803 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:31.228899 12523 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:31.247824 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:31.249163 12734 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:31.248769 12822 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.264794  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:31.260852 12853 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:31.275405 12853 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:31.275952 12853 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.274014  9912 leader_election.cc:400] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:31.278611  9912 leader_election.cc:403] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
W20260709 12:24:31.278734 12854 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:24:31.279857  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.280303 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2574) took 1247 ms. Trace:
I20260709 12:24:31.280639 12812 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6.
I20260709 12:24:31.281426 12812 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:31.280745 10024 rpcz_store.cc:276] 0709 12:24:30.032919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.629882 (+596963us) service_pool.cc:224] Handling call
0709 12:24:30.643414 (+ 13532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.643848 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.643890 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:30.650450 (+  6560us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.023323 (+372873us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:31.067417 (+ 44094us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.068255 (+   838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.068333 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:31.280031 (+211698us) raft_consensus.cc:1685] finished
0709 12:24:31.280061 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.280232 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4006,"prepare.queue_time_us":44133,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":4998,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":45787,"thread_start_us":323440,"threads_started":1,"wal-append.queue_time_us":501571}
I20260709 12:24:31.285404 12816 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:31.295671 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47915.4us
I20260709 12:24:31.305524  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:31.308156  9945 raft_consensus.cc:2376] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6. Re-sending same reply.
I20260709 12:24:31.306360  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.309453 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.304541 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:31.317268  9948 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:31.322685 12394 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.706s	user 0.007s	sys 0.004s
I20260709 12:24:31.323282 12394 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.326637  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.327942  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.328490  9941 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:31.329514  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:31.332450  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.333235  9949 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
W20260709 12:24:31.337760 12870 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:24:31.343434  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63650.1us
I20260709 12:24:31.344606  9946 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:31.269485 12523 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:31.350157  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:31.351044 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:31.386104  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.274542 12450 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:31.391004 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81622us
I20260709 12:24:31.392514 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.393316  9944 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:31.395872  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.396627 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.387564 12726 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:31.431394 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 202083us
I20260709 12:24:31.371243 12785 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.375079 12062 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:31.374754 12865 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:31.438499 10642 leader_election.cc:336] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.819s (SENT)
I20260709 12:24:31.460467 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:31.394017 12871 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:31.518080  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:31.519061  9944 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.467450  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.388075 12734 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:24:31.433501 12696 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.371476 12687 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.073s
I20260709 12:24:31.537572 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.539579 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.540436 10025 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:24:31.541221 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:31.527807 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 89340.8us
I20260709 12:24:31.528343 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.375567 12822 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.076s
I20260709 12:24:31.524236 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:31.537817 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:31.548416 10642 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.986s (SENT)
I20260709 12:24:31.548910 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:31.551174 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 609) took 2083 ms. Trace:
I20260709 12:24:31.551656 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.455250 12394 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.552404 10025 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:31.554983 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.556464 10025 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:31.557940 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.561955  9944 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:31.517027 12450 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.551621 10676 rpcz_store.cc:276] 0709 12:24:29.455283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:29.833540 (+378257us) service_pool.cc:224] Handling call
0709 12:24:30.241241 (+407701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.241994 (+   753us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.538799 (+1296805us) spinlock_profiling.cc:243] Waited 407 ms on lock 0x7b54007f4750. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:31.538943 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936688896}
I20260709 12:24:31.567607 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.571381 10025 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:31.572675 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.579366  9945 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:31.467504 12732 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.102s
I20260709 12:24:31.587580 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.588500 10024 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 3.
I20260709 12:24:31.518375 12726 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:31.584352  9943 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:31.578775 12856 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:31.451136 12845 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.118s
I20260709 12:24:31.593547 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:31.594465 10673 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:31.547164 12858 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:31.597724 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:31.600039 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.601732 10017 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 4: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:31.602520 12747 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.507s	user 0.006s	sys 0.004s
I20260709 12:24:31.602766 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.604074 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.604624 10017 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:31.605365 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.606097 10017 raft_consensus.cc:2376] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6. Re-sending same reply.
I20260709 12:24:31.576251 12450 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.607653 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.609318 10017 raft_consensus.cc:2376] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7. Re-sending same reply.
I20260709 12:24:31.609632 12450 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.610178 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:31.611245 12875 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:24:31.613255 12810 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.560s	user 0.007s	sys 0.005s
I20260709 12:24:31.569967 12847 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.613732 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 416) took 1058 ms. Trace:
I20260709 12:24:31.592664 12860 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
W20260709 12:24:31.614790 12612 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.002s
I20260709 12:24:31.538333 12871 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:31.616519 12827 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.000s
I20260709 12:24:31.614159 10019 rpcz_store.cc:276] 0709 12:24:30.555487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.025413 (+469926us) service_pool.cc:224] Handling call
0709 12:24:31.046170 (+ 20757us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:31.046674 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.046718 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:31.090890 (+ 44172us) log.cc:844] Serialized 52 byte log entry
0709 12:24:31.252098 (+161208us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:31.265315 (+ 13217us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.266115 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.266187 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:31.613534 (+347347us) raft_consensus.cc:1685] finished
0709 12:24:31.613554 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.613678 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21409,"prepare.queue_time_us":12790,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":8463,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":15740,"thread_start_us":160095,"threads_started":1,"wal-append.queue_time_us":303157}
I20260709 12:24:31.616952 12827 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.617467 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.618816 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.619624 10017 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:31.620496 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.621018 10017 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:31.621271 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.605237 12821 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:31.621938 10672 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:31.468462 12785 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.624440 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:31.566792 12861 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:31.614282 12856 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.627637 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.637253 10024 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.639077 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:31.642575  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.647908  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0479702390002785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17270000000000002s for a response
I20260709 12:24:31.650341 10019 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:31.650820 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 611) took 1524 ms. Trace:
I20260709 12:24:31.638511 12821 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:31.623571 12787 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:31.640424 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:31.653927 10670 rpcz_store.cc:276] 0709 12:24:30.148108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.162382 (+ 14274us) service_pool.cc:224] Handling call
0709 12:24:30.688911 (+526529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.693495 (+  4584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.647780 (+954285us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b5400610150. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:31.650739 (+  2959us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177213184}
I20260709 12:24:31.655895  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:31.659746 10676 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:31.662161 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:31.640859 12747 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:31.614104 12847 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:31.636548 12774 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:24:31.667033 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:31.664059 12845 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:31.666203 12747 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.692481 12888 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:24:31.672585 12062 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:31.676695 12524 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:31.677735 12789 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.661530 12732 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:24:31.665365 12787 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.666534 12734 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:31.686203 12696 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:31.695061 12747 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.694703 12572 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:31.732636  9945 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:31.685940 12698 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:24:31.691874 12845 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:31.718564 12524 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:31.717495 12827 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:31.747859 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 150181us
I20260709 12:24:31.752898 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:31.756916 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.756392  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 100599us
I20260709 12:24:31.693379 12726 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.760030  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.734298 12572 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:31.738034 12889 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:31.743537 12845 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.772958  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.723419 12687 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:31.748842 12524 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:31.728727 12789 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.743765 12696 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:31.771198 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.774091  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.721907 12821 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.784094 12785 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.792788  9944 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:31.793584 10673 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:31.776188  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.784369 12062 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:31.786005 12802 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.862s	user 0.001s	sys 0.010s
I20260709 12:24:31.768172 12853 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.775136 12893 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.783205 12806 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:31.784256 12572 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.800194 10017 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:31.759583 12803 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.820760 12856 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:31.808308 12893 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.821282 12827 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.821842 12827 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.822932 12803 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:31.823087 12893 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:31.823962  9945 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:31.824074 12789 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:31.824939 12880 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 12:24:31.825465  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:31.811059 12882 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:24:31.813154 12853 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:31.813956 12883 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:24:31.814981 12696 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:31.833288 12689 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.002s
I20260709 12:24:31.833724 12689 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:31.834956 12572 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:31.836691 12689 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.837170 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:31.837831 10025 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:31.840191 12700 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.783224 12755 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:31.843262 12755 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:31.842090 12845 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.848991  9941 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:31.817245 12787 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.781036 12862 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.849916 12862 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:31.850256 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:31.829804  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:31.830508 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 716) took 1142 ms. Trace:
I20260709 12:24:31.830742  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:31.851999 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.819876 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:31.837725 12821 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:31.854605 10017 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:31.859740  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 83595.4us
I20260709 12:24:31.823980 12827 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:31.826407 10024 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:31.862317 10025 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:31.868477 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:31.869364 10093 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:31.872668 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:31.873584 10096 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:31.875476 10090 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:31.851492 10677 rpcz_store.cc:276] 0709 12:24:30.688196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.688312 (+   116us) service_pool.cc:224] Handling call
0709 12:24:30.704982 (+ 16670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.058605 (+353623us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:31.252308 (+193703us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.539363 (+287055us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.550218 (+ 10855us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.550357 (+   139us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:31.550390 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.550464 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:31.830278 (+279814us) raft_consensus.cc:1685] finished
0709 12:24:31.830309 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:31.830450 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2517,"prepare.queue_time_us":95997,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":223000,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":6600,"thread_start_us":749329,"threads_started":3,"wal-append.queue_time_us":380481}
I20260709 12:24:31.883759  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:31.844343 12755 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:32.244045 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.019981 10677 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 12:24:32.040025  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.894268 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:32.605505  9915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.006s	sys 0.027s
I20260709 12:24:31.893249 10093 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:32.668077  9915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.006s	sys 0.032s
I20260709 12:24:32.600546 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 619) took 1063 ms. Trace:
I20260709 12:24:32.647370 10670 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:32.669023 10672 rpcz_store.cc:276] 0709 12:24:31.537364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.537584 (+   220us) service_pool.cc:224] Handling call
0709 12:24:32.600245 (+1062661us) spinlock_profiling.cc:243] Waited 75.3 ms on lock 0x7b54004d35d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:32.600452 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":173288704}
I20260709 12:24:32.571883 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 401) took 1981 ms. Trace:
I20260709 12:24:31.884132  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:32.572293  9991 leader_election.cc:336] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.567s (SENT)
I20260709 12:24:32.670750  9940 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
W20260709 12:24:32.669008  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.976307  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:31.985808  9949 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
W20260709 12:24:32.169258 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:32.673894 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.674403  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:32.670469 10092 rpcz_store.cc:276] 0709 12:24:30.590509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.590684 (+   175us) service_pool.cc:224] Handling call
0709 12:24:30.603113 (+ 12429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:30.956379 (+353266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.571524 (+1615145us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b5400305a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:32.571797 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810587392}
I20260709 12:24:32.670868  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:31.860745 12523 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:32.000041 12895 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 12:24:32.256634 12905 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.268s	user 0.000s	sys 0.000s
W20260709 12:24:32.447167 12890 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
W20260709 12:24:32.438834 12747 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.692s	user 0.004s	sys 0.007s
I20260709 12:24:31.916253 12845 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:32.677657  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 701422us
I20260709 12:24:32.675266 12622 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:32.721042  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:32.687264 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:31.880328 12853 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.700394  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:31.895181 12450 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.708119  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 135875us
I20260709 12:24:32.048404 12789 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:31.926939 12774 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.680776 12901 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:24:31.891390 12886 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:32.569536 12819 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:32.649039 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:32.649133 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:31.941637 12806 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.721585 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:32.729750 12622 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:31.895196 12862 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:32.655995 12847 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:32.715615 10096 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.820s	user 0.007s	sys 0.010s
W20260709 12:24:32.761683 10096 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.008s	sys 0.010s
I20260709 12:24:32.675091 12865 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:32.728490  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:32.732149 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:32.852188 10093 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:32.732880 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.733084 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:32.820173  9944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.847s	user 0.003s	sys 0.034s
W20260709 12:24:32.853292  9944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.003s	sys 0.034s
I20260709 12:24:32.730983  9942 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:31.894487 12687 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.826s
I20260709 12:24:32.700409 12523 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.679553 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:31.915680 12889 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.815s
I20260709 12:24:31.925997 12524 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:32.719687 12572 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.738513 12822 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:32.739022 12893 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:32.729549 12747 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:32.793362 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:32.886335 10092 raft_consensus.cc:2376] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5. Re-sending same reply.
W20260709 12:24:32.784122 12726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.038s	sys 0.050s
I20260709 12:24:32.755158 10676 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:32.798723 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:32.739480 12853 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:32.799901 12862 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.762475 12806 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:32.830260 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:32.894541 10020 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:32.761178 10669 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:32.839309 12622 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:32.756332 12819 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:32.745415 12789 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:32.750157 12886 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:32.748701 12774 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:32.854003  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 125627us
I20260709 12:24:32.747711 12450 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:32.879927 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.884121 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:32.901611 10093 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:32.902973 10093 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:24:32.888243  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:32.904068  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.904496  9942 raft_consensus.cc:2376] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6. Re-sending same reply.
I20260709 12:24:32.888695 10096 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:32.851303 12843 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.799727 12865 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:32.877074 12815 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:32.888033 12726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.025s	user 0.039s	sys 0.050s
W20260709 12:24:32.897881  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 178625us
I20260709 12:24:32.871505 12901 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:32.908702 10092 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:32.951227 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1616) took 1096 ms. Trace:
I20260709 12:24:32.951572 10092 rpcz_store.cc:276] 0709 12:24:31.856188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.891738 (+1035550us) service_pool.cc:224] Handling call
0709 12:24:32.903971 (+ 12233us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.950942 (+ 46971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.951170 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:32.919133 10017 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.002s	sys 0.006s
W20260709 12:24:32.952602 10017 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.003s	sys 0.006s
I20260709 12:24:32.901949 10091 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:32.921324 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1619) took 1011 ms. Trace:
I20260709 12:24:32.900700 12886 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:32.947703 10096 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:32.952251 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2582) took 1599 ms. Trace:
I20260709 12:24:32.961059 10019 rpcz_store.cc:276] 0709 12:24:31.353378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.618543 (+265165us) service_pool.cc:224] Handling call
0709 12:24:32.937491 (+1318948us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b54001f7f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:32.952190 (+ 14699us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70241024}
I20260709 12:24:32.920758 12622 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:32.967653 12821 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.099s	user 0.009s	sys 0.004s
I20260709 12:24:32.969835 12726 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.953447  9990 connection.cc:256] RPC call timeout handler was delayed by -1.6819820720002099s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
I20260709 12:24:32.933485  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:32.931664 12822 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:32.955314 10093 rpcz_store.cc:276] 0709 12:24:31.910398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.902274 (+991876us) service_pool.cc:224] Handling call
0709 12:24:32.902674 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:32.910739 (+  8065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.921231 (+ 10492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:32.971932  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:32.955298 10095 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2666) took 2582 ms (client timeout 1914 ms). Trace:
I20260709 12:24:32.934492 12687 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:32.972642  9990 connection.cc:256] RPC call timeout handler was delayed by -1.3993820720002077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1826s for a response
I20260709 12:24:32.930696 12815 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:32.897420 12862 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:32.955960 12572 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:32.950516 12901 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:32.992887  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:32.996615  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:32.975934 10095 rpcz_store.cc:269] 0709 12:24:30.354535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.395614 (+ 41079us) service_pool.cc:224] Handling call
0709 12:24:30.622329 (+226715us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:32.936372 (+2314043us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54002fc454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:32.936531 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":325546496}
W20260709 12:24:32.998476  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:32.930931 12843 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:32.975320 12910 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:24:33.044040 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:33.043987 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:33.013331  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:32.970986 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1611) took 1117 ms. Trace:
W20260709 12:24:33.047760 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 108607us
I20260709 12:24:32.973762 12915 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:32.973604 12912 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.175s
I20260709 12:24:33.077678 12915 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:33.079548 12857 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.917s	user 0.000s	sys 0.010s
I20260709 12:24:33.080627 12819 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.081140 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 722) took 1434 ms. Trace:
I20260709 12:24:33.081655 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2660) took 2735 ms. Trace:
I20260709 12:24:33.081549 10673 rpcz_store.cc:276] 0709 12:24:31.647003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.647140 (+   137us) service_pool.cc:224] Handling call
0709 12:24:31.793227 (+146087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.794119 (+   892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.080914 (+1286795us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54007d7b50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:33.081074 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":314928512}
I20260709 12:24:33.082167 10094 rpcz_store.cc:276] 0709 12:24:30.346331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.350090 (+  3759us) service_pool.cc:224] Handling call
0709 12:24:30.350477 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:30.350947 (+   470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.350986 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:30.351751 (+   765us) log.cc:844] Serialized 52 byte log entry
0709 12:24:31.020525 (+668774us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:31.268228 (+247703us) log.cc:844] Serialized 14 byte log entry
0709 12:24:31.268961 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.269155 (+   194us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:31.604957 (+335802us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:24:33.078950 (+1473993us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:33.081462 (+  2512us) raft_consensus.cc:1685] finished
0709 12:24:33.081480 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.081601 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":11951,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":3467,"raft.run_cpu_time_us":7812,"raft.run_wall_time_us":1256968,"spinlock_wait_cycles":51125248,"thread_start_us":914196,"threads_started":2,"wal-append.queue_time_us":724249}
I20260709 12:24:33.082679 12524 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:33.072837 12789 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.074170 12663 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:33.085397 12745 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.086050 12663 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:33.086438 12663 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:33.087299 12663 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:33.054536 12914 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:24:32.970448 12726 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.055214 12853 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.052531 10093 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:33.057678  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1398) took 1737 ms. Trace:
I20260709 12:24:33.058712 10091 rpcz_store.cc:276] 0709 12:24:31.853494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.871272 (+ 17778us) service_pool.cc:224] Handling call
0709 12:24:32.966068 (+1094796us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:33.059362 12916 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:24:33.060252  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:33.062196 12913 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.333s	user 0.000s	sys 0.001s
I20260709 12:24:33.062978 12523 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:33.067044 12843 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.067836 12774 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.014474 12572 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:33.069739 12873 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.769s	user 0.006s	sys 0.004s
I20260709 12:24:33.003930 12822 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:33.069816 12747 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:32.999925 12687 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:33.013903 12901 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.072825  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1396) took 1754 ms. Trace:
I20260709 12:24:33.000177 12922 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.224185 12726 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.223832 12922 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:33.089319 12450 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.226157 12922 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.087733 12865 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:32.999529 12889 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.227454 12889 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:33.228258 12862 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:33.228330  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.229079  9943 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
W20260709 12:24:33.056654  9914 connection.cc:256] RPC call timeout handler was delayed by -2.310334894999869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22250000000000003s for a response
I20260709 12:24:33.230195  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.234910  9943 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:33.117575 10020 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:33.236153  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.177692  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:33.114315 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 182378us
I20260709 12:24:33.201638  9947 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.201936  9941 rpcz_store.cc:276] 0709 12:24:31.320892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.329274 (+  8382us) service_pool.cc:224] Handling call
0709 12:24:33.057455 (+1728181us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b5400207e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:33.057615 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192813312}
I20260709 12:24:33.240823  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.010839  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:33.176154  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.202523 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.203615 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:33.243265 10095 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:33.205852  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.206498 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:33.244175 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:33.074887  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1408) took 1270 ms. Trace:
I20260709 12:24:33.212513  9948 rpcz_store.cc:276] 0709 12:24:31.318690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.318823 (+   133us) service_pool.cc:224] Handling call
0709 12:24:31.319175 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.654668 (+335493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.654705 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.692188 (+ 37483us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.855648 (+163460us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:31.855682 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.072429 (+1216747us) spinlock_profiling.cc:243] Waited 56.1 ms on lock 0x7b54001b8854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:33.072480 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.072527 (+    47us) raft_consensus.cc:1685] finished
0709 12:24:33.072542 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.072765 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68,"raft.queue_time_us":207098,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8292,"spinlock_wait_cycles":129051520,"thread_start_us":405888,"threads_started":2,"wal-append.queue_time_us":217198}
W20260709 12:24:33.247418  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 236696us
I20260709 12:24:33.247924  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.248718  9948 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:33.249672  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:33.249889  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:33.215935 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:33.256625 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:33.219810  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:33.221437 12815 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:33.221976  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.245236  9944 rpcz_store.cc:276] 0709 12:24:31.804443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.804940 (+   497us) service_pool.cc:224] Handling call
0709 12:24:31.805313 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:31.871375 (+ 66062us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.871421 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:31.886008 (+ 14587us) log.cc:844] Serialized 52 byte log entry
0709 12:24:32.855117 (+969109us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:32.855169 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.855235 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.074665 (+219430us) raft_consensus.cc:1685] finished
0709 12:24:33.074685 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.074829 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53688,"raft.queue_time_us":75213,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":11411,"thread_start_us":881967,"threads_started":1,"wal-append.queue_time_us":1028872}
I20260709 12:24:33.257711  9940 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:33.222288 12915 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:33.261135  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.012238 12893 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:33.230381 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 714) took 2573 ms. Trace:
I20260709 12:24:33.231199 12806 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:33.232018  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 218753us
I20260709 12:24:33.232528 12810 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:33.232872 12572 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.012801  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.233150 12871 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:33.251370  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.255329  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:33.256422 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 625) took 1490 ms. Trace:
I20260709 12:24:33.288309 10674 rpcz_store.cc:276] 0709 12:24:31.766077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.766254 (+   177us) service_pool.cc:224] Handling call
0709 12:24:31.781543 (+ 15289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.853077 (+ 71534us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.853107 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.853511 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.311081 (+457570us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:32.311152 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.075545 (+764393us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:33.075592 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.234013 (+158421us) raft_consensus.cc:1685] finished
0709 12:24:33.234031 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.256351 (+ 22320us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5897,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":2587,"spinlock_wait_cycles":164562432,"thread_start_us":442060,"threads_started":1,"wal-append.queue_time_us":891425}
I20260709 12:24:33.215960 12933 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.291178 12933 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.258526  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.287729  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:33.222575 12932 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.257237 12450 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:24:33.264587 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.268710 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:33.233155 12663 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.306825 12924 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:33.307328 12924 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.307657 12924 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:33.307807 12687 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.261392  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.327297  9943 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:33.263676  9948 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:33.261940 12847 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:33.278235 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:33.301090 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:33.301254 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.301309  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:33.261190 12726 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.294000 12933 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.294811 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:33.316506 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.315495 12622 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:33.308643 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2585) took 1936 ms. Trace:
I20260709 12:24:33.324447  9947 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:33.359706  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.360386  9947 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:33.277940 12816 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:24:33.278502 12524 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:33.281265 12912 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.265462 10671 rpcz_store.cc:276] 0709 12:24:30.657299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.693367 (+ 36068us) service_pool.cc:224] Handling call
0709 12:24:30.725214 (+ 31847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:30.952409 (+227195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:30.952446 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:30.959026 (+  6580us) log.cc:844] Serialized 26 byte log entry
0709 12:24:31.174713 (+215687us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:31.174752 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:31.174820 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.069013 (+1894193us) spinlock_profiling.cc:243] Waited 4.49 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:33.230054 (+161041us) raft_consensus.cc:1685] finished
0709 12:24:33.230079 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.230307 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14443,"raft.queue_time_us":191946,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":16622,"spinlock_wait_cycles":10321792,"thread_start_us":340769,"threads_started":2,"wal-append.queue_time_us":243367}
I20260709 12:24:33.331879 10095 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:33.266292 12789 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:33.261677 12774 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:24:33.354949  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.361384  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.357925 10025 rpcz_store.cc:276] 0709 12:24:31.372185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.863504 (+491319us) service_pool.cc:224] Handling call
0709 12:24:31.863803 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.871631 (+  7828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.871662 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.872509 (+   847us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.603402 (+730893us) spinlock_profiling.cc:243] Waited 2.91 ms on lock 0x7b180045ca00. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:24:32.998943 (+395541us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:32.999002 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.999099 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.308347 (+309248us) raft_consensus.cc:1685] finished
0709 12:24:33.308365 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.308581 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4664,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":13247,"spinlock_wait_cycles":6697728,"thread_start_us":395288,"threads_started":1,"wal-append.queue_time_us":1371829}
I20260709 12:24:33.296731  9944 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:33.405076 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:33.410457 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:33.411249 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:33.329656 12810 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.352245 12763 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:33.366886  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.371564 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2583) took 2016 ms. Trace:
I20260709 12:24:33.330641 12739 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:33.328779 12847 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:33.353142 12919 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:24:33.328732 12747 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.296023 12806 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 12:24:33.350487 12886 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.464439 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2584) took 2092 ms. Trace:
I20260709 12:24:33.420612  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:33.443785 12849 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.324s	user 0.022s	sys 0.006s
W20260709 12:24:33.518317 12849 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.399s	user 0.031s	sys 0.006s
W20260709 12:24:33.420305  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 165081us
I20260709 12:24:33.519560  9943 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:33.520390  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:33.518379 12763 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.431713 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:33.521569  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.443411 12937 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:33.493377  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 192145us
I20260709 12:24:33.521663 10024 rpcz_store.cc:276] 0709 12:24:31.371795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.862154 (+490359us) service_pool.cc:224] Handling call
0709 12:24:32.670636 (+808482us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:32.671036 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.671064 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:32.671740 (+   676us) log.cc:844] Serialized 52 byte log entry
0709 12:24:32.893887 (+222147us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:33.265521 (+371634us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.279084 (+ 13563us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.463989 (+184905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.464095 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.464152 (+    57us) raft_consensus.cc:1685] finished
0709 12:24:33.464170 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.464360 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12361,"prepare.queue_time_us":312652,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":53260,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":38898,"spinlock_wait_cycles":95856256,"thread_start_us":767842,"threads_started":3,"wal-append.queue_time_us":341222}
I20260709 12:24:33.456126 12774 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:33.431571 12915 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:33.525736 12774 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:33.484990 12930 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.194s	user 0.001s	sys 0.000s
I20260709 12:24:33.517088 12810 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:33.523738 12954 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:33.523993  9941 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:33.362015 12622 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:33.525141 12886 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.526383 12774 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:33.488039 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.531503 12952 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.543442 12622 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:33.432937 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2591) took 1495 ms. Trace:
I20260709 12:24:33.543990 12952 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.544337 10016 rpcz_store.cc:276] 0709 12:24:31.937762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.432016 (+1494254us) service_pool.cc:224] Handling call
0709 12:24:33.432315 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.432518 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.432555 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:33.432612 (+    57us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:33.432661 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.432732 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.432872 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:33.545317 12954 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:33.545807 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:33.546469 10016 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:33.546794 12952 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.519595 12847 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:33.547451 12952 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.547708 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:33.548854 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:33.549414 10016 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:33.366106 12450 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:33.521106  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:33.463240 10675 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:33.525978 12915 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:33.431423 12776 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:33.499893 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2586) took 2127 ms. Trace:
I20260709 12:24:33.546418 12954 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.531080 12763 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:33.421173 10642 leader_election.cc:336] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.691s (SENT)
I20260709 12:24:33.549958 12952 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:33.357614 12726 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.455313 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:33.420820  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.526432 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2592) took 1588 ms. Trace:
I20260709 12:24:33.409515 12862 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.154s
I20260709 12:24:33.550398 10017 rpcz_store.cc:276] 0709 12:24:31.354732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.860110 (+505378us) service_pool.cc:224] Handling call
0709 12:24:31.860443 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:31.860666 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.860695 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:31.883422 (+ 22727us) log.cc:844] Serialized 26 byte log entry
0709 12:24:32.314987 (+431565us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:32.954933 (+639946us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.370921 (+415988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.371218 (+   297us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.371283 (+    65us) raft_consensus.cc:1685] finished
0709 12:24:33.371306 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.371482 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1867,"prepare.queue_time_us":883803,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":18383,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":18213,"thread_start_us":1360501,"threads_started":3,"wal-append.queue_time_us":746829}
I20260709 12:24:33.562378 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:33.421666 12931 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:33.564970 12931 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.317894 12932 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:33.557636  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.559132 10024 rpcz_store.cc:276] 0709 12:24:31.938037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.525632 (+1587595us) service_pool.cc:224] Handling call
0709 12:24:33.525940 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.526102 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.526125 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:33.526162 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:33.526195 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.526243 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.526368 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:33.558460  9944 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:33.586189  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.521579  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:33.553942 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:33.550999  9943 raft_consensus.cc:2368] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6.
I20260709 12:24:33.423346 12912 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.551697 12747 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.551366 12776 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.597167 12822 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.597826 12948 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:33.598506  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:33.598891  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.650858  9943 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:33.614404  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.551065 12915 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:33.648144 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 405) took 3052 ms (client timeout 1691 ms). Trace:
I20260709 12:24:33.571623 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:33.656723 10099 rpcz_store.cc:269] 0709 12:24:30.595760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.622962 (+ 27202us) service_pool.cc:224] Handling call
0709 12:24:33.647903 (+3024941us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b5400163854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:33.648053 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":635435392}
I20260709 12:24:33.551904 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:33.558387 12810 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:33.632462 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 211321us
I20260709 12:24:33.602658 12937 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:33.655090 12948 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:33.661480 12943 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:33.650127 12776 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:33.670565 12940 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 12:24:33.671597 10092 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:33.657963 12922 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.672724 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2587) took 2299 ms. Trace:
W20260709 12:24:33.670186 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.662777 12844 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:33.672544 12922 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.673198 10020 rpcz_store.cc:276] 0709 12:24:31.372830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.896842 (+1524012us) service_pool.cc:224] Handling call
0709 12:24:32.907755 (+ 10913us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.908228 (+   473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.908266 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:32.948591 (+ 40325us) log.cc:844] Serialized 564 byte log entry
0709 12:24:33.117411 (+168820us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:33.456982 (+339571us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.457248 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.457350 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.672461 (+215111us) raft_consensus.cc:1685] finished
0709 12:24:33.672490 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.672653 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36430,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":111549,"threads_started":1,"wal-append.queue_time_us":396926}
W20260709 12:24:33.687300  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 165779us
I20260709 12:24:33.687947  9947 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:33.688222  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:33.600070  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.673761 10092 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:33.522858  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.705515 10017 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:33.717963  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:33.659209 12822 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:33.761152 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.767558 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.551623 10018 rpcz_store.cc:276] 0709 12:24:31.372515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.595524 (+1223009us) service_pool.cc:224] Handling call
0709 12:24:32.828171 (+232647us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:32.828441 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:32.828873 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:32.828915 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:32.829526 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.009978 (+180452us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:33.276241 (+266263us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.499252 (+223011us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:33.499371 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.499446 (+    75us) raft_consensus.cc:1685] finished
0709 12:24:33.499603 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.499810 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2970,"prepare.queue_time_us":189333,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":44760,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":213,"spinlock_wait_cycles":15532800,"thread_start_us":606422,"threads_started":3,"wal-append.queue_time_us":473131}
I20260709 12:24:33.682245 12810 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:33.668246 12948 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:33.788488 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.767683 12822 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:33.769220 12952 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.807646 12952 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:33.808164 12952 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:33.808604 12952 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:33.780943 12943 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:24:33.809904 12915 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.831247 12932 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.833418 12912 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.837611 12937 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.851768 12922 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.852560 12922 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:33.855825 12747 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.781842 12810 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.866156 12810 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.875587 12745 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:33.878266 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:33.878906 12745 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.881053 12745 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:33.813016 12822 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:33.885524 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:33.889551 12967 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:24:33.890252 10674 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:33.892452 12789 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:33.892989 12789 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:33.895253 12906 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.221s	user 0.001s	sys 0.001s
I20260709 12:24:33.894011 12789 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:33.897814 12789 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:33.901396 10642 leader_election.cc:336] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.057s (SENT)
I20260709 12:24:33.902786 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:33.905881  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1407) took 2114 ms. Trace:
I20260709 12:24:33.906447  9946 rpcz_store.cc:276] 0709 12:24:31.791611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.791913 (+   302us) service_pool.cc:224] Handling call
0709 12:24:31.792320 (+   407us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:31.818034 (+ 25714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:31.818068 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:31.820734 (+  2666us) log.cc:844] Serialized 52 byte log entry
0709 12:24:32.656123 (+835389us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:32.656180 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:32.656260 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:33.889888 (+1233628us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:33.898362 (+  8474us) raft_consensus.cc:1685] finished
0709 12:24:33.898393 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:33.905760 (+  7367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41368,"raft.queue_time_us":35654,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"spinlock_wait_cycles":4301184,"thread_start_us":131916,"threads_started":1,"wal-append.queue_time_us":852614}
I20260709 12:24:33.909061  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:33.920674 12554 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:33.903522 12622 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:33.931825  9941 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:33.932662 12622 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.933779 12726 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:33.939975  9946 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:33.941700  9946 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:33.933769  9941 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:24:33.953845 10022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.003s	sys 0.031s
W20260709 12:24:33.954563 10022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.003s	sys 0.031s
W20260709 12:24:33.966944 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.961795 12924 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.000s
I20260709 12:24:33.993589 10671 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:33.989647  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 357668us
W20260709 12:24:34.008066  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:33.992449 12663 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.006s
I20260709 12:24:34.009425 12924 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.037153 12924 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.025238 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:34.043478 12776 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.044863 12948 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.019366 12931 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.040995 12523 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:34.088183  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.129050  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 536) took 2356 ms. Trace:
I20260709 12:24:34.129613  9945 rpcz_store.cc:276] 0709 12:24:31.772136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.772693 (+   557us) service_pool.cc:224] Handling call
0709 12:24:31.792662 (+ 19969us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:31.824448 (+ 31786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.128671 (+2304223us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b54001a31d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:34.128958 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71213824}
W20260709 12:24:34.125037 12739 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.572s	user 0.005s	sys 0.006s
W20260709 12:24:34.124805 12774 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.581s	user 0.006s	sys 0.005s
I20260709 12:24:34.151808  9943 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [4.6-6.8]
W20260709 12:24:34.139655 12954 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.037s	sys 0.095s
W20260709 12:24:34.167466 12954 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.037s	sys 0.096s
W20260709 12:24:34.168078 12965 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:24:34.170455  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:34.161135 12917 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.168452 12958 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:34.182375  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 464586us
I20260709 12:24:34.160701 12774 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.166834 12864 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.013s
W20260709 12:24:34.184345 12820 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.008s	sys 0.011s
I20260709 12:24:34.189848  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:34.183275 12958 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.198035  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:34.204561  9945 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:34.190958 12864 consensus_queue.cc:1243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:34.197305 12958 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.206532  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:34.234798  9941 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:34.198328 12948 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.194810 12822 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.276751  9941 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:34.264503 12523 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.280915  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.015s	sys 0.010s
W20260709 12:24:34.282366  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.015s	sys 0.010s
W20260709 12:24:34.282850  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.286495  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.011s	sys 0.016s
W20260709 12:24:34.287087  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.012s	sys 0.016s
I20260709 12:24:34.205587 10018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:45918
I20260709 12:24:34.290848 10018 raft_consensus.cc:606] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Received request to transfer leadership
I20260709 12:24:34.291742 10018 raft_consensus.cc:614] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:34.354323 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.178669 12917 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:34.394090  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.395490  9945 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:24:34.418262  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.420557 12622 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.421113 12622 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.423904 12922 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:34.424701 12922 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.427203 12700 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:34.429636 10090 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:34.431177 12966 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:24:34.433280 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:34.434067 10024 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:34.437040 12915 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:34.439479 12957 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:24:34.441143  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 543) took 1197 ms. Trace:
I20260709 12:24:34.441726 12622 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:34.442708 12975 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
W20260709 12:24:34.443879 12976 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
W20260709 12:24:34.393169 12925 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
W20260709 12:24:34.441668 12962 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.263s	user 0.000s	sys 0.000s
I20260709 12:24:34.442498 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.430394 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1615) took 2576 ms. Trace:
I20260709 12:24:34.464463 10090 rpcz_store.cc:276] 0709 12:24:31.854423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:31.876753 (+ 22330us) service_pool.cc:224] Handling call
0709 12:24:34.429249 (+2552496us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:34.429453 (+   204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.430112 (+   659us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.430330 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32546688}
I20260709 12:24:34.399713 12820 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:34.471681  9942 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:34.474201  9914 connection.cc:256] RPC call timeout handler was delayed by -1.8613111320000826s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2182s for a response
W20260709 12:24:34.475023  9914 connection.cc:256] RPC call timeout handler was delayed by -1.3940690770000401s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3063s for a response
W20260709 12:24:34.475426  9914 connection.cc:256] RPC call timeout handler was delayed by -1.061775900000157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19140000000000001s for a response
I20260709 12:24:34.476846 10024 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:34.483600  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:34.455302 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 106515us
I20260709 12:24:34.441565  9947 rpcz_store.cc:276] 0709 12:24:33.243403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.420353 (+176950us) service_pool.cc:224] Handling call
0709 12:24:34.440930 (+1020577us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540021dc54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:34.441074 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":387026304}
I20260709 12:24:34.463862 10095 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:34.443935 12922 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:34.485070  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:34.485996 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:34.488160 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:34.486713 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1629) took 1483 ms. Trace:
I20260709 12:24:34.488783 10095 rpcz_store.cc:276] 0709 12:24:33.003381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.557573 (+554192us) service_pool.cc:224] Handling call
0709 12:24:34.442343 (+884770us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:34.486582 (+ 44239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48851840}
I20260709 12:24:34.486135 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:34.490257 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.490275 10024 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:34.491663  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:34.492014 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:34.492682 10021 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:34.455104 12969 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.391263 12523 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.488741 12663 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:24:34.468144 12915 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:24:34.488574 12953 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:34.472021 12820 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:34.515938 12789 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.002s
I20260709 12:24:34.525880 12726 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.526268 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:34.518879  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1420) took 1033 ms. Trace:
I20260709 12:24:34.527820 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:34.528542 10018 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:34.527643  9946 rpcz_store.cc:276] 0709 12:24:33.485601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.945726 (+460125us) service_pool.cc:224] Handling call
0709 12:24:33.975187 (+ 29461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.978932 (+  3745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.978974 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.979568 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:24:34.207439 (+227871us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:34.207482 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.207544 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.518662 (+311118us) raft_consensus.cc:1685] finished
0709 12:24:34.518683 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.518819 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9823,"spinlock_wait_cycles":2877824,"thread_start_us":179536,"threads_started":1,"wal-append.queue_time_us":469878}
I20260709 12:24:34.539831 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2664) took 4192 ms. Trace:
W20260709 12:24:34.544112 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.546891 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:34.548444 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:34.551165 10099 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:24:34.542585 12923 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.501236 12981 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:34.513095 12901 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.256s	user 0.010s	sys 0.008s
W20260709 12:24:34.509016 12978 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:24:34.554236 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:34.511142 12774 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.517051 12524 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:34.547843 12853 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.005s	sys 0.010s
I20260709 12:24:34.513397 12523 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:34.553169 12749 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.507851 12931 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:24:34.517232 12889 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:34.589489 10021 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:34.526520 12726 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.555343 12924 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:34.584705 12853 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.005s	sys 0.011s
I20260709 12:24:34.520726 12969 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.593011 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:34.579596 12923 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.584617 12901 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.592168 12938 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:24:34.592073 12062 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:34.582691 12983 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.585115 12821 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:34.591255 12952 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.782s	user 0.013s	sys 0.000s
I20260709 12:24:34.605666 10021 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:34.585891 12981 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:34.623974  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1644) took 1408 ms. Trace:
I20260709 12:24:34.624432  9942 rpcz_store.cc:276] 0709 12:24:33.215203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.915947 (+700744us) service_pool.cc:224] Handling call
0709 12:24:34.451000 (+535053us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.472296 (+ 21296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.623756 (+151460us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b540021dc50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:34.623911 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229522304}
I20260709 12:24:34.591068 12749 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:34.630435  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 213422us
I20260709 12:24:34.616357 12822 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:34.598279 12924 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:34.626874 12970 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.422s	user 0.003s	sys 0.000s
I20260709 12:24:34.635051 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:34.642198 10021 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:34.645506  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:34.648537  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:34.650266  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:34.646499  9942 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:34.623683 12952 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.595064 12820 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.665375 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2594) took 1127 ms. Trace:
I20260709 12:24:34.666003 10016 rpcz_store.cc:276] 0709 12:24:33.538398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.550122 (+ 11724us) service_pool.cc:224] Handling call
0709 12:24:33.554017 (+  3895us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:33.554566 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.554618 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:33.710357 (+155739us) log.cc:844] Serialized 590 byte log entry
0709 12:24:34.158999 (+448642us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:34.165865 (+  6866us) log.cc:844] Serialized 14 byte log entry
0709 12:24:34.173217 (+  7352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.173319 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.665059 (+491740us) raft_consensus.cc:1685] finished
0709 12:24:34.665090 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.665294 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4628,"prepare.queue_time_us":6697,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":28091,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":5591,"thread_start_us":378294,"threads_started":1,"wal-append.queue_time_us":439465}
I20260709 12:24:34.675935 10021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:34.678560  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:34.680781  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:34.682078 10021 raft_consensus.cc:606] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:34.682646 10021 raft_consensus.cc:614] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:34.619223 12983 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.621974 12821 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.629366 12958 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.669165 12923 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:34.643748 12942 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.612624 12524 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.689797  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:34.684469 12871 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:34.722512  9991 leader_election.cc:400] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6.
I20260709 12:24:34.725584  9991 leader_election.cc:403] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:24:34.726655  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:34.714964 12988 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.124s	user 0.001s	sys 0.001s
I20260709 12:24:34.710781 12572 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:34.724834  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.074s	user 0.021s	sys 0.028s
W20260709 12:24:34.733978  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.021s	sys 0.028s
W20260709 12:24:34.734398  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 1.13589e+06us
I20260709 12:24:34.735599  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:34.728207  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:34.644229 12062 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.066s
I20260709 12:24:34.742002 12062 raft_consensus.cc:2764] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:24:34.706996 12948 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:34.742743 12948 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:34.743139 12948 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:34.710943 12820 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.720781 12947 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:34.751240 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2588) took 2852 ms. Trace:
I20260709 12:24:34.723502 12700 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:34.738178 12984 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:24:34.753497 10023 rpcz_store.cc:276] 0709 12:24:31.936720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.043623 (+1106903us) service_pool.cc:224] Handling call
0709 12:24:33.043991 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.044418 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.044464 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.045075 (+   611us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.602366 (+557291us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b1800422440. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:24:34.001125 (+398759us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:34.001168 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.001234 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.552020 (+550786us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:34.750999 (+198979us) raft_consensus.cc:1685] finished
0709 12:24:34.751020 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.751179 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7209,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":54963,"spinlock_wait_cycles":88327296,"thread_start_us":388580,"threads_started":1,"wal-append.queue_time_us":1131855}
I20260709 12:24:34.730890 12954 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6.
I20260709 12:24:34.730027 12871 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:34.758311 12989 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:24:34.750113 12822 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:34.767347 12901 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:34.767841 12822 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.757354 12947 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.768311 12822 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.768543 12947 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:34.767383  9974 heartbeater.cc:646] Failed to heartbeat to 127.8.137.254:38721 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.8.137.254:38721 timed out after 15.000s (SENT)
I20260709 12:24:34.740224 12572 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:34.770874 12572 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:34.772539 12936 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.773348 12936 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:34.775414 12924 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.729748 12981 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.776048 12936 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:34.776566 12992 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:34.779759  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:34.783484 12622 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:34.759020 12982 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:34.757458 12979 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:34.786051  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:34.720009 12942 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.788895  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:34.789870  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:34.778622 12847 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.230s	user 0.007s	sys 0.007s
W20260709 12:24:34.793447 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 199175us
I20260709 12:24:34.794342 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:34.796046 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 203133us
I20260709 12:24:34.797612 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 730) took 1546 ms. Trace:
I20260709 12:24:34.800527 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:34.804983 12774 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:34.807053 12871 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:34.798436 10670 rpcz_store.cc:276] 0709 12:24:33.251635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.569129 (+317494us) service_pool.cc:224] Handling call
0709 12:24:33.596808 (+ 27679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.769638 (+172830us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.769667 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.787786 (+ 18119us) log.cc:844] Serialized 26 byte log entry
0709 12:24:34.268566 (+480780us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:34.268605 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.268680 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.797418 (+528738us) raft_consensus.cc:1685] finished
0709 12:24:34.797441 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.797559 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35819,"raft.queue_time_us":64182,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":578438,"threads_started":2,"wal-append.queue_time_us":734953}
I20260709 12:24:34.816643 10017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:34.820960 10017 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:34.821379 10017 raft_consensus.cc:614] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:34.817324 10096 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:34.763877 12787 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:34.825084 12787 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:34.825369 12958 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:34.825481 12787 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:34.826633 12821 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.778344 12700 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.829377 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.830077 10095 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:34.831636 12822 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:34.832185 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.832805 10091 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:34.833771 12822 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:34.819358 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.793088 12622 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:34.823865 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:34.836867 10097 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:34.823603 12774 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:34.833662 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.797096 12726 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.841284 10092 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:34.849502  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:34.838469 12893 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:34.839984  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1637) took 1936 ms. Trace:
I20260709 12:24:34.795396 12936 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:34.844097 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.853070 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.854102  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:34.854987 10642 leader_election.cc:336] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.505s (SENT)
I20260709 12:24:34.857854 10096 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:34.857997 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:34.791195 12982 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 12:24:34.839196  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.812129 12969 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.854928  9944 rpcz_store.cc:276] 0709 12:24:32.903422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.761560 (+858138us) service_pool.cc:224] Handling call
0709 12:24:33.774200 (+ 12640us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.777474 (+  3274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.777502 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.816653 (+ 39151us) log.cc:844] Serialized 26 byte log entry
0709 12:24:34.201753 (+385100us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:34.201811 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.201910 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:34.839705 (+637795us) raft_consensus.cc:1685] finished
0709 12:24:34.839733 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.839909 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2906,"raft.queue_time_us":4155,"spinlock_wait_cycles":5788544,"thread_start_us":332372,"threads_started":1,"wal-append.queue_time_us":618887}
I20260709 12:24:34.838762 12924 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:34.857158 12936 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.870477 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.866281  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:34.838850 12622 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:34.872040 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 632) took 2191 ms. Trace:
I20260709 12:24:34.872443 10677 rpcz_store.cc:276] 0709 12:24:32.681101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.681320 (+   219us) service_pool.cc:224] Handling call
0709 12:24:34.871380 (+2190060us) spinlock_profiling.cc:243] Waited 2.4 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:34.871571 (+   191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.871737 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:34.871760 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.871799 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:34.871829 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.871882 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.871988 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5527936}
I20260709 12:24:34.800817 12847 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.871313 10090 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:34.870765 12924 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:34.875867  9894 auto_rebalancer.cc:893] Removing replace marker failed: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:24:34.876057 12789 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:34.876416  9894 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
I20260709 12:24:34.876250 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:34.876823  9894 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 12:24:34.877588 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:34.878094 12936 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:34.878106 10090 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:34.865512 12958 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:34.875902 12924 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:34.886540 12622 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:34.857048 12819 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:34.906692  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 410113us
W20260709 12:24:34.924046  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51331.4us
I20260709 12:24:34.864977 12827 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:34.939421  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.826435 12787 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:34.936663 10090 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:34.874069 12954 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:34.944186 10018 raft_consensus.cc:2368] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5.
I20260709 12:24:34.945544 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:34.891333 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.950572  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:34.946025  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:34.933094 10097 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:24:34.948274  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 205217us
W20260709 12:24:34.941350 12996 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:24:34.865815 12893 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:34.974385 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:34.984568  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:34.989384 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.991503 10090 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:34.992528 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.993161 10090 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:34.994040 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.989511  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:34.995934 10090 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:35.009104 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.009711 10097 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:34.992117 12810 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.021776 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.943977 12572 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.029503 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:34.955734 12952 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.943018 12789 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.070s
I20260709 12:24:35.011348 12822 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:35.026643 12516 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:34.946215 12827 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.014323 12787 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:24:35.043418 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.043483  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54083.4us
I20260709 12:24:35.026458 12810 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.026890 12958 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:35.042579 10091 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:35.054049 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.055023 10090 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:35.052631 10097 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:35.056607 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.043526 12983 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.077775 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:35.078341 10677 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.068392 10017 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:35.081306 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:35.081961 10017 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.011524 12997 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:24:35.074658 10024 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:24:35.075307 10021 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:35.075433 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.085250 10090 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:35.076319 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.050874 12810 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.049535 12822 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:35.088857  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 93588.4us
W20260709 12:24:35.097822 12998 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:24:35.099906 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:35.100577 10024 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.102442 10672 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEARNER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:35.104598 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:35.068913 12901 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:35.106199 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.107052 10672 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:35.107950 10676 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:35.106303 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.111665 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.114166 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.115842 10670 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:35.117825 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.138159 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.143000 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.143900 10676 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:35.067407 12958 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.148315 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.154953 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:35.105783 10091 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.139138 10021 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:35.167508 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.168221 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:35.169206 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.170382 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:35.171368  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 92874.9us
I20260709 12:24:35.123407 12922 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:35.161550 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.161798 12822 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.179481 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.180213 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:35.180800 12917 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.167615 12948 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.182164 10670 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:35.157723 10017 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
W20260709 12:24:35.175410 10062 connection.cc:256] RPC call timeout handler was delayed by -2.2677770800000872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1509s for a response
I20260709 12:24:35.183373 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.177910 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.168376 12827 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.178781 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.180339 13003 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.099754 12942 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:35.183769 10062 connection.cc:256] RPC call timeout handler was delayed by -2.0940770799998063s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 12:24:35.185838 10062 connection.cc:256] RPC call timeout handler was delayed by -2.0922770799998034s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1704s for a response
W20260709 12:24:35.186158 10062 connection.cc:256] RPC call timeout handler was delayed by -2.0634770800002116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:24:35.186429 10062 connection.cc:256] RPC call timeout handler was delayed by -1.9878770800000893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18200000000000002s for a response
W20260709 12:24:35.186761 10062 connection.cc:256] RPC call timeout handler was delayed by -1.9662770800000544s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1844s for a response
W20260709 12:24:35.187032 10062 connection.cc:256] RPC call timeout handler was delayed by -1.9455770800000209s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 12:24:35.187295 10062 connection.cc:256] RPC call timeout handler was delayed by -1.9158770799999729s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19s for a response
W20260709 12:24:35.187589 10062 connection.cc:256] RPC call timeout handler was delayed by -1.823177079999823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
W20260709 12:24:35.187888 10062 connection.cc:256] RPC call timeout handler was delayed by -1.7835770800002138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20470000000000002s for a response
W20260709 12:24:35.188151 10062 connection.cc:256] RPC call timeout handler was delayed by -1.5504770799998369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23060000000000003s for a response
W20260709 12:24:35.188414 10062 connection.cc:256] RPC call timeout handler was delayed by -1.5216770799997903s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2338s for a response
W20260709 12:24:35.188694 10062 connection.cc:256] RPC call timeout handler was delayed by -1.4019770800000515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24710000000000001s for a response
W20260709 12:24:35.188957 10062 connection.cc:256] RPC call timeout handler was delayed by -1.3245770799999264s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2557s for a response
W20260709 12:24:35.189217 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0590770799999518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2852s for a response
I20260709 12:24:35.198282 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.198282 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:35.246272 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:35.254021 10668 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:35.254736 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:35.255661 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57448.9us
I20260709 12:24:35.256016 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.256450 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:35.256851 10090 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:35.257747 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.258008 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:35.255678 10669 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:35.267258 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:35.168532 12847 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:35.187384 12915 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:35.258307 12524 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:35.281009 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 110723us
I20260709 12:24:35.274674 12923 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:35.274755 12816 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:35.283772 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:35.291556 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:35.288396 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.294054 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.295688 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.293957 10093 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:35.156212 10024 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:35.268747 12853 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:35.331043 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.329093 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.282656 12923 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:35.284994 13000 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
W20260709 12:24:35.285754 13013 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
W20260709 12:24:35.288647 12980 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.782s	user 0.004s	sys 0.000s
I20260709 12:24:35.351612  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1633) took 2504 ms. Trace:
I20260709 12:24:35.357968 10677 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:35.358023  9943 rpcz_store.cc:276] 0709 12:24:32.847596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.665074 (+817478us) service_pool.cc:224] Handling call
0709 12:24:34.130475 (+465401us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:34.154064 (+ 23589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.351306 (+1197242us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54001a31d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:35.351518 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1069429504}
I20260709 12:24:35.285130 12776 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:35.354789 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63486us
I20260709 12:24:35.338296 12524 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.282567 12827 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.355540 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.278448 12915 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:35.258621 13003 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.297190 12816 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:35.360934 12953 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.841s	user 0.013s	sys 0.000s
I20260709 12:24:35.278625 12924 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.356653 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:35.362414 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:35.390868 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:35.371094 12450 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:35.372812 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2590) took 3435 ms. Trace:
I20260709 12:24:35.379918 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.353900 12923 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.381091 12915 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.385285 13003 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:35.385919 10090 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:35.379731 12827 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:35.379518 12524 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:35.411219 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:35.411940 10024 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.358182 12822 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:35.403640 10093 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:35.417523 10093 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:35.419273 10093 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:35.420780  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 122107us
I20260709 12:24:35.390034 12953 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:35.394083 12816 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:35.426578 10093 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:35.401566 12853 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:35.436038 10024 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:35.438053 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45070us
I20260709 12:24:35.438699 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.439857 10024 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:35.440667 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:35.441022 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.441512 12776 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:35.441988 12776 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:35.442397 12776 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.442489 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:35.443367 12917 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:35.166311 12901 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Committing config change with OpId 5.5: config changed from index 3 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:35.419713 12827 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.419922 12524 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:35.442809 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.422190 12822 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.424710 13003 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
W20260709 12:24:35.472384 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.420116 13005 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 12:24:35.451944 12924 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:35.459177 12958 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.453078 12776 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.451983 10677 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:24:35.452998 12820 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.706s	user 0.006s	sys 0.004s
I20260709 12:24:35.444038 13031 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 12:24:35.484277 12776 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.463748 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:35.403595 12726 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:24:35.465361 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1549) took 3942 ms. Trace:
W20260709 12:24:35.469125  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.467142 12700 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.474246 10024 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:35.522918 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.523172 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 640) took 1992 ms. Trace:
I20260709 12:24:35.523736 10024 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:35.525056 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.429971 12816 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:35.526139 12816 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.395733 10025 rpcz_store.cc:276] 0709 12:24:31.937456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.408547 (+1471091us) service_pool.cc:224] Handling call
0709 12:24:33.408921 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.409183 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:33.455882 (+ 46699us) log.cc:844] Serialized 14 byte log entry
0709 12:24:35.287715 (+1831833us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b180037b160. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:24:35.370164 (+ 82449us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:35.370542 (+   378us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:35.370767 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.371642 (+   875us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.372297 (+   655us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101148,"prepare.queue_time_us":45729,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":17534,"raft.queue_time_us":304,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":121,"spinlock_wait_cycles":6220032,"thread_start_us":81238,"threads_started":1,"wal-append.queue_time_us":1935790}
W20260709 12:24:35.481372 13024 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:24:35.481807 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:35.482143 13019 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:35.485594 10020 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 12:24:35.486786  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.492926 10676 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:35.490417 10019 rpcz_store.cc:276] 0709 12:24:31.522874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:32.966844 (+1443970us) service_pool.cc:224] Handling call
0709 12:24:32.976872 (+ 10028us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.225933 (+249061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.225964 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.226439 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.502943 (+276504us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:33.655337 (+152394us) log.cc:844] Serialized 14 byte log entry
0709 12:24:33.655548 (+   211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.464980 (+1809432us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b540038dbd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:35.465033 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.465077 (+    44us) raft_consensus.cc:1685] finished
0709 12:24:35.465173 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.465298 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":131506,"prepare.queue_time_us":178012,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":116012,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":68777,"spinlock_wait_cycles":571924864,"thread_start_us":276066,"threads_started":1,"wal-append.queue_time_us":276371,"wal-append.run_cpu_time_us":3198,"wal-append.run_wall_time_us":1175098}
I20260709 12:24:35.466908 12923 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:35.477334 12819 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:35.501264 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.549623 10016 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:35.526585 12622 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.001s
W20260709 12:24:35.503872  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 82354.2us
I20260709 12:24:35.504570 13033 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:35.504794 12845 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:35.508944 12936 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.630s	user 0.007s	sys 0.001s
I20260709 12:24:35.510908 13008 raft_consensus.cc:1064] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:24:35.478434 13005 consensus_queue.cc:1243] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:24:35.517304 10640 connection.cc:256] RPC call timeout handler was delayed by -1.4458392850001474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1636s for a response
I20260709 12:24:35.474308 12865 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:34.540396 10098 rpcz_store.cc:276] 0709 12:24:30.347620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:30.373240 (+ 25620us) service_pool.cc:224] Handling call
0709 12:24:32.329080 (+1955840us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:33.600218 (+1271138us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:33.600408 (+   190us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:33.600625 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:33.600919 (+   294us) log.cc:844] Serialized 14 byte log entry
0709 12:24:34.539399 (+938480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:34.539468 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:34.539518 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:34.539590 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:34.539754 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":565831,"prepare.queue_time_us":31039,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":575653,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":149,"spinlock_wait_cycles":43670144,"thread_start_us":360059,"threads_started":1,"wal-append.queue_time_us":571998}
I20260709 12:24:35.521641 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.527684 10669 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:35.528498 10096 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:35.523571 10671 rpcz_store.cc:276] 0709 12:24:33.530714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.638122 (+107408us) service_pool.cc:224] Handling call
0709 12:24:33.654652 (+ 16530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.001338 (+346686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.522950 (+1521612us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:35.523112 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":778945408}
I20260709 12:24:35.614950  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:35.618479  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:35.533306 12822 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:35.501859 12952 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:35.552132 10016 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:24:35.539142 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:35.536875 12947 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.768s	user 0.010s	sys 0.000s
I20260709 12:24:35.534901 10091 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:35.567152 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.566463 12523 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:35.529109 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2589) took 3592 ms. Trace:
I20260709 12:24:35.669629 10022 rpcz_store.cc:276] 0709 12:24:31.937132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.053791 (+1116659us) service_pool.cc:224] Handling call
0709 12:24:33.054196 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:33.300793 (+246597us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:33.300840 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:33.330728 (+ 29888us) log.cc:844] Serialized 26 byte log entry
0709 12:24:33.955095 (+624367us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:33.955154 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.528710 (+1573556us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b54001c5f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:35.528754 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:35.528811 (+    57us) raft_consensus.cc:1685] finished
0709 12:24:35.528825 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:35.529046 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11333,"raft.queue_time_us":67676,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4527,"spinlock_wait_cycles":539992320,"thread_start_us":545366,"threads_started":1,"wal-append.queue_time_us":799454,"wal-append.run_cpu_time_us":3265,"wal-append.run_wall_time_us":1366736}
I20260709 12:24:35.672925 10022 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:35.534832 12816 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:35.675437 12816 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:35.578258 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.656740 12969 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.676415 10090 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:35.676653 12969 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.677621 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.679221 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.680413 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.681028 12572 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:35.681715 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.688004 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.689531 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.690109 10097 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:35.690963 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.580878 13028 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:35.575302 13033 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:35.577993 12845 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:35.547379  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 137629us
I20260709 12:24:35.631400 10023 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:35.616736  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:35.617374 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.615834  9948 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:35.569218 12700 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:35.579095 12787 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.001s
I20260709 12:24:35.661948  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:35.664199 10025 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:35.582301 13027 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:35.568718 12819 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.660351 12912 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:35.666478 12947 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:35.667979 10016 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:35.668144 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.668910 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:35.676083 13036 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:24:35.655656 10640 connection.cc:256] RPC call timeout handler was delayed by -1.3207391639998605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2837s for a response
I20260709 12:24:35.673951 12524 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:35.577423 12622 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:35.691449 10097 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:35.801600 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:35.865105 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63593.4us
I20260709 12:24:35.865684 12450 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:35.866004 13005 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:35.866227 12450 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.866631 13005 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:35.866778 13027 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:35.867621 13005 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:35.869444 13027 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.757859  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:35.878463  9943 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:35.789961 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:35.878996  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 121366us
I20260709 12:24:35.814567 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:35.880398  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:35.882397  9945 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:35.883968 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:35.833894 10022 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:35.884707 10023 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:35.734103 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:35.886847  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:35.871505  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:35.798653 10021 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:35.878578 10640 connection.cc:256] RPC call timeout handler was delayed by -1.2550392850002936s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 12:24:35.656955 13008 raft_consensus.cc:1068] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Unable to promote non-voter 6c7bd3fa1bfa431c89fc9c9096452a76: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:24:35.898810 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:35.899528 10017 pending_rounds.cc:85] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 3
W20260709 12:24:35.844671 12810 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.728s	user 0.011s	sys 0.000s
I20260709 12:24:35.899504 10022 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:35.900359 10017 pending_rounds.cc:107] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:24:35.900171  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:35.904066 10017 raft_consensus.cc:2889] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:35.901280 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:35.882390 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:35.903687 10024 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:35.905540 12810 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.907866 12810 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:35.661283 12952 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:35.864503 12893 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.616144 12936 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.781522 12969 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:35.908493 12622 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:35.697943 12923 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:35.849871 12827 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.892623 12699 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:24:35.878024 13036 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:35.830155 12816 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:35.734627 12572 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:35.923043 10096 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:35.892880 13027 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:35.927297  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:35.956336  9943 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
W20260709 12:24:35.956615  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56521.5us
I20260709 12:24:35.965942 12523 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:35.958722 10097 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:35.968798 12938 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:35.918139 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:35.904542 12954 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:35.967998 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:36.035351  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 148594us
I20260709 12:24:35.660349 12865 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.977751 10023 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:35.899344 13035 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:35.908887 13032 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:35.851502 12915 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.954847 12917 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:35.897300 12900 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:24:35.903214 12861 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:24:35.893389 13028 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:35.892696 12912 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.022251 10022 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:36.032135  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:36.062445  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:36.064116 12810 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:36.064189  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:36.064973 12810 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:36.065373 12523 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:36.065804 12523 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:35.957679 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 75362.5us
I20260709 12:24:36.066874 12969 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:36.067292 12969 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:35.968425 13031 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:36.066156 12523 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.067689 12969 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.067919 13031 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.070796 13038 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:36.072489 13038 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:24:36.039724  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.043382 10098 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:36.036760 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 118696us
I20260709 12:24:35.898253 12700 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.075484 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:36.070320 13031 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:36.077831 12700 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.047555 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:36.080112 10669 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
W20260709 12:24:36.051218  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 162624us
I20260709 12:24:36.058754 10674 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:36.083011  9987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.957s (SENT) [suppressed 23 similar messages]
I20260709 12:24:36.084513 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.085834 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.084966 10668 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 3 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:36.086479 10023 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:36.087024 10022 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:36.087491 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.088161 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.089082 10021 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:24:36.090377  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:36.036808 12938 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:36.092031 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:36.066151 12810 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.039945 12699 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:36.036264 12954 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.092517  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:36.098606 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:36.099468 10676 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.088173 10023 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:36.046228 13032 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.103984 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:36.106674  9912 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:36.034466 12572 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.034919 12816 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.118114 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:36.105603 13022 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:36.003739 12845 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.101972 12954 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.096150 12699 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:36.127532 12699 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:36.127985 12947 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.103433 13032 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:36.045774 12952 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.129515 12952 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:36.131174 13042 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:24:36.031024 13033 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.103121 12700 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:36.136317 12969 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.137036 12815 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.140844 10669 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:36.096616 12938 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:36.143945 12954 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.144644 12749 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.138444 12969 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.110464 13021 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:36.092593 12917 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:36.168736  9990 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:36.178006  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:36.179262  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:36.126772  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:36.193704  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:36.194525  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:36.191500  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64832.5us
I20260709 12:24:36.199832  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:36.200604  9943 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:36.201401 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.201737  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:36.202756  9945 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:36.217092 10023 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:36.204799  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:36.222132  9945 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:36.223692  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:36.149703 12822 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:36.251320  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:36.243675  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79976us
I20260709 12:24:36.245719 10676 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:36.196237 13046 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:36.197743 12820 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:36.199177 12516 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:36.228560  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:36.196010 13022 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:36.250468  9943 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:36.193763 12947 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:24:36.253504  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59879.5us
I20260709 12:24:36.252833  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:36.274076  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:36.260967 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:36.203755 12954 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:36.266711 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 199464us
I20260709 12:24:36.147958 12450 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:36.252317 13040 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 12:24:36.276178  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:36.192446 12523 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.278532 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:36.281226 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 205811us
I20260709 12:24:36.251227 12524 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:36.272575 13046 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:36.296829  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:36.297600 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.275390 13022 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.095502 13036 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.274091 12787 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:36.278059 12936 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:36.321137  9947 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:36.276115 12516 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:36.324381 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:36.342252  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45493.2us
I20260709 12:24:36.343327  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:36.299991 12893 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:36.305294 13022 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:36.350555  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 120938us
I20260709 12:24:36.353994  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:36.290441 12524 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:24:36.387538 10676 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:36.320539 12787 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:36.301838 12958 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:24:36.303920 13052 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
W20260709 12:24:36.395752  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 121735us
I20260709 12:24:36.352108 12947 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:36.402438  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48524.2us
W20260709 12:24:36.391587 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 100288us
I20260709 12:24:36.398931  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:36.351703 13045 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.349056 12953 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:36.355366 12969 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:36.320221 12822 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
W20260709 12:24:36.384223 13040 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:24:36.412452  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:36.340375 12749 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 12:24:36.414765  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:36.346267 12936 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:24:36.413081  9948 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:36.300483 12917 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.380090 12915 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.386029 12900 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:36.420979  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:36.365023 13050 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:36.390636 12787 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.417080  9945 pending_rounds.cc:85] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 5
I20260709 12:24:36.418965 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.419651  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:36.429529  9945 pending_rounds.cc:107] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 3.6
W20260709 12:24:36.422717 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78731.7us
I20260709 12:24:36.423766  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:36.426276  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:36.429905  9945 raft_consensus.cc:2889] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:36.431329  9941 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:36.435643 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.411017 12922 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:36.410642 12622 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:36.443418 10023 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:36.412621 12865 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:36.322722 13032 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.398537 13063 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:36.427217 12900 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:36.464649  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:36.479763  9949 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:36.479152 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.426155 12816 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:36.465047  9944 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:36.484699  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85850.7us
I20260709 12:24:36.463922 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:36.471752  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:36.485476  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:36.476473 12523 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.477247 13032 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.485584 10022 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:36.421229 12912 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.490025 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:36.490553  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:36.469048  9948 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:36.471531 13065 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:36.417486 12969 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.519235 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:36.484073 12900 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:36.487118 12952 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.420866 13055 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260709 12:24:36.481819 12622 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:36.528270 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:36.495111 13050 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:36.481824 12749 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:24:36.533124 12938 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.435150 12516 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.500195 13065 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:36.531287 13056 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:36.543732 10022 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:36.524715 13021 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:36.523171 13072 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:36.566427 10670 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:24:36.526686 13032 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:36.534538 12912 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:36.482040 13068 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.083s
W20260709 12:24:36.558756 13049 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.257s	user 0.002s	sys 0.000s
I20260709 12:24:36.586627 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:36.591259 10673 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:24:36.561060 13043 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:24:36.429626 13046 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.605067 12062 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.605585 12062 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.604892 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:36.599792 12922 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:36.610464 10673 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:36.620023 13027 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.580s	user 0.010s	sys 0.001s
I20260709 12:24:36.630306 13027 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:36.631062 12776 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:36.631520 12776 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:36.632391 12776 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.586303 12912 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:36.634900 13053 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.223s	user 0.000s	sys 0.001s
I20260709 12:24:36.635121 12912 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.622068 12922 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:36.641453 13071 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:24:36.536275 12952 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:36.634948 12699 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.002s
I20260709 12:24:36.661413 12699 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.663756 12699 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:36.642427 13061 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:24:36.619500 12450 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.678885 12622 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.588263 13072 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.587359 13032 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.713766 12856 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:36.714401 12856 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:36.714771 12912 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.716006 12856 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.717391 12856 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:36.703377 13032 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.671851 13065 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.605055 13050 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:36.924224 12912 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:36.931593 12815 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:36.844054 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:36.923346 12953 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.703410 12900 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.932893 12815 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.932343 12912 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.933583 12953 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.935559 10676 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:36.925555 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.934988 12912 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:36.732771 12952 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:36.938395 12953 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:36.890980 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2693) took 1139 ms. Trace:
I20260709 12:24:36.939307 12953 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:36.875110 10097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.005s	sys 0.014s
I20260709 12:24:36.941151 12912 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:36.939252 10098 rpcz_store.cc:276] 0709 12:24:35.751140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.073810 (+322670us) service_pool.cc:224] Handling call
0709 12:24:36.074157 (+   347us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:24:36.074550 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.074576 (+    26us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:24:36.075262 (+   686us) log.cc:844] Serialized 78 byte log entry
0709 12:24:36.300033 (+224771us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:36.300078 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.300163 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.739824 (+439661us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b2400223988. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a7fb0 00007f4e0911e488 00007f4e0911ebe8 00007f4e090f6398 00007f4e090f176a 00007f4e090f0bb6
0709 12:24:36.890002 (+150178us) raft_consensus.cc:1685] finished
0709 12:24:36.890053 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.890709 (+   656us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":169919,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":15642,"spinlock_wait_cycles":5398784,"thread_start_us":224379,"threads_started":1,"wal-append.queue_time_us":321399}
W20260709 12:24:36.941500 10097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.009s	sys 0.021s
I20260709 12:24:36.841488  9940 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
W20260709 12:24:36.897804 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.850901 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1576) took 1030 ms. Trace:
I20260709 12:24:36.897938  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:24:36.845890  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:36.889631 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 417) took 2043 ms. Trace:
I20260709 12:24:36.890281 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:37.114727 10668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.003s	sys 0.011s
I20260709 12:24:36.915697 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:36.901072 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:36.865542 10090 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:36.931885 12893 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:36.940267 12953 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:37.092931 10642 leader_election.cc:336] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.608s (SENT)
W20260709 12:24:37.139617  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 241993us
I20260709 12:24:37.141317  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.142666 12516 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:37.143559 12516 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:37.145423 12516 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:37.146294 12947 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.734s	user 0.002s	sys 0.018s
W20260709 12:24:37.146431 10064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.005s	sys 0.013s
I20260709 12:24:37.118732 10025 rpcz_store.cc:276] 0709 12:24:35.819964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.887587 (+ 67623us) service_pool.cc:224] Handling call
0709 12:24:35.887917 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:36.038426 (+150509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.038470 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:36.039067 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 12:24:36.350850 (+311783us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:36.350886 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.350957 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.850180 (+499223us) raft_consensus.cc:1685] finished
0709 12:24:36.850234 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.850636 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":653,"raft.queue_time_us":3532,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":6849,"thread_start_us":394939,"threads_started":2,"wal-append.queue_time_us":392632}
I20260709 12:24:37.150374  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.152232 12900 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:37.133688 10668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.004s	sys 0.012s
W20260709 12:24:37.156401 10064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.005s	sys 0.013s
I20260709 12:24:37.156641 10021 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:37.170670 12815 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.170874  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.171721 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2694) took 1418 ms. Trace:
I20260709 12:24:37.160100 10671 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.121824 10095 rpcz_store.cc:276] 0709 12:24:34.845935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.851136 (+  5201us) service_pool.cc:224] Handling call
0709 12:24:35.675501 (+824365us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:24:35.675711 (+   210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:35.681439 (+  5728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.681469 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:35.682001 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:24:35.976048 (+294047us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:35.976091 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:35.976158 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:36.875094 (+898936us) raft_consensus.cc:1685] finished
0709 12:24:36.875144 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:36.889219 (+ 14075us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5300,"raft.queue_time_us":6961,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"spinlock_wait_cycles":4230016,"thread_start_us":293654,"threads_started":1,"wal-append.queue_time_us":539083}
I20260709 12:24:36.912937 12450 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.268s
I20260709 12:24:37.200383 13036 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:37.172945 12969 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:37.204046 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1565) took 2092 ms (client timeout 1922 ms). Trace:
I20260709 12:24:37.204921  9949 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:24:37.205746 13048 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:37.205883 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:37.207901 12953 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.204116  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:37.208621 13045 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.790s	user 0.002s	sys 0.024s
I20260709 12:24:37.209790 12952 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.211316  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.227267 12952 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.228288  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.229598 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2669) took 4116 ms. Trace:
I20260709 12:24:37.229915 12952 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.214475  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:37.216738 10020 rpcz_store.cc:269] 0709 12:24:35.111278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.245971 (+134693us) service_pool.cc:224] Handling call
0709 12:24:37.203567 (+1957596us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b5400216bd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:37.203810 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":548040960}
I20260709 12:24:37.186257 13072 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:37.172678 10097 rpcz_store.cc:276] 0709 12:24:35.753472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.079107 (+325635us) service_pool.cc:224] Handling call
0709 12:24:36.079373 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.079564 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:36.345348 (+265784us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.170355 (+825007us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:37.170568 (+   213us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:37.170788 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.171008 (+   220us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.171625 (+   617us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":493724,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":20797,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":256,"thread_start_us":1039860,"threads_started":2}
W20260709 12:24:37.222153 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.175333 12947 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:37.220273  9942 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:37.229861  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.213255 12820 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:37.218681 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.007s	sys 0.014s
I20260709 12:24:37.233560  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.231835 10099 rpcz_store.cc:276] 0709 12:24:33.112969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:33.657723 (+544754us) service_pool.cc:224] Handling call
0709 12:24:34.539720 (+881997us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.551685 (+ 11965us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.229198 (+2677513us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b5400341050. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:37.229425 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1995026688}
I20260709 12:24:37.239976  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:37.231986  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.011s	sys 0.016s
W20260709 12:24:37.241111 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.008s	sys 0.014s
W20260709 12:24:37.244690 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 966158us
I20260709 12:24:37.247015 12820 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.248871 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:37.249203 12774 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.250213 12853 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:37.251514 12776 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.252301 12853 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:37.255046  9941 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.255461 12819 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.257037  9941 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.259479 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:37.262713 13045 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.264814 13035 raft_consensus.cc:1064] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: attempting to promote NON_VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 to VOTER
I20260709 12:24:37.266000 12853 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:37.266618 12853 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:37.268384 12938 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:37.268066 12853 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.269404 12938 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:37.278849 13086 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:37.280752  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.794s	user 0.013s	sys 0.016s
I20260709 12:24:37.283027 10021 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:37.226233 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:37.191032 12893 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.223018 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 746) took 1131 ms. Trace:
I20260709 12:24:37.297775 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:37.240118 13048 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.246304 12572 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:37.311753 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 418) took 2465 ms (client timeout 2837 ms). Trace:
I20260709 12:24:37.287393 13086 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:37.312216 10092 rpcz_store.cc:269] 0709 12:24:34.848767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.848890 (+   123us) service_pool.cc:224] Handling call
0709 12:24:37.225698 (+2376808us) spinlock_profiling.cc:243] Waited 1.55 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:37.311640 (+ 85942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3557120}
I20260709 12:24:37.316534 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:37.323419  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 182281us
I20260709 12:24:37.314694 13086 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.332834 10671 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:37.333870 10020 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:37.335088 10025 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:37.339520 12924 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:24:37.337857  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.391141 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:37.394821 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1577) took 1517 ms. Trace:
I20260709 12:24:37.395459 10017 rpcz_store.cc:276] 0709 12:24:35.820254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.889300 (+ 69046us) service_pool.cc:224] Handling call
0709 12:24:35.899340 (+ 10040us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:35.905140 (+  5800us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:35.905168 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:35.909793 (+  4625us) log.cc:844] Serialized 52 byte log entry
0709 12:24:36.093078 (+183285us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:36.283363 (+190285us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.197984 (+914621us) spinlock_profiling.cc:243] Waited 8.6 ms on lock 0x7b18003a4560. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:24:37.234561 (+ 36577us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.321601 (+ 87040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.330505 (+  8904us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.334755 (+  4250us) raft_consensus.cc:1685] finished
0709 12:24:37.334775 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.337631 (+  2856us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39228,"prepare.queue_time_us":2025769,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":88,"raft.queue_time_us":26910,"raft.run_cpu_time_us":517,"raft.run_wall_time_us":41180,"spinlock_wait_cycles":19769216,"thread_start_us":319519,"threads_started":2,"wal-append.queue_time_us":305030}
W20260709 12:24:37.342876 12936 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.879s	user 0.006s	sys 0.003s
W20260709 12:24:37.399003 12936 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.006s	sys 0.005s
I20260709 12:24:37.399638 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 411) took 2579 ms. Trace:
W20260709 12:24:37.401023 13064 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.874s	user 0.001s	sys 0.000s
I20260709 12:24:37.400097 10094 rpcz_store.cc:276] 0709 12:24:34.820056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.820175 (+   119us) service_pool.cc:224] Handling call
0709 12:24:34.831092 (+ 10917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:34.831298 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:35.042476 (+211178us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.228593 (+2186117us) spinlock_profiling.cc:243] Waited 3.77 ms on lock 0x7b180034b1c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:24:37.350685 (+122092us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:37.368303 (+ 17618us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:37.399133 (+ 30830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.399244 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.399440 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4564,"prepare.queue_time_us":181125,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":5555,"raft.queue_time_us":19833,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":52995,"spinlock_wait_cycles":8675584,"thread_start_us":234190,"threads_started":2}
I20260709 12:24:37.402760 12820 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:37.403228 10090 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2687) took 1821 ms (client timeout 2188 ms). Trace:
W20260709 12:24:37.403684 10090 rpcz_store.cc:269] 0709 12:24:35.581624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.678897 (+ 97273us) service_pool.cc:224] Handling call
0709 12:24:37.402883 (+1723986us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400094554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:37.403048 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2725463808}
I20260709 12:24:37.413909  9944 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:37.414770 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 322095us
W20260709 12:24:37.416316 10642 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.752s (SENT)
W20260709 12:24:37.417857 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 81016.9us
I20260709 12:24:37.420001 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:37.420130 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:37.420400 10018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 432) took 2873 ms (client timeout 2974 ms). Trace:
I20260709 12:24:37.420885 10092 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:37.421095 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:37.421610 13076 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:37.422322 10099 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:37.423027 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:37.422384 13076 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.351068 13035 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.426864 13072 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:37.428297 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2600) took 2331 ms. Trace:
I20260709 12:24:37.428452 13072 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.428880 13072 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:37.430127 13072 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.431767 13072 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.422226  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1676) took 1216 ms. Trace:
I20260709 12:24:37.435676  9947 rpcz_store.cc:276] 0709 12:24:36.205649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.205800 (+   151us) service_pool.cc:224] Handling call
0709 12:24:37.421688 (+1215888us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b540009bad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:37.421998 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158720896}
I20260709 12:24:37.438529 12947 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.439317 12947 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.439949 10022 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:37.440632 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 438) took 1274 ms. Trace:
I20260709 12:24:37.440914 10022 rpcz_store.cc:276] 0709 12:24:36.166544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.438987 (+1272443us) service_pool.cc:224] Handling call
0709 12:24:37.439463 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.440385 (+   922us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.440556 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.441561 12947 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.441814 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:37.442838 10096 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:37.426440 13076 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:37.444856 10674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 632) took 2265 ms (client timeout 2948 ms). Trace:
W20260709 12:24:37.445282 10674 rpcz_store.cc:269] 0709 12:24:35.179539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.183152 (+  3613us) service_pool.cc:224] Handling call
0709 12:24:37.444437 (+2261285us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b5400615b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:37.444649 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011307392}
I20260709 12:24:37.447278 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:37.447492 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:37.420849 10094 raft_consensus.cc:2376] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6. Re-sending same reply.
I20260709 12:24:37.449301 10095 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:24:37.450922 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:37.451807 10096 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:37.421022 10018 rpcz_store.cc:269] 0709 12:24:34.546514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:34.548201 (+  1687us) service_pool.cc:224] Handling call
0709 12:24:37.419965 (+2871764us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b540038e5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:37.420210 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894592768}
I20260709 12:24:37.452481 12062 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:37.454244 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:37.442692 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:37.455873 12827 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.456665 10090 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:37.437209  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.459193  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.460053 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:37.460784 10099 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.441398 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:37.428730 10019 rpcz_store.cc:276] 0709 12:24:35.096476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.549249 (+452773us) service_pool.cc:224] Handling call
0709 12:24:35.562578 (+ 13329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:35.903514 (+340936us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:36.162948 (+259434us) log.cc:844] Serialized 14 byte log entry
0709 12:24:36.388848 (+225900us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:36.388943 (+    95us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:36.389036 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.400053 (+1011017us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:37.427821 (+ 27768us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54002d5d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:37.427885 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.428112 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":838832,"prepare.queue_time_us":396057,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":3554,"raft.queue_time_us":26448,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8170,"spinlock_wait_cycles":786801792,"thread_start_us":455706,"threads_started":2,"wal-append.queue_time_us":315936}
I20260709 12:24:37.463343 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:37.464509 12958 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.464951 12958 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:37.465104 12900 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:37.465207 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:37.465771 12900 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.466250 12900 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.466490 12856 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:37.468246 12856 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:37.472441 12979 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:37.473143 12979 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:37.474118 12922 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.835s	user 0.010s	sys 0.019s
I20260709 12:24:37.473327 10019 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:37.475160 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2621) took 1080 ms. Trace:
W20260709 12:24:37.475840 13051 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.982s	user 0.001s	sys 0.000s
I20260709 12:24:37.450071 10022 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 3. Current term is 4. Ops: []
I20260709 12:24:37.477898 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2686) took 1898 ms. Trace:
I20260709 12:24:37.478324 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2618) took 1093 ms. Trace:
I20260709 12:24:37.479899 10022 rpcz_store.cc:276] 0709 12:24:36.385235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.441775 (+1056540us) service_pool.cc:224] Handling call
0709 12:24:37.449400 (+  7625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.477612 (+ 28212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.478172 (+   560us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.480605 10020 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:37.481292 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:37.482165 10017 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.475531 10019 rpcz_store.cc:276] 0709 12:24:36.394938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.464174 (+1069236us) service_pool.cc:224] Handling call
0709 12:24:37.472772 (+  8598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.474882 (+  2110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.475081 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:37.489100 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:37.472451  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 987006us
I20260709 12:24:37.478271 10093 rpcz_store.cc:276] 0709 12:24:35.578901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.870677 (+291776us) service_pool.cc:224] Handling call
0709 12:24:35.871061 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:35.872879 (+  1818us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:35.872913 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:35.888589 (+ 15676us) log.cc:844] Serialized 26 byte log entry
0709 12:24:36.198306 (+309717us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:36.198349 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.198419 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.263103 (+1064684us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:24:37.397597 (+134494us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:37.477477 (+ 79880us) raft_consensus.cc:1685] finished
0709 12:24:37.477507 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.477712 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1328,"raft.queue_time_us":1445,"raft.run_cpu_time_us":3649,"raft.run_wall_time_us":1390068,"spinlock_wait_cycles":160929408,"thread_start_us":249474,"threads_started":1,"wal-append.queue_time_us":447471}
I20260709 12:24:37.499938 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:37.500541 10092 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:37.502846 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 148153us
I20260709 12:24:37.506729 10025 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.513487 10090 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:37.496357 13022 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.905s	user 0.014s	sys 0.012s
W20260709 12:24:37.519574 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.522580 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57494.5us
I20260709 12:24:37.535269 12953 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.536360 10019 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:37.536108 12953 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.536525  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:37.537149 13094 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:37.538013 13094 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.537206  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.536612  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.545645  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.546372 13094 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.546665 13048 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.536135 12952 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.547199  9941 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.547523 12952 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.548874 12958 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.550233 12958 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.550663 13076 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:37.551121  9944 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 4: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:37.552382  9944 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.494401 12922 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.554858 12953 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.555603 12922 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.556717 12952 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.557572 12865 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.557983 12865 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:37.548914  9941 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.568069  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.568902 13073 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:37.569219 12917 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.021s	user 0.011s	sys 0.005s
I20260709 12:24:37.569482 13073 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:37.509308 10674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:24:37.575188 10674 raft_consensus.cc:606] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:37.577773 12787 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.061s	user 0.014s	sys 0.011s
I20260709 12:24:37.578349 12787 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.580868 12787 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.581658 10020 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.582576  9946 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 7. Ops: []
I20260709 12:24:37.585436 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1568) took 2473 ms. Trace:
I20260709 12:24:37.585831 10024 rpcz_store.cc:276] 0709 12:24:35.111995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.524693 (+412698us) service_pool.cc:224] Handling call
0709 12:24:37.585047 (+2060354us) spinlock_profiling.cc:243] Waited 378 ms on lock 0x7b54001c5a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:37.585268 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868837248}
I20260709 12:24:37.581715 12820 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:24:37.595697 12700 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.596148 12700 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:37.569690 12917 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.092s	user 0.011s	sys 0.005s
I20260709 12:24:37.597150 12700 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.568934  9945 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.605118 10017 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:37.607694 10096 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:37.609582 10092 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
W20260709 12:24:37.618005 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.600481 12700 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:37.620393 10642 leader_election.cc:336] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.884s (SENT)
I20260709 12:24:37.623224  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.624019  9946 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:37.626240 13095 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:24:37.626884 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67616.9us
I20260709 12:24:37.627861 12893 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.621789 10022 pending_rounds.cc:85] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 7
W20260709 12:24:37.629184 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.629096 10022 pending_rounds.cc:107] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:24:37.629671 10022 raft_consensus.cc:2889] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:37.559151 12865 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.632367 12865 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.634920 13028 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.634418 13073 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.294261 10668 rpcz_store.cc:276] 0709 12:24:36.084099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.084230 (+   131us) service_pool.cc:224] Handling call
0709 12:24:36.084571 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:36.084822 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:36.534039 (+449217us) log.cc:844] Serialized 14 byte log entry
0709 12:24:37.213739 (+679700us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:37.213851 (+   112us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:37.214766 (+   915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.214885 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.215172 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26786,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":229,"thread_start_us":623771,"threads_started":1,"wal-append.queue_time_us":790665}
I20260709 12:24:37.636616 13073 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:37.640995  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.644835  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.645381  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.645607 13066 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.057s	user 0.002s	sys 0.000s
I20260709 12:24:37.648236 13065 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.648721 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1575) took 1829 ms. Trace:
I20260709 12:24:37.649122 10016 rpcz_store.cc:276] 0709 12:24:35.819685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:35.886977 (+ 67292us) service_pool.cc:224] Handling call
0709 12:24:35.919811 (+ 32834us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:36.176626 (+256815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:36.176681 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:36.180334 (+  3653us) log.cc:844] Serialized 52 byte log entry
0709 12:24:36.401933 (+221599us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:24:36.401967 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:36.402032 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.644806 (+1242774us) spinlock_profiling.cc:243] Waited 936 us on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:37.648255 (+  3449us) raft_consensus.cc:1685] finished
0709 12:24:37.648278 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.648559 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3624,"raft.queue_time_us":46825,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":30042,"spinlock_wait_cycles":2151680,"thread_start_us":404069,"threads_started":2,"wal-append.queue_time_us":358586}
I20260709 12:24:37.651196  9946 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:37.651811  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 555) took 1233 ms. Trace:
I20260709 12:24:37.652166  9940 rpcz_store.cc:276] 0709 12:24:36.418942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.419152 (+   210us) service_pool.cc:224] Handling call
0709 12:24:36.419446 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.128654 (+709208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.651500 (+522846us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b54000b7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:37.651712 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":968308992}
I20260709 12:24:37.527854 13022 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.656327 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:37.622560 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:37.658107 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:37.667124  9943 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:37.674413  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.678633 10025 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:37.678922 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:37.679715 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60373.8us
I20260709 12:24:37.680858 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:37.687023  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.690495  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.688561 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:37.678350 12969 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.696472 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:37.701002  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.701828  9944 raft_consensus.cc:2376] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8. Re-sending same reply.
I20260709 12:24:37.703138 13055 raft_consensus.cc:2764] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:37.703317 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:37.704063 10025 raft_consensus.cc:606] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:37.701046 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:37.706007 10674 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:37.697546 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:37.707027 12969 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.707475 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:37.708236 10668 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:37.709376 12969 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.711920  9942 raft_consensus.cc:2410] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:37.712299  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:37.713243  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.713994  9940 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:37.715044 12787 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:37.715745 12787 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:37.716765 12787 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.697108 10672 raft_consensus.cc:2376] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7. Re-sending same reply.
I20260709 12:24:37.719827 12622 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.720011  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.720340 12622 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:37.721472 12622 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.714982  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.701197 12523 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.672780 12900 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.721490  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.693130 13022 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:37.728089  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 191819us
I20260709 12:24:37.731153 10099 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:37.733757 10016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:37.734463 10016 raft_consensus.cc:606] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:37.734817 10016 raft_consensus.cc:614] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:37.735227  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.722494 12622 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.684180 12979 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.738762  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.688251 13081 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:37.743809 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:37.736079  9944 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:37.740159  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:37.744374 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59047us
I20260709 12:24:37.746184  9940 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:37.747900  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.749406  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.751282  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.750808 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:37.757304 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:37.758054 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 142570us
I20260709 12:24:37.760705 10676 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:37.761708 10019 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:37.766690 10671 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:37.767408  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:37.769723 13050 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.771391 10018 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:37.733418 12523 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.767164 10672 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:37.763096  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:37.748289 12516 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:37.776849 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 441) took 1609 ms (client timeout 1633 ms). Trace:
W20260709 12:24:37.777101 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.777281 10023 rpcz_store.cc:269] 0709 12:24:36.167446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:36.275477 (+108031us) service_pool.cc:224] Handling call
0709 12:24:37.776486 (+1501009us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54001392d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:37.776666 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333412608}
I20260709 12:24:37.778369 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:37.779855 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:37.781368 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:37.782100 10673 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:37.775106 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:37.778242 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:37.787604  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.788147  9945 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
W20260709 12:24:37.788453 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.793156  9944 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:37.783661 13055 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:37.794528  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.796907 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:37.798509  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.786756 13022 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:37.800814 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52548.7us
I20260709 12:24:37.803174 10674 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:37.792428 12917 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:37.805095  9940 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:37.770356 13050 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.800643 13081 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:37.810756  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.803097 13022 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:37.809823 12861 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:37.817162 12900 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.807591 12958 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:37.833256  9940 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:37.795213 12516 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.815853 13022 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.842525 12861 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:37.845045  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 96976.9us
I20260709 12:24:37.854789 12958 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.856457  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:37.857262 12958 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.858669 13028 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:37.858511 12861 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.859176 13028 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.859114 12969 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:37.859740 13028 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.860459 12969 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:37.861692 13008 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:24:37.862003 12938 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:24:37.862643  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.862062 13008 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:37.862972 13006 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:37.863943 13006 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:37.863983 13008 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.865262 13006 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:37.865496  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:37.866397  9948 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:37.866513 13003 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.867014 13003 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:37.862159 13055 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:24:37.867267 12062 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.867664 12062 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:37.867864 13055 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:37.868366 13055 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:37.868058 13003 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.868599 12062 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.869415 13003 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.869827 12062 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:37.869349 13055 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.871104  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.871825  9941 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:37.872156 12516 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:37.824353 13081 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:37.873387 13081 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.866881 12923 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.862864 12981 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:37.878947 12923 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:37.879113 12981 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:37.880052 12923 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.880205 12981 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.881405 12923 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.881632 12981 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:37.865659 13008 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:37.866149 13006 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:37.885270 13006 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 7.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:24:37.885898 12524 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:37.884387 12523 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.886300 12524 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:37.890755  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:37.899166  9940 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:37.907348  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:37.907862 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:37.916462 10677 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:37.918509 12524 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:37.920077 12524 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:37.926011 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:37.926564 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.899255 13055 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:37.928579  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:37.922641 12948 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:37.909615 13006 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:24:37.931250 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:37.934897 12523 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:37.957775  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:37.959055 12861 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:37.960443 12861 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:37.961679 12861 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.965788 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 445) took 1069 ms. Trace:
I20260709 12:24:37.966270 10022 rpcz_store.cc:276] 0709 12:24:36.896546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.491893 (+595347us) service_pool.cc:224] Handling call
0709 12:24:37.527957 (+ 36064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:37.658996 (+131039us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:37.659026 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:37.681775 (+ 22749us) log.cc:844] Serialized 26 byte log entry
0709 12:24:37.895275 (+213500us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:37.895313 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:37.895380 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:37.965541 (+ 70161us) raft_consensus.cc:1685] finished
0709 12:24:37.965560 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:37.965719 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":162142,"threads_started":1,"wal-append.queue_time_us":268713}
I20260709 12:24:37.972272  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:37.938941 12979 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:37.973945 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66163.9us
I20260709 12:24:37.974880 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:37.944083 12917 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:37.944380 12819 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:37.987329  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:37.994748 12523 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:38.018510 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43758.8us
I20260709 12:24:38.018244 12523 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.019330 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.019797 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:38.020179 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.020583 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.020869 10098 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:38.021272 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:38.021763 12776 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:24:38.022182 12776 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:38.022011 12900 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:38.022572 12776 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:24:38.024915 12827 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:38.025477 12827 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:37.981926 12819 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:37.999260  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:38.006850  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 118407us
I20260709 12:24:38.011081 10668 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:38.013411 13050 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.013074 12917 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.028383 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.018417 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:38.027076 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 103006us
I20260709 12:24:38.004067 13022 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:37.976449 12893 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:38.028366 12819 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.032224 13046 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:38.033635 12917 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:38.035491 12827 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:38.030545 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45342.1us
W20260709 12:24:38.058817 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.061319  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:38.061843 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.068403  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:38.031165 12865 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:38.068186 10093 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:38.075147 10090 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
W20260709 12:24:38.076284 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.041555 13046 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.040267 12893 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.080662  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:38.087248 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.093089  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:38.097378 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.100795 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.101651 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.081903 13046 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:38.081863 12893 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:38.105219 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85493.3us
I20260709 12:24:38.084503 12865 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:38.060249 13073 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:24:38.107261  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:38.082930 12622 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:24:38.114760 10093 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.113487  9942 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:38.117357 13081 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:38.117789 13081 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:38.120570  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 215825us
I20260709 12:24:38.123703 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:38.126655  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.128149  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.128641 13033 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.130005 12516 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:38.130692 13050 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:38.130924 12516 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.131117 13050 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:38.132205 13050 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.133450 13050 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:38.135672 12865 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.139932  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:38.141304  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:38.141922 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.129580  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.124207  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:38.144515  9940 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
W20260709 12:24:38.153196  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.174341 10093 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:38.174954 10090 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.173352 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:38.177531 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 449) took 1005 ms. Trace:
I20260709 12:24:38.177932 10019 rpcz_store.cc:276] 0709 12:24:37.172641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.537449 (+364808us) service_pool.cc:224] Handling call
0709 12:24:37.537758 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.762114 (+224356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.177307 (+415193us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400217ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:38.177476 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509640576}
I20260709 12:24:38.173763 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.185952 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.173451 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:38.188453 10023 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:38.188726 10025 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:38.190115 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.185870 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.190836 10016 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:38.194445 13081 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.195927 13081 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.199496 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:38.151209 12622 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:38.201443 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77798.2us
W20260709 12:24:38.202203 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.203361 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.204173 10093 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:38.205817 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.200258 12820 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.207896 13022 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.189704 13072 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:38.209445 13072 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:38.202148 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:38.210508 13072 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.118880 13028 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.218609 10020 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:38.223415  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.225670  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.225767 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:38.227536 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.231369 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.232370 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.228161 10674 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:38.211726 12622 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.235260 10677 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:38.243376 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.243978 10098 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:38.193750 13055 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:38.219825 12524 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.249037 10094 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:38.258162 13111 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:24:38.258975 13055 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.259883 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.260975 13113 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:38.261523 13113 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:38.262509 13113 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.264600 12900 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:24:38.265755 12900 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:38.266124 12900 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:38.266831  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 137375us
I20260709 12:24:38.268675  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.269477 12820 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:38.269858  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:38.270994 10091 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:38.272059 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.272256 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.273097 10095 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:38.273231 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.273414 12861 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:24:38.273909 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.274276 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.275074 10096 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:38.274276 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2619) took 1881 ms. Trace:
I20260709 12:24:38.275758 10018 rpcz_store.cc:276] 0709 12:24:36.393426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.452748 (+1059322us) service_pool.cc:224] Handling call
0709 12:24:37.771226 (+318478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:37.774871 (+  3645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.274078 (+499207us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54001f7050. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:38.274218 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730212736}
I20260709 12:24:38.281762 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:38.283041 13112 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 12:24:38.283381 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.283948 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.283087 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.285362 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.286145 10092 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:38.285914  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.286810 12948 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:38.289181 10090 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:38.290899  9944 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:38.284128 10094 raft_consensus.cc:2376] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8. Re-sending same reply.
I20260709 12:24:38.218367 13022 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:24:38.215600 12776 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.246443 12622 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:38.298348  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:38.289723 12787 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:38.275233 12516 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:38.307945 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:38.316426 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.313432 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.315152 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.261722 13055 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.317008 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.318542 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.318686 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.320957 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:38.322345  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.323323  9941 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:38.325572 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.326179 10677 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:38.289819 12819 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:38.332355  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 564) took 1491 ms (client timeout 1748 ms). Trace:
I20260709 12:24:38.333972  9949 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:38.335086 12062 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:38.335575 12062 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:38.343302 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:38.344558 13055 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.345264 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:38.346453 13055 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:38.332937  9943 rpcz_store.cc:269] 0709 12:24:37.128901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.140487 (+ 11586us) service_pool.cc:224] Handling call
0709 12:24:38.332071 (+1191584us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b540003e154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:38.332278 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192916224}
I20260709 12:24:38.351706 10094 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.355998 10096 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:38.359218 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.360301 10090 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:38.321493 12820 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.369395 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:38.357004 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.370972 12523 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:38.372635 13006 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:24:38.373229 13006 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:38.374157 13006 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.363088  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:38.370190 12820 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:38.361250 12062 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.370671 10670 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:38.371171 10674 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:38.368691 12819 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.363718 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.293843 12524 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.390856 10095 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:38.394248 12062 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.398327  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:38.438249 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.439754 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:38.441757  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:38.442930  9943 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:38.444697 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.445286 10091 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:38.445712 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.446455 10090 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:38.451500 10092 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:38.452710 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:38.399325 12523 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:24:38.454658 12523 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:38.455026 12523 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:38.455894 12523 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.457207 12523 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.458716 13014 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:38.459183 13014 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.459580 13014 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.459811 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.460392 13036 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:38.461100 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.474001 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.475294 10095 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:38.477108 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.435602 13022 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 12:24:38.495074  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 172835us
W20260709 12:24:38.493309 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.494352 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.494544 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.499810 10092 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:38.496635  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:38.502358 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.478948 12923 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:38.510957 10094 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:38.505306  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:38.488974 12856 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.512418 10092 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:38.507087  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:38.520555 12819 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.520318 12856 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.522789 10674 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:38.522224  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:38.528882 10094 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:24:38.529455 10094 pending_rounds.cc:85] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 7
I20260709 12:24:38.529749 10094 pending_rounds.cc:107] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:24:38.530027 10094 raft_consensus.cc:2889] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:38.514804 13028 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.541347 12901 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.541930 12901 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.543375 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.544201 10093 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:38.545219 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.545992 10093 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:24:38.545895  9947 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:24:38.539443 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.506232 12787 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:38.522418 12923 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:38.561641 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.580495 10023 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:38.582767 10677 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:38.561666 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.562523 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:38.565752 10019 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:24:38.565798 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.539073 12819 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:38.528853 13008 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:38.567142 10020 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:38.592564 10021 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:38.535602 12776 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:38.568466 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.570621 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.571744 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:38.574030 12900 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.575259 12787 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.575944 12524 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.545872 12901 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:38.578254 10016 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:38.579252 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.587613 10094 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 12:24:38.566694 13114 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.314s	user 0.002s	sys 0.000s
I20260709 12:24:38.596618 13008 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:38.582391 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.580380 12845 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.621316 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:38.619894 10062 connection.cc:256] RPC call timeout handler was delayed by -1.5822230989997479s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1922s for a response
I20260709 12:24:38.621773 12938 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.595522 10677 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:38.579391 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:38.650331 10674 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.596716 10018 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:38.614039 10092 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:38.546911 13022 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:38.614965  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79224.6us
I20260709 12:24:38.635669 10024 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:38.615860 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.597306 10096 raft_consensus.cc:2376] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5. Re-sending same reply.
W20260709 12:24:38.618044  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.619222 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.621548 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:38.576711 13036 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:38.613902 13003 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:38.651787 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.630843  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1442) took 1097 ms. Trace:
I20260709 12:24:38.640316 10673 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:38.643282 10023 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
W20260709 12:24:38.643519 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0007159599999795s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17600000000000002s for a response
I20260709 12:24:38.642859 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.618290 12820 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.643759 10019 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:38.645756 10090 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:38.632714 10016 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:38.634194 10094 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:38.654460  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:38.661509  9947 rpcz_store.cc:276] 0709 12:24:37.533780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.539239 (+  5459us) service_pool.cc:224] Handling call
0709 12:24:37.547548 (+  8309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.546676 (+999128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.630538 (+ 83862us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b540042d954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:38.630766 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2295267200}
I20260709 12:24:38.662150 10093 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:38.666033 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.641258 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.656123 10670 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:38.661935 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.666970 10672 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEARNER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:38.670832 10018 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:38.630540 13046 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.510s	user 0.006s	sys 0.001s
I20260709 12:24:38.679133  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:38.640694 12893 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.526s	user 0.009s	sys 0.003s
I20260709 12:24:38.661478 13036 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:38.644248 12938 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.677299 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:38.635828 13128 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.644407 12819 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:38.675310 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59527.7us
W20260709 12:24:38.637975 13050 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.504s	user 0.012s	sys 0.001s
I20260709 12:24:38.678604 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:38.691416 10016 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:38.643937 12856 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:38.696583 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.634574 13008 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:38.643733 13124 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:38.699972 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.701033  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.692610 10096 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
W20260709 12:24:38.705022  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50679.4us
I20260709 12:24:38.727823  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:38.684818 13003 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.706280 13128 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.707912 12819 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.712392 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:38.714993 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:38.717231 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.679894 12979 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.723845 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:38.724632 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46120.6us
I20260709 12:24:38.703049 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:38.732038  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52979.4us
I20260709 12:24:38.704165 13081 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:38.702315 12901 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:38.740053 13006 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:38.740926 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.678785 12820 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.742318 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:38.722895 13022 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.745757 10020 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:38.736294 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 232109us
I20260709 12:24:38.733769  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:38.743772 10095 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:38.664647 12622 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.749583  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:38.738464 12856 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:38.691708 13046 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.694224 12893 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.754115 10670 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:38.758895 12726 raft_consensus.cc:2764] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:24:38.747334 13050 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:38.760092 13003 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.759934 13050 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:38.760790 13050 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:38.761103 10674 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
W20260709 12:24:38.762946 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66445.7us
I20260709 12:24:38.765170 10092 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 12:24:38.765848 12822 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.766999 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:38.768709  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:38.770262 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:38.771037 10016 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:38.771471 10018 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:38.761175 13050 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:38.763660 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:38.746835 13033 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.783825 10020 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:38.781183  9941 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:38.785580 13033 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.786675 10099 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:38.787909 13033 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.783020 12827 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:24:38.776042 13003 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.738144 13124 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.745107 12923 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.785540 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:38.799935 10016 raft_consensus.cc:2368] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5.
I20260709 12:24:38.792730 10019 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:38.779203 10017 raft_consensus.cc:2368] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:38.794266  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:38.798005 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:38.801592 10671 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:38.798931 10025 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:38.798367 10091 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 12:24:38.782105  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.794028 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:38.802274  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 68304.1us
I20260709 12:24:38.785109  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:38.851788  9947 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:24:38.855832  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 68690.5us
I20260709 12:24:38.856782  9941 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:38.747020 12979 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:38.855906 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72654.7us
I20260709 12:24:38.861363 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:38.821036 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:38.863384 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 99753.7us
I20260709 12:24:38.777267 13050 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:38.864639  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:38.788618 13015 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:38.770393 12787 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:38.788503 13033 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:38.790019 12820 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:38.875057  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:38.875735  9946 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
W20260709 12:24:38.871351 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.872602 10022 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:38.770102 12822 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.868460  9941 raft_consensus.cc:2368] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5.
I20260709 12:24:38.874308  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:38.870362 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:38.875890  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 90856.2us
I20260709 12:24:38.877569 10016 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:38.880604 10090 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:24:38.884509  9947 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:38.772433 13022 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 12:24:38.788800 13028 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.084s
I20260709 12:24:38.789484 13003 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.898418 13136 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:38.898977 13135 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:38.899346 13135 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:38.900013 13136 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:38.900204 13135 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.894200 10017 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:38.901569 13135 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:38.901795 12815 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:38.903432 10017 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:38.894647 12062 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:38.882632 10090 pending_rounds.cc:85] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 2
I20260709 12:24:38.905668 10090 pending_rounds.cc:107] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:24:38.905576 12516 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:38.906077 10090 raft_consensus.cc:2889] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:38.906212 12516 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.902491 12815 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:38.803829 13006 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:24:38.897845 12820 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:38.925393 10016 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:38.907765 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46493.4us
I20260709 12:24:38.896733  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1435) took 1670 ms. Trace:
I20260709 12:24:38.878955 13137 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:38.873625 13033 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:38.873968 13072 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:38.877988 10021 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:38.869748 13015 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:38.893438 13055 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:38.873245 12856 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:24:38.797405 13014 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:38.925407 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:38.928400 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.928939 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:38.926209  9949 rpcz_store.cc:276] 0709 12:24:37.225927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.257392 (+ 31465us) service_pool.cc:224] Handling call
0709 12:24:38.333719 (+1076327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:38.349999 (+ 16280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:38.885144 (+535145us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540003e150. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:38.896660 (+ 11516us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2474600320}
I20260709 12:24:38.932335 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:38.931861  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48704.3us
I20260709 12:24:38.989427 10672 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:38.932535 10025 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:24:38.952467 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 82192.4us
I20260709 12:24:38.990684 10674 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:38.990990 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:38.952618 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:38.955657 10677 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
I20260709 12:24:38.955309 10673 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
W20260709 12:24:38.953063  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 83073.6us
W20260709 12:24:38.994488 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.995092  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:38.900029 13003 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:39.005846 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:38.929200 12861 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.009220 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:39.013437 10017 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:39.015827 10640 leader_election.cc:400] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5.
I20260709 12:24:39.016173 10640 leader_election.cc:403] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:24:39.016667 10674 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:39.017881  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 151770us
I20260709 12:24:38.994144 13033 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.019972 10673 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:24:39.029877 10016 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:38.953857 12822 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:38.988440 13072 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:38.958721  9843 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:38.927870 12981 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.089s
I20260709 12:24:39.039882 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:39.040844 10672 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:24:39.001447 13127 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:39.042034  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.010155 12892 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:38.909503 12726 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.105s
I20260709 12:24:39.000917 13137 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:39.047670 10017 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:39.014585 12787 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.045607 10090 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:39.001143 13028 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:24:39.043040 13003 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 12:24:39.041968 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 89442.8us
W20260709 12:24:39.076855  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 86956.7us
I20260709 12:24:39.040697 13072 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.004065 13008 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.039734 12917 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5.
I20260709 12:24:39.070410  9843 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 2 to 3, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:24:39.003072 13138 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:24:39.055116 13141 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:39.101243 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:39.073067 12787 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:39.106858  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.110517  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 115490us
W20260709 12:24:39.115958  9839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 328) took 19028 ms (client timeout 15000 ms). Trace:
I20260709 12:24:39.109382  9843 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 4 to 7, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:39.122980 13003 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.064949 12822 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:24:39.124878 12822 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:39.125378 12822 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:39.126720 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.070998 12861 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:24:39.128921 12516 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.113157 13015 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.130949 13015 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.138322 13138 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.139110 13138 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.139209 13137 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:39.147804 12523 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.690s	user 0.011s	sys 0.002s
I20260709 12:24:39.148468 12523 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:24:39.149704 13008 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.150090 13028 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.098567 13046 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.150671 13028 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.106107 13141 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:39.153499 13028 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.116583 12901 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.114617 12787 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.161480 13014 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:39.175277 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74143.4us
I20260709 12:24:39.177215 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:39.174270  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 62805.8us
I20260709 12:24:39.179739  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:39.126446 12822 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.184486 12822 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.179867 12815 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:39.152892 13141 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.187960 13141 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:39.192346 10642 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.633s (SENT)
W20260709 12:24:39.188145 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.192312 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 144965us
I20260709 12:24:39.192902  9843 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 3 to 4, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 4 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:39.182436 12938 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:39.196301  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:39.116437  9839 rpcz_store.cc:269] 0709 12:24:20.042633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:20.042933 (+   300us) service_pool.cc:224] Handling call
0709 12:24:20.706160 (+663227us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094c0090 00007f4e0955ef3a 00007f4e0f314e9c 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc 00007f4e0d8d3392 00007f4e0789148c 00007f4e078908c6
0709 12:24:39.070694 (+18364534us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:28.503437 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:28.504351 (+   914us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:28.504406 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:24:28.504423 (+    17us) tablet.cc:662] Decoding operations
0709 12:24:28.769624 (+265201us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:28.786425 (+ 16801us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:28.791028 (+  4603us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:24:29.041032 (+250004us) tablet.cc:701] Row locks acquired
0709 12:24:29.041093 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:24:29.041304 (+   211us) write_op.cc:270] Start()
0709 12:24:29.041485 (+   181us) write_op.cc:276] Timestamp: P: 1783599869041225 usec, L: 0
0709 12:24:29.041515 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:24:29.287565 (+246050us) log.cc:844] Serialized 7996 byte log entry
0709 12:24:30.183856 (+896291us) op_driver.cc:464] REPLICATION: finished
0709 12:24:30.219240 (+ 35384us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09551e28 00007f4e0cf8c581 00007f4e0cf8bd58 00007f4e0cf909f1 00007f4e0cf909a9 00007f4e0cf90939 00007f4e0cf90901 00007f4e0cf8fb2d 0000000000388634 0000000000388479
0709 12:24:30.701144 (+481904us) write_op.cc:301] APPLY: starting
0709 12:24:30.701283 (+   139us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:30.701675 (+   392us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:30.701723 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:31.342557 (+640834us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:31.578769 (+236212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=20
0709 12:24:31.586823 (+  8054us) write_op.cc:312] APPLY: finished
0709 12:24:31.601654 (+ 14831us) log.cc:844] Serialized 139 byte log entry
0709 12:24:31.755525 (+153871us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:31.770485 (+ 14960us) write_op.cc:454] Released schema lock
0709 12:24:31.791984 (+ 21499us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:31.836850 (+ 44866us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:31.836879 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3246328,"reported_tablets":33,"spinlock_wait_cycles":5728128,"child_traces":[["op",{"apply.queue_time_us":508848,"apply.run_cpu_time_us":23423,"apply.run_wall_time_us":1135702,"num_ops":20,"prepare.queue_time_us":634296,"prepare.run_cpu_time_us":25305,"prepare.run_wall_time_us":1716175,"raft.queue_time_us":224455,"raft.run_cpu_time_us":51471,"raft.run_wall_time_us":387328,"replication_time_us":1142168,"spinlock_wait_cycles":23585536,"thread_start_us":1123912,"threads_started":5,"wal-append.queue_time_us":454275,"wal-append.run_cpu_time_us":60293,"wal-append.run_wall_time_us":3387484}]]}
I20260709 12:24:39.199865  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:39.201598 10643 leader_election.cc:400] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 4 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 4. Current term is 5.
I20260709 12:24:39.095175 12893 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.208876 10097 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:39.204028 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:39.211045 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.204250 12516 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.212724 12856 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.213145 12856 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.213505 12856 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.201311 13003 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:39.202332 13046 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.201464 13150 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.219309  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:39.231231  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 209794us
I20260709 12:24:39.170575 13036 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.227015 13127 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:24:39.233680 13130 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:39.234102 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:39.235913 12900 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.228863 13046 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.239778  9843 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 4 to 5, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 5 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:24:39.229038 13150 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.243631 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:39.248747 10672 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:39.249835 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:39.250531 10673 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:39.245621 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:39.245162  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 156327us
I20260709 12:24:39.238787 13003 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.257206 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:39.257776 10674 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.260134  9947 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:24:39.260715  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:39.261654  9944 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:39.262573 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:39.262522  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.266113 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2706) took 1042 ms. Trace:
I20260709 12:24:39.266435 10095 rpcz_store.cc:276] 0709 12:24:38.223722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.501028 (+277306us) service_pool.cc:224] Handling call
0709 12:24:38.501513 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:38.751918 (+250405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.265913 (+513995us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54003aded4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:39.266067 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552679168}
I20260709 12:24:39.264737 12815 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.267683 12815 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:39.267899 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:39.268029  9943 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:39.269549 13036 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:39.269711  9946 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:24:39.270117  9946 pending_rounds.cc:85] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 6
I20260709 12:24:39.270354  9946 pending_rounds.cc:107] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:24:39.270565  9946 raft_consensus.cc:2889] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:39.271201 13055 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.272202 12845 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:39.272809 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:39.273047 12062 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:39.273787 13153 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.272922 12845 consensus_queue.cc:1243] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:39.268086 12815 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.243705 13150 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.243804 13046 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.275318 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:39.275198 10098 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:39.284192 12062 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:39.284595 12062 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:39.279613  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:39.290696 10674 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:39.290344  9946 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:39.293485 10095 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:39.294996 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:39.295909 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:39.296605 10670 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:39.296881 10671 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:39.297994 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:39.298823 10672 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:39.299108 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:39.274976 13135 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.265300 13130 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:39.303742  9843 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 4 to 5, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 5 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:39.304620 10095 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 12:24:39.305465  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.306041  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.293452 12892 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.301848 12861 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.274911 12923 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:24:39.315667 12861 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.316401 12861 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:24:39.318742 12893 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:39.318572  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:39.275815 13127 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:24:39.319787  9943 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
W20260709 12:24:39.324404  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59103.2us
I20260709 12:24:39.323802  9843 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 3 to 5, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 5 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:24:39.325435  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:39.323819 12062 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.331580 12062 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.303077 12938 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 12:24:39.340742 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:39.313617 13135 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.342943  9946 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:39.343075 12622 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.343659 13135 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.343765 12622 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.344492 12787 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.344700 12622 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:39.348769  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:39.350914 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51932.9us
I20260709 12:24:39.351666 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:39.351971  9843 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 4 to 5, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 5 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:39.362013  9843 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 4, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 4 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:39.366255  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:39.374652 12845 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.375322  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:39.375298 12845 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.377380  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:39.377493 12822 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:39.377549 13135 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:24:39.378508 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89463.1us
I20260709 12:24:39.379070  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:39.405697 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53028.5us
I20260709 12:24:39.409706 13165 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.412077 10092 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 12:24:39.412500 10092 pending_rounds.cc:85] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 6
I20260709 12:24:39.412712 10092 pending_rounds.cc:107] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:24:39.412945 10092 raft_consensus.cc:2889] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:39.378175 13135 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:39.334560 12820 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:24:39.365832 12787 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:39.378434 13073 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:39.388888  9946 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:24:39.405368 13036 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:39.381304  9843 catalog_manager.cc:5697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 2, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:39.378221 12822 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:39.421893  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80996.9us
I20260709 12:24:39.418494 13135 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.437755  9843 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:39.378226 13157 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:24:39.383237 13015 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.441306  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:39.450162  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:39.415282 13006 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:24:39.452942  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:39.466423  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:39.405841 13072 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:39.441957 13050 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.567s	user 0.011s	sys 0.001s
I20260709 12:24:39.435494 13073 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:39.469434  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:39.439126 13036 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:39.479656 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65525.8us
I20260709 12:24:39.478577  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:39.382980 13156 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:24:39.474143  9843 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 5 to 6, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 6 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:39.463744 12820 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.482249 10670 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:39.448765 13015 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.424544 13166 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 12:24:39.473793 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 65853.6us
W20260709 12:24:39.535821  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45883.1us
W20260709 12:24:39.476344 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 96444.6us
I20260709 12:24:39.537868  9912 leader_election.cc:400] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:39.538229  9912 leader_election.cc:403] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:24:39.540272 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:39.441499 12787 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:24:39.474177 12981 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:39.467779 13160 raft_consensus.cc:2764] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:39.539958 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:39.586256 12062 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.483795 12901 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.641427 12062 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.415248 13130 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.472383 12700 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:39.644173 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 104288us
I20260709 12:24:39.669721 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:39.642760 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105090us
I20260709 12:24:39.663234 12062 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:39.670516 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:39.671077 12062 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:39.671654 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:39.672266 10019 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:24:39.668274  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 245927us
I20260709 12:24:39.541088  9843 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 3 to 4, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 4 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:39.643507  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 105793us
I20260709 12:24:39.481693 13149 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:39.687065 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:39.475241 13050 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.697695 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:39.698561 10019 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:39.472470 13154 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.677652 12700 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.541471 13015 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.692312 12062 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:39.664108 12981 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.700276 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:39.701078 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:39.701844 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:39.723846 10021 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:39.692441 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:39.706343 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:39.676082 12516 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.545194 13137 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.675860 12901 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.706707 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:39.713840 12062 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.706882 13173 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.078s
W20260709 12:24:39.688529 13033 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.000s
I20260709 12:24:39.676332 13168 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.718899  9940 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:39.686457 13149 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:39.724454 10023 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:39.721575 10024 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:39.540526 12822 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.713294 12981 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:39.706992 13154 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:39.708117 12893 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:39.761816 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2644) took 1115 ms. Trace:
I20260709 12:24:39.762235 10020 rpcz_store.cc:276] 0709 12:24:38.646090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.785005 (+138915us) service_pool.cc:224] Handling call
0709 12:24:39.176160 (+391155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.176388 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.176413 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:39.176472 (+    59us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:39.176519 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.176579 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:39.761630 (+585051us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54001f7550. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:39.761759 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898847616}
W20260709 12:24:39.765594 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94811.6us
W20260709 12:24:39.740890 13159 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.362s	user 0.001s	sys 0.000s
I20260709 12:24:39.706219 13172 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 12:24:39.741601 13128 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:39.777655  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.771198 10021 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:24:39.755820 13149 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:39.758845 13154 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.740576 13033 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.719013 13073 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.735877 12516 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.738481 13137 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:39.796258 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 109282us
I20260709 12:24:39.809911 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:39.720911 13036 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.807660 13072 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.815161  9942 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:39.789417 12893 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.789407 13180 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 12:24:39.838656 13055 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.001s
I20260709 12:24:39.840456 13006 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:39.859515 13055 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:39.859910 12979 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:39.840729 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:39.840975 13181 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:39.861632 13050 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.841708 13174 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:39.863097 12062 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.863547 12062 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:39.842186 13169 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:24:39.842983 13129 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:39.863902 12062 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:39.864177 13129 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:39.865250 13129 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:39.866415 13124 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:39.864677 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 451) took 1597 ms. Trace:
I20260709 12:24:39.867812 10097 rpcz_store.cc:276] 0709 12:24:38.266966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.497842 (+230876us) service_pool.cc:224] Handling call
0709 12:24:38.516475 (+ 18633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.213479 (+697004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.864414 (+650935us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b5400164754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:39.864606 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582332800}
I20260709 12:24:39.863130 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:39.849797 13141 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.661s	user 0.011s	sys 0.000s
W20260709 12:24:39.885900  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:39.894134 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.846364  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:39.854183 12845 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:39.755556 13168 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:39.873073 13173 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:24:39.876668 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:39.883464 12622 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:24:39.849921 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2714) took 1571 ms (client timeout 1631 ms). Trace:
W20260709 12:24:39.900470 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 132941us
W20260709 12:24:39.901464  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55170.8us
I20260709 12:24:39.929075 12062 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:39.878520 13153 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.604s	user 0.004s	sys 0.003s
W20260709 12:24:39.928671 10098 rpcz_store.cc:269] 0709 12:24:38.278986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.282801 (+  3815us) service_pool.cc:224] Handling call
0709 12:24:39.275048 (+992247us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b540023a120. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e09559e18 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6
0709 12:24:39.849745 (+574697us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b540014a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:39.849858 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1539843840}
I20260709 12:24:39.936035  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:39.940419  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:39.941159  9947 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:39.940304  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:39.941498  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:39.899623 12923 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.958415  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:39.966478  9941 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:39.967649  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:39.969005  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:39.927526 12979 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:39.932510 12622 consensus_queue.cc:1243] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:39.976647  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:39.943908 13046 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.637s	user 0.005s	sys 0.005s
I20260709 12:24:39.927314 13173 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:39.975096 12062 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:39.932322 13129 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:39.945024 13153 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:39.943392 13184 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.970265  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:39.994369  9941 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:39.994711 12062 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.899173 13124 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 12:24:39.987460 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57957.6us
W20260709 12:24:39.995076 13135 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.000s
W20260709 12:24:39.988564  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 339258us
I20260709 12:24:39.997598 13135 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.001379  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:40.004017 13141 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:40.004635 13141 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.005065 13141 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:40.015262  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:40.016173 13130 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:40.015895 12787 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:40.019518 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:40.021905 10020 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:40.027354 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:40.028247 10019 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:40.039315 10019 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:40.041098 12062 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:40.043677 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:40.044839 10021 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:40.015475 13135 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:40.048172 10021 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:39.985190 13150 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.683s	user 0.010s	sys 0.000s
I20260709 12:24:39.942494 12819 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:24:40.053941  9946 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:40.053712 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:39.977890 12979 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.052502  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:40.056725 10673 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:39.956342 13072 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:24:39.987308 12917 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:39.993218 12856 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.765s	user 0.010s	sys 0.001s
W20260709 12:24:40.075416 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78135.4us
I20260709 12:24:40.070393 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:39.987213 13173 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:39.992522 13165 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.582s	user 0.005s	sys 0.005s
W20260709 12:24:39.988624 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:40.094025 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 95676.6us
I20260709 12:24:40.094627  9944 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:40.096138 12062 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:40.096971  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:40.107169  9946 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:40.115018 10019 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:40.117274 10674 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:40.070680 12979 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:40.121410  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:40.122321 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:40.057881 13181 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:39.989739 12923 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.162400 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:39.990621 13046 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.171329 12062 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.172143 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:40.175374 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:40.172153 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:40.100884 13165 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:40.105722 13135 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:40.174120 12062 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:40.175976 10020 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:40.176429 10098 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:40.113579 13014 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:40.113104 13006 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:40.158855 12815 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.865s	user 0.011s	sys 0.000s
I20260709 12:24:40.109012 13127 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.072s
I20260709 12:24:40.206856 10023 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:24:40.100211 12856 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.100015 12917 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.211596 12062 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:40.215664 12062 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.216249 12062 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.218981 12923 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:40.190018 13153 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.060683 13150 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.053884 13184 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.225925 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:40.179941 13033 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:40.229190 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 106966us
I20260709 12:24:40.190491 13014 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.231171 10677 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:39.861627 13055 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 3 to 6, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:40.240402 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:40.213663 13138 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:40.246578 12062 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:40.219903 12516 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:40.251677 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:40.223419 13135 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:40.307201 10670 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:24:40.309456 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69154.5us
I20260709 12:24:40.206400 12815 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.165557 13181 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:40.311811 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:40.231451 12923 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.312337 12062 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:40.231243 13187 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:40.249625 13014 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:40.320518 12062 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:40.321147 12062 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.321503 12062 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:40.237870 13153 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.317063 13138 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:40.371273 10674 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:40.324260 12827 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.381443 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:40.406955 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:40.320664 12923 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.406138 13195 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:40.386420 13033 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:40.387899 13168 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:40.327239 13006 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:40.425499 12700 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:40.426477 13194 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
W20260709 12:24:40.427106 13182 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
W20260709 12:24:40.427430 13196 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:24:40.428879 13150 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:40.429217 13188 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 12:24:40.429838 13033 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.431471 13153 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:40.432786 13073 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:40.433522 13146 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:40.552994 10099 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:40.487815 12981 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.001s
I20260709 12:24:40.440448 12923 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:40.441372 13186 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
W20260709 12:24:40.442577 10640 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.633s (SENT)
I20260709 12:24:40.578586 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:40.443014 13204 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 12:24:40.446234 13198 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:24:40.447167 13185 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:40.375226 13187 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.453418 12062 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:40.456187 10098 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:40.404968 12827 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.460544 10094 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:24:40.461095 12815 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:40.411360 13195 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:40.462648 12900 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.219s	user 0.013s	sys 0.000s
W20260709 12:24:40.459455  9947 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.516s	user 0.010s	sys 0.000s
I20260709 12:24:40.491464 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:40.493530 10021 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:24:40.446247 13015 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.729s	user 0.008s	sys 0.000s
I20260709 12:24:40.521271  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:40.553926 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:40.403719 13138 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.434624 13141 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:40.559443 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:40.584064 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:40.449028 13135 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:40.904605  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 318541us
I20260709 12:24:40.587220  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:40.851539  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1463) took 1608 ms. Trace:
I20260709 12:24:40.949115 13195 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:40.949296 13036 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:40.567327 12981 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:40.988363 10676 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
W20260709 12:24:40.608143 12901 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.868s	user 0.010s	sys 0.003s
W20260709 12:24:40.637336 13149 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.830s	user 0.011s	sys 0.000s
I20260709 12:24:40.962123 13138 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:40.791605 10097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.009s	sys 0.017s
W20260709 12:24:40.791342 13166 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.002s	sys 0.025s
W20260709 12:24:40.997326 10097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.012s	sys 0.017s
I20260709 12:24:40.785353 13046 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:40.608723 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 166172us
I20260709 12:24:40.957451 13141 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:40.944880  9947 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:41.004638  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1703) took 1066 ms. Trace:
I20260709 12:24:41.005107  9947 rpcz_store.cc:276] 0709 12:24:39.937970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.940202 (+  2232us) service_pool.cc:224] Handling call
0709 12:24:41.004544 (+1064342us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:40.989498 13081 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.956034 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 683) took 1415 ms. Trace:
I20260709 12:24:40.715819 13208 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:40.863394 13203 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.400s	user 0.000s	sys 0.000s
I20260709 12:24:40.652779 13150 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.960312 12787 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.963127 10095 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:40.864281 13197 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.279s	user 0.000s	sys 0.000s
I20260709 12:24:40.585860 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:40.862056  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:40.945567 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2651) took 1302 ms. Trace:
W20260709 12:24:40.965490  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 378347us
I20260709 12:24:40.947705  9945 rpcz_store.cc:276] 0709 12:24:39.243849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.243969 (+   120us) service_pool.cc:224] Handling call
0709 12:24:39.260622 (+ 16653us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.278728 (+ 18106us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.278765 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.300677 (+ 21912us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.879952 (+579275us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b18004aa160. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:24:40.100612 (+220660us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.100651 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.100721 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.784385 (+683664us) raft_consensus.cc:1685] finished
0709 12:24:40.784413 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.851444 (+ 67031us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22769,"raft.queue_time_us":21663,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":5295,"spinlock_wait_cycles":7661824,"thread_start_us":220467,"threads_started":1,"wal-append.queue_time_us":905654}
I20260709 12:24:40.952291 13187 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:40.945791 10094 pending_rounds.cc:85] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 6
I20260709 12:24:40.964140 12948 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.971247 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:40.972023 13033 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:40.964484 12923 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:41.030584 10094 pending_rounds.cc:107] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:24:41.027688 10018 rpcz_store.cc:276] 0709 12:24:39.643456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.669530 (+ 26074us) service_pool.cc:224] Handling call
0709 12:24:39.670015 (+   485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.678142 (+  8127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.678175 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.691569 (+ 13394us) log.cc:844] Serialized 564 byte log entry
0709 12:24:39.810912 (+119343us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:39.810950 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.811018 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.945101 (+1134083us) raft_consensus.cc:1685] finished
0709 12:24:40.945123 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.945383 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":118951,"threads_started":1,"wal-append.queue_time_us":298836}
I20260709 12:24:40.973302 13181 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:40.973821 12822 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:40.973865 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:41.032968 10672 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:40.964756 13130 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.634001 12700 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:40.974627 12827 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:40.553462 13168 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:41.043728  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:40.863081 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 652) took 1482 ms. Trace:
I20260709 12:24:40.676036 12892 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:40.559715 13153 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.975409 12726 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:40.591159 13073 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.976078 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:40.569507 12856 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:40.977047 13055 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.531s	user 0.004s	sys 0.006s
W20260709 12:24:40.977173 13154 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.164s	user 0.006s	sys 0.004s
W20260709 12:24:40.865660 13214 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:24:40.586659 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:40.991895 12062 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:40.621115 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:40.865013 13174 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:40.952813 10090 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:40.862486 13204 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:40.990279 13173 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.871s	user 0.004s	sys 0.007s
I20260709 12:24:40.995415 12901 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:41.013228  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:41.001163 13166 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.006s	sys 0.025s
I20260709 12:24:41.039305 12948 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.025772 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.021864 10671 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:41.027153  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:41.090473  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 63413.9us
I20260709 12:24:41.031230 10094 raft_consensus.cc:2889] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:41.092381 10091 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:41.093881 10095 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:41.037498 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:41.095224 10098 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:40.581157 13146 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:40.998406 13149 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:40.980175 13185 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.016039 13081 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.026139 12787 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.021109 13150 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.050582 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:41.021499 13208 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.079454 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.115911 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:41.116361 10093 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:41.049371 12892 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:40.981258 13015 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.052835 12726 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:41.128369 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:41.081578 13204 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.095341 12901 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:41.081461 13174 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.087517 13217 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.089653 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1601) took 1070 ms. Trace:
I20260709 12:24:41.040911 13130 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.101373 10018 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:41.101768  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105674us
I20260709 12:24:41.040567 12923 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:24:41.049304 12979 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.868s	user 0.011s	sys 0.002s
I20260709 12:24:41.137179 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:41.142064 10090 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
W20260709 12:24:41.112188 10092 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.022s	sys 0.052s
I20260709 12:24:41.048240 13168 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.097491 13189 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.118242 13205 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.110s	user 0.001s	sys 0.001s
I20260709 12:24:41.135782 10024 rpcz_store.cc:276] 0709 12:24:40.019666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.026877 (+  7211us) service_pool.cc:224] Handling call
0709 12:24:40.027326 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.027688 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.028978 (+  1290us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.029676 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.176976 (+147300us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:40.365922 (+188946us) log.cc:844] Serialized 14 byte log entry
0709 12:24:40.458926 (+ 93004us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.459019 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.089468 (+630449us) raft_consensus.cc:1685] finished
0709 12:24:41.089485 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.089598 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11201,"prepare.queue_time_us":48194,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":7980,"raft.queue_time_us":4880,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":207,"spinlock_wait_cycles":26496,"thread_start_us":303890,"threads_started":2,"wal-append.queue_time_us":432832}
I20260709 12:24:41.132962 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:41.137374 12062 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.134914  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:41.079634 13154 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.055249 13153 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.118432 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:41.143172 10092 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.022s	sys 0.052s
I20260709 12:24:41.084299 13173 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.111181 13187 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
W20260709 12:24:41.142831  9914 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.763s (SENT) [suppressed 33 similar messages]
I20260709 12:24:41.147753 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:41.161195 10671 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:41.160919 12062 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:41.158215 10016 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:41.140007 12700 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.178195 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.144130 13036 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.176774  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:41.182277  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:41.183662 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:41.177176 10020 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:41.184064 10098 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:41.173424 13216 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.165s	user 0.002s	sys 0.000s
I20260709 12:24:41.186144  9947 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:41.167550 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:41.187897 12062 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:41.190111 10090 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:41.142738 12979 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.163264 12901 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:41.177802 13046 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:41.182416 12981 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.010377 10668 rpcz_store.cc:276] 0709 12:24:39.540552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.663088 (+122536us) service_pool.cc:224] Handling call
0709 12:24:39.723136 (+ 60048us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.758732 (+ 35596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.760916 (+  2184us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.816408 (+ 55492us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.054261 (+237853us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.059422 (+  5161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.073422 (+ 14000us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.955801 (+882379us) raft_consensus.cc:1685] finished
0709 12:24:40.955831 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.955969 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":360362,"raft.queue_time_us":195214,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":29565,"thread_start_us":136048,"threads_started":1,"wal-append.queue_time_us":308743}
I20260709 12:24:41.213086 13008 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:41.213564 13050 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.213886 13008 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.214066 13050 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.217001 12917 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.216674 13050 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.219930 13137 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:41.220961 10640 leader_election.cc:336] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.779s (SENT)
I20260709 12:24:41.224632 13028 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.225214 13028 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.226027 13172 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.226585 13172 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.227514 13028 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.228075 13028 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.228739 13172 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.230520 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:41.228514 13028 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.234900 10019 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:41.239609  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:41.240825 10090 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:41.245965 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2647) took 1603 ms. Trace:
I20260709 12:24:41.246517 10025 rpcz_store.cc:276] 0709 12:24:39.642234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.672745 (+ 30511us) service_pool.cc:224] Handling call
0709 12:24:39.693307 (+ 20562us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.972973 (+279666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.973002 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.984815 (+ 11813us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.215760 (+230945us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:40.215801 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.462985 (+247184us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54001f8954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:40.463036 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.235420 (+772384us) raft_consensus.cc:1685] finished
0709 12:24:41.235452 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.245879 (+ 10427us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":420,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":11557,"spinlock_wait_cycles":642345216,"thread_start_us":143700,"threads_started":1,"wal-append.queue_time_us":952361}
W20260709 12:24:41.196537 13213 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.135s	user 0.001s	sys 0.001s
I20260709 12:24:41.142189 13217 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.272429  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:41.273708  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:41.155210 12726 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.186379 12700 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:41.242463 13129 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.250s	user 0.006s	sys 0.007s
I20260709 12:24:41.185458 13036 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.156059 13189 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.201223 13136 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:41.286244 13168 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.287482 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 774) took 2028 ms. Trace:
W20260709 12:24:41.291385  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 109173us
I20260709 12:24:41.287194 13073 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.254813 12917 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.046141 10669 rpcz_store.cc:276] 0709 12:24:39.380979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.382865 (+  1886us) service_pool.cc:224] Handling call
0709 12:24:39.426189 (+ 43324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.444614 (+ 18425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.444667 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:39.445241 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:24:39.941028 (+495787us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:39.955924 (+ 14896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:39.972893 (+ 16969us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:40.851551 (+878658us) raft_consensus.cc:1685] finished
0709 12:24:40.851578 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:40.862996 (+ 11418us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":699,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":458252,"threads_started":1,"wal-append.queue_time_us":625501}
I20260709 12:24:41.295400 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:41.332970  9945 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:41.297637 13220 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:24:41.214365 13008 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.699417 12923 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:41.388175 13204 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.388761 13174 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:41.389489 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 476) took 1069 ms. Trace:
W20260709 12:24:41.510960 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:41.464776 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.711295 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 777) took 1485 ms. Trace:
I20260709 12:24:41.711758 10674 rpcz_store.cc:276] 0709 12:24:40.226064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.401453 (+175389us) service_pool.cc:224] Handling call
0709 12:24:40.402635 (+  1182us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.405281 (+  2646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.405345 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.421754 (+ 16409us) log.cc:844] Serialized 564 byte log entry
0709 12:24:40.784228 (+362474us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:40.784290 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.784712 (+   422us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.391002 (+606290us) raft_consensus.cc:1685] finished
0709 12:24:41.391030 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.711226 (+320196us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":361954,"threads_started":1,"wal-append.queue_time_us":586186}
I20260709 12:24:41.341745  9948 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:41.715829 10671 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:41.699467 10673 rpcz_store.cc:276] 0709 12:24:39.259038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.265768 (+  6730us) service_pool.cc:224] Handling call
0709 12:24:39.289513 (+ 23745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:40.071700 (+782187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.287227 (+1215527us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b54007d7b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:41.287418 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1763806080}
I20260709 12:24:41.709870 10092 rpcz_store.cc:276] 0709 12:24:40.321254 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.321442 (+   188us) service_pool.cc:224] Handling call
0709 12:24:40.372019 (+ 50577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.379669 (+  7650us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:41.158414 (+778745us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.259120 (+100706us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.259624 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.259773 (+   149us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:41.259807 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.259880 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.389217 (+129337us) raft_consensus.cc:1685] finished
0709 12:24:41.389242 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.389410 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20443,"prepare.queue_time_us":485871,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"raft.queue_time_us":22994,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":17678,"thread_start_us":821439,"threads_started":2,"wal-append.queue_time_us":105948}
W20260709 12:24:41.716271 10016 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.000s
I20260709 12:24:41.295539 13128 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.781291 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:41.700273 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2741) took 1122 ms. Trace:
I20260709 12:24:41.806528 10016 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:41.737520 10094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.009s	sys 0.002s
W20260709 12:24:41.333917 13222 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 12:24:41.550333 12726 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:41.807652 10095 rpcz_store.cc:276] 0709 12:24:40.578087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.095939 (+517852us) service_pool.cc:224] Handling call
0709 12:24:41.096226 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.096418 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:41.096658 (+   240us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.699790 (+603132us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.699942 (+   152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:41.699981 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:41.700030 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.700094 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.700213 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4506,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"raft.queue_time_us":792,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":133,"thread_start_us":235647,"threads_started":1}
W20260709 12:24:41.808548 10094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.008s
I20260709 12:24:41.626153 13146 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:41.341209 13213 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:24:41.389216 13230 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:24:41.670619 13153 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.676617 13173 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.645893 13154 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:41.372761 13226 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
W20260709 12:24:41.667392 12892 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.536s	user 0.007s	sys 0.004s
I20260709 12:24:41.612402 12700 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.646162 12787 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:41.379406 13219 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 12:24:41.626343 13233 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:24:41.333110 13166 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:41.813875  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 112778us
I20260709 12:24:41.662637 13189 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:41.337769 13172 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:41.338006 12856 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:41.542840 13129 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.379990 13015 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.542797 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:41.614581 13150 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.387113 12948 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.808736 13036 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.780853 13204 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.681663 12979 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.625524 13185 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:41.781338 12917 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.836207  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:41.823236 12981 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.838574  9945 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:41.825240 13153 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:41.812773 13128 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.827399 13028 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.819667 13050 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 12:24:41.843566 13014 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.518s	user 0.008s	sys 0.005s
I20260709 12:24:41.838223 13154 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:24:41.844806 13184 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.383165 12901 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.848328  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:41.828122 13208 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:41.861685  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:41.849612 13207 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.418s	user 0.011s	sys 0.012s
I20260709 12:24:41.830677 12892 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.841476 13129 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.841871 12856 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.879809  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:41.875995  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1713) took 1028 ms. Trace:
I20260709 12:24:41.864046 13036 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:41.848750 13172 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.882380  9941 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:41.838774 13166 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.866528 13217 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.893023  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:41.894022  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1456) took 3105 ms. Trace:
I20260709 12:24:41.897223 10099 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:41.895819  9942 rpcz_store.cc:276] 0709 12:24:38.779319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.779486 (+   167us) service_pool.cc:224] Handling call
0709 12:24:38.852255 (+ 72769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.820506 (+968251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.884235 (+2063729us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b540037ffd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:41.884474 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2213972864}
I20260709 12:24:41.904191  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:41.876587 13028 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:41.882881  9944 rpcz_store.cc:276] 0709 12:24:40.847974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.954705 (+106731us) service_pool.cc:224] Handling call
0709 12:24:40.955074 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:40.958386 (+  3312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.958447 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:40.959398 (+   951us) log.cc:844] Serialized 52 byte log entry
0709 12:24:41.107012 (+147614us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:41.107054 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.107117 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.875780 (+768663us) raft_consensus.cc:1685] finished
0709 12:24:41.875799 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.875939 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2655,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":99,"thread_start_us":147176,"threads_started":1,"wal-append.queue_time_us":183368}
I20260709 12:24:41.911401 10674 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:41.866149 12981 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.862748 13153 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.870172 13014 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:41.929443 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:41.865207 13204 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:41.931867 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.934381 10092 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:41.936484  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:41.936753 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.936484 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.938963  9941 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:41.939247 10093 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:41.937987 10098 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:41.939986 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.941713 10095 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:41.896829 13152 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:41.945744 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.947382  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1460) took 3166 ms. Trace:
I20260709 12:24:41.947729 13129 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.949899 12981 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.994035 10668 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:41.995127 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1602) took 1975 ms. Trace:
I20260709 12:24:41.995644 10668 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:41.995517 10023 rpcz_store.cc:276] 0709 12:24:40.020010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.027811 (+  7801us) service_pool.cc:224] Handling call
0709 12:24:40.028135 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.207524 (+179389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.994907 (+1787383us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b540038e0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:41.995072 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410246528}
W20260709 12:24:41.951186 13241 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:24:41.997843 13128 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.951116 13130 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.999014 13152 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:41.952371 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2736) took 1458 ms. Trace:
I20260709 12:24:41.897006 13208 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:41.933739 13219 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:41.954888 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:41.963431 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:42.008767 10098 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.009748 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.010577 10092 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:42.011878 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:41.963961  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:42.014673  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.015748  9947 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:41.969074 12856 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.912912 13008 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.970427 13242 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:41.973436 13195 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.024s	user 0.010s	sys 0.000s
I20260709 12:24:41.974607 13081 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:41.975450 12917 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.904045 13234 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:41.976511 12892 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:41.979851  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:41.980769 13014 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.974328  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 601) took 2038 ms. Trace:
W20260709 12:24:42.036927  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57163.6us
I20260709 12:24:41.984874  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:41.986521 10090 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:42.023953 12938 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:41.989450 10093 raft_consensus.cc:2376] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 7. Re-sending same reply.
I20260709 12:24:41.989637  9940 rpcz_store.cc:276] 0709 12:24:38.780772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:38.780882 (+   110us) service_pool.cc:224] Handling call
0709 12:24:38.793787 (+ 12905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:39.738107 (+944320us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.947089 (+2208982us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b540003fa54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:41.947312 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2127048960}
I20260709 12:24:41.990909  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:41.967021 13153 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:41.987775 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.012585 10092 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:42.021291  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:42.023242 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:41.999939 10091 rpcz_store.cc:276] 0709 12:24:40.494205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.100767 (+606562us) service_pool.cc:224] Handling call
0709 12:24:41.101077 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.106578 (+  5501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.107413 (+   835us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.114914 (+  7501us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.195756 (+ 80842us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:24:41.195792 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.195861 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.952081 (+756220us) raft_consensus.cc:1685] finished
0709 12:24:41.952112 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.952292 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":647,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":9004,"thread_start_us":54523,"threads_started":1,"wal-append.queue_time_us":65834}
I20260709 12:24:41.942909 13028 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.037998  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.131893 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:42.096973 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 475) took 1975 ms. Trace:
I20260709 12:24:42.032076 12845 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:42.098698  9941 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:42.070946 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 689) took 1068 ms. Trace:
I20260709 12:24:42.037168  9943 rpcz_store.cc:276] 0709 12:24:39.936241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.936862 (+   621us) service_pool.cc:224] Handling call
0709 12:24:39.963528 (+ 26666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.984636 (+ 21108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.984672 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:40.055762 (+ 71090us) log.cc:844] Serialized 26 byte log entry
0709 12:24:40.238903 (+183141us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:40.238981 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:40.239169 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:41.832228 (+1593059us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b2400193118. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a809d 00007f4e090da10e 00007f4e090f188c 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221
0709 12:24:41.974062 (+141834us) raft_consensus.cc:1685] finished
0709 12:24:41.974083 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:41.974273 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":11880,"spinlock_wait_cycles":42279296,"thread_start_us":123667,"threads_started":1,"wal-append.queue_time_us":329432}
I20260709 12:24:42.024061 13173 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:42.068374 13204 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:41.970428 13172 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:42.017678 13208 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.035821 13195 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:42.133733  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 95858.2us
I20260709 12:24:42.101532 10098 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:42.067014  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.169050  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.178133 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.188359 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.192198  9940 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:42.192334  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.134436 10097 rpcz_store.cc:276] 0709 12:24:40.121136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.121272 (+   136us) service_pool.cc:224] Handling call
0709 12:24:40.174973 (+ 53701us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:40.198188 (+ 23215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:40.198223 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:40.209837 (+ 11614us) log.cc:844] Serialized 52 byte log entry
0709 12:24:40.997889 (+788052us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:41.012192 (+ 14303us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.012500 (+   308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.012604 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.076329 (+1063725us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b180046a9e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:42.078987 (+  2658us) raft_consensus.cc:1685] finished
0709 12:24:42.096721 (+ 17734us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.096901 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7108,"prepare.queue_time_us":18172,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":134187,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":17619,"spinlock_wait_cycles":6927744,"thread_start_us":749632,"threads_started":1,"wal-append.queue_time_us":857070}
W20260709 12:24:42.109069 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.189311  9941 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:42.070387 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 697) took 1032 ms. Trace:
W20260709 12:24:42.193760  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 2.31535e+06us
I20260709 12:24:42.199101 10668 rpcz_store.cc:276] 0709 12:24:41.038550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.996507 (+957957us) service_pool.cc:224] Handling call
0709 12:24:41.996829 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.006917 (+ 10088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.008479 (+  1562us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.026983 (+ 18504us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:42.032953 (+  5970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.040129 (+  7176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.070283 (+ 30154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.199904  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:42.200570 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:42.200774 13130 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:42.192660 10095 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:42.046030 13015 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:42.196249  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.205420  9941 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
W20260709 12:24:42.138833 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0201548519999051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
I20260709 12:24:42.136061 10671 rpcz_store.cc:276] 0709 12:24:41.002244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.833361 (+831117us) service_pool.cc:224] Handling call
0709 12:24:41.847072 (+ 13711us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.847407 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.847426 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.847804 (+   378us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.893803 (+ 45999us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:41.893859 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.903180 (+  9321us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.070675 (+167495us) raft_consensus.cc:1685] finished
0709 12:24:42.070703 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.070875 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":490,"raft.queue_time_us":1923,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":6478,"thread_start_us":18062,"threads_started":1,"wal-append.queue_time_us":150205}
I20260709 12:24:42.101518 13234 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:42.217231 10093 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:42.137045 12981 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:42.204435  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.080667 13149 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:42.166627 13135 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.186s	user 0.015s	sys 0.048s
I20260709 12:24:42.041303 13008 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:42.131074 12917 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:42.195858  9914 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.631s (SENT)
I20260709 12:24:42.211057  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.232311  9940 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:42.201614 12901 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:42.201345 13130 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.206538 10668 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.131978 12938 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:42.232244 13135 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.266s	user 0.019s	sys 0.049s
I20260709 12:24:42.070643 13152 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:42.236196 13128 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.198850 13173 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.217419 13168 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:42.222754 12856 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:24:42.222546 13129 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:24:42.239568 13208 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.243889  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 598) took 2346 ms. Trace:
I20260709 12:24:42.245357  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.237371 13130 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.246197 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 684) took 1245 ms. Trace:
I20260709 12:24:42.247077 13204 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.207170 12787 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.253067  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:42.225729 12981 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.202199 13154 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.104s
I20260709 12:24:42.332557 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.198539 12845 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.219007 13172 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:24:42.295938  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:42.248292  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52485.9us
I20260709 12:24:42.339215  9940 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:42.316807 10016 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:42.315337  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.314517  9944 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:42.341472  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.233302 12892 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:24:42.042052 13242 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.331286 10023 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:24:42.234601 12901 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:42.289340 13128 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.350802 12892 consensus_queue.cc:1243] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:24:42.351358 12892 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:42.351691 12892 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:42.352108 13242 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:42.352305 13128 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.258621 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:42.333447 10017 raft_consensus.cc:2368] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7.
I20260709 12:24:42.257532 12726 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.354075 12726 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:42.354875 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.355512 13173 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.356065 12726 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:42.356204 10024 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:42.334441 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.258306 13234 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:42.334537 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.205204 13015 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.339188 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.253675 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:42.305639 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:42.235124 12917 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.259912 12938 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.254828 13028 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.261011 13195 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.348538 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.323300 13150 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:42.352504 12892 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.302515 13168 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:42.314774 13008 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.331773 13204 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.332571 12981 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.329962  9949 rpcz_store.cc:276] 0709 12:24:39.897014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.897175 (+   161us) service_pool.cc:224] Handling call
0709 12:24:39.897490 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:39.904855 (+  7365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:39.904894 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.840198 (+1935304us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b2c00012928. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:24:41.847987 (+  7789us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.945969 (+ 97982us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:41.946001 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.946067 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.243576 (+297509us) raft_consensus.cc:1685] finished
0709 12:24:42.243597 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.243822 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1670,"raft.queue_time_us":9944,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":53175168,"thread_start_us":83388,"threads_started":1,"wal-append.queue_time_us":333232}
I20260709 12:24:42.492925  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.360633  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.467495 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.359014 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:42.387050  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.471912  9944 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:42.514344 10673 raft_consensus.cc:2410] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:24:42.514564  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1473) took 1276 ms. Trace:
I20260709 12:24:42.514997  9944 rpcz_store.cc:276] 0709 12:24:41.238571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.390175 (+1151604us) service_pool.cc:224] Handling call
0709 12:24:42.471656 (+ 81481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.514321 (+ 42665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.514514 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.456513 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:42.508378 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 254814us
I20260709 12:24:42.343737 13172 consensus_queue.cc:1243] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:24:42.513721  9941 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:42.516942 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:42.456890 13248 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:24:42.455976 13249 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:24:42.526489  9949 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:42.533425  9944 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:42.502514 13015 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:42.557551 13251 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:24:42.560030 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2737) took 2055 ms (client timeout 1631 ms). Trace:
I20260709 12:24:42.560323 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 691) took 1557 ms. Trace:
I20260709 12:24:42.559999 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.517417 13174 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:42.546797  9947 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:42.502262 12901 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.566922 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:42.534318 12938 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:42.549867  9940 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:42.332789 12787 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:42.560786 10672 rpcz_store.cc:276] 0709 12:24:41.003094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.977678 (+974584us) service_pool.cc:224] Handling call
0709 12:24:41.978127 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.978542 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.978578 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.979112 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:24:42.198054 (+218942us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:42.198091 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.198154 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.560144 (+361990us) raft_consensus.cc:1685] finished
0709 12:24:42.560163 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.560273 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7079,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":2044,"thread_start_us":218561,"threads_started":1,"wal-append.queue_time_us":347327}
I20260709 12:24:42.555858 10671 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:42.570840 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 780) took 1337 ms. Trace:
I20260709 12:24:42.571214 10671 rpcz_store.cc:276] 0709 12:24:41.233299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.512962 (+1279663us) service_pool.cc:224] Handling call
0709 12:24:42.555616 (+ 42654us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.570585 (+ 14969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.570790 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.571609 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:42.555081 12892 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:42.562883  9991 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.005s	sys 0.012s
W20260709 12:24:42.575064  9991 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.005s	sys 0.013s
I20260709 12:24:42.464273 12845 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.572244 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.579631 10025 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:42.573154  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1479) took 1209 ms. Trace:
I20260709 12:24:42.580497  9940 rpcz_store.cc:276] 0709 12:24:41.363523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.517687 (+1154164us) service_pool.cc:224] Handling call
0709 12:24:42.549688 (+ 32001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.572877 (+ 23189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.573087 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:42.560508 10099 rpcz_store.cc:269] 0709 12:24:40.504809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.553976 (+ 49167us) service_pool.cc:224] Handling call
0709 12:24:42.559831 (+2005855us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b540014a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:42.559965 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2182776064}
I20260709 12:24:42.573345 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.546433  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1476) took 1254 ms. Trace:
I20260709 12:24:42.574076 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.575742 13173 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:42.576030 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.576941 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1604) took 2459 ms. Trace:
I20260709 12:24:42.589376 10019 rpcz_store.cc:276] 0709 12:24:40.117139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.175719 (+ 58580us) service_pool.cc:224] Handling call
0709 12:24:40.176006 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:41.235620 (+1059614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.576668 (+1341048us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b54001c5054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:42.576858 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2434618752}
W20260709 12:24:42.583325  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 601641us
I20260709 12:24:42.567690 13174 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.584229 10022 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
I20260709 12:24:42.584967 10672 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:42.586798 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.572845 12938 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.511210 13150 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.583240  9944 rpcz_store.cc:276] 0709 12:24:41.292428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.532280 (+1239852us) service_pool.cc:224] Handling call
0709 12:24:42.533066 (+   786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:42.538830 (+  5764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.546369 (+  7539us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:42.603411 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.604552 10099 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:42.597585 13174 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.601150 12938 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:42.639549 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:42.649051 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69333.5us
I20260709 12:24:42.658986 12917 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:42.660789  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56016.4us
I20260709 12:24:42.605633 13128 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:24:42.633153 13168 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:42.665309 10670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 653) took 2678 ms (client timeout 1867 ms). Trace:
W20260709 12:24:42.665752 10670 rpcz_store.cc:269] 0709 12:24:39.986501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.988870 (+  2369us) service_pool.cc:224] Handling call
0709 12:24:42.665096 (+2676226us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b54004d35d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:42.665243 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":571652480}
I20260709 12:24:42.669243 13168 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:24:42.670125 13253 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:24:42.597218 12726 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Committing config change with OpId 3.4: config changed from index 2 to 4, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:42.669379 13008 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:42.680730 13206 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 2.159s	user 0.007s	sys 0.005s
I20260709 12:24:42.681716 13128 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:42.682317 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.314649 10669 rpcz_store.cc:276] 0709 12:24:41.000948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.809434 (+808486us) service_pool.cc:224] Handling call
0709 12:24:41.837730 (+ 28296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.871211 (+ 33481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.875237 (+  4026us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.896785 (+ 21548us) log.cc:844] Serialized 26 byte log entry
0709 12:24:42.132472 (+235687us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:42.136998 (+  4526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.159563 (+ 22565us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.245958 (+ 86395us) raft_consensus.cc:1685] finished
0709 12:24:42.245975 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.246136 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4886,"raft.queue_time_us":9483,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":10786,"thread_start_us":194509,"threads_started":1,"wal-append.queue_time_us":300057}
I20260709 12:24:42.663763 10668 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:42.683288 10099 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:42.686372  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1716) took 1730 ms. Trace:
I20260709 12:24:42.686834  9945 rpcz_store.cc:276] 0709 12:24:40.955539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.030919 (+ 75380us) service_pool.cc:224] Handling call
0709 12:24:42.686208 (+1655289us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b54000163d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:42.686312 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":662108928}
I20260709 12:24:42.695040 13258 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.695588 13258 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.698720 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.699400 10097 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
W20260709 12:24:42.702728 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.703199  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.631825 12979 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:24:42.711556  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.718742  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.721362  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.731729 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.732812 10024 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:42.736979 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:42.700984 12787 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:42.682392 13128 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:42.734279 10021 pending_rounds.cc:85] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 5
I20260709 12:24:42.743311 10021 pending_rounds.cc:107] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:24:42.743621 10021 raft_consensus.cc:2889] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:42.748776 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.728461 13187 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:42.750185 13187 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:42.739629 13234 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:42.751406 13234 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.751809 13234 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.757085 13015 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:42.719462 13258 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:42.762944  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:42.720491 12917 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:42.738471 13152 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:42.782955  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71547.8us
I20260709 12:24:42.791417  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.739693 13154 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:24:42.773209 12787 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:42.781426 13152 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:42.741297 13130 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.789425 13128 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.803165 13204 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.815577  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.817821  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.827523  9944 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:42.827355  9947 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:42.790418 13172 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:42.828609  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.829977  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:42.831763 13128 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:42.792897 13224 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:42.837823 13254 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:24:42.839219 10095 raft_consensus.cc:2410] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260709 12:24:42.838624 13263 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:24:42.784418 12917 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.751142 13187 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.802348 13152 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:42.855036  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:42.870833 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:42.917075 10091 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:42.919914  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 603) took 1307 ms. Trace:
I20260709 12:24:42.920589  9941 rpcz_store.cc:276] 0709 12:24:41.612189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.571065 (+958876us) service_pool.cc:224] Handling call
0709 12:24:42.579362 (+  8297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.579665 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:42.628380 (+ 48715us) log.cc:844] Serialized 14 byte log entry
0709 12:24:42.919468 (+291088us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.919532 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:42.919593 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.919682 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.919837 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43091,"prepare.queue_time_us":89408,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":277887,"threads_started":2}
W20260709 12:24:42.869637 13260 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:24:42.941537  9945 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:24:42.849076 13172 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.960229 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.961002 10098 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:42.947336 12523 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:42.961951 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:42.938874 13269 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:42.877255 13270 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:42.950173  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.966008 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:42.967993  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:42.854105 13224 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:42.969386  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:42.969760 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:42.948026 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 247029us
I20260709 12:24:42.947595 13173 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:42.974969  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1475) took 1735 ms. Trace:
I20260709 12:24:42.976168  9949 rpcz_store.cc:276] 0709 12:24:41.255605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.525670 (+1270065us) service_pool.cc:224] Handling call
0709 12:24:42.526097 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:42.526343 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:42.850087 (+323744us) log.cc:844] Serialized 14 byte log entry
0709 12:24:42.974615 (+124528us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:42.974670 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:42.974722 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.974798 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:42.974915 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":123902,"threads_started":1}
I20260709 12:24:42.965086 12523 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:42.979915 13274 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:24:42.979691 12901 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:42.981109 13273 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.142s	user 0.000s	sys 0.000s
I20260709 12:24:42.980863 12523 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:42.972841 10098 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:42.985168 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2748) took 1086 ms. Trace:
I20260709 12:24:42.985994 10090 rpcz_store.cc:276] 0709 12:24:41.899219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.900612 (+  1393us) service_pool.cc:224] Handling call
0709 12:24:42.985005 (+1084393us) spinlock_profiling.cc:243] Waited 47.5 ms on lock 0x7b540014afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:42.985111 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109180800}
I20260709 12:24:42.989171 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:24:42.985082 13268 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:24:42.966825 12787 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:43.004766 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.005573 10090 raft_consensus.cc:2368] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6.
I20260709 12:24:42.974674 13154 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:42.969913 13270 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:43.001650 10092 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:43.014789  9990 leader_election.cc:400] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6.
I20260709 12:24:43.016654  9990 leader_election.cc:403] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:43.014312 10098 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:43.017187 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:43.018019 10090 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.018807 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:43.022272 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:43.024820 13072 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.025269 13072 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.025616 13072 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.026010 13172 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:43.027583 12901 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:42.972301 13269 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.019933 13270 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.031649 13269 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:43.000775 12856 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:24:43.035943 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:42.993630 13015 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:43.032240 13269 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:43.054795 13269 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.055380 13269 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:43.046428 13281 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:24:43.057291 13270 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.036733 10097 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:43.059511 13270 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:43.060206 13270 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:43.039738 12981 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.062702 12981 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:43.045605 13277 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:24:43.008265 12523 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:43.064023  9947 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:43.055393 13174 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.067355 13174 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:43.068544 10093 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:43.046890 13229 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:24:43.038352 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:43.039460 13284 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:43.050351 13224 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:43.037603 13172 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.059427 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.062811 13130 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:43.046155 12787 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.102321 13172 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.104604 13172 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:43.042794 10676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 669) took 5251 ms (client timeout 1975 ms). Trace:
I20260709 12:24:43.071763 10098 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:43.106042  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1702) took 3151 ms (client timeout 1532 ms). Trace:
I20260709 12:24:43.019863 13283 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:43.089916 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.016172 13154 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:43.096225 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.098891 12938 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.101910 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2730) took 2609 ms. Trace:
I20260709 12:24:43.062330 10091 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
W20260709 12:24:43.105337 10676 rpcz_store.cc:269] 0709 12:24:37.771257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:37.771473 (+   216us) service_pool.cc:224] Handling call
0709 12:24:38.714794 (+943321us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:24:43.022266 (+4307472us) spinlock_profiling.cc:243] Waited 2.24 s on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:43.022405 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5156795008}
W20260709 12:24:43.106475  9948 rpcz_store.cc:269] 0709 12:24:39.937637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:39.940027 (+  2390us) service_pool.cc:224] Handling call
0709 12:24:43.088513 (+3148486us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540042e354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:43.088641 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2635190144}
I20260709 12:24:43.113986 10677 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:43.116029 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.228883 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:43.229789 10091 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:43.170956 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.176831 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:43.127238 10674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 782) took 1893 ms (client timeout 2024 ms). Trace:
I20260709 12:24:43.233821 10099 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 12:24:43.233691 10674 rpcz_store.cc:269] 0709 12:24:41.233958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.780981 (+547023us) service_pool.cc:224] Handling call
0709 12:24:43.126957 (+1345976us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540045ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:43.127124 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":238394624}
W20260709 12:24:43.169109  9942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1482) took 1557 ms (client timeout 1910 ms). Trace:
W20260709 12:24:43.235785  9942 rpcz_store.cc:269] 0709 12:24:41.611509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.945282 (+333773us) service_pool.cc:224] Handling call
0709 12:24:43.168896 (+1223614us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b5400206f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:43.169040 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1443480960}
I20260709 12:24:43.235776 10097 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 12:24:43.237468 10642 connection.cc:256] RPC call timeout handler was delayed by -1.3413186970001334s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1855s for a response
I20260709 12:24:43.240892  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.242244  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.253890  9949 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:43.125622 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2739) took 2565 ms. Trace:
I20260709 12:24:43.254678 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:43.132952 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.244060 13146 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:43.244179  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:43.244290 13183 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.244660  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.265323  9945 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:43.119688 13283 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.255241 10094 rpcz_store.cc:276] 0709 12:24:40.560602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.092973 (+532371us) service_pool.cc:224] Handling call
0709 12:24:41.117583 (+ 24610us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.154646 (+ 37063us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:41.174887 (+ 20241us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.813692 (+638805us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.816408 (+  2716us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.829677 (+ 13269us) log.cc:844] Serialized 564 byte log entry
0709 12:24:41.829854 (+   177us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:41.829901 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.027229 (+1197328us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b18001a63e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:43.125182 (+ 97953us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b5400094054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:43.125263 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.125329 (+    66us) raft_consensus.cc:1685] finished
0709 12:24:43.125350 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.125543 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4439,"prepare.queue_time_us":25154,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":117,"raft.queue_time_us":7712,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":5223,"spinlock_wait_cycles":94840704,"thread_start_us":620387,"threads_started":1,"wal-append.queue_time_us":652031,"wal-append.run_cpu_time_us":3520,"wal-append.run_wall_time_us":1160606}
I20260709 12:24:43.245103 12979 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:43.235353 13154 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.246035 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:43.246780 10065 connection.cc:256] RPC call timeout handler was delayed by -1.2365961259997675s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:24:43.248826  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.320322  9943 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:43.249308 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 260180us
I20260709 12:24:43.249549  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.321305  9948 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:43.121970 10098 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:43.230453 10096 rpcz_store.cc:276] 0709 12:24:40.492508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.492619 (+   111us) service_pool.cc:224] Handling call
0709 12:24:40.558738 (+ 66119us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:40.559128 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:41.003770 (+444642us) log.cc:844] Serialized 14 byte log entry
0709 12:24:41.194609 (+190839us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.244560 (+ 49951us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.244718 (+   158us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:41.244800 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.244868 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.693528 (+1448660us) spinlock_profiling.cc:243] Waited 29.3 ms on lock 0x7b18003c2020. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:42.813164 (+119636us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b18003c2020. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:43.101653 (+288489us) raft_consensus.cc:1685] finished
0709 12:24:43.101675 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.101844 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36674,"prepare.queue_time_us":306694,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":42432,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":80938,"spinlock_wait_cycles":72690304,"thread_start_us":526501,"threads_started":2,"wal-append.queue_time_us":177604}
W20260709 12:24:43.179005 13276 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.209s	user 0.000s	sys 0.002s
I20260709 12:24:43.128577 10673 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 3 to 6, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:43.140064 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:43.324608 10095 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.252801 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:43.233901  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.328393 12979 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:43.328869 12979 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:43.329421 13168 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.257941 10090 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:43.106305 13234 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.330760 13128 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.259160  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.331547  9947 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:43.331288 13128 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.332451  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.333165  9944 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:24:43.333336 13128 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:43.163961 13240 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.276s	user 0.000s	sys 0.004s
I20260709 12:24:43.333915 13128 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:43.300208  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:43.300875  9941 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:43.334862 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.335703  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.336421  9941 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:43.301200  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:43.160804 13135 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:24:43.303766 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49224.3us
I20260709 12:24:43.305455 13173 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 3.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:24:43.305867 12892 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:43.119716 13166 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:43.129297 13270 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6.
I20260709 12:24:43.309250 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.344499  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.245443  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:43.351986  9948 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:43.352386 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.310918 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.184943 13150 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:43.166749 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 776) took 2941 ms. Trace:
I20260709 12:24:43.235507 12917 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:43.328904  9987 connection.cc:256] RPC call timeout handler was delayed by -1.3006575819999853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20270000000000002s for a response
I20260709 12:24:43.146173 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.329926 13168 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.332482 13234 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:43.119345 13221 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 2.056s	user 0.003s	sys 0.000s
I20260709 12:24:43.336053 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.333626  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.340852  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1469) took 2116 ms. Trace:
I20260709 12:24:43.374871 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.251816 13174 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:43.334358 13128 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.264531 13006 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.113327 13269 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.351282 10097 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:43.354140 10091 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:43.303050  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:43.355036 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.355887 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.264242 13184 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:43.343902 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:43.359408  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 114042us
I20260709 12:24:43.390190 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.391328 10023 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:43.391958 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.360414 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.361322 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:43.318246 13172 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:43.361933 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.365664  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:43.397508 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.398612 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.399435 10093 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:43.371438 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.362946 10092 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:43.369709 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:43.375267  9946 rpcz_store.cc:276] 0709 12:24:41.224432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:41.229336 (+  4904us) service_pool.cc:224] Handling call
0709 12:24:41.229698 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:41.271267 (+ 41569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:41.271298 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:41.281378 (+ 10080us) log.cc:844] Serialized 26 byte log entry
0709 12:24:41.714152 (+432774us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:41.714179 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:41.714228 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:43.163386 (+1449158us) spinlock_profiling.cc:243] Waited 3.74 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:43.336756 (+173370us) raft_consensus.cc:1685] finished
0709 12:24:43.336775 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.340780 (+  4005us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":29370,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":7659,"spinlock_wait_cycles":8598784,"thread_start_us":367854,"threads_started":1,"wal-append.queue_time_us":507767}
I20260709 12:24:43.329980 12979 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.320079 13204 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:43.262907 13152 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.384737  9949 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:43.388579  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 85628.4us
I20260709 12:24:43.377573 13150 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.389335 10672 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:43.422487 10021 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:43.425925 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:43.406270 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.426698 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.406698 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.427816 10022 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:43.428227 10016 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:24:43.363675 13028 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.308154 13146 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:43.444198 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100358us
I20260709 12:24:43.409647 10018 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:43.444995 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:43.445676 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:43.446482 10024 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.413870 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:43.419335 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:43.422286 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:43.341746 13135 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:43.370630 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:43.429806 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.451449 10674 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:43.394238 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.433079 10099 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:43.458194 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.458567 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.458971 10672 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:43.435030  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 427995us
I20260709 12:24:43.452478  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:43.372483 13166 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:24:43.453567 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.466024 10095 raft_consensus.cc:2410] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 8 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:24:43.316085 13183 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.454008 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.455014 10023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df211fae593420e88f685abc8d618ec"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:43.432770 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.467231 10023 raft_consensus.cc:606] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:24:43.456646 10019 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:43.467724 10023 raft_consensus.cc:614] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:43.467662 10092 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:43.476451 10095 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:24:43.476552 10099 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:43.478380 10095 pending_rounds.cc:85] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 4
I20260709 12:24:43.480274 10095 pending_rounds.cc:107] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:24:43.424216 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.389695 13174 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.396960 13006 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.417948 13184 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.479600 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:43.480500  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:43.712199 13152 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:43.662287 12892 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:43.712577 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.712829 12892 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:43.713621 10019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:43.714152 10019 raft_consensus.cc:606] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:43.713766 12892 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:43.714483 10019 raft_consensus.cc:614] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:43.715579  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 161639us
I20260709 12:24:43.556692 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.716464  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:43.566766 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.717141 10095 raft_consensus.cc:2889] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.517727 10099 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:43.716560 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:43.618403 10092 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:43.636214 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 317129us
I20260709 12:24:43.645779 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:43.497386 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52471.5us
I20260709 12:24:43.481016 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.433786 13028 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:43.720712 10021 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:43.507552 10020 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:43.556877 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.727231 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:43.721939 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1683) took 1002 ms. Trace:
I20260709 12:24:43.740314 10099 rpcz_store.cc:276] 0709 12:24:42.721321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.477669 (+756348us) service_pool.cc:224] Handling call
0709 12:24:43.478037 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.721686 (+243649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.721886 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:43.721468  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 241081us
I20260709 12:24:43.717692 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:43.722407  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:43.722666 10025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.023s	sys 0.066s
W20260709 12:24:43.742170 10025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.024s	sys 0.066s
I20260709 12:24:43.721659 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:43.457751 12917 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.557581 10094 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:24:43.336031 13050 raft_consensus.cc:2955] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 6 LEADER]: Committing config change with OpId 6.7: config changed from index 4 to 7, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:43.554787 13014 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.220s
I20260709 12:24:43.457934 13172 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:43.726665 10090 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:24:43.715407 12892 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:43.734462 13152 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:43.765960 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.806782 13152 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:43.807058 10022 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:43.807258 13152 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.808049 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.808868 10022 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:24:43.808931 10090 pending_rounds.cc:85] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 4
I20260709 12:24:43.809334 10090 pending_rounds.cc:107] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:24:43.809592 10090 raft_consensus.cc:2889] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.572324 13287 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.809796 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:43.810197 13287 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.739653 10023 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:43.812188 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.752488 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:43.444682 13150 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.459396 12979 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:43.429884 13204 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:43.499716 13149 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.571975 13218 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:43.748845 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 482) took 1178 ms. Trace:
I20260709 12:24:43.465816 13022 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.377s
I20260709 12:24:43.749466 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.499435 10668 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:44.052793 10676 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.002s	sys 0.013s
I20260709 12:24:43.496869 13234 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:44.339217 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:44.339954 10020 raft_consensus.cc:2376] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6. Re-sending same reply.
I20260709 12:24:44.340802 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:44.341760 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:43.579775 13180 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:43.780723 13166 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:43.734143 13028 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.896612 10091 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:24:44.355508 10091 pending_rounds.cc:85] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 6
I20260709 12:24:43.739866 13290 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:43.750512  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:43.810535 10022 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:43.804598 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:43.911856 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:43.818188 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:44.357633 10671 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:43.756940 10099 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:24:44.358690 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1688) took 1599 ms. Trace:
I20260709 12:24:44.359035 10099 rpcz_store.cc:276] 0709 12:24:42.759428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.756308 (+996880us) service_pool.cc:224] Handling call
0709 12:24:43.756615 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.358420 (+601805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.358636 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:43.981487 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1696) took 1220 ms. Trace:
I20260709 12:24:43.912186 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:43.766448 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:43.974007 10024 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.000s
I20260709 12:24:44.327106 10672 raft_consensus.cc:2376] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7. Re-sending same reply.
I20260709 12:24:44.328469  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.362406 10024 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:44.362995  9948 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:44.009568 10093 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:44.363662 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2765) took 1601 ms. Trace:
I20260709 12:24:43.773488 13269 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:44.364058 10093 rpcz_store.cc:276] 0709 12:24:42.765390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.895072 (+1129682us) service_pool.cc:224] Handling call
0709 12:24:43.982058 (+ 86986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.363427 (+381369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.363611 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.364563 13290 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:43.814118 12892 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.365172 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:44.365327 13150 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:44.365907 10093 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:44.367022 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:44.367652 10668 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.368374 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:44.369166 10093 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:44.369999 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:43.760001 13174 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:43.572144 13235 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:43.810307 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1675) took 1102 ms. Trace:
I20260709 12:24:44.336695 13128 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:43.725481 13072 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
I20260709 12:24:43.593397 13283 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:43.721580 12938 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:44.354166 13300 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:43.813264  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:43.613000 13183 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:44.345149  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:44.405254  9944 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.108712 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2768) took 1325 ms. Trace:
I20260709 12:24:44.348121 10020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:44.406836 10020 raft_consensus.cc:606] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:43.867182 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 403969us
I20260709 12:24:43.792465 13291 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:43.753990 13299 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.348778  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.006s	sys 0.011s
W20260709 12:24:44.419716  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.006s	sys 0.011s
I20260709 12:24:44.422755  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:44.351011  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.424269  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.329643 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:44.284873 13294 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.574s	user 0.002s	sys 0.001s
I20260709 12:24:43.817618 12917 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:44.352807  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:44.433530 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.029s	sys 0.042s
I20260709 12:24:44.433789  9949 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:44.433877 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.029s	sys 0.042s
I20260709 12:24:44.353425  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.355759 10091 pending_rounds.cc:107] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:24:44.434492 10091 raft_consensus.cc:2889] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:43.897938 10098 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:44.328642 10094 rpcz_store.cc:276] 0709 12:24:42.569937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.466855 (+896918us) service_pool.cc:224] Handling call
0709 12:24:43.557288 (+ 90433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:43.748431 (+191143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.748765 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.363060 10096 rpcz_store.cc:276] 0709 12:24:42.761438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.896817 (+1135379us) service_pool.cc:224] Handling call
0709 12:24:43.980808 (+ 83991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.981047 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:43.981084 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:43.981138 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:43.981185 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.981257 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.981407 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:43.944214 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:44.360199 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:44.457715 10099 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:43.774287 13168 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:43.775775 13146 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:44.313815 13006 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:44.374683 10090 rpcz_store.cc:276] 0709 12:24:42.707890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.475284 (+767394us) service_pool.cc:224] Handling call
0709 12:24:43.716694 (+241410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.810063 (+ 93369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:43.810239 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:44.333318 10676 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.996s	user 0.002s	sys 0.021s
I20260709 12:24:44.382350 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:44.467123 10093 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.351945 13180 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.348676 13166 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.389612  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.472865  9948 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:44.389987  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.066293 10092 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:44.474905 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1686) took 1716 ms. Trace:
I20260709 12:24:44.475224 10092 rpcz_store.cc:276] 0709 12:24:42.758884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.580216 (+821332us) service_pool.cc:224] Handling call
0709 12:24:43.618237 (+ 38021us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.474656 (+856419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.474850 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.473868  9948 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:44.316422  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.000s	sys 0.010s
I20260709 12:24:44.398025  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.343324 13184 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:44.418648 13014 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:44.400692  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 587466us
I20260709 12:24:44.426082 10021 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:44.452584 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 725387us
I20260709 12:24:44.489753 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:44.453722 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1694) took 1692 ms. Trace:
I20260709 12:24:44.455986 10668 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:44.339330 13149 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:44.459240 10671 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:24:44.459589 10062 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.004s	sys 0.013s
I20260709 12:24:44.376531 13235 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:44.463044  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:44.492761 10062 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.004s	sys 0.013s
I20260709 12:24:44.464560 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:44.493072 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 773598us
I20260709 12:24:44.493546 10094 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.457859  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:44.471377 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2764) took 1709 ms. Trace:
W20260709 12:24:44.472245 10095 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.005s	sys 0.015s
W20260709 12:24:44.477216  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.000s	sys 0.011s
I20260709 12:24:44.410197 10097 rpcz_store.cc:276] 0709 12:24:42.783487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.106165 (+1322678us) service_pool.cc:224] Handling call
0709 12:24:44.106647 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.106862 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:44.106886 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.106930 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:44.108356 (+  1426us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.108454 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.108632 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.401187 13187 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:44.467628 13180 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.420279 13283 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:44.461642 13154 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.120s	user 0.016s	sys 0.001s
W20260709 12:24:44.494935  9942 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.009s	sys 0.013s
W20260709 12:24:44.501081  9942 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.010s	sys 0.013s
I20260709 12:24:44.502174  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1730) took 1264 ms. Trace:
I20260709 12:24:44.490444 10098 rpcz_store.cc:276] 0709 12:24:42.760909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.897342 (+1136433us) service_pool.cc:224] Handling call
0709 12:24:43.897708 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.453418 (+555710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.453623 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.497546 10091 rpcz_store.cc:276] 0709 12:24:42.762414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.895760 (+1133346us) service_pool.cc:224] Handling call
0709 12:24:43.896170 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.471110 (+574940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.471319 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:44.497678 10095 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.005s	sys 0.016s
W20260709 12:24:44.497959  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 756472us
I20260709 12:24:44.394400 13072 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.499552 10670 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:44.511641 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:44.510169 10097 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:44.517457 10094 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
W20260709 12:24:44.432135 13289 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.000s	sys 0.017s
W20260709 12:24:44.494891 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.512238  9946 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:44.480346 13166 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.524117 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:44.532768 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:44.526173 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2769) took 1736 ms. Trace:
I20260709 12:24:44.534190 10097 rpcz_store.cc:276] 0709 12:24:42.790256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.509700 (+1719444us) service_pool.cc:224] Handling call
0709 12:24:44.509994 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.525837 (+ 15843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.526088 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:44.536253 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.000s	sys 0.024s
W20260709 12:24:44.528201 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.484457 13234 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:44.539628 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:44.540972 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:44.520236  9942 rpcz_store.cc:276] 0709 12:24:43.237685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.249442 (+ 11757us) service_pool.cc:224] Handling call
0709 12:24:43.249711 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.249911 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:43.263543 (+ 13632us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.719222 (+455679us) log.cc:844] Serialized 14 byte log entry
0709 12:24:43.781249 (+ 62027us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b180034ea60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:43.817004 (+ 35755us) spinlock_profiling.cc:243] Waited 15.8 ms on lock 0x7b180034ea60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:44.501567 (+684563us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.501744 (+   177us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:44.501837 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.501937 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.502101 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2912,"prepare.queue_time_us":14882,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":75,"raft.queue_time_us":138186,"raft.run_cpu_time_us":3418,"raft.run_wall_time_us":19478,"spinlock_wait_cycles":93200896,"thread_start_us":1144694,"threads_started":2,"wal-append.queue_time_us":490665}
W20260709 12:24:44.530932 13289 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.000s	sys 0.017s
I20260709 12:24:44.544899 10093 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:44.500378 13235 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:44.523072 13184 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.549539 10096 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:44.553182 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 492) took 1292 ms. Trace:
W20260709 12:24:44.530149 10673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.004s	sys 0.023s
I20260709 12:24:44.500201 13006 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:44.505898 13303 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:44.553620 10096 rpcz_store.cc:276] 0709 12:24:43.260532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.533254 (+1272722us) service_pool.cc:224] Handling call
0709 12:24:44.549312 (+ 16058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.552879 (+  3567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.553119 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:44.555968 10093 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:44.557061 13150 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:43.362003 10675 rpcz_store.cc:276] 0709 12:24:40.225770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:40.237778 (+ 12008us) service_pool.cc:224] Handling call
0709 12:24:42.674136 (+2436358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:42.679296 (+  5160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.679345 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:42.680196 (+   851us) log.cc:844] Serialized 52 byte log entry
0709 12:24:42.793417 (+113221us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:42.794411 (+   994us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:42.801692 (+  7281us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:42.838976 (+ 37284us) raft_consensus.cc:1685] finished
0709 12:24:42.847970 (+  8994us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:43.166410 (+318440us) spinlock_profiling.cc:243] Waited 2.35 s on lock 0x7b54004d35d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:43.166643 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37975,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":17757,"spinlock_wait_cycles":5404637952,"thread_start_us":101673,"threads_started":1,"wal-append.queue_time_us":152650,"wal-append.run_cpu_time_us":1735,"wal-append.run_wall_time_us":1025895}
I20260709 12:24:44.566480 10093 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:44.571334 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.824s	user 0.000s	sys 0.024s
I20260709 12:24:44.504446 13014 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:44.511351 13187 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.511216 13154 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.458806 13299 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:44.552726 12917 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:44.554147 10673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.847s	user 0.004s	sys 0.023s
I20260709 12:24:44.547699 13301 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:44.578267 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.006s	sys 0.023s
W20260709 12:24:44.592126 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.007s	sys 0.023s
I20260709 12:24:44.569361 10093 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:44.494414 13172 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.564565 13183 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:44.524953 13288 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.005s	sys 0.021s
I20260709 12:24:44.544091 13283 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:24:44.601074 13135 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.422521 13300 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:44.565963 13006 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.339478 13218 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:44.547909 13234 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.596963  9945 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:44.565233 12938 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:44.604408 10094 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:44.604804 10093 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:44.563941 13184 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.584173 13146 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:44.608547 12523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.013s	sys 0.014s
I20260709 12:24:44.607280 10643 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:44.602353 13135 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:44.571230 13303 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.591176 13301 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:44.592641 13287 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:44.593204 13316 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:44.603449 10090 raft_consensus.cc:2955] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 4 to 7, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:24:44.578771 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 857139us
I20260709 12:24:44.606305 12917 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:44.606030 13204 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:44.625022 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:24:44.657153 10062 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.625s (SENT)
W20260709 12:24:44.714758 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57659.9us
W20260709 12:24:44.715658 10062 leader_election.cc:336] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.644s (SENT)
W20260709 12:24:44.716320 10062 leader_election.cc:336] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.706s (SENT)
W20260709 12:24:44.716910 10062 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.718s (SENT)
W20260709 12:24:44.717480 10062 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.718s (SENT)
I20260709 12:24:44.717890 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:44.675398 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2772) took 1805 ms. Trace:
I20260709 12:24:44.718837 10094 rpcz_store.cc:276] 0709 12:24:42.869488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.531113 (+1661625us) service_pool.cc:224] Handling call
0709 12:24:44.531512 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.614581 (+ 83069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.674227 (+ 59646us) spinlock_profiling.cc:243] Waited 72.5 ms on lock 0x7b5400163d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:44.674425 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166810880}
I20260709 12:24:44.720345 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:44.623092 13288 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.005s	sys 0.022s
I20260709 12:24:44.614504 12938 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:44.702178 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:44.734400 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 109445us
I20260709 12:24:44.741403 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:44.708045  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 389723us
W20260709 12:24:44.624024 12523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.013s	sys 0.015s
I20260709 12:24:44.720929 10094 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:44.700402 13149 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:44.722481 10093 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:24:44.728842 10062 leader_election.cc:336] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.867s (SENT)
I20260709 12:24:44.625913 13183 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:44.742774  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 992271us
W20260709 12:24:44.739104 13152 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.931s	user 0.008s	sys 0.004s
W20260709 12:24:44.746470 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44392.1us
I20260709 12:24:44.690405  9944 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:44.700002 13301 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:44.747936 10094 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:44.713964 12917 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:44.747653 13204 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:44.771262 10093 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
W20260709 12:24:44.772207  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 263313us
I20260709 12:24:44.714844 13316 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:44.774861 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:44.733721 13312 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:44.781432  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:44.742292 13072 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:44.737937 13306 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:44.795960 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:44.811952 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 83181.4us
I20260709 12:24:44.764271 13022 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:44.814561 13022 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:44.813993  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:44.815570 13022 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.797302 13309 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:24:44.816155  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:44.813076 10062 leader_election.cc:336] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.888s (SENT)
I20260709 12:24:44.813925 13072 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:44.817021 13072 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:44.814081 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:44.814550 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 478) took 2110 ms. Trace:
I20260709 12:24:44.818220 10025 rpcz_store.cc:276] 0709 12:24:42.704227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.705399 (+  1172us) service_pool.cc:224] Handling call
0709 12:24:42.735125 (+ 29726us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:42.974283 (+239158us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:42.974318 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:42.974907 (+   589us) log.cc:844] Serialized 52 byte log entry
0709 12:24:44.329043 (+1354136us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:44.803665 (+474622us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.814249 (+ 10584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.814323 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.814360 (+    37us) raft_consensus.cc:1685] finished
0709 12:24:44.814371 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.814494 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37007,"raft.queue_time_us":55530,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":8810,"thread_start_us":1984299,"threads_started":3,"wal-append.queue_time_us":935402}
I20260709 12:24:44.778316 13183 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:24:44.822553 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 281802us
I20260709 12:24:44.823606 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:44.834548 13329 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.839743 13321 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:44.801395 12917 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:44.796950 12523 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:44.839826 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 98518.8us
I20260709 12:24:44.898751 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:44.900408 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:44.901933 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:44.903764 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:44.843750 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1684) took 2111 ms. Trace:
I20260709 12:24:44.878499 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:44.817479 13072 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:44.897423 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:44.905409  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.911170 10668 raft_consensus.cc:2368] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
W20260709 12:24:44.851253  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 341531us
I20260709 12:24:44.915096 10671 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:44.910676 10095 rpcz_store.cc:276] 0709 12:24:42.732963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.477720 (+744757us) service_pool.cc:224] Handling call
0709 12:24:43.478112 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:44.524861 (+1046749us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:44.524911 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:44.525668 (+   757us) log.cc:844] Serialized 52 byte log entry
0709 12:24:44.825834 (+300166us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:44.825899 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.826015 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:44.843479 (+ 17464us) raft_consensus.cc:1685] finished
0709 12:24:44.843500 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.843687 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":72556,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":1021817,"threads_started":2,"wal-append.queue_time_us":279828}
W20260709 12:24:44.922317  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:44.795228 13152 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.821830 13312 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:44.840816 13317 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:44.941141  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 159819us
I20260709 12:24:44.848841 13306 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:44.955561 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:44.873485 13183 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:44.961122 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:44.963491 13014 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:44.964763 13329 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:44.965680  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:44.888041 12892 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:44.974814 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:44.975124  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:44.977322 10672 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:44.812211 13308 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.151s
W20260709 12:24:44.999454 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 150127us
W20260709 12:24:45.020298  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 212642us
W20260709 12:24:45.023996  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49063.3us
W20260709 12:24:44.971755 13310 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 12:24:45.025298  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:44.983968  9943 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:24:44.978225 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 667) took 1662 ms. Trace:
I20260709 12:24:45.034540 10671 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:45.030676 10676 rpcz_store.cc:276] 0709 12:24:43.315804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.315940 (+   136us) service_pool.cc:224] Handling call
0709 12:24:43.316238 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.316447 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:44.467083 (+1150636us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.977749 (+510666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.977825 (+    76us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:44.977894 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.977983 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:44.978149 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5884,"prepare.queue_time_us":1206761,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":2749,"raft.queue_time_us":45938,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":1629431,"threads_started":3,"wal-append.queue_time_us":365768}
I20260709 12:24:45.038944 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.039721 10676 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:45.040767 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:45.041625 10676 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:45.044269 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.047143 10671 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:44.908932 13289 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:44.896081 13172 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 12:24:45.061801 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:45.021061 13345 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:24:44.919101 13149 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:45.070648 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.089046 10671 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:45.020390 13305 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:44.919682 12523 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.920929 13146 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:45.096577  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 280471us
I20260709 12:24:44.981714 13325 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:45.061724 13302 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.024261 13033 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:45.059386 13154 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.104749  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:45.024040 13331 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:44.841567 12938 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:44.919495 13341 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.192s
I20260709 12:24:45.066987 13218 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.072674 13234 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.071502 13329 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:45.116865  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:45.094805 13150 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.132645 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.098320 13006 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.233011 13302 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:45.233295 13316 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.233652 13302 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:45.234323 13150 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.235575 13150 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:45.234354 13187 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.236085 13150 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:45.152197 13204 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 12:24:45.127820 13033 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.240485 13033 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.241678 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.244105 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.246898 13146 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.248035 13128 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:45.248868 13128 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.130232  9912 leader_election.cc:400] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:45.110294 13351 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.104031 13300 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:45.111279 10669 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:24:45.110690 13325 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.262451 13325 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.237035 13150 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.081199 12892 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 12:24:45.176620  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59834.6us
I20260709 12:24:45.269447 13300 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:45.269870 13300 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:45.178812 13303 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.184226 13329 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.270730 13300 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.272353 13300 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.184897 13072 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.186928 13352 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.101511 13149 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.160562 13154 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:24:45.137624 13321 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:45.234992 13302 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.280130 13302 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.280260 10672 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:45.280580 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:45.281307 10025 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:45.281582 13072 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.281790 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:45.191004 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 672) took 1625 ms. Trace:
I20260709 12:24:45.282429 10674 rpcz_store.cc:276] 0709 12:24:43.566124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.566411 (+   287us) service_pool.cc:224] Handling call
0709 12:24:45.190798 (+1624387us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540058f2d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:45.190943 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2577631104}
I20260709 12:24:45.282785 13321 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:45.283754 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.283787 13321 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.284498 10674 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:45.285231 13321 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.285722 10674 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:45.192076 13349 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.077s	user 0.003s	sys 0.000s
W20260709 12:24:45.192629 13235 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.637s	user 0.008s	sys 0.000s
W20260709 12:24:45.287269 13028 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.913s	user 0.008s	sys 0.004s
I20260709 12:24:45.287139 13235 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:45.193934 12979 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.855s	user 0.007s	sys 0.002s
I20260709 12:24:45.194016 13301 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.377879 10017 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:45.122563 13312 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:45.201104 13166 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.662s	user 0.008s	sys 0.001s
W20260709 12:24:45.203738 13180 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.674s	user 0.010s	sys 0.003s
I20260709 12:24:45.211825 13184 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.213655 13152 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:45.159255 13290 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.794s	user 0.010s	sys 0.000s
I20260709 12:24:45.252730  9912 leader_election.cc:403] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:45.138509 13050 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.332s	user 0.011s	sys 0.001s
I20260709 12:24:45.259629 13351 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:45.276492 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.419943 10670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 677) took 1664 ms (client timeout 1888 ms). Trace:
W20260709 12:24:45.278381  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.432816 10670 rpcz_store.cc:269] 0709 12:24:43.755277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.911869 (+156592us) service_pool.cc:224] Handling call
0709 12:24:45.419746 (+1507877us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b5400490a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:45.419874 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":272356096}
I20260709 12:24:45.279217 13283 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:45.257757 13128 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.291049 10674 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:45.380560 13348 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.280319 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 661) took 3279 ms. Trace:
I20260709 12:24:45.300130 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.301158 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:45.442915 10019 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.374740 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2771) took 2580 ms. Trace:
I20260709 12:24:45.381201 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2776) took 1050 ms. Trace:
W20260709 12:24:45.382299 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 478588us
W20260709 12:24:45.283809 13291 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.864s	user 0.010s	sys 0.000s
I20260709 12:24:45.450137 13291 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:45.453017 10019 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:45.454095 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.456338 10672 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:45.457828 10019 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:45.390462 13318 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.002s	sys 0.008s
W20260709 12:24:45.380954  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 276261us
I20260709 12:24:45.376597 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.468312 10025 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:45.469458 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.400784 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 673) took 1784 ms. Trace:
I20260709 12:24:45.294205 12938 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:45.403455 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 336919us
W20260709 12:24:45.471692  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 446460us
I20260709 12:24:45.412837 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.287719 13028 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.377144 12726 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.376132 13204 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:45.419745  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 289742us
I20260709 12:24:45.561210 13204 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.561957  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:45.562769 13363 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:45.562098 13128 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.563328 13204 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:45.563385  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:45.564217 13363 raft_consensus.cc:2764] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:45.564368  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:45.565156 13363 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:45.565517 13363 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:45.375725 13174 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.564172 13028 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.566852 13128 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.565834 13363 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:45.424016 10017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1616) took 2045 ms (client timeout 1644 ms). Trace:
W20260709 12:24:45.568331 10017 rpcz_store.cc:269] 0709 12:24:43.378617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.378777 (+   160us) service_pool.cc:224] Handling call
0709 12:24:45.423703 (+2044926us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54001c6454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:45.423900 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4551853312}
I20260709 12:24:45.569633 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:45.570436 10017 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:45.571298 13301 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.571435 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.572050 10017 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:45.573060 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:45.573483 12900 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:45.574056 12900 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:45.436597 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:45.437803 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:45.296136 13299 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:45.278426 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.435087 10670 raft_consensus.cc:2955] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 4 to 7, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:45.441943 10673 rpcz_store.cc:276] 0709 12:24:42.000911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:42.546027 (+545116us) service_pool.cc:224] Handling call
0709 12:24:43.128144 (+582117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:43.128430 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:43.610515 (+482085us) log.cc:844] Serialized 14 byte log entry
0709 12:24:44.590509 (+979994us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:44.590578 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:44.590671 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:44.590759 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.280145 (+689386us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b540045ae50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:45.280263 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":190775,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":17337,"spinlock_wait_cycles":1337936640,"thread_start_us":883474,"threads_started":1,"wal-append.queue_time_us":1026277}
I20260709 12:24:45.444844 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:45.586031 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 141258us
I20260709 12:24:45.587245 13303 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:45.587307 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:45.587788 13303 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:45.588313 13303 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.588590 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:45.589871 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:45.443683 10091 rpcz_store.cc:276] 0709 12:24:42.794586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.510493 (+1715907us) service_pool.cc:224] Handling call
0709 12:24:44.510801 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.511015 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:24:44.835820 (+324805us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.181518 (+345698us) spinlock_profiling.cc:243] Waited 8.84 ms on lock 0x7b18001e3dc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db
0709 12:24:45.191619 (+ 10101us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:45.374276 (+182657us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b18001e3dc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287
0709 12:24:45.374369 (+    93us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:45.374416 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:45.374457 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.374521 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.374677 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22796,"prepare.queue_time_us":318991,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":237477,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":104,"spinlock_wait_cycles":32322176,"thread_start_us":523974,"threads_started":3,"wal-append.queue_time_us":222870}
I20260709 12:24:45.591370 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:45.591555 13317 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.592620 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:45.593256 10091 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 4.
W20260709 12:24:45.593560 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.594762 13302 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.443813 10097 rpcz_store.cc:276] 0709 12:24:44.325434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.535175 (+209741us) service_pool.cc:224] Handling call
0709 12:24:44.535623 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.535892 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:44.853613 (+317721us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.109513 (+255900us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.374751 (+265238us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:45.374812 (+    61us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:45.374879 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.375759 (+   880us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.375916 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19765,"prepare.queue_time_us":295245,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":273750,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":145,"thread_start_us":784142,"threads_started":3,"wal-append.queue_time_us":332945}
I20260709 12:24:45.449429 13351 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.458796 10674 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:45.458863 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:45.596637 10019 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.461565 10094 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 12:24:45.403954  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 283621us
I20260709 12:24:45.472386 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.474318 10016 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:45.600621  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:45.601797 13028 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:45.602301 13028 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:45.591373 13368 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.602970 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:45.603247 13028 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.604070 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.604661 10016 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:45.605468 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.607167 10025 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:45.607897 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.431041 13290 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:45.479104 10640 leader_election.cc:336] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.863s (SENT)
I20260709 12:24:45.439188 13339 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.609197 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:45.610316 13302 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:45.613101 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.613783 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.614310 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.614828 10022 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:45.615908 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.616477 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1636) took 1843 ms. Trace:
I20260709 12:24:45.390713 12917 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:45.610323 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 131262us
I20260709 12:24:45.619112 12917 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:45.485678 10098 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.003s	sys 0.018s
W20260709 12:24:45.619750 10098 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.003s	sys 0.018s
I20260709 12:24:45.619889 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.620407 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2770) took 2827 ms. Trace:
I20260709 12:24:45.620811 10098 rpcz_store.cc:276] 0709 12:24:42.792897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.506096 (+1713199us) service_pool.cc:224] Handling call
0709 12:24:44.506396 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.506619 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:44.822502 (+315883us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.620071 (+797569us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:45.620127 (+    56us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:45.620181 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.620245 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.620354 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":323163,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":180273,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"thread_start_us":1085963,"threads_started":3,"wal-append.queue_time_us":362920}
I20260709 12:24:45.486443 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:45.623869 10093 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260709 12:24:45.624257 13308 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.004s	sys 0.000s
I20260709 12:24:45.624344 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:45.624800 13308 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.004s	sys 0.000s
I20260709 12:24:45.625300 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:45.625988 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:45.626819 10025 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:45.397672 13166 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.402444 13180 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.649672 13180 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.481841 13306 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:45.650851 13306 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:45.477428 13319 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.000s	sys 0.018s
W20260709 12:24:45.516078 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.444296 13283 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.495723 13364 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.655503 13364 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.495481 13325 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.457523 13348 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.500078 13183 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.534404 13235 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.532835 13149 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.658968 10024 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:45.661201 13022 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:45.663995 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 483) took 2166 ms. Trace:
I20260709 12:24:45.664533 10018 rpcz_store.cc:276] 0709 12:24:43.498088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.499244 (+  1156us) service_pool.cc:224] Handling call
0709 12:24:43.711652 (+212408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:44.773437 (+1061785us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:44.773481 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:44.774048 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:24:44.837307 (+ 63259us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:45.028264 (+190957us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.401112 (+372848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.663488 (+262376us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:45.663556 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:45.663619 (+    63us) raft_consensus.cc:1685] finished
0709 12:24:45.663734 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.663916 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43595,"prepare.queue_time_us":125559,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":212130,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":147,"spinlock_wait_cycles":1491137920,"thread_start_us":965616,"threads_started":4,"wal-append.queue_time_us":33824}
I20260709 12:24:45.666137 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:45.666853 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.667557 10018 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:45.668596 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.668886 13305 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:45.669389 13305 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.669884 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.671310 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.672181 10025 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.672585 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.673753 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:45.674754 10018 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:45.674886 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.675719 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:45.549976 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 487248us
W20260709 12:24:45.425343  9945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1491) took 2165 ms (client timeout 2063 ms). Trace:
W20260709 12:24:45.678764  9945 rpcz_store.cc:269] 0709 12:24:43.260078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.338114 (+ 78036us) service_pool.cc:224] Handling call
0709 12:24:45.425053 (+2086939us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540042de54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:45.425239 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2756800768}
I20260709 12:24:45.680598  9945 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:45.682857 12523 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:45.683023 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:45.683977 12726 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.684187 13180 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:45.691851 13325 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.565807  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:45.696969 13235 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.573539 10017 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.426540  9941 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:45.580540 10099 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:45.580947 10021 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:45.703698 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.704459  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 616) took 2443 ms. Trace:
I20260709 12:24:45.581663  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:45.704982  9941 rpcz_store.cc:276] 0709 12:24:43.260924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.381024 (+120100us) service_pool.cc:224] Handling call
0709 12:24:45.426000 (+2044976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.701771 (+275771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.704188 (+  2417us) spinlock_profiling.cc:243] Waited 2.02 s on lock 0x7b540042de50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:45.704383 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4640243456}
I20260709 12:24:45.706748  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:45.707448  9943 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:45.580071 10095 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:45.580089 12900 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.711943 13289 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:45.713280 13325 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:45.719239 12900 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.470979 10677 rpcz_store.cc:276] 0709 12:24:43.616031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.616319 (+   288us) service_pool.cc:224] Handling call
0709 12:24:43.616696 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:44.320699 (+704003us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:24:44.732993 (+412294us) log.cc:844] Serialized 14 byte log entry
0709 12:24:45.061735 (+328742us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:45.061800 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:45.061859 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.400535 (+338676us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400723654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:45.400583 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:45.400714 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29045,"prepare.queue_time_us":276689,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":3228,"raft.queue_time_us":40642,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":11739,"spinlock_wait_cycles":481014656,"thread_start_us":687289,"threads_started":2,"wal-append.queue_time_us":444684}
I20260709 12:24:45.722078 10677 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:45.723443 10677 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:45.725780 10677 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:45.627451 13356 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:45.726786 13356 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:45.727759 13356 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:45.728926 13356 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:45.676486 10018 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:45.629303  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.630267  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:45.613812 10674 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:45.631481 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.747303 10673 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:45.632143  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.602308  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:45.748903  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 146659us
I20260709 12:24:45.749974  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:45.633241 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:45.634426  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:45.751905  9947 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:45.467486 13318 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.003s	sys 0.008s
I20260709 12:24:45.625995 10672 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.616920 10022 rpcz_store.cc:276] 0709 12:24:43.772800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.357179 (+584379us) service_pool.cc:224] Handling call
0709 12:24:44.379168 (+ 21989us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:45.615265 (+1236097us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:45.616237 (+   972us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54001f8e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:45.616414 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2841312640}
I20260709 12:24:45.642721 10669 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:45.658223 10675 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:45.675971 10024 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:45.730497  9942 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:45.536600 10062 leader_election.cc:336] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.888s (SENT)
I20260709 12:24:45.775430 12523 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:45.782444  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:45.745697 13289 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:45.790373 10677 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:45.812237 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:45.730252 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.770665 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:45.813743  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 248007us
I20260709 12:24:45.814085 10019 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:45.705026 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.838570 10021 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:45.606500 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.819031  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:45.846218 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:45.847309 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 310717us
W20260709 12:24:45.827142 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.834074 10018 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:45.848067 10062 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.910s (SENT)
I20260709 12:24:45.848477 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:45.852010  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:45.854491  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 77907.5us
I20260709 12:24:45.856406 10016 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:45.857507  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.858533 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:45.862267  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:45.862788 10025 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:45.864068  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:45.865692  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:45.869964 10062 leader_election.cc:336] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.986s (SENT)
I20260709 12:24:45.875532  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:45.876878 13362 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:45.879369 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:45.880124 10024 raft_consensus.cc:606] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:45.883699  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:45.884495  9942 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:45.885283  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.886727  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.889643 10019 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:45.891364 10019 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: [4.5-4.5]
I20260709 12:24:45.893383  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:45.789921 13234 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:45.682648  9842 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:45.878235 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:45.912839 10022 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:45.913619  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.898427 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:45.903167 13174 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.903643 12938 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:45.853924 12726 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.854171 13308 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.868144 13329 raft_consensus.cc:2764] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:45.905835 13006 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:45.883105 13150 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:45.877853 13154 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:45.878067 13149 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:24:45.907406  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:45.909598  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.790177 13235 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 12:24:45.792378 13289 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:45.870339 13382 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.898707 13291 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:45.919008  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:45.947362  9948 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:45.921680  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.947588 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:45.912477 13299 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.921957 10016 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:45.840462 13359 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:45.838291 12523 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:45.932042  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 117236us
I20260709 12:24:45.789631 13187 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.127s
I20260709 12:24:45.910650  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.916266 13183 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:45.952673  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:45.914372  9947 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
W20260709 12:24:45.945915  9843 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 320) took 17590 ms (client timeout 15000 ms). Trace:
I20260709 12:24:45.914928 10017 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:24:45.929248 12892 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:45.912030 13234 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:45.952919  9842 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 4, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 4 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:45.927460 10677 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:45.954921 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:45.976114  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.995236  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.996009  9943 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 4.
I20260709 12:24:46.001963  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:45.951915 13172 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.006240  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46937.6us
I20260709 12:24:45.929881 13174 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.989183 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:45.932853 12938 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:45.956365 13150 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:45.979070 13309 raft_consensus.cc:2764] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:24:46.012501 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:45.972457 12726 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:46.014819 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:46.031169 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.031134  9842 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 4, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 4 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:45.982791 13234 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:45.943392 13318 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
W20260709 12:24:46.041433  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 148137us
W20260709 12:24:45.960210  9843 rpcz_store.cc:269] 0709 12:24:28.356041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:28.356385 (+   344us) service_pool.cc:224] Handling call
0709 12:24:45.945840 (+17589455us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:41.079184 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:41.288590 (+209406us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:41.295073 (+  6483us) write_op.cc:435] Acquired schema lock
0709 12:24:41.295093 (+    20us) tablet.cc:662] Decoding operations
0709 12:24:41.624671 (+329578us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:41.624804 (+   133us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:41.624846 (+    42us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:24:41.681394 (+ 56548us) tablet.cc:701] Row locks acquired
0709 12:24:41.681448 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:24:41.681650 (+   202us) write_op.cc:270] Start()
0709 12:24:41.681820 (+   170us) write_op.cc:276] Timestamp: P: 1783599881681567 usec, L: 0
0709 12:24:41.681844 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:24:41.682567 (+   723us) log.cc:844] Serialized 5231 byte log entry
0709 12:24:42.246131 (+563564us) op_driver.cc:464] REPLICATION: finished
0709 12:24:42.649986 (+403855us) write_op.cc:301] APPLY: starting
0709 12:24:42.664121 (+ 14135us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:42.750709 (+ 86588us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:42.750866 (+   157us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:43.055427 (+304561us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:43.055764 (+   337us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 12:24:43.055809 (+    45us) write_op.cc:312] APPLY: finished
0709 12:24:43.056210 (+   401us) log.cc:844] Serialized 95 byte log entry
0709 12:24:43.056407 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:43.056550 (+   143us) write_op.cc:454] Released schema lock
0709 12:24:43.056616 (+    66us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:43.100654 (+ 44038us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b54003297a8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f9129e1 00007f4e0f91164c 00007f4e0cfa23fd 00007f4e0cf8e389 00007f4e0cf8db53 00007f4e0cf91d61 00007f4e0cf91d19 00007f4e0cf91ca9 00007f4e0cf91c71 00007f4e0cf90e9d 0000000000388634
0709 12:24:43.101201 (+   547us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:43.101235 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":561102,"reported_tablets":41,"child_traces":[["op",{"apply.queue_time_us":374137,"apply.run_cpu_time_us":8982,"apply.run_wall_time_us":471732,"num_ops":13,"prepare.queue_time_us":815790,"prepare.run_cpu_time_us":35491,"prepare.run_wall_time_us":776859,"raft.queue_time_us":255411,"raft.run_cpu_time_us":54070,"raft.run_wall_time_us":281315,"replication_time_us":564122,"spinlock_wait_cycles":4158208,"thread_start_us":1088221,"threads_started":4,"wal-append.queue_time_us":193793,"wal-append.run_cpu_time_us":30497,"wal-append.run_wall_time_us":2193182}]]}
I20260709 12:24:46.051100  9842 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 4 to 5, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 5 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:46.056555 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44148.4us
I20260709 12:24:46.062343 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:46.063289  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 194263us
I20260709 12:24:46.063680 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:46.066080  9942 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:46.000464 13382 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:46.070777 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:45.980684 13373 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:46.060436 10017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:46.131525 10017 raft_consensus.cc:606] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:46.131907 10017 raft_consensus.cc:614] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:45.895469 13351 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:24:46.009341 13317 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:46.071624  9947 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:46.025108 12892 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:46.095315 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:45.981459 13321 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:24:46.025081 13348 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:46.050894 13308 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:46.003000 13359 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 12:24:46.130421  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 158298us
I20260709 12:24:46.006771 13152 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:24:46.098405  9842 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:24:46.145891  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52890.4us
I20260709 12:24:46.074111 13204 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:46.078156 13299 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:46.019378 13172 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.162436  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:46.035506 13028 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:24:46.079182 13364 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.004652 13289 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.049651 13050 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
W20260709 12:24:46.172418 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 92088.8us
I20260709 12:24:46.100255 12726 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:46.146183 13287 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:46.181365  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:46.207537  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:46.208451  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:46.169826 13339 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.560s	user 0.012s	sys 0.002s
I20260709 12:24:46.175040 13348 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:46.211514  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 204326us
I20260709 12:24:46.170096 13382 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:46.193054  9842 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.151222 12892 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:46.223325 12892 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:46.174247 13368 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.003s
I20260709 12:24:46.226938 13368 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:46.202512 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:46.228828 13283 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:46.229471 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:46.231488 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:46.232499 13154 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:46.163144 13308 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.073242 12523 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:46.234529 13308 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:46.239660 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.241354 13174 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.175355 13299 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:46.242887 13299 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:46.245896 13291 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:46.178377 13374 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:46.170457 13318 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
W20260709 12:24:46.246739 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.247723 13364 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:46.248584 13364 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:46.248836 13303 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.660s	user 0.006s	sys 0.004s
I20260709 12:24:46.249390 13364 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:46.249310 13303 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:46.214146  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 51833.1us
I20260709 12:24:46.250921  9987 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:46.251396 13204 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.251933 13204 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:46.253916 13204 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:46.221756 13382 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:46.255566 13382 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:46.222409 13149 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:46.218757 13348 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:46.258314 13318 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:46.259378 13128 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:46.260231 13128 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:46.260545 13128 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:46.261294 13363 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.693s	user 0.008s	sys 0.000s
I20260709 12:24:46.261408 13128 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.261760 13363 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.264038 13172 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:46.211531 13329 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.265028 13329 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:46.267369 13329 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:46.154045 13370 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:24:46.268383 10670 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:46.260438  9842 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 4, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 4 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:46.271982 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:46.242390 12938 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.272166 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:46.232880 13359 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:46.246374 13291 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:46.274757 10675 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:46.274068 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:46.279345  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:46.281080 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:46.211156 12726 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:46.283773 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.283636 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:46.268404 13370 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:46.281025 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:46.285266  9945 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:46.285177  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:46.285931  9948 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:24:46.221102 13339 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
W20260709 12:24:46.273916 13356 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.545s	user 0.001s	sys 0.014s
I20260709 12:24:46.303859 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.288802 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.232928 13166 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:46.281097 10672 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.528s	user 0.007s	sys 0.002s
I20260709 12:24:46.309567 10672 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:46.276867 13128 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:46.294337 10675 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:46.300030 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:46.294807  9940 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:46.306448 10023 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:46.298237 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:46.275671 13359 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:46.297866 10062 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.789s (SENT) [suppressed 26 similar messages]
I20260709 12:24:46.286726 12726 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:46.284924 13183 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:46.312619 10676 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:46.289480 13368 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:46.330611 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.341866 10670 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:46.329527 10677 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:46.300961 13329 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.053s
W20260709 12:24:46.333488 13394 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:24:46.309002 13022 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.647s	user 0.013s	sys 0.004s
I20260709 12:24:46.286011 13172 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:24:46.277314 13318 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:24:46.332824 13348 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:46.302878 13362 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:46.354756 13291 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.363030 10023 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:46.364595 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.293715 13370 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:46.337570 13006 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:46.380280 10670 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:46.370885 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.381325 10017 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:46.371694 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:46.382462 10016 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:46.373644 10020 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:46.330055 13303 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:46.282109 13365 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:24:46.364192 13172 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:46.386560 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.212144 13150 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.390713 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:46.381752  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2675877120000223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
I20260709 12:24:46.383526  9946 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:46.383899 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.374547 13309 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.389184 13172 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:46.396078 13291 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:46.403873 10024 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:46.394531 13348 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:46.405987  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2009877119999146s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2242s for a response
I20260709 12:24:46.392623 13329 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.014s
W20260709 12:24:46.418323 12900 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.002s
I20260709 12:24:46.405155 13308 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:46.441701 10668 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:46.443959 13362 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:46.444514 13362 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:46.407327 13309 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.420732 13363 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.403252 13370 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.416805 12892 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:46.456794  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.415266 13348 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.463665  9944 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
W20260709 12:24:46.458253  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 361809us
I20260709 12:24:46.464849  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:46.468542  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:46.470202  9941 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:46.474466  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:46.472848  9942 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:46.475369  9945 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:46.476562  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:46.477461  9949 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:24:46.487862  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:46.497861  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:46.516448  9941 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:46.481608 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:46.518425 12938 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:46.518836 12938 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.519263 12938 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:46.523259 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.523659  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:46.511322  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:46.501650 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 211572us
W20260709 12:24:46.529387 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.531734 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:46.516260 13305 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.846s	user 0.009s	sys 0.010s
I20260709 12:24:46.496882 13321 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:46.537190 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:46.565582 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:46.566402 10677 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:46.528436 13367 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:46.535983 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:46.566784 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:46.536499 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:46.499083 13006 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:24:46.602211 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:46.602903 10672 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:46.597904  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 187988us
W20260709 12:24:46.600071 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 302251us
I20260709 12:24:46.506124 12900 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:24:46.491228 13396 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:24:46.610033 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:46.514662 13329 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:24:46.614019 13403 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:46.614136 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:46.615224 13402 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:46.601192 13287 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
W20260709 12:24:46.616959 13395 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:24:46.621374  9947 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:46.623546 10677 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:46.624111 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52991.5us
W20260709 12:24:46.627586  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 116232us
W20260709 12:24:46.632791  9912 connection.cc:256] RPC call timeout handler was delayed by -1.1190877119997822s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23330000000000004s for a response
W20260709 12:24:46.633397  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.634330  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:46.647538 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:46.650010 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:46.619360 12938 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.611896 13321 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:46.565858 13305 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.607256 13290 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:46.611023 13174 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:46.656878  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:46.667650 13399 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:24:46.668507 13401 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:46.669019 13401 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:46.669061 13348 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.669508 13401 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:46.623253 13377 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:46.671507 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2691) took 1166 ms. Trace:
I20260709 12:24:46.678103 10025 rpcz_store.cc:276] 0709 12:24:45.504746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.863588 (+358842us) service_pool.cc:224] Handling call
0709 12:24:45.863903 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:45.864119 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:45.864159 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:45.931215 (+ 67056us) log.cc:844] Serialized 26 byte log entry
0709 12:24:46.293013 (+361798us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:46.293068 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:46.293321 (+   253us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:46.671295 (+377974us) raft_consensus.cc:1685] finished
0709 12:24:46.671320 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:46.671448 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":268631,"threads_started":1,"wal-append.queue_time_us":461319}
I20260709 12:24:46.672986  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:46.680573  9947 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:46.691444  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:46.692664 12726 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.693315 13363 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:46.693735 13363 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.694069 13363 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:46.696832 13362 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.700876  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:46.658241  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 134658us
I20260709 12:24:46.697989 13172 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.706899 10676 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:46.665009 13234 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.000s
I20260709 12:24:46.666447 13321 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.654820 12900 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.727638 10670 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:46.654517 13351 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:46.732213 13362 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.698060 12726 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.725525 13397 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:46.734161 13172 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:46.655033 13405 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:46.735162 13235 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:24:46.736812  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:46.737736  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:46.744220 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.685285 13306 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:46.799973  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62347.9us
I20260709 12:24:46.665947 13290 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:46.730023 13303 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:46.736495 13321 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:46.748876 13406 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:24:46.805187 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:46.743600 13348 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.681694 13174 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:46.833244 13414 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 12:24:46.834308 13413 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:24:46.835155 13287 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.819114 13303 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:46.835716 13287 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.837152  9945 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:46.838446 13348 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.839408 13290 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.840478 13290 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:46.841260 13348 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.757481 13362 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.725531 13022 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:24:46.736757 13234 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:46.854668 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:46.716317 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64817.8us
I20260709 12:24:46.819340 13312 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:46.832245 10022 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:24:46.739228 13166 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.800808 13402 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:46.873685 10022 pending_rounds.cc:85] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 4
I20260709 12:24:46.799577 13306 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.744366 12900 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.876588  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:46.883859  9945 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:46.884922  9947 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:46.885593  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:46.888243 10022 pending_rounds.cc:107] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:24:46.888636 10022 raft_consensus.cc:2889] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:46.890300  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:46.890937  9949 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:24:46.894574 10024 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:46.751957 13374 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.126s
I20260709 12:24:46.884112 13166 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.899636 13234 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:46.901710 13166 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:46.869956 12892 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.871136 13287 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.872292 13312 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:46.920876 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.923322 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:46.914230 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:46.918642 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:46.877066 13174 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:46.887773 13306 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:46.892643 13022 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:46.927534 10021 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:24:46.928362 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 58678.2us
W20260709 12:24:46.953151 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 303227us
I20260709 12:24:46.875851 13362 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:46.933480  9949 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:46.927048 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:46.956212  9942 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:46.940667 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:46.944852 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:46.956276 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:46.957064 10024 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:46.957108 10019 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:46.959586 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:46.968230 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:46.969051 10099 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:46.970021 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:46.970821 10098 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:46.970957 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:46.971803 10095 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:46.917452 12938 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:46.957516 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:46.977639 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:46.978681 10675 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:46.926965 13312 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:46.892833 13402 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:46.990873  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:47.001700 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:47.005352 10016 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:46.998221 10095 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:47.006394 10096 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:47.007742 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:47.008561 10093 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:46.917188 13150 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:24:47.014933 10092 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:47.020038 10098 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:46.953521 13022 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:47.024628 13426 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.025256 13426 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.025791 13362 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:47.027427 13426 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:47.028601 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:47.033239 10099 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:47.028926 10019 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:47.029361 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:46.975236 13174 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:47.034961 10091 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:47.035646 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.039593 10024 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:46.978909 13006 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.040875 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.043063 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:47.044786 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:46.982035 12892 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:47.048456 10022 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:47.050303 10090 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:47.027571 13422 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 12:24:47.018879 13305 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:47.000330 13421 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:47.060355 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.034942 12726 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.040867 12938 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:47.046777 13377 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:47.048239 13301 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:47.053689 12892 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.078131 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.080276 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:47.069478 13299 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.826s	user 0.013s	sys 0.000s
W20260709 12:24:47.072816 13411 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.319s	user 0.000s	sys 0.001s
I20260709 12:24:47.092272 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.094862 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.099195 10021 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 4. Current term is 5. Ops: []
W20260709 12:24:47.102169 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.072679 13174 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:47.103041 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.076555 12938 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.053253 13006 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:47.103907 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 124536us
I20260709 12:24:47.097569 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:47.093825 12892 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.132808 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.134518 10022 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.132436 10023 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:47.133276 10093 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:47.136384 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:47.138072 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:47.139001 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:47.132336 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.140228 10021 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:47.141348 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.141945 10021 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:47.149367 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:47.078445 12726 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:47.159109 13433 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:24:47.160584 13385 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.169s	user 0.004s	sys 0.003s
I20260709 12:24:47.170967 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:47.117893 12938 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.096823 13299 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.165372 13424 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
I20260709 12:24:47.153551 13426 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:47.218626 13426 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:47.219566 13426 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.187875 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.152733 13425 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:47.232419 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:47.233392 10670 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:47.194226 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.235848 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.236629 10674 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.201930 13306 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.203394 13424 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:47.205714 13291 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.797s	user 0.009s	sys 0.000s
I20260709 12:24:47.205552 13299 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:47.206679 13437 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:24:47.206492 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.207399 13006 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:47.208204 10669 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.370s	user 0.012s	sys 0.001s
I20260709 12:24:47.208292 12726 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:47.193321 13434 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:24:47.211326 13368 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.856s	user 0.008s	sys 0.003s
I20260709 12:24:47.211488 13402 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:47.163271 12900 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:47.212754 13321 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.162245 13363 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:47.215505 13401 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.546s	user 0.006s	sys 0.004s
I20260709 12:24:47.184301 13431 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:24:47.217931 13439 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:47.153148 13305 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:47.229518 13204 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.256733 13421 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:47.266410 13424 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:47.268649 13368 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:47.261621 13312 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.197186 13301 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:47.261592 13308 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.277805 13439 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.281219 13305 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:47.275727 13401 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.295691 13359 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.295957 13305 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.297415 13397 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:47.290191 13321 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:47.291208 13309 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:47.291496 13363 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:47.299033 13381 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.220s	user 0.001s	sys 0.009s
I20260709 12:24:47.290385 13312 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:47.300962 13401 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:47.287600 13439 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.296837 13359 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.290886 13424 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.306527 13351 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:47.306981 13351 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:47.285624 13425 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:47.318545 13425 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.322193 13439 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.181324 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:47.186291  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.367647 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.370484 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.371799 10021 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:47.373466 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.374794 10021 raft_consensus.cc:2376] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8. Re-sending same reply.
I20260709 12:24:47.375387 10019 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:47.376624 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.378177 10023 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:47.380250 10024 raft_consensus.cc:2410] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:24:47.385388 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.387099 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.388091 10023 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:24:47.389379 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 218321us
I20260709 12:24:47.391122 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.266754 10669 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:47.427378 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 717) took 1803 ms. Trace:
I20260709 12:24:47.427887 10669 rpcz_store.cc:276] 0709 12:24:45.623759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.625678 (+  1919us) service_pool.cc:224] Handling call
0709 12:24:47.427148 (+1801470us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:47.429543 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:47.268357 10677 raft_consensus.cc:2376] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9. Re-sending same reply.
I20260709 12:24:47.392987 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:47.282193 13301 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:47.396958 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.398799 12892 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.280318 13204 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.400012 13351 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.307030 13306 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:47.407666 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:47.408680  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:47.414070 13392 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.414971 13364 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:47.413277 10018 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:47.431675 10669 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 5: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:47.451586 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59465.5us
I20260709 12:24:47.451988 13308 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:47.453168 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:47.454355 13308 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:47.455078 13308 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:47.455044 13364 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.456933 13308 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.458175 13364 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.460899 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.451604 13439 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:47.458727 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:47.462929  9946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 626) took 1835 ms (client timeout 1970 ms). Trace:
W20260709 12:24:47.463450  9946 rpcz_store.cc:269] 0709 12:24:45.627854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.629978 (+  2124us) service_pool.cc:224] Handling call
0709 12:24:47.462475 (+1832497us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b5400015ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:47.462714 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1341887872}
I20260709 12:24:47.464329 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.465796 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.465951 13351 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.469821 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.447737 13301 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.471239 13392 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.482785 10023 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:47.486454 10668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 683) took 1872 ms (client timeout 2039 ms). Trace:
W20260709 12:24:47.486912 10668 rpcz_store.cc:269] 0709 12:24:45.613445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:45.613561 (+   116us) service_pool.cc:224] Handling call
0709 12:24:47.485934 (+1872373us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b540058e654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:47.486255 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1902321024}
I20260709 12:24:47.487746 13204 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.488581 13444 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:47.472127  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 77281.2us
I20260709 12:24:47.463461 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:47.468721 10673 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.472525 10675 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:47.477609 12892 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:47.480252  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.451929  9944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1739) took 2546 ms (client timeout 3069 ms). Trace:
W20260709 12:24:47.488752  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.308840 13421 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.499086 10677 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:47.501106 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.503804  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:47.504529 13022 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.486176 13392 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:47.495558  9944 rpcz_store.cc:269] 0709 12:24:44.904924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:44.905083 (+   159us) service_pool.cc:224] Handling call
0709 12:24:47.451467 (+2546384us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b540018b0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:47.451687 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3560606976}
I20260709 12:24:47.507966 13444 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:47.506011 13022 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.514822 13204 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.515642 10019 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:47.523066 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 722) took 1149 ms. Trace:
I20260709 12:24:47.523658 10676 rpcz_store.cc:276] 0709 12:24:46.373152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.380545 (+  7393us) service_pool.cc:224] Handling call
0709 12:24:46.380828 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:46.707448 (+326620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.507086 (+799638us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54006a0254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:47.522823 (+ 15737us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749292288}
I20260709 12:24:47.525612 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.527024 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:47.531651 13367 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:47.532233 13367 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:47.533648 10676 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:47.519834 13444 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:47.536342 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.540520 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:47.541121 13380 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.394s	user 0.000s	sys 0.011s
I20260709 12:24:47.541375 10019 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:47.541939 13204 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:47.544021 13387 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.407s	user 0.005s	sys 0.006s
I20260709 12:24:47.544579 13204 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.538530 13444 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.549952 13370 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.587064 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:47.588085 13362 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:47.588733 12892 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:47.589367 12892 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.538689 13364 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.590656 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:47.590514 13364 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.590864 13374 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:47.591863 13306 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:47.592272 13306 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.592387 13374 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:47.579099 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:47.593070 13374 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:47.593264 13364 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.593679 10668 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:47.533605 13367 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.593887 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:47.594509 13367 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.572775 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.594728 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:47.595469 10021 raft_consensus.cc:2368] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:47.595502 10668 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:47.582437 13283 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:47.588699 13362 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:47.599193 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.599442 13431 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:47.599970 13431 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:47.600395 13431 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.589762 12892 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:47.562904 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.592669 13306 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.615785 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:47.617285 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.600189 13362 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.618254 10020 raft_consensus.cc:2376] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 7. Re-sending same reply.
I20260709 12:24:47.607246 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.618904 13362 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.594130 13374 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.598160 13283 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:47.622887 13439 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:47.623548 13283 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.624703 13283 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.627007 10022 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:47.634110 10017 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:47.585367 13301 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.703723 10023 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:47.704384 10673 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:47.699215 13431 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.710759 13356 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.711596 13356 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.712610 13431 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.714937 13339 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.715145  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:47.715816 13339 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.716351 13050 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:47.717981 13050 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:47.719820 10677 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:47.720230 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:47.724740 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.714290 13356 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.726786 10668 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
W20260709 12:24:47.730000 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61605.7us
W20260709 12:24:47.730943 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 115312us
I20260709 12:24:47.726400 13374 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:47.734402 10021 raft_consensus.cc:2376] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 8. Re-sending same reply.
I20260709 12:24:47.737763 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.616214 12938 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:47.749805 10674 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.512s	user 0.017s	sys 0.000s
I20260709 12:24:47.750309 13301 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.751461 10674 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:47.735313 13356 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:47.750823 10668 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:47.759120 10669 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:47.766927 13339 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:47.768029 13006 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.768272  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53209.3us
I20260709 12:24:47.769013 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.788429 10021 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:47.790758 13006 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.793246 13006 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.784961 10673 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:47.804353 10090 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:47.807934 13382 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.808498 10020 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:47.745254 13444 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:47.810882 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57463.8us
I20260709 12:24:47.811236 13444 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:47.812445 10640 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.716s (SENT)
I20260709 12:24:47.812927 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:47.813823 10016 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:47.813889 13356 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.814973 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.816184 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.816259 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.816556 13356 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:47.816926 10016 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 5: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:47.816984 10017 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:47.818256 13150 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.818797 13150 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.820196  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:47.821029 13303 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.822333 13303 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.775624 12892 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.824569 13235 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:47.824767 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.825042 13235 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:47.825507 13235 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.825999 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.833763 10677 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:47.834343 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.826583 10019 raft_consensus.cc:2376] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 7. Re-sending same reply.
W20260709 12:24:47.844321  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.824767 12892 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.851625 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.853976 10017 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.854915 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:47.853977 10016 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:47.858131 10019 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
W20260709 12:24:47.858928  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:47.859520  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.859863 12938 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:47.860137 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:47.864521  9912 leader_election.cc:400] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:47.869388 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.841635 13356 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.851853 13370 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.897774  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:47.863674 13362 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:47.905597  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48981.8us
I20260709 12:24:47.909612 13367 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:47.909487 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:47.917163 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:47.913753 13367 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:47.924690 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:47.938998  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:47.942503 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 130155us
I20260709 12:24:47.926868 13367 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:47.912849 13362 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:47.946982 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:47.948580 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:47.951396 10020 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:47.950687 10092 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:47.950641 10096 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
W20260709 12:24:47.953267 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.956367 13359 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:47.956576  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 120991us
I20260709 12:24:47.957023 13359 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:47.958393  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:47.959146 13426 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:47.960335  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.959398 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 818) took 1420 ms. Trace:
I20260709 12:24:47.923269 13370 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.964602 13444 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:47.962421 10095 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:47.964937 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:47.965010 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:47.925360 13356 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:47.961997 10675 rpcz_store.cc:276] 0709 12:24:46.541044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.991060 (+450016us) service_pool.cc:224] Handling call
0709 12:24:47.450150 (+459090us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.500610 (+ 50460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.959149 (+458539us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b5400528fd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:47.959310 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011450112}
W20260709 12:24:47.971021  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 118544us
I20260709 12:24:47.976748 10019 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:47.958001 13359 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:47.981662  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:47.983098 10099 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:47.947358 13309 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 12:24:47.962376 13146 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:47.986253 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:47.992841  9948 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:47.994096 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:47.984179 10640 leader_election.cc:336] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.780s (SENT)
I20260709 12:24:47.977988 13444 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:47.997988 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:47.999281  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 71566.7us
I20260709 12:24:48.000237 10668 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.003810 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.008170 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.010345 13444 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:48.018800 10095 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:48.021862 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 821) took 1218 ms. Trace:
I20260709 12:24:48.022420 10669 rpcz_store.cc:276] 0709 12:24:46.803431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.528524 (+725093us) service_pool.cc:224] Handling call
0709 12:24:47.758816 (+230292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.778821 (+ 20005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.021578 (+242757us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54007396d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:48.021763 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521270144}
I20260709 12:24:48.025724 10675 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:48.028060 10669 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:48.029129 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.030215 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.030099 13150 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:48.030793 13356 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:48.031641 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.032459 10094 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:48.032599 13396 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:48.033406 10022 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 12:24:48.033152 13396 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:48.033582 10094 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:48.033777 10022 pending_rounds.cc:85] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 3
I20260709 12:24:48.034080 10022 pending_rounds.cc:107] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 7.4
I20260709 12:24:48.034365 10022 raft_consensus.cc:2889] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:48.046425 10099 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:48.047482  9944 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:48.037348  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.052531 10092 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:48.040093 13362 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:48.041076 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:48.042876  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:48.044183 10671 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260709 12:24:48.040143 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.064484 10096 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:48.064694 10021 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:48.042790 10093 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:24:48.067047 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 819) took 1525 ms. Trace:
I20260709 12:24:48.069437 10677 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:48.067498 10671 rpcz_store.cc:276] 0709 12:24:46.541405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.031131 (+1489726us) service_pool.cc:224] Handling call
0709 12:24:48.031502 (+   371us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:48.066668 (+ 35166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.066973 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:48.002264 13426 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:48.074821 13283 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:48.081928 10670 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:48.084808 10092 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:24:48.043823 13396 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:48.085860 10017 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
W20260709 12:24:48.078464 13022 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.001s
I20260709 12:24:48.089079 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.090555  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.091495  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.092435  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.094761 12892 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:48.094980  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.095585 10094 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:24:48.095851 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54229.3us
I20260709 12:24:48.095662  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:48.097385  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.103317 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:48.114167 10093 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.115280  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.116067  9942 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:48.116268 13235 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.116986  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.118121  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.114216 12892 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.114382  9941 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.123383  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.123584 13287 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:48.124569  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.124727 13287 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:24:48.125077 13287 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:48.125705  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.127413 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.128563 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.129325 10023 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
W20260709 12:24:48.141727  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.142256  9941 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:48.142967  9946 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:48.143404 10017 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:24:48.146152  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.125959 13287 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.147269  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.149607 10093 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:48.147269 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.148121  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.118462 13235 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:48.156101  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.157397  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61805.5us
I20260709 12:24:48.159816  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.160450 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 501) took 1081 ms. Trace:
I20260709 12:24:48.160303  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.162810 10640 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.163920 10022 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:48.160929 10024 rpcz_store.cc:276] 0709 12:24:47.072438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.072786 (+   348us) service_pool.cc:224] Handling call
0709 12:24:48.127352 (+1054566us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54001f5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:48.153795 (+ 26443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646198144}
I20260709 12:24:48.075536 13146 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.176362  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.178102 10094 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 9. (index mismatch)
I20260709 12:24:48.153664 13309 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:48.186892 10093 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:24:48.189955  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.190960  9943 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:48.185418 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:48.160799 13396 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.196538 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.177986 13426 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:48.209147 10017 raft_consensus.cc:2955] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Committing config change with OpId 6.7: config changed from index 4 to 7, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:48.210541  9942 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:48.214711  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.206723 12892 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:48.221942 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77655.1us
I20260709 12:24:48.223876 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.226693 13287 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:24:48.227018  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:48.230006 13022 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:48.230510 13022 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.230957 13022 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.235447 13374 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:48.235953 13374 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.235666 13028 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:48.236349 13374 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.237627 13028 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:48.256808 10091 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:48.259398  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.260382 10093 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:48.259042 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.262058 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:48.265729  9941 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:48.270167  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.208214 13396 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.275646 13396 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:48.276363 13439 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:48.276468 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 727) took 1133 ms. Trace:
I20260709 12:24:48.268061 13303 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 12:24:48.277628 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:48.277895 10098 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:48.279263 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.280179 10098 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:48.281154 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.281296 12892 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.281796 10098 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.285027 10092 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:48.279291 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:24:48.288182  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.289390 10094 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:48.290139 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66357.1us
I20260709 12:24:48.247768 13370 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:48.295324  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:48.294611 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48050.8us
I20260709 12:24:48.302819 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.300935 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.307231 10668 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:48.309767 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.291340 13431 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:48.282074 12892 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.254487 13028 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 12:24:48.325057 10091 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:48.334115 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.277019 10676 rpcz_store.cc:276] 0709 12:24:47.143374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.534579 (+391205us) service_pool.cc:224] Handling call
0709 12:24:47.698810 (+164231us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:47.781146 (+ 82336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:47.781196 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.835069 (+ 53873us) log.cc:844] Serialized 26 byte log entry
0709 12:24:47.969188 (+134119us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:47.969250 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:47.969369 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.276088 (+306719us) raft_consensus.cc:1685] finished
0709 12:24:48.276107 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.276397 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35717,"raft.queue_time_us":9716,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":20330,"thread_start_us":116422,"threads_started":1,"wal-append.queue_time_us":330893}
I20260709 12:24:48.342402 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:48.345317 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66111.4us
W20260709 12:24:48.342830  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 113205us
I20260709 12:24:48.359055 10091 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:48.359443  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.360083 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 488) took 1411 ms. Trace:
I20260709 12:24:48.360654 10091 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:24:48.361150  9991 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:48.362025 10091 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:48.362077 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:24:48.364081 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.359087  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.366852 13303 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:48.367305 13303 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.367451  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.369558 13180 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:48.375850 13146 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:48.377224 13317 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:48.377643 13317 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:48.378500 13317 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.379877 13317 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:48.381315 13367 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:48.382355 13382 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.574s	user 0.007s	sys 0.000s
I20260709 12:24:48.367673 13303 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:48.351929 13306 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.738s	user 0.011s	sys 0.003s
I20260709 12:24:48.384629 10093 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:24:48.398850 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.399596 10093 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:48.386926 13235 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:48.387104 12892 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:48.387732  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.389196 13289 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:48.389772 10092 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:48.390537 10019 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
W20260709 12:24:48.405850 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43910.8us
I20260709 12:24:48.370311 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.395064 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:48.396152 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.409490  9944 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:48.350962 13356 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:48.350787 13439 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:48.410454  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49402.3us
I20260709 12:24:48.414134  9949 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:48.415174 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:48.416862 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.419767  9942 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:48.419945 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.417737 10093 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:48.420722 10091 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:48.425705 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.426369 10093 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:48.427004 10094 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:24:48.427192 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.405269 13235 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:48.428318 10093 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:48.360471 10025 rpcz_store.cc:276] 0709 12:24:46.892778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:46.892886 (+   108us) service_pool.cc:224] Handling call
0709 12:24:46.893204 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:46.893569 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:46.893597 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:47.831534 (+937937us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b2c00046bc8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:24:47.832169 (+   635us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.048226 (+216057us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:48.048262 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.061143 (+ 12881us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:48.283065 (+221922us) raft_consensus.cc:1685] finished
0709 12:24:48.283092 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:48.304165 (+ 21073us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33650,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":15496,"spinlock_wait_cycles":4786560,"thread_start_us":215653,"threads_started":1,"wal-append.queue_time_us":351650}
I20260709 12:24:48.439420 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.439810  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.440280 10094 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:48.409682 12938 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.440464  9941 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:48.443552  9947 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:48.404678 13022 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.447754 13022 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:48.448553  9945 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:48.422791 13370 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:48.453979 13180 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:48.454442 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:48.454782 13180 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.457119 13180 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:48.457858 13166 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:48.454442 13428 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:48.460117 13428 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.462246 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:48.462778 13428 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:48.463459 10021 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:48.467453  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55682.9us
I20260709 12:24:48.470985 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.473383 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:48.481706  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.484018 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.416041 13439 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:48.486251 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 91301.8us
I20260709 12:24:48.488945  9943 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:48.489558 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 93486.6us
I20260709 12:24:48.496461 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.408138 13306 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:48.487515 12938 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:48.505283 10094 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:48.506407 13305 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:24:48.506242 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.516681 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:48.491865 13439 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.524806 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.525714 10091 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:48.530227 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.531538 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.532119 10094 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:48.535430 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:48.536094  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.538801 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.540532 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:48.541087 10094 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:48.541898 10097 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:48.544025 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.536442  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:48.545670 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54780.5us
I20260709 12:24:48.544673 10091 raft_consensus.cc:2376] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 7. Re-sending same reply.
I20260709 12:24:48.548300 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:48.549320 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:48.548808 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.549625 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.550321 10094 raft_consensus.cc:2376] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 10. Re-sending same reply.
I20260709 12:24:48.523411 13431 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:48.552407 13235 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.554003 13235 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.555050 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.555541 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.555207 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:48.559662 13428 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:48.555478 13431 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.560527 13166 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:48.561367 13425 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:48.559461 10094 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:48.562968 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:48.564214 13464 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:24:48.570190 13367 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:48.572469 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.596312 10092 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 6 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:48.596243 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.569753 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.596921 10091 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:48.552109 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.519919 13305 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:48.541574 13404 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.566730 13317 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:48.555884 10019 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:48.559667 10016 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:48.562232 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.520313 13370 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260709 12:24:48.556722 13183 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:24:48.602478  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 109476us
I20260709 12:24:48.602695 13475 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:48.603237  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:48.603531 10094 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:48.603595 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.602480 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:48.633241 10092 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:48.611832 10091 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
W20260709 12:24:48.613067 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 260674us
I20260709 12:24:48.560531 13428 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:48.613632 12892 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:48.613687 10096 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:48.614449 13382 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:48.617745 10021 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:48.616433 13356 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:48.619055 10098 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:48.628911 13317 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:48.629129 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 203932us
I20260709 12:24:48.644233 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:48.644984 10094 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 4. Current term is 5. Ops: []
I20260709 12:24:48.645949 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:48.628325 13440 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:48.642998  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:48.652681 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:48.655458 10023 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
W20260709 12:24:48.653012  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 131750us
W20260709 12:24:48.647197  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44101.6us
I20260709 12:24:48.633893 13475 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:48.595829 13287 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:24:48.659085 10098 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:24:48.645284 13351 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:48.662935 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 207079us
I20260709 12:24:48.663218 10092 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:48.656622 13475 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.655923 13317 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:48.629369 13404 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:48.688259 10096 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260709 12:24:48.638139 13289 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 12:24:48.693964 10018 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:24:48.671224 13283 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:48.709460 10023 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:48.670681 13146 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:48.714090 10020 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:48.697283 12892 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:48.720005  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 88337.5us
I20260709 12:24:48.722990  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:48.670312 13382 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:24:48.660729 13440 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.711580 13283 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:48.728601 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79487.4us
I20260709 12:24:48.702296 13425 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:48.730351  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:48.727842 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 168489us
I20260709 12:24:48.719249 13363 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:48.698549 13028 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:24:48.699285 13351 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:48.723752 12892 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:48.741441 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 97270.6us
I20260709 12:24:48.711629 13166 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:24:48.726923 13146 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:48.771570 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:48.749899  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.773025  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:48.773722 13469 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
I20260709 12:24:48.774142 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:48.774550 13425 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:48.776054  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:48.779318 10016 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:48.739323 13306 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:48.740523 13283 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:48.796437  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 73528.9us
I20260709 12:24:48.797385  9945 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:48.743537 13356 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:48.741099 13396 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.775946 12892 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:48.856019 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60889.6us
W20260709 12:24:48.857748 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 86258.8us
I20260709 12:24:48.864281 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.741197 13363 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:48.838020  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.862542 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2724) took 1156 ms. Trace:
I20260709 12:24:48.791667 13028 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:48.769677 13339 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 12:24:48.714236 13305 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:48.866539  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 121999us
I20260709 12:24:48.867575 13306 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:48.868332 10018 rpcz_store.cc:276] 0709 12:24:47.705878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.025418 (+319540us) service_pool.cc:224] Handling call
0709 12:24:48.025804 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.694578 (+668774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.862305 (+167727us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54002d5354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:48.862486 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1535854848}
W20260709 12:24:48.869648  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:48.872311 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 150345us
W20260709 12:24:48.886607  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.890512 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.792245 13309 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:24:48.904295 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.904529 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:48.905213  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:48.912631 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.739485 13428 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:48.914088  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54334.5us
I20260709 12:24:48.915352 10671 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:48.915531 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:48.784920 13477 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.226s
I20260709 12:24:48.920548 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.922003 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:48.871809 13028 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:48.966912 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:48.891618 13306 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:48.917176 13303 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.005s
I20260709 12:24:48.971021 10668 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:48.912225 13382 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:24:48.972420 13472 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.265s	user 0.000s	sys 0.000s
I20260709 12:24:48.967834 13028 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:48.985459 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:48.985740 10676 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
W20260709 12:24:48.986842  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 105611us
I20260709 12:24:48.973261 13431 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:48.978689 13306 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:48.970551 13339 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:24:48.990953 13022 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:48.989187 13477 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:48.997951 13493 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:48.995397 13382 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:48.983381 13303 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.011461 13404 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:49.054972 13312 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:24:49.056461 13471 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.342s	user 0.000s	sys 0.000s
W20260709 12:24:49.057816 13487 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:24:49.055128 13491 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.058413 13483 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.143s
I20260709 12:24:49.066584 13426 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:49.067073 13426 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:49.068032 13426 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.068960 12938 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:49.069275 13426 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.077724 13312 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:49.078931 13486 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:49.079471 13486 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:49.079581  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:48.994048 13146 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:49.118930 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 131661us
I20260709 12:24:49.022851 13022 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:49.059878 13351 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:49.111917 13382 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.139206  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.114562 12938 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.141016  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:49.142836  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 219262us
I20260709 12:24:49.141650  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.146121  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:48.996913 13363 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:49.148728  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 155379us
I20260709 12:24:49.117796 13404 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:49.154093  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:49.155673 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 233769us
I20260709 12:24:49.080657 13312 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:24:49.119967 13166 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:49.154800  9947 raft_consensus.cc:2376] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8. Re-sending same reply.
I20260709 12:24:49.166380 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:49.170022 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 388689us
I20260709 12:24:49.170713 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:49.176167 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:49.179504 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:49.166760 13287 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.170758 13363 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:49.215680 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:49.215890 13309 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.217466 13309 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:49.218020 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:49.211628  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:49.151191 13382 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.239720 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:49.196990  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:49.155162 12938 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:49.299609  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 264248us
W20260709 12:24:49.300622  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.159809 13339 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
W20260709 12:24:49.311976 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94108.5us
I20260709 12:24:49.179230 13493 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:49.160980 13404 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.172006 13351 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:49.178738 13477 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:49.192658 13374 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.946s	user 0.012s	sys 0.008s
W20260709 12:24:49.249037 13490 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
W20260709 12:24:49.256771 13488 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 12:24:49.197177 13431 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:49.241511 13484 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.210s	user 0.001s	sys 0.000s
W20260709 12:24:49.263557 13501 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:24:49.323133 13493 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:49.325484  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 113963us
I20260709 12:24:49.351102  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:49.326213  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 142506us
I20260709 12:24:49.328635 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:49.352197 10098 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.329622 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:49.336464 13382 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:49.255105 13370 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.338162  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.218573 13309 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:49.248704 13356 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.310870 13166 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:49.361457 13166 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.362748 13370 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.364148 13166 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.365242 13370 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:49.340804  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.160203 13486 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:49.324069 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 166681us
I20260709 12:24:49.352844 10094 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:49.326792 13492 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.295240 13287 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.261128 12892 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:49.357074  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.346483 13235 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.792s	user 0.013s	sys 0.000s
I20260709 12:24:49.264206 13499 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:24:49.382856 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:49.383632 10023 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:49.354882 13508 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:49.240569 13498 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.368906 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:49.379186 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:49.193929 13022 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.390900 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:49.358871 13312 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:49.355495 13493 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:49.394531 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:49.395145 10094 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:49.342207 13374 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.399102 10022 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:49.405509 10019 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:24:49.406337 13404 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:49.373271 13303 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:49.348192 13301 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:24:49.399734 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:49.410712 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:49.422611 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:49.351198 13477 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:24:49.423403 10025 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
W20260709 12:24:49.454053  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:49.397011 13439 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.856s	user 0.012s	sys 0.000s
I20260709 12:24:49.459972 10676 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:24:49.390214 13370 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:49.380843 13486 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.438714 10098 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:49.374437 13382 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:24:49.466922 13507 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:24:49.467181 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67956.8us
I20260709 12:24:49.468111 13374 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:49.468045  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.468859  9945 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:49.408926 13235 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:49.466964 10020 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:49.470970 13235 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:49.471342 13481 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.732s	user 0.007s	sys 0.018s
I20260709 12:24:49.467397 13507 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:49.409142 13305 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.408342 13505 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:49.407943 13492 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.482372 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:49.474071  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.483917 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:49.484592 10019 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.480638 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:49.408430 13329 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:49.490093 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:49.428362 13301 consensus_queue.cc:1243] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:24:49.474444 13339 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:24:49.483240 13183 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.844s	user 0.008s	sys 0.004s
I20260709 12:24:49.503022 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:49.401930 13498 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.509323 10092 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.473634 13431 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:49.507108  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 154676us
I20260709 12:24:49.514254  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.485114 13492 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.536942  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.563316  9943 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:49.540920  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.571427 10019 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:49.499220 13303 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:49.567010  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:49.397326 13508 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.465372 13439 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:49.533730 13514 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:24:49.540221 13492 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.499686 13370 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.533970 13183 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:24:49.513397 13505 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:49.510248 13477 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:49.475098 13235 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.512965 13339 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:49.598321  9941 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:49.503443 13356 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:24:49.541837 13359 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.621327 10092 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:49.612553 13359 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.624826 13440 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:49.624903 13359 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:49.504223 13305 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.628647 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:49.597546 13404 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:49.611455 13475 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.906s	user 0.012s	sys 0.000s
I20260709 12:24:49.537935 13329 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.597190 13382 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:49.620312 13339 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.629535 10093 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:49.600059 12938 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:49.635151  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 120599us
I20260709 12:24:49.611025 12892 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:49.629789 10669 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
W20260709 12:24:49.631461 13317 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.920s	user 0.004s	sys 0.009s
I20260709 12:24:49.647763  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:49.673702 10093 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:24:49.675511  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.700843  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:49.647758 13504 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.707026 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2789) took 1773 ms. Trace:
I20260709 12:24:49.631834 13477 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:24:49.709631 10093 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:49.709487 10097 rpcz_store.cc:276] 0709 12:24:47.924166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:47.946772 (+ 22606us) service_pool.cc:224] Handling call
0709 12:24:49.696950 (+1750178us) spinlock_profiling.cc:243] Waited 576 ms on lock 0x7b5400093154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:49.697093 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1325496064}
W20260709 12:24:49.673538 13396 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.805s	user 0.012s	sys 0.000s
W20260709 12:24:49.713227  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 118989us
I20260709 12:24:49.647634 13382 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.572466 13309 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.656893 13503 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.719815 10099 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:49.649636 13515 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.642695 13404 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.695595 13235 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.696343 13497 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.597242 13507 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:49.715970 13514 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:24:49.730540  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.709245 13504 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.636484 13475 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.723706 13396 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:49.682070 13317 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.722209 13287 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:49.727281 13520 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.743701 13504 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.755930 13329 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:49.756174 13521 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:24:49.756647  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.751184 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:49.757081  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56360.1us
I20260709 12:24:49.730460 13529 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:49.756598 13513 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.275s	user 0.001s	sys 0.000s
I20260709 12:24:49.765306  9944 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:49.742733 13146 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:49.680732 13431 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.785816 13531 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:24:49.786705  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.786928 13370 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:49.764693  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.787896  9948 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:49.788532 13404 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:49.747134 13517 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 12:24:49.771145  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:49.774142  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:49.775949 13475 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:49.775795 13367 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:49.776443 13529 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:49.741920 13505 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.732967 13515 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.779554 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:49.781507 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:49.783775 10668 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:24:49.805251 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1713) took 1413 ms. Trace:
I20260709 12:24:49.805516 10676 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:49.784577  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.771600 12938 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:49.764483 13502 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:49.790557 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:49.798015  9946 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
W20260709 12:24:49.795890 13028 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.761s	user 0.006s	sys 0.006s
I20260709 12:24:49.737780 13522 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.257s
I20260709 12:24:49.818758 13502 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:49.819391 13489 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.958s	user 0.004s	sys 0.012s
I20260709 12:24:49.819003 13028 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.821245 13502 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:49.821852 13028 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:49.824653  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61302.8us
I20260709 12:24:49.825433  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:49.825886  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.826330  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:49.806263 13146 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:49.827468 13146 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.828874 10676 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:49.829571 13370 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:49.829651  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.808296  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.732921 13503 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.809754  9948 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:24:49.813788 13283 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.953s	user 0.012s	sys 0.000s
W20260709 12:24:49.813959 13528 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:24:49.817267 13289 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.809283 12938 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.792994 13439 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:49.860793 10672 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
W20260709 12:24:49.877025  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.005s	sys 0.010s
I20260709 12:24:49.832417 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:49.811411 13507 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.803294 13404 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:49.802924 13367 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:49.861833  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:49.805658 10090 rpcz_store.cc:276] 0709 12:24:48.383604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.615465 (+231861us) service_pool.cc:224] Handling call
0709 12:24:48.621192 (+  5727us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.711489 (+ 90297us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.711534 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.731780 (+ 20246us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.167312 (+435532us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:24:49.339118 (+171806us) log.cc:844] Serialized 14 byte log entry
0709 12:24:49.342476 (+  3358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.342642 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.794394 (+451752us) raft_consensus.cc:1685] finished
0709 12:24:49.794415 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.796649 (+  2234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29799,"prepare.queue_time_us":242529,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":54683,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":31576,"thread_start_us":591752,"threads_started":3,"wal-append.queue_time_us":639316}
I20260709 12:24:49.828318  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:49.818770 13317 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:49.830227 13370 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.854534  9945 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:49.878021  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.005s	sys 0.010s
I20260709 12:24:49.855695 13329 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.859568 13396 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:49.879596 12938 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.872609  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.890318 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 833) took 1120 ms. Trace:
W20260709 12:24:49.878357 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46045.5us
I20260709 12:24:49.898571 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:49.856009 13477 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.811241 13498 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:49.855084 12892 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:49.910771  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 559719us
I20260709 12:24:49.877276 13289 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.786381 13531 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.879689 13166 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.916376  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.916816 10676 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:49.883057 13404 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:49.916215 10677 rpcz_store.cc:276] 0709 12:24:48.770208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.774533 (+  4325us) service_pool.cc:224] Handling call
0709 12:24:48.774887 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:49.314556 (+539669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:49.319401 (+  4845us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:49.341148 (+ 21747us) log.cc:844] Serialized 26 byte log entry
0709 12:24:49.459482 (+118334us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:49.459614 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:49.459689 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:49.890079 (+430390us) raft_consensus.cc:1685] finished
0709 12:24:49.890100 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:49.890255 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10421,"raft.queue_time_us":222896,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":12621,"thread_start_us":488113,"threads_started":2,"wal-append.queue_time_us":234398}
I20260709 12:24:49.773475 13520 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.940889 13439 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.950668 13520 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:49.950762  9946 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:49.916676 13508 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:49.951046 13439 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:49.951879 13312 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.001s
I20260709 12:24:49.952284 13312 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:49.953152 13439 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:49.953531  9948 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 12:24:49.953890  9948 pending_rounds.cc:85] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 6
I20260709 12:24:49.954092  9948 pending_rounds.cc:107] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:24:49.954306  9948 raft_consensus.cc:2889] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:49.919673 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:49.955225 10669 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:49.955165 13542 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:24:49.957114 13404 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:49.922147 13485 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:49.921741 13367 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.957796 13485 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.958066 13367 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:49.958240 13485 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.923262  9945 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
W20260709 12:24:49.923390 13538 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:24:49.913945 13396 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:49.966637 13396 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:49.967444 13374 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:49.968005 13374 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:49.932248 13306 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.913s	user 0.012s	sys 0.000s
I20260709 12:24:49.968860 13306 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:49.969120 13374 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.919610 13502 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:49.923760 13512 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.970993 13512 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.972045 13374 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:49.973197 13512 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:49.974889 13498 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:24:49.944509 13363 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:24:49.924599  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:49.925284 13309 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:49.988016 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:49.990525 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:49.990998 10677 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:49.862646 13283 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.912901 13522 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:49.946584 13523 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.947947 13531 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:49.948377 13166 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:49.999660  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:50.000244 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:50.001056 10677 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:49.925918 13505 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:49.990859 10669 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:49.927552 13303 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:49.992416  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 164203us
W20260709 12:24:49.994206  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:49.977843  9839 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:49.998242 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:49.941496 13289 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.006373 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.007454 10668 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:50.001260 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.003264 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.003541 10669 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:50.003440  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:49.984612 13475 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:49.978407 13306 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:50.024032 13287 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.023603 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.007365 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:50.006341  9842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 309) took 30348 ms (client timeout 15000 ms). Trace:
W20260709 12:24:49.962340 13479 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.248s	user 0.007s	sys 0.000s
W20260709 12:24:50.009549  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 85015.7us
I20260709 12:24:50.011161 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:49.990096 13329 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.033756 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.020255 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.034497 10676 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 4.
I20260709 12:24:50.021900 13526 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:50.000391 13235 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:50.022979 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.023594 13283 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:49.993108 13522 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:50.018013 13525 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.024886  9945 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:50.014207  9839 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.035264  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:49.940999 12892 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.059867 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.054030 13522 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.037730 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:50.038956  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:50.041292 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.086987 10021 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.039680 10671 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:50.036077 12938 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.094264 13370 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.085421 13444 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:50.095165 13444 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:50.095338 12938 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:50.096114 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:50.097379 13544 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.199s	user 0.005s	sys 0.000s
I20260709 12:24:50.097105 13396 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.056083 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:50.050868 13329 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.066884 10676 raft_consensus.cc:2376] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6. Re-sending same reply.
I20260709 12:24:49.919907 13477 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.070704 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:50.027089 13339 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.079262 13525 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.080170  9839 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 5, term changed from 2 to 6, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to <none>, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 6 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:50.086102 10022 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:50.024573 13520 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:50.070315 10018 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:50.086577  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51420.1us
I20260709 12:24:50.092878 10020 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:50.109965  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:50.111531 10023 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:50.024608 13287 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:50.114786 13543 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:24:50.115129 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.115667  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:50.115883 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:50.140852  9948 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.126928 10021 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:50.131886  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:50.100169 13426 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.030s	user 0.013s	sys 0.007s
W20260709 12:24:50.137588  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.140910 10668 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.119065 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.140071  9839 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 7, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 7 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.142303  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:49.978356 13529 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.145522 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.146274 10669 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
W20260709 12:24:50.147435 13482 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.418s	user 0.003s	sys 0.001s
W20260709 12:24:50.130113 13486 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.644s	user 0.003s	sys 0.008s
I20260709 12:24:50.110854 13303 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:50.152645  9941 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:50.162846 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:50.153645  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.154654 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:50.042272 13475 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.158296 10670 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
W20260709 12:24:50.168666 13493 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.706s	user 0.011s	sys 0.000s
I20260709 12:24:50.168843 13301 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:50.054344 13526 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.170742 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1710) took 1804 ms. Trace:
I20260709 12:24:50.174142 13022 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.173786 13289 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.175271 13515 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:50.176955 12892 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.142261 13359 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:50.298347 10668 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:50.178457 13396 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.115988 13503 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.179422  9945 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:50.152369 13363 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:50.180858 13309 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:24:50.183468  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 149213us
W20260709 12:24:50.331645 13504 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.555s	user 0.010s	sys 0.000s
I20260709 12:24:50.350994  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:50.182855 13444 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.151755 12938 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.187678 13287 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.000555 13523 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.192519 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.192371  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:50.193665  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:50.359774  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:50.194068 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:50.142434 13370 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 12:24:50.195050 13305 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:50.348780 13502 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:50.195673 10675 pending_rounds.cc:85] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 7
I20260709 12:24:50.196925 13404 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.197408 13477 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.198580 13166 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:50.352978 13367 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.200119 13306 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.200783 13339 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.362089 13503 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:50.387766 10019 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:50.163575 10672 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 4.
I20260709 12:24:50.201589 10671 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
W20260709 12:24:50.208838 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54298.9us
I20260709 12:24:50.389724 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:50.213231 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.215868  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:50.212376  9839 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 1 to 6, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 6 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.149534 13283 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.220291 10095 rpcz_store.cc:276] 0709 12:24:48.366362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.445112 (+ 78750us) service_pool.cc:224] Handling call
0709 12:24:50.170508 (+1725396us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b5400192dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:50.170678 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1620645632}
I20260709 12:24:50.215148 13301 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:50.244616  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:50.393589  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.202903 13303 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.369453 13502 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.130430 13520 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:50.304519  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1761) took 1957 ms. Trace:
W20260709 12:24:50.237818 13492 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.608s	user 0.004s	sys 0.008s
I20260709 12:24:50.303063  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:50.336678  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:50.226805 13312 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:50.026456  9842 rpcz_store.cc:269] 0709 12:24:19.667964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:19.689478 (+ 21514us) service_pool.cc:224] Handling call
0709 12:24:21.022399 (+1332921us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094c0090 00007f4e0955ef3a 00007f4e0f314e9c 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc 00007f4e0d8d3392 00007f4e0789148c 00007f4e078908c6
0709 12:24:50.003320 (+28980921us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:46.738834 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:46.925997 (+187163us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:46.937276 (+ 11279us) write_op.cc:435] Acquired schema lock
0709 12:24:46.947705 (+ 10429us) tablet.cc:662] Decoding operations
0709 12:24:47.161075 (+213370us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:47.161160 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:47.161189 (+    29us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:24:47.209033 (+ 47844us) tablet.cc:701] Row locks acquired
0709 12:24:47.209058 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:24:47.209276 (+   218us) write_op.cc:270] Start()
0709 12:24:47.209449 (+   173us) write_op.cc:276] Timestamp: P: 1783599887209201 usec, L: 0
0709 12:24:47.209477 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:24:47.210358 (+   881us) log.cc:844] Serialized 2855 byte log entry
0709 12:24:47.315357 (+104999us) op_driver.cc:464] REPLICATION: finished
0709 12:24:47.322957 (+  7600us) write_op.cc:301] APPLY: starting
0709 12:24:47.323044 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:47.323224 (+   180us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:47.323359 (+   135us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:47.324090 (+   731us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:47.324249 (+   159us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 12:24:47.324277 (+    28us) write_op.cc:312] APPLY: finished
0709 12:24:47.324483 (+   206us) log.cc:844] Serialized 59 byte log entry
0709 12:24:47.324662 (+   179us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:47.324745 (+    83us) write_op.cc:454] Released schema lock
0709 12:24:47.324773 (+    28us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:47.466769 (+141996us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:47.466845 (+    76us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1008264,"reported_tablets":31,"spinlock_wait_cycles":171945344,"child_traces":[["op",{"apply.queue_time_us":7284,"apply.run_cpu_time_us":3233,"apply.run_wall_time_us":125592,"num_ops":7,"prepare.queue_time_us":322028,"prepare.run_cpu_time_us":16222,"prepare.run_wall_time_us":548911,"raft.queue_time_us":18298,"raft.run_cpu_time_us":1231,"raft.run_wall_time_us":2440,"replication_time_us":105606,"thread_start_us":248927,"threads_started":4,"wal-append.queue_time_us":75552,"wal-append.run_cpu_time_us":9816,"wal-append.run_wall_time_us":1118131}]]}
I20260709 12:24:50.360743 10020 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:50.433099  9948 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:50.220508 13493 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.195361 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:50.369735 10675 pending_rounds.cc:107] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:24:50.434790 10675 raft_consensus.cc:2889] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:50.199631 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47518.1us
I20260709 12:24:50.422187 13523 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.357051 13363 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:50.385721 13478 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.612s	user 0.005s	sys 0.001s
I20260709 12:24:50.364205 12938 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.455734 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.238628  9944 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:50.383747 13235 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:50.397747 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:50.401970  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:50.404724  9839 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 2 to 4, term changed from 3 to 4, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to <none>, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 4 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:50.459947  9839 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:50.406988 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:50.388345 13522 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:50.410651  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 74103.5us
I20260709 12:24:50.410940 13329 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.411545 13287 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:50.398159 13303 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:50.416416 13550 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:24:50.398118 13306 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.397889 13289 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:50.418855 13507 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.529s	user 0.007s	sys 0.004s
I20260709 12:24:50.419232 13492 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.395848 13301 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.408588 13503 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:50.429523 13525 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:50.358530 13504 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:50.360061  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 217847us
I20260709 12:24:50.399721  9940 rpcz_store.cc:276] 0709 12:24:48.340312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.340451 (+   139us) service_pool.cc:224] Handling call
0709 12:24:48.365600 (+ 25149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:48.404212 (+ 38612us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:48.404246 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:48.430453 (+ 26207us) log.cc:844] Serialized 26 byte log entry
0709 12:24:48.636053 (+205600us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:48.636119 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:48.636208 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:50.146784 (+1510576us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:50.236874 (+ 90090us) raft_consensus.cc:1685] finished
0709 12:24:50.236903 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:50.297061 (+ 60158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25324,"raft.queue_time_us":38702,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":13072,"spinlock_wait_cycles":33070976,"thread_start_us":181288,"threads_started":1,"wal-append.queue_time_us":223022}
I20260709 12:24:50.350406 13309 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:50.435527 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:24:50.483387 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47935.2us
I20260709 12:24:50.441815 10018 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:50.398262 13477 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:50.403394 13502 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.455672  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:50.385314 13485 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.408905 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.491529 10673 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:50.461655 10098 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:50.387600 13367 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.465266  9839 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 7, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 7 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.464337  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:50.438030 13404 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:50.468765 10669 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:50.434916 13526 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:50.458854 13235 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:50.480232  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:50.436988 13312 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.488209 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.506338 10672 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:50.494114  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:50.506680  9839 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 6, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 6 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } }
I20260709 12:24:50.507833  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:50.495538 13426 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:50.496397 13523 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.496558 13504 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:50.479795 13287 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:50.497992 13305 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:50.500475 13525 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:50.503042 13367 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.512208  9839 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 6, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 6 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.509151 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:50.521831 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:50.556623 10672 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:24:50.556524 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47924.8us
I20260709 12:24:50.522925 13363 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.496914 13486 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 12:24:50.506116 13235 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.528946 13312 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.523820 13329 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:50.529153 13503 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.529969 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.530464  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:50.531824 13475 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.532356 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.533694 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:50.534436 13485 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:50.539451 13551 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.195s	user 0.001s	sys 0.001s
W20260709 12:24:50.583063  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72054.8us
I20260709 12:24:50.548789 13309 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.477145 13520 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.521885 13287 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.594748 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:50.595387 12938 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.562474 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.515765 13404 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:50.507135 13477 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:50.502584 13515 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:24:50.564411 13305 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:50.564664 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:50.509079 13553 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.569979 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:50.578222  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47833.6us
I20260709 12:24:50.617726  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:50.570331 13558 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:50.572669  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62246.4us
I20260709 12:24:50.573493 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.570667 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:50.548064 13306 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.586823  9839 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 4, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 4 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:50.593683 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:50.598275 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.651682 10020 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.570132 13504 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:50.572595 13431 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.608733 10099 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:50.520391 13523 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.614154 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:50.573247 13329 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:50.664287 13529 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:50.571712 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.586001 13312 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:50.665522 13522 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.665889 13312 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:50.577095 13525 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:50.666121 13301 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.666416 13523 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:50.667071 13525 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.667208 13312 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.668248 13301 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:50.621655 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:50.672984 10098 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:50.650933 10093 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 5.
I20260709 12:24:50.654282  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:50.674381 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:50.675096 10093 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.661118 10017 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
W20260709 12:24:50.677959 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 107436us
I20260709 12:24:50.621250 10091 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:24:50.664628 13329 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.685197 10096 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:50.672046 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:50.620399 10092 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:50.676311  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:50.619364 10024 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:50.622273 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:50.616947 13493 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:50.750448 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.714622 10020 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:50.607579 13404 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:50.722630  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.014s	sys 0.031s
I20260709 12:24:50.664808 13529 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.748548  9839 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 7, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 7 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.616825 13553 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.698498  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:50.783653  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 96710.2us
W20260709 12:24:50.784190  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 85807.8us
W20260709 12:24:50.756711  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.015s	sys 0.031s
I20260709 12:24:50.784466  9839 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:24:50.785595  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 604684us
I20260709 12:24:50.695086 10097 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:50.786909 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45540.6us
I20260709 12:24:50.757704 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:50.788734 10024 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:50.791570 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.792174 10025 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.657881 13504 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:50.649886 13283 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:50.790627 13529 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.756395 12938 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:50.795481 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:50.813310 10097 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.796231 13526 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:50.782183 13404 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.797640  9839 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 5, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 5 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.798475 13370 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:50.807231 10023 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:50.808975 10017 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:50.840260  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:50.749010 13523 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:50.805388 13504 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:50.810283 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:50.812238  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:50.823047  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:50.851696  9948 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:50.757171 13339 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:50.831391  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.835489  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:50.836126  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:50.837260  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.839598 10022 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:50.842308 10093 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:50.841887  9839 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 5, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 5 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:50.843202 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:50.799611 13498 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:50.792747 13301 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:50.846163  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:50.860489  9944 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:50.698145 13537 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:50.756304 13028 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 12:24:50.800349 13553 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:50.850884 10096 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.687069 13431 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.800938 13306 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.053s
I20260709 12:24:50.861044 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:50.863652 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:50.878253 10024 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:50.866775 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 333143us
I20260709 12:24:50.873939  9839 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 5, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 5 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.841200 13351 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:50.885196 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:50.886121 10017 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:24:50.888288 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:50.872138 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:50.856093 13339 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.687865 13558 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.847133 13523 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.881186  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:50.917766  9839 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 5 to 8, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 8 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:50.884644 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.812045 13303 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.923022 10090 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.915778 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:50.860064 13498 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:50.918954  9945 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:50.896054 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:50.847971 13499 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260709 12:24:50.928404 10672 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:50.928885 10093 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:50.865640 13537 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:50.921675 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:50.929768 10674 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:50.871865 13564 raft_consensus.cc:2764] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:50.930649 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:50.936471  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 151228us
I20260709 12:24:50.835057 12938 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.937266  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:50.902208 13370 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:50.913201 13339 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.941545 10093 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:24:50.946758  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:50.948419 10673 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:50.823482 13477 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.924347 13523 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.933634 13498 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:50.854317 13305 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:50.947854 13370 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:50.858379 13561 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.087s
I20260709 12:24:50.872444 13301 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.955024  9839 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 4, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 4 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:50.920631 13351 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:50.934185 13537 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:50.958252 13439 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:51.036888  9839 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:51.040362  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 200164us
I20260709 12:24:51.041917  9839 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:51.084030  9839 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 8, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 8 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:24:51.039793 13554 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:24:51.039723 13235 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:51.113683 10640 leader_election.cc:336] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.643s (SENT)
I20260709 12:24:51.113888  9839 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:51.115931  9943 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:51.130259  9839 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 2 to 6, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 6 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:51.083227 13439 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.175720  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:51.176196  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 239017us
W20260709 12:24:51.177734 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64076.4us
W20260709 12:24:51.178423 10640 leader_election.cc:336] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.646s (SENT)
I20260709 12:24:51.180419 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:51.141467 13499 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:24:51.223232 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44837.2us
W20260709 12:24:51.230859  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55204.5us
I20260709 12:24:51.232057  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:51.191250 13166 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:51.245147  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:51.187183 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:51.246402 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59427.6us
I20260709 12:24:51.247238 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:51.129974 13370 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:51.244000  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:24:51.236858 13382 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:51.236757 13312 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:24:51.205395 13580 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:24:51.298280 10025 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.506s	user 0.005s	sys 0.004s
I20260709 12:24:51.327967 10025 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:51.294375 10090 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:51.327136  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1545) took 1396 ms. Trace:
I20260709 12:24:51.246068 13583 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.279639  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 95045us
I20260709 12:24:51.279650 13526 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 12:24:51.250051 13303 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:51.188985 13582 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.078s
W20260709 12:24:51.275261 13575 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:24:51.360136  9945 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:24:51.235813 13493 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:51.329035  9949 rpcz_store.cc:276] 0709 12:24:49.931201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:49.952197 (+ 20996us) service_pool.cc:224] Handling call
0709 12:24:49.952449 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:49.952632 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:49.953032 (+   400us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.444728 (+491696us) spinlock_profiling.cc:243] Waited 4.11 ms on lock 0x7b18004aafa0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:50.964101 (+519373us) spinlock_profiling.cc:243] Waited 5.53 ms on lock 0x7b18004aafa0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:24:51.134683 (+170582us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.326765 (+192082us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:51.326821 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:51.326869 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.326937 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.327063 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195141,"prepare.queue_time_us":25746,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":80,"raft.queue_time_us":45553,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":4029,"spinlock_wait_cycles":22176256,"thread_start_us":282619,"threads_started":2}
I20260709 12:24:51.264922 13431 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:51.271936 13235 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:51.279248 13301 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:51.367693 13563 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.405s	user 0.000s	sys 0.001s
I20260709 12:24:51.371095 13525 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:51.334340 13531 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:51.350761 13382 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:51.255481 13166 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:51.394042 13574 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 12:24:51.394621 13525 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:51.372609 13580 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:51.371503 13287 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:51.416165  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 171139us
W20260709 12:24:51.378394  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.388144 13303 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.387068 13431 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:51.414647 13573 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.374s	user 0.001s	sys 0.000s
I20260709 12:24:51.390578 13351 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:51.428929 10096 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.557s	user 0.011s	sys 0.000s
I20260709 12:24:51.392630 13301 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:51.430016 10097 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.607s	user 0.009s	sys 0.000s
I20260709 12:24:51.443248 10096 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:24:51.359972 13583 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:51.380008 13493 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.451457 10097 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:51.421468 13503 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:51.445259 13431 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:51.456477 13592 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:24:51.458644 13582 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:51.446717  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.462821  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:51.425730 13580 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:51.423885 13339 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.464078 10097 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:51.437094 13499 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:51.445497 12938 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.445698 13303 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:51.496191 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:51.544623 13382 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.544852 13351 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.545446 13351 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.546211 13515 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.544517 13301 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:51.545284 13382 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.546947 13306 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:51.547503 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:51.547807 13576 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:51.476902 13439 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:51.548974 13351 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.550983 13525 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.536640 10097 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:51.555599 13166 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:51.556155 13493 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.558005 13303 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:51.483762 10024 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.580s	user 0.011s	sys 0.002s
W20260709 12:24:51.559579  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 97901.9us
W20260709 12:24:51.463800 13180 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.007s	sys 0.013s
I20260709 12:24:51.517288 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 706) took 1322 ms. Trace:
I20260709 12:24:51.446190 13590 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:51.462579 13503 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:51.491194 13577 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
W20260709 12:24:51.489487 13570 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
W20260709 12:24:51.586887  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 251143us
I20260709 12:24:51.570163 13512 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.517933 13558 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:51.489235 13580 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:51.619315 13180 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.011s	sys 0.016s
I20260709 12:24:51.579846 12938 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:51.619415 10675 rpcz_store.cc:276] 0709 12:24:50.194979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.195127 (+   148us) service_pool.cc:224] Handling call
0709 12:24:50.195457 (+   330us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:50.440896 (+245439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.440928 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:50.455822 (+ 14894us) log.cc:844] Serialized 52 byte log entry
0709 12:24:50.758232 (+302410us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:50.758294 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.758383 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:51.516994 (+758611us) raft_consensus.cc:1685] finished
0709 12:24:51.517022 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.517204 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1665,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14696,"thread_start_us":279796,"threads_started":1,"wal-append.queue_time_us":420798}
W20260709 12:24:51.584163 13502 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.077s	user 0.015s	sys 0.000s
W20260709 12:24:51.645857  9987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 2.142s (SENT) [suppressed 32 similar messages]
I20260709 12:24:51.577304 13576 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:51.629595 10024 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:24:51.644171 13022 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.405s	user 0.008s	sys 0.006s
I20260709 12:24:51.629496 13590 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.569909 13515 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.492268 13582 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:51.664950 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 417786us
W20260709 12:24:51.667024 13498 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.696s	user 0.007s	sys 0.011s
I20260709 12:24:51.578790 13306 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.673961 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:51.677784 10670 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:51.654114 13601 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:51.648614 13604 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:24:51.662951 13529 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.826s	user 0.010s	sys 0.002s
I20260709 12:24:51.687844 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 755) took 1168 ms. Trace:
I20260709 12:24:51.671581 13608 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:24:51.688676 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:51.684196 13498 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.662943 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2838) took 1059 ms. Trace:
I20260709 12:24:51.676889 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:51.579414 13301 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.675273 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:51.674212 10017 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:51.672839 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:51.688248 10668 rpcz_store.cc:276] 0709 12:24:50.519483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.520412 (+   929us) service_pool.cc:224] Handling call
0709 12:24:50.522834 (+  2422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.782240 (+259406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:50.782293 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.782954 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.937931 (+154977us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:50.937981 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.938079 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:51.687635 (+749556us) raft_consensus.cc:1685] finished
0709 12:24:51.687652 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.687788 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":65972,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9021,"thread_start_us":369120,"threads_started":2,"wal-append.queue_time_us":286878}
I20260709 12:24:51.692967 10675 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:51.685321 10021 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:24:51.686054 10019 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:51.693418 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:51.671658 13146 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.699869 13146 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:51.673686 13339 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.702569 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:51.674525 13596 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:24:51.703469 10024 raft_consensus.cc:2376] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5. Re-sending same reply.
I20260709 12:24:51.704775 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:51.669473 13586 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.706161 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2823) took 1116 ms. Trace:
I20260709 12:24:51.706712  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:51.707780 10098 rpcz_store.cc:276] 0709 12:24:50.590061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.606193 (+ 16132us) service_pool.cc:224] Handling call
0709 12:24:50.623345 (+ 17152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.673365 (+ 50020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.705933 (+1032568us) spinlock_profiling.cc:243] Waited 49.5 ms on lock 0x7b540014f0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:51.706103 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":113760000}
I20260709 12:24:51.654278 13512 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.674927 12938 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:51.693744 10092 rpcz_store.cc:276] 0709 12:24:50.603050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.459849 (+856799us) service_pool.cc:224] Handling call
0709 12:24:51.460166 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.460401 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:51.566525 (+106124us) log.cc:844] Serialized 14 byte log entry
0709 12:24:51.662530 (+ 96005us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:51.662610 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:51.662669 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.662743 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.662881 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47834,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":697,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":281,"thread_start_us":175150,"threads_started":2}
I20260709 12:24:51.715889 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:51.671636 13590 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.717173 10024 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:24:51.669751 13502 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.719465 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:51.720351 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:51.720415 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:51.693266 13603 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.724062  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:51.701854 13022 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:51.701983 13493 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:51.726338 10024 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 5.
I20260709 12:24:51.723464 13589 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:51.727484 13609 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:24:51.727639 13329 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.966s	user 0.012s	sys 0.000s
I20260709 12:24:51.727967 13609 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.728132 13329 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.729060 13614 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:24:51.730649 13329 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.731120 13614 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:51.731556 13370 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:51.732704 13603 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.735114 13603 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.707623 13586 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:51.737630 13586 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:51.737860 10677 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:51.739452 13502 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.740278 10023 raft_consensus.cc:2368] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8.
I20260709 12:24:51.740917 13593 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:51.701773 13529 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.743237 13529 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.743939  9946 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:51.747438 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1683) took 1184 ms. Trace:
I20260709 12:24:51.747530 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2827) took 1156 ms. Trace:
I20260709 12:24:51.748128 10090 rpcz_store.cc:276] 0709 12:24:50.591449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.591577 (+   128us) service_pool.cc:224] Handling call
0709 12:24:51.747314 (+1155737us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b540002dd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:51.747454 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":753454848}
W20260709 12:24:51.752997 13553 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.857s	user 0.010s	sys 0.001s
I20260709 12:24:51.753545 13553 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.754644 10020 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:51.663686 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:51.759441 10025 raft_consensus.cc:606] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:51.759954 10025 raft_consensus.cc:614] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:51.766414 13583 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:51.768796 13146 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:51.741051 10671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 747) took 1627 ms (client timeout 2142 ms). Trace:
I20260709 12:24:51.728513 13609 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:24:51.788684  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64731.2us
I20260709 12:24:51.791249  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:51.731634 13614 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:51.782541 10671 rpcz_store.cc:269] 0709 12:24:50.113352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.118713 (+  5361us) service_pool.cc:224] Handling call
0709 12:24:51.740884 (+1622171us) spinlock_profiling.cc:243] Waited 57.7 ms on lock 0x7b54006ff754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:51.740994 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132619136}
I20260709 12:24:51.794956  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:51.812264 13306 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.673503 13503 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:51.813786 13382 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.814466 13503 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.814837 13492 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:51.815923 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2751) took 1253 ms. Trace:
I20260709 12:24:51.816277 10022 rpcz_store.cc:276] 0709 12:24:50.562300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.563018 (+   718us) service_pool.cc:224] Handling call
0709 12:24:50.563352 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:50.857520 (+294168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:51.815699 (+958179us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b54001f8954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:51.815866 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":634714368}
I20260709 12:24:51.691383 13601 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:51.819583 13444 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:51.820849 13601 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:51.825712 13301 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.683053 13499 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.830282 13492 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.781888 13283 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Committing config change with OpId 5.5: config changed from index 3 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:24:51.769904 13404 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.917s	user 0.012s	sys 0.000s
I20260709 12:24:51.830833 13359 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:51.835330  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:51.835382  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:51.835467 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:51.836149 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:51.892938 10090 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:51.797248  9946 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:24:51.747905 10018 rpcz_store.cc:276] 0709 12:24:50.563159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.563553 (+   394us) service_pool.cc:224] Handling call
0709 12:24:50.563884 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:50.564257 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:50.564550 (+   293us) log.cc:844] Serialized 14 byte log entry
0709 12:24:50.977575 (+413025us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:50.992778 (+ 15203us) log.cc:844] Serialized 26 byte log entry
0709 12:24:50.992923 (+   145us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:50.992961 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:50.993034 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:51.747245 (+754211us) raft_consensus.cc:1685] finished
0709 12:24:51.747263 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:51.747380 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38150,"prepare.queue_time_us":698,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":37115,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":5313,"thread_start_us":396440,"threads_started":2,"wal-append.queue_time_us":448861}
I20260709 12:24:51.803258  9940 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:51.762179 10640 leader_election.cc:336] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.784s (SENT)
I20260709 12:24:51.898212 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:51.844516  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:51.794857 13531 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:51.845677 13565 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:51.845989 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:51.846640 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:51.847198 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:51.847415 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:51.848233 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:51.933003 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 84840.1us
I20260709 12:24:51.932152  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:51.848750 13180 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:51.845667 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:51.854748  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:51.934396 10091 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:51.934459  9946 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:51.794385 13599 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:24:51.965251 10095 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:51.860230  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:51.786221 13493 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:51.861512 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:51.966987 10097 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:51.856561  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66348.8us
I20260709 12:24:51.863842  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:51.968345  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 107295us
I20260709 12:24:51.969250  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:51.858363 10022 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:51.935256 10094 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:51.928694  9941 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:51.780006 13515 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:51.991986 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:51.992178  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:51.988745 10024 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260709 12:24:51.993739 10024 pending_rounds.cc:85] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 7
I20260709 12:24:51.994050 10024 pending_rounds.cc:107] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 12:24:51.994720 10024 raft_consensus.cc:2889] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:51.781936 13582 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:51.987017 13542 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:51.998087  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:51.999084  9941 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:52.000545  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.001461  9946 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:52.002574  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.003328  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:52.005846 13287 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:52.006387 13287 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:52.006847 13561 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:52.007042 13601 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:52.007516 13561 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.008028 13561 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.008061 13601 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:52.009347 13303 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:24:52.011494 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:52.012038 13553 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:51.932245 13312 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.012898 13614 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:52.013375 13312 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.014760 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:52.015570 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:52.016191 10020 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
W20260709 12:24:52.017755 13579 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.750s	user 0.008s	sys 0.003s
I20260709 12:24:52.020882 13580 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:52.022027 10016 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:52.023134 13180 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:51.833961 13593 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:51.850324 13301 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.044478  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:52.045509  9946 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.046872  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.047154 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:52.047940  9947 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:24:52.015574 10668 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:52.030532  9945 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:52.031738  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.032191  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.049750 10020 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:52.047955 10671 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:24:52.056979  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.057647  9941 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:52.055186 13542 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:52.066347 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:51.927089 13531 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:51.933799 13359 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.067590 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:52.079587 10020 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.080168 10674 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:24:52.068279 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 306173us
W20260709 12:24:52.082144 10640 leader_election.cc:336] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.898s (SENT)
I20260709 12:24:52.074472 10021 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:52.077991  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:52.078433  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.061496 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:52.083990 10025 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
I20260709 12:24:52.084501 10025 pending_rounds.cc:85] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 4
I20260709 12:24:52.084748 10025 pending_rounds.cc:107] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:24:52.085022 10025 raft_consensus.cc:2889] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:52.085863 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:52.086939 10090 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:51.795557 13022 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:52.027390 13493 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.006841 13287 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:51.861572 13492 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:52.062420 13504 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.200s	user 0.013s	sys 0.001s
I20260709 12:24:52.103597 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:52.104063 10019 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:52.067915 13301 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.096060 10021 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:52.141891 10020 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:52.124383 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:52.108351  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:52.136246 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:52.155570 10090 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:51.987633 13565 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:52.175024  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66759.6us
I20260709 12:24:52.079118 13531 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.178839  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:52.137403 10017 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:52.191323  9948 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 8
W20260709 12:24:52.111721 13537 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.072s	user 0.013s	sys 0.000s
I20260709 12:24:51.962831 13404 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.067819 13235 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 12:24:52.098498 13515 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:52.207269 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:52.107349 13529 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.212042 10099 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:52.212164 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 130063us
I20260709 12:24:51.778199 13576 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:52.232213 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:52.219223 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:52.099402 13475 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
I20260709 12:24:52.197989 13582 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.243589 10020 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:52.099840 13146 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.118s
I20260709 12:24:52.269543  9948 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:52.285799 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:52.311278 10090 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:52.259406 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:52.274396 10018 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:52.199410 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:52.313760 10024 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:52.136112 13028 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:24:52.211212 13339 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:24:52.315258  9839 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.025s	sys 0.126s
W20260709 12:24:52.320118  9839 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.026s	sys 0.126s
I20260709 12:24:52.207708 13382 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.318904 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:52.321022 10020 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:52.065567 13593 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.055996 12938 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:52.198256 13504 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.328058 13498 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:52.203016 13608 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.099s
W20260709 12:24:52.306190 13587 consensus_queue.cc:1175] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aa345bcd43b44088114b7b6f2830b17" responder_term: 8 status { last_received { term: 7 index: 5 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:52.334720 13382 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.336417 13609 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.337312 13591 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.409s
I20260709 12:24:52.337919 13535 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:52.338640  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.338835 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:52.339869 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:52.348448 13359 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:52.386823 13507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.038s	sys 0.082s
I20260709 12:24:52.352759 13493 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:52.224457 13515 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.356477 10671 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:52.235020 13166 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.230826 13431 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:52.233479 13580 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:52.231174 13363 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 12:24:52.366297 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:52.305493 13542 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.369668 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:52.395213 13535 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:52.225322 13303 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:52.387022 10674 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:52.213868 13537 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.391458  9940 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:52.350160  9946 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:52.396844 10096 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:52.398443 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.007s	sys 0.035s
W20260709 12:24:52.395618  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.423264 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.007s	sys 0.035s
W20260709 12:24:52.399626 13367 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.005s	sys 0.035s
W20260709 12:24:52.423600 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 703217us
I20260709 12:24:52.405102 13515 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:52.423483 13522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.536s	user 0.011s	sys 0.027s
W20260709 12:24:52.398721  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.400135 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:52.327149 13582 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:52.394351 13523 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.401s	user 0.014s	sys 0.000s
W20260709 12:24:52.395406 13477 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.428s	user 0.022s	sys 0.002s
I20260709 12:24:52.411715 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:52.390321 13512 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:52.309746 13576 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.414377 10022 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:52.414366 10090 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
W20260709 12:24:52.371732  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.395262 13565 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:52.418380 10024 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
W20260709 12:24:52.399437 13507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.038s	sys 0.083s
I20260709 12:24:52.447135  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:52.428381 13367 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.005s	sys 0.036s
I20260709 12:24:52.420239 13535 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:52.459664  9943 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:52.461920 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:52.400254 13605 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 12:24:52.438444 13522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.561s	user 0.011s	sys 0.028s
W20260709 12:24:52.463105 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.019s	sys 0.027s
W20260709 12:24:52.439807 13525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.030s	sys 0.007s
W20260709 12:24:52.599889 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.020s	sys 0.027s
I20260709 12:24:52.458734 13610 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.519644  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.603186 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:52.460692 13301 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:52.609257  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 162199us
I20260709 12:24:52.610312  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:52.511801 13586 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.515563  9946 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:52.511593 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:52.516215  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.516636  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:52.465426  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.526253  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:52.457644 13477 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.490s	user 0.022s	sys 0.003s
I20260709 12:24:52.427597 13303 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.508770 10025 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:24:52.412914 13166 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:52.600260 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 835229us
I20260709 12:24:52.603120 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:52.701552  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:52.526273 13492 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:52.716480 13582 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:52.716631  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.717841  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:52.508723 13359 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:52.719380 10093 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:52.718125 10065 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:52.489554 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:52.600544 13580 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:52.692101 13609 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:52.689086 13180 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:52.603441 13525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.031s	sys 0.007s
I20260709 12:24:52.726652 13431 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.457451 13022 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:52.461833 13523 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:52.706404 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 194920us
I20260709 12:24:52.716321 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:52.738758  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:52.739144 10669 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:24:52.696323  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.701717 10090 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:24:52.728158 10020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
W20260709 12:24:52.740097  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:52.740183 10020 raft_consensus.cc:606] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:52.740567 10020 raft_consensus.cc:614] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:52.735276 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:52.694058 13287 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:52.720005 13370 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.988s	user 0.013s	sys 0.001s
I20260709 12:24:52.778623 13306 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:52.727283 13508 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:52.762902 10677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.057s	sys 0.102s
W20260709 12:24:52.836750 10677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.058s	sys 0.102s
I20260709 12:24:52.748342  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:52.839481  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 96932.8us
I20260709 12:24:52.840024 10098 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:24:52.771354  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.718333 13582 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:52.741247 13359 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:24:52.857458 13591 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:52.855648 13404 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:52.859007 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2835) took 2256 ms. Trace:
I20260709 12:24:52.859395 10093 rpcz_store.cc:276] 0709 12:24:50.602074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.953969 (+351895us) service_pool.cc:224] Handling call
0709 12:24:50.954340 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.719778 (+1765438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:52.858785 (+139007us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b540028fd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:52.858951 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4058834816}
I20260709 12:24:52.860759 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:52.861511 10093 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:52.861918 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1741) took 1023 ms. Trace:
I20260709 12:24:52.840577 13382 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:52.841521 13561 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.833s	user 0.007s	sys 0.004s
I20260709 12:24:52.863595 13561 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:52.865164  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1553) took 1074 ms. Trace:
I20260709 12:24:52.865779 13561 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:52.865552  9948 rpcz_store.cc:276] 0709 12:24:51.790566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.790745 (+   179us) service_pool.cc:224] Handling call
0709 12:24:52.865006 (+1074261us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b5400016dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:52.865106 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":871206656}
I20260709 12:24:52.866709 10021 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:52.871124 13622 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:52.766340 13524 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:52.778581 13287 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:52.779059 13619 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:52.780251 13590 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.993s	user 0.010s	sys 0.000s
I20260709 12:24:52.765923  9947 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:52.762701 10675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.929s	user 0.018s	sys 0.005s
I20260709 12:24:52.727334 13580 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:52.790295 10640 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.009s	sys 0.018s
W20260709 12:24:53.159044 10640 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.009s	sys 0.019s
W20260709 12:24:53.159379 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 938975us
I20260709 12:24:52.843393 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.162931 10090 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:52.735110 13477 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:52.849582 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:52.707327 13586 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:52.854907 13493 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:52.781158 13028 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:24:52.855602 13022 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:52.645504 13610 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:52.777731 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59672.5us
I20260709 12:24:53.159664 13580 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:52.782183 13504 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:52.782025 13593 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:53.010776 13283 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.053s	sys 0.116s
I20260709 12:24:52.972177  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:52.847249 13520 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:52.846540 13475 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:52.961463 13524 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.081787 13508 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:52.768057 13431 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:52.879496 13529 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.000s
W20260709 12:24:53.142153 13535 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.004s
I20260709 12:24:53.142988 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:53.199263  9946 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:53.144234 13303 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.205034 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.205750 10668 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:53.205945 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:53.145931  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 535718us
W20260709 12:24:53.117738  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.207978 10090 raft_consensus.cc:2410] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
W20260709 12:24:53.148268 13515 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.686s	user 0.005s	sys 0.011s
W20260709 12:24:53.142179 13503 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.327s	user 0.007s	sys 0.012s
I20260709 12:24:53.214958 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:53.096879 10674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.000s	sys 0.010s
I20260709 12:24:52.940966 13309 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:53.223003 10674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.000s	sys 0.012s
I20260709 12:24:53.224000 10090 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:53.151396  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 308065us
I20260709 12:24:53.226754  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:53.153668 13523 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:53.157249 10675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.323s	user 0.018s	sys 0.005s
I20260709 12:24:53.194845 13367 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:53.152153 13537 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.155170 13287 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.156668 13531 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.945s	user 0.005s	sys 0.013s
I20260709 12:24:52.862211 10093 rpcz_store.cc:276] 0709 12:24:51.838761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.860496 (+1021735us) service_pool.cc:224] Handling call
0709 12:24:52.861864 (+  1368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:53.151880 13166 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:53.165117 13180 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.165304 13512 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.157963 13590 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:53.205127 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 62212.8us
W20260709 12:24:53.243322 10673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 753) took 2719 ms (client timeout 2234 ms). Trace:
W20260709 12:24:53.144601 13576 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.576s	user 0.004s	sys 0.011s
I20260709 12:24:52.885603 13622 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.215030 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:52.766479 13301 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.107s
W20260709 12:24:53.198784 13283 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.064s	sys 0.156s
I20260709 12:24:53.218029 13493 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.218528 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:53.158217  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:53.231920  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.260800  9946 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:53.260922 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:53.211333 13529 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:53.236964 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.263902 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.264489 10093 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:53.181553 13477 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.204820 13475 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:53.238948 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 761) took 1020 ms. Trace:
I20260709 12:24:53.239398  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:53.212189 13542 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:53.270908 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.216537 13504 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:53.267361 10677 rpcz_store.cc:276] 0709 12:24:52.218348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.218505 (+   157us) service_pool.cc:224] Handling call
0709 12:24:52.218916 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.219193 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:52.837292 (+618099us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.238645 (+401353us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.238693 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:53.238738 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.238796 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.238896 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1762,"prepare.queue_time_us":933425,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":114,"raft.queue_time_us":24071,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":15939,"thread_start_us":1009990,"threads_started":3,"wal-append.queue_time_us":399026}
I20260709 12:24:53.270187 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.247745 13404 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.243707 10673 rpcz_store.cc:269] 0709 12:24:50.518931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:50.521079 (+  2148us) service_pool.cc:224] Handling call
0709 12:24:53.237719 (+2716640us) spinlock_profiling.cc:243] Waited 369 ms on lock 0x7b54006126d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.237857 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":848103680}
I20260709 12:24:53.230548 13367 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:53.236662 13028 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 12:24:53.250026 10021 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:53.250211 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.250952 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.228560 13632 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:53.164716 13619 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:53.293779 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.294467 10021 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:53.294991 13619 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:53.295305 13632 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.297724 13632 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:53.276961 13636 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:24:53.272526 13603 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.255491 13522 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.241642 13287 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.253142 13512 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:53.245245 10642 connection.cc:256] RPC call timeout handler was delayed by -1.1531475649999265s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1748s for a response
I20260709 12:24:53.264393 13303 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:24:53.197638 13610 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.258409 13624 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.329241 13624 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.266021 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:53.240608 13503 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:53.271754 10092 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:53.330667 10025 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
W20260709 12:24:53.330962 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65065.1us
I20260709 12:24:53.331794 13477 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.273633 10093 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:53.245172 13166 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.206924 13535 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.245518 10090 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:53.229152 13515 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.272665 13569 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:24:53.279974 13404 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.252072 13359 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:53.251601 13582 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.248307  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:53.285387  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:53.256191 13622 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:53.290846 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 51505us
I20260709 12:24:53.268961 13492 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:53.301584 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.446254 10024 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:53.301717 10020 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:53.240471 13531 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:53.254997 13631 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:24:53.271718 13306 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:53.271842 13475 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.214313 13508 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.478474 13632 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:53.479315 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1731) took 1644 ms (client timeout 1671 ms). Trace:
I20260709 12:24:53.472546 13306 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:53.480548 13306 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.308211 13561 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:53.481336 13561 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:53.482237 13561 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.247090 13180 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.260823  9947 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:53.479741 10099 rpcz_store.cc:269] 0709 12:24:51.834988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.835122 (+   134us) service_pool.cc:224] Handling call
0709 12:24:53.479113 (+1643991us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400164254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.479255 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":729312640}
I20260709 12:24:53.288323 13593 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:53.332734 13477 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:53.332039 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1733) took 1496 ms (client timeout 1734 ms). Trace:
W20260709 12:24:53.507836 10092 rpcz_store.cc:269] 0709 12:24:51.836079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.845743 (+  9664us) service_pool.cc:224] Handling call
0709 12:24:53.331864 (+1486121us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54003ae3d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.331984 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3080065408}
I20260709 12:24:53.255915 13526 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.300747 13028 consensus_queue.cc:1243] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:24:53.333577 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.473313 13492 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.320046 13603 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.430891 13586 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.432432 13590 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.289863 13641 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.152s
I20260709 12:24:53.532508  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.434528 13431 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.434252 13146 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.438680 13524 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.278275 13542 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.333485 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2841) took 1476 ms. Trace:
I20260709 12:24:53.533591 13146 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.533939  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:53.535411 13542 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:53.466446  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 218202us
I20260709 12:24:53.536293 13146 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.401247 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 849) took 1740 ms. Trace:
I20260709 12:24:53.335255 13535 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.410719 13503 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:53.550464 13503 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.333132 13166 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:53.477440 13602 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.777s	user 0.008s	sys 0.009s
I20260709 12:24:53.477880 13531 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:53.429059 13637 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:24:53.554879 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:53.555979 13431 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:53.487381 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 58206.7us
I20260709 12:24:53.556463 13431 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.557430 13580 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.558415 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2761) took 1567 ms. Trace:
I20260709 12:24:53.558851 13431 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:53.321228 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:53.559448 13431 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:53.559873 13431 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:53.560019 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 238920us
I20260709 12:24:53.501159 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.507418 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.393478 13382 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.515147 10093 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:53.376693  9990 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.671s (SENT)
I20260709 12:24:53.560276 13603 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.560245 13431 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.589982 13303 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:53.589311 13475 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.591400  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:53.592133  9948 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:53.592309  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:53.592890  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:53.410856 13486 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:53.523981 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.533831 10090 rpcz_store.cc:276] 0709 12:24:51.856824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.214608 (+1357784us) service_pool.cc:224] Handling call
0709 12:24:53.333430 (+118822us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:53.479337 13632 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:53.538378 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.335722 13515 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.539170 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.595662  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:53.514045  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.595993 10021 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:53.596726  9946 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:53.536904 10675 rpcz_store.cc:276] 0709 12:24:51.661321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.713150 (+ 51829us) service_pool.cc:224] Handling call
0709 12:24:51.725945 (+ 12795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.784618 (+ 58673us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:24:53.230661 (+1446043us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.400767 (+170106us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.400844 (+    77us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:53.400914 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.401007 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.401168 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30052,"prepare.queue_time_us":1367731,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":24348,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":23331,"thread_start_us":1540255,"threads_started":2,"wal-append.queue_time_us":300007}
I20260709 12:24:53.541908 13624 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.518537 13526 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.543989 13287 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.512629 13647 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:24:53.449868 13367 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.429076 13359 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.524892 13312 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.525524 13492 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.323549 13522 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.549115 13180 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:53.558358 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:53.427932 13404 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.592449 13475 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.592971  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.665052  9947 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:53.337739 13512 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.597086 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.666811 10021 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:53.507325 13593 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.558966 10022 rpcz_store.cc:276] 0709 12:24:51.990931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.991033 (+   102us) service_pool.cc:224] Handling call
0709 12:24:51.991338 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.552485 (+561147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.558197 (+1005712us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b540038d6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:53.558358 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972496256}
I20260709 12:24:53.671864 10024 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:24:53.489921 13582 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.599274 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:53.599426 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 111301us
I20260709 12:24:53.673323 10677 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.599918 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1742) took 1761 ms. Trace:
I20260709 12:24:53.599273 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:53.674201 10099 rpcz_store.cc:276] 0709 12:24:51.838981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.507047 (+1668066us) service_pool.cc:224] Handling call
0709 12:24:53.599852 (+ 92805us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:53.560169  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1786) took 1047 ms. Trace:
I20260709 12:24:53.677670 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.601573  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.601677 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.603811 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1746) took 1261 ms. Trace:
I20260709 12:24:53.612655 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.613525 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.679195 10668 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 12:24:53.679582 10673 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:53.679807 10675 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:53.679193  9946 raft_consensus.cc:2410] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:24:53.624114 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.680493 10676 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.625182 13359 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.625790 13499 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:53.598443 13641 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:24:53.629753 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.685703 10097 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:53.602937 13486 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.630926 13635 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:53.687251  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.638626  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1558) took 1775 ms. Trace:
I20260709 12:24:53.640586  9948 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:53.602767 13382 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.642230 13508 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:53.642693 13367 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.613346 13235 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:24:53.618752 13535 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:53.643549 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2839) took 3040 ms. Trace:
W20260709 12:24:53.652486  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 275815us
I20260709 12:24:53.661492 10093 raft_consensus.cc:2410] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:24:53.663885 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 105603us
I20260709 12:24:53.619323 13287 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:53.670836 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:53.711971 10677 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:53.712275 10021 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.672333 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:53.614327 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.697647 10096 rpcz_store.cc:276] 0709 12:24:50.603454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.461225 (+857771us) service_pool.cc:224] Handling call
0709 12:24:51.461806 (+   581us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.423222 (+961416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.643275 (+1220053us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b54003aded4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:53.643465 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2149861120}
I20260709 12:24:53.682693 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.679008 10094 rpcz_store.cc:276] 0709 12:24:52.342651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.500846 (+1158195us) service_pool.cc:224] Handling call
0709 12:24:53.603739 (+102893us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:53.713709 10022 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:53.684448 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 85239.8us
I20260709 12:24:53.713716 10090 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:53.714630 10092 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:53.687357 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.675343  9942 rpcz_store.cc:276] 0709 12:24:52.512482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.517287 (+  4805us) service_pool.cc:224] Handling call
0709 12:24:52.874126 (+356839us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.874403 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:53.213107 (+338704us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.245516 (+ 32409us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.245618 (+   102us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:53.245674 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.245742 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.560009 (+314267us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b540009c9d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.560113 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":372793,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":13758,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":8080,"spinlock_wait_cycles":715712384,"thread_start_us":91744,"threads_started":2,"wal-append.queue_time_us":110535}
I20260709 12:24:53.688388  9943 rpcz_store.cc:276] 0709 12:24:51.863360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.863497 (+   137us) service_pool.cc:224] Handling call
0709 12:24:51.863821 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.597151 (+733330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.638286 (+1041135us) spinlock_profiling.cc:243] Waited 596 ms on lock 0x7b54000168d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:53.638520 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1369795840}
I20260709 12:24:53.714504 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:53.691893  9947 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:53.716980 10674 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:24:53.717413  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.692796  9946 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:53.717931  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.717815 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.718116  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1793) took 1154 ms. Trace:
I20260709 12:24:53.718504  9946 rpcz_store.cc:276] 0709 12:24:52.563752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.680632 (+1116880us) service_pool.cc:224] Handling call
0709 12:24:53.692617 (+ 11985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.717894 (+ 25277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.718067 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:53.670246 13180 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:53.690435 13635 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:53.618904 13648 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:24:53.727288 10673 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:53.712910 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1747) took 1370 ms. Trace:
I20260709 12:24:53.717335  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1792) took 1202 ms. Trace:
I20260709 12:24:53.692953  9948 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:53.719102 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.728161 10093 rpcz_store.cc:276] 0709 12:24:52.342952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.333057 (+990105us) service_pool.cc:224] Handling call
0709 12:24:53.712831 (+379774us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:53.728826  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1565) took 1026 ms. Trace:
W20260709 12:24:53.719591  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 180689us
I20260709 12:24:53.706712 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:53.728385  9947 rpcz_store.cc:276] 0709 12:24:52.514447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.672982 (+1158535us) service_pool.cc:224] Handling call
0709 12:24:53.689728 (+ 16746us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.717074 (+ 27346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.717286 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:53.718420  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.687716 13499 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:53.672787 13593 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.722723 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.722891 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.692344 13508 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:53.727336 10021 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:53.709831 10676 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:24:53.687994 13515 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:53.729161  9948 rpcz_store.cc:276] 0709 12:24:52.702900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.692296 (+989396us) service_pool.cc:224] Handling call
0709 12:24:53.692716 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.728567 (+ 35851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.728768 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37120}
I20260709 12:24:53.730988 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.721333 13642 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:53.733062 13542 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:53.727196 13569 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:53.733793 13526 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.736001 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.729830 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:53.661082 13632 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.738348 10674 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:53.720543 13647 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:24:53.747987 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.768519 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.768889 10676 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:53.715358 13512 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:24:53.732028  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:53.749670 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.719772 13235 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:53.751931 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.752092 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:53.778667 10021 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:53.778510  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46566us
W20260709 12:24:53.739002  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.779666  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:53.737974 13644 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:24:53.730412 13523 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.765036 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.767874 10093 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
W20260709 12:24:53.773097  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.723940 13303 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:24:53.737701 13022 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.775756 10676 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:53.755564 13309 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.001s
I20260709 12:24:53.768281 13569 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.765132 13542 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.855271 13542 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.788009 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.856003 10022 raft_consensus.cc:2368] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:53.856957 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.858031 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.789702 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.859076 10093 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:53.859491 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2849) took 1411 ms. Trace:
I20260709 12:24:53.859783 10093 rpcz_store.cc:276] 0709 12:24:52.448542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.789441 (+1340899us) service_pool.cc:224] Handling call
0709 12:24:53.859436 (+ 69995us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:53.860495 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:53.786790 13650 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:24:53.858549 10022 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:53.794678  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:53.805459 10021 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:24:53.863871  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 69394us
I20260709 12:24:53.864181 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.864575  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:53.865335 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.865773  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:53.796842 13339 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 12:24:53.796831 13499 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.867306 13339 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:53.868110 13339 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:53.806912 13553 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:53.813398 13569 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:53.868710 13553 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.869112 13553 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.869604 13339 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.769434 13642 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:53.875070 13642 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.790850 13404 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.876161  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:53.876302 13404 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.798012 13523 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.878895 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.879941 10025 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:24:53.814832 13508 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.817171 13582 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.880952 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:53.881444 13582 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:53.818446 13522 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.882793 10097 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:53.883154 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2759) took 2196 ms. Trace:
I20260709 12:24:53.883616 10016 rpcz_store.cc:276] 0709 12:24:51.686712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.689816 (+  3104us) service_pool.cc:224] Handling call
0709 12:24:51.690078 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.022417 (+332339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.882844 (+1860427us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b540038d1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:53.883050 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763007232}
I20260709 12:24:53.884233 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.885047 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.880769 13523 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:53.885991 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.886562 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.818691 13166 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:53.821825 13493 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.000s
I20260709 12:24:53.822168 13635 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.887975 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.819414 13235 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.792338 13515 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:53.888444 13635 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:53.888803 13515 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:53.889246 13515 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:53.889791 10018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2760) took 1900 ms (client timeout 1714 ms). Trace:
I20260709 12:24:53.825274 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:53.890179 10018 rpcz_store.cc:269] 0709 12:24:51.989450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.989655 (+   205us) service_pool.cc:224] Handling call
0709 12:24:53.889600 (+1899945us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54001f7a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.889723 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475881728}
I20260709 12:24:53.890873 10017 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:53.891363 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.891712 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.891959 10018 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:53.892256 10017 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 5.
I20260709 12:24:53.892854 13404 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.893630 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.894207 10017 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:53.828269 13593 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:53.830025 13653 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:24:53.829917 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 846) took 2173 ms. Trace:
I20260709 12:24:53.831175 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.894959 10670 rpcz_store.cc:276] 0709 12:24:51.656694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.678574 (+ 21880us) service_pool.cc:224] Handling call
0709 12:24:51.704632 (+ 26058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:51.785686 (+ 81054us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:51.785728 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:51.785782 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:51.785834 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.829655 (+2043821us) spinlock_profiling.cc:243] Waited 80.6 ms on lock 0x7b5400690d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:53.829700 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.829840 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2793,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":10838,"spinlock_wait_cycles":185402240}
I20260709 12:24:53.896103 13593 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.792249 13526 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.828822 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.838264 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 716) took 1410 ms. Trace:
I20260709 12:24:53.896418  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:53.843406 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.841550 10019 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:53.897265 10095 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:53.897459 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.898052 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.808504 13507 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.845124 13306 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.898720 13507 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.899251 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2846) took 1454 ms. Trace:
I20260709 12:24:53.899631 10092 rpcz_store.cc:276] 0709 12:24:52.445330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.508952 (+1063622us) service_pool.cc:224] Handling call
0709 12:24:53.899114 (+390162us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400163d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.899201 (+    87us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275196288}
I20260709 12:24:53.900969 13306 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.901104 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:53.898692  9942 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:53.902104 13507 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.845543 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:53.902791  9944 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:53.902814  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1798) took 1044 ms. Trace:
I20260709 12:24:53.903724  9942 rpcz_store.cc:276] 0709 12:24:52.858142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.716542 (+858400us) service_pool.cc:224] Handling call
0709 12:24:53.730767 (+ 14225us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.901920 (+171153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.902471 (+   551us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540009b5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:53.902758 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385600768}
I20260709 12:24:53.860615 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.836975 13561 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.904474 13486 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:53.904693 10093 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:53.724555 13535 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.787402 13580 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:53.906854 13615 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.981s	user 0.011s	sys 0.019s
I20260709 12:24:53.907074 13580 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:53.810288 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.908128 13535 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.878491 13309 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:53.908439 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.908921 13309 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:53.909379 13309 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.746447 13382 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:53.888034 13493 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:53.827308 13022 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.911806 13643 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:53.912200 13643 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:53.913169 13022 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:53.896250 10020 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:53.896941 10671 rpcz_store.cc:276] 0709 12:24:52.427402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.428237 (+   835us) service_pool.cc:224] Handling call
0709 12:24:52.428495 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:52.742852 (+314357us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:53.070225 (+327373us) log.cc:844] Serialized 14 byte log entry
0709 12:24:53.230865 (+160640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:53.230939 (+    74us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:53.231007 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.838032 (+607025us) spinlock_profiling.cc:243] Waited 314 ms on lock 0x7b5400615b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:53.838076 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:53.838207 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":410924,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":52,"raft.queue_time_us":37373,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":7059,"spinlock_wait_cycles":722460416,"thread_start_us":373360,"threads_started":3,"wal-append.queue_time_us":107916}
I20260709 12:24:53.914270 13363 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:53.914326 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.898258 10094 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:53.914809 13363 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:53.868140 13499 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:53.831571 13520 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.625s	user 0.011s	sys 0.000s
I20260709 12:24:53.880748  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:53.898682 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 530) took 1559 ms. Trace:
I20260709 12:24:53.915630 13363 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:53.916165 10019 rpcz_store.cc:276] 0709 12:24:52.338802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.339014 (+   212us) service_pool.cc:224] Handling call
0709 12:24:53.841302 (+1502288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:53.897714 (+ 56412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:53.898480 (+   766us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001f5c50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.898623 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3454062336}
I20260709 12:24:53.865370 13287 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:53.917359 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:53.917604 13287 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.917982 13287 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:53.896476 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.918982 10670 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
W20260709 12:24:53.919085 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 73678.2us
I20260709 12:24:53.866019 10021 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:53.920296 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.882326 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:53.915747  9949 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:53.904449 10018 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:53.908159 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:53.839848 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 518) took 2135 ms (client timeout 1643 ms). Trace:
I20260709 12:24:53.904428  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1564) took 1906 ms. Trace:
I20260709 12:24:53.923976  9944 rpcz_store.cc:276] 0709 12:24:51.998225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.028576 (+ 30351us) service_pool.cc:224] Handling call
0709 12:24:53.904258 (+1875682us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b5400189cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.904375 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4184013184}
I20260709 12:24:53.915723 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:53.924710  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.865826 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:53.898273 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.926229 10095 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.925606 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:53.927314  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.916625 13363 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.945727 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.902263 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:53.946496 10092 raft_consensus.cc:2376] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 8. Re-sending same reply.
I20260709 12:24:53.946511 10019 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:53.921078 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:53.929032  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.925606 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:53.921047 10021 raft_consensus.cc:2410] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:53.935724 10093 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:24:53.863380 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.949102 10022 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:53.943390 13339 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.944059 13493 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:53.944428 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1734) took 2107 ms (client timeout 1991 ms). Trace:
W20260709 12:24:53.952296 10098 rpcz_store.cc:269] 0709 12:24:51.836809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.836916 (+   107us) service_pool.cc:224] Handling call
0709 12:24:53.944128 (+2107212us) spinlock_profiling.cc:243] Waited 908 ms on lock 0x7b5400305554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.944330 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2087406464}
W20260709 12:24:53.923089 10023 rpcz_store.cc:269] 0709 12:24:51.704328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.704482 (+   154us) service_pool.cc:224] Handling call
0709 12:24:53.839512 (+2135030us) spinlock_profiling.cc:243] Waited 32.8 ms on lock 0x7b54001f5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.839715 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75360512}
I20260709 12:24:53.945648  9948 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 12:24:53.953296  9948 pending_rounds.cc:85] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 5
I20260709 12:24:53.913093 13643 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 6.9, Last appended by leader: 9, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:53.953480 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 223714us
I20260709 12:24:53.953660 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:53.955123  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74827.1us
I20260709 12:24:53.956851  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:53.953534  9948 pending_rounds.cc:107] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 7.6
I20260709 12:24:53.959349  9948 raft_consensus.cc:2889] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:53.946017  9912 leader_election.cc:400] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:53.947171 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.960860  9912 leader_election.cc:403] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:24:53.961143 10025 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:53.962338 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:53.963888 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 98134.7us
I20260709 12:24:53.964107  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:53.927574 10096 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:24:53.964754  9945 raft_consensus.cc:2376] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8. Re-sending same reply.
I20260709 12:24:53.948475  9941 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:53.938656  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:53.969128 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 43609.2us
W20260709 12:24:53.970515 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49488.1us
I20260709 12:24:53.971558 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:53.971868  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:53.970676 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.973157 10677 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:53.981916  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:53.982195  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.982239 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.983127  9945 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:53.983417 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:53.981926  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.985297 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:53.985994 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.991098 10669 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 6: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:53.992087  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.992301  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:53.945104 13444 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:53.994823 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:53.988001  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:53.995577 10670 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:53.946489 13591 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:53.989537 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:53.989909 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:53.997053  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:53.984602 13166 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:53.986614  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:53.993621  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:53.986483  9949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1556) took 2132 ms (client timeout 1558 ms). Trace:
I20260709 12:24:53.980755 13542 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:53.995675 10019 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:53.898672 13522 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:53.988094  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:53.994527 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:54.021003  9945 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:24:53.993299 13523 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:53.994515 13582 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.001377 13605 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:53.995306 13022 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8.
I20260709 12:24:54.005456  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:54.002887 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:53.985031 13647 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:54.006368 13622 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:24:54.032130 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:53.985158 13593 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:24:53.921578 13608 consensus_queue.cc:1175] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aa345bcd43b44088114b7b6f2830b17" responder_term: 9 status { last_received { term: 8 index: 4 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:24:54.007495  9949 rpcz_store.cc:269] 0709 12:24:51.854238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:51.854401 (+   163us) service_pool.cc:224] Handling call
0709 12:24:53.986324 (+2131923us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54003a0554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:53.986421 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4557040896}
I20260709 12:24:53.992045 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:54.008051 13529 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.002799 12938 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.008289  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:54.002655 13583 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:54.008291 13477 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:54.025323 13523 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:53.988831 13404 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:54.001248 13444 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.062282 10672 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:54.017124  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:53.991550 13643 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:53.994315 13498 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.002420 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:54.073205 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:54.074301 13583 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:54.023092 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:54.074712  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.075237 10017 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 6: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:54.075313 13583 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.076067 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:54.076882 10017 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:24:54.077347 13367 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.077988 10017 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:54.077823 13367 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.005175  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.079991 13367 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:54.081269 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:54.081895 13477 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:54.082906 13477 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.086542  9948 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:54.089213 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:54.090680 13553 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.093240 13553 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:54.062322 13404 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:54.095170 13404 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.075434  9940 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:54.096285  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.097031 13526 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.097982 10020 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:54.099306 13287 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.101917 13287 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:54.102772 13498 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.034578  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.109030 10095 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:54.020570 13503 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:54.048359  9944 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.049077  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:54.024827 13580 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.052317 10097 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [5.5-7.6]
I20260709 12:24:54.052976 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:54.010623 13614 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:24:54.016801 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:54.131946 10668 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:54.133152 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1743) took 2277 ms. Trace:
I20260709 12:24:54.133786 10097 rpcz_store.cc:276] 0709 12:24:51.856035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.029295 (+2173260us) service_pool.cc:224] Handling call
0709 12:24:54.051750 (+ 22455us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:54.132550 (+ 80800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.133068 (+   518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.004262 13492 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.047945 13529 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:54.114491 13503 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.140157 13529 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.140264 13503 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:54.066354 13655 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.276s	user 0.001s	sys 0.000s
I20260709 12:24:54.062041 13523 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.027822 13605 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.067494 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:54.143399 10098 raft_consensus.cc:2376] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9. Re-sending same reply.
I20260709 12:24:54.144378 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:54.069228  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52172.4us
I20260709 12:24:54.004621 13591 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.021960  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.152905  9945 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:24:54.153127  9944 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:54.153512  9990 leader_election.cc:400] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:24:54.074445 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:54.032002 13647 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:53.992682 13306 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:54.106987  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:54.167883 10098 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:54.168365 10020 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: [10.5-10.5]
W20260709 12:24:54.130757 13431 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.001s
I20260709 12:24:54.177402  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.175066 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:54.178552  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71650.5us
I20260709 12:24:54.179059 10092 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:54.179582  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:54.131057 13526 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.035674 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:54.109489  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.113673 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:54.192883 10095 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
W20260709 12:24:54.115095  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66093.2us
I20260709 12:24:54.194018  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:54.195150  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:54.119486  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:54.196125  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 76708.7us
I20260709 12:24:54.196811  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:54.120375 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 543) took 1659 ms. Trace:
I20260709 12:24:54.135391  9943 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:24:54.198968  9943 pending_rounds.cc:85] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 4
I20260709 12:24:54.199393  9943 pending_rounds.cc:107] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:24:54.199668  9943 raft_consensus.cc:2889] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:54.137930 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:54.201098 10093 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:54.202497 10093 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.203140 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2840) took 2346 ms. Trace:
I20260709 12:24:54.203485 10093 rpcz_store.cc:276] 0709 12:24:51.856581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.202046 (+2345465us) service_pool.cc:224] Handling call
0709 12:24:54.202332 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.202922 (+   590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.203088 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.205122 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2842) took 2348 ms. Trace:
I20260709 12:24:54.139477 10024 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:54.206511 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:54.207281 10024 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:54.111297 13498 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.154803 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:54.211160 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:54.211580 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56905.6us
I20260709 12:24:54.211879 10095 raft_consensus.cc:2376] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9. Re-sending same reply.
I20260709 12:24:54.156680 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:54.143477 13614 consensus_queue.cc:1243] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:24:54.212927 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56348.3us
I20260709 12:24:54.214655 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:54.030453  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:54.175063  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.180845  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:54.073316 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:54.187340 10091 raft_consensus.cc:2368] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9.
I20260709 12:24:54.172708 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:54.072568 13522 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:54.234789 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62181.3us
I20260709 12:24:54.236078 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:54.174373 13647 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.198844 10096 rpcz_store.cc:276] 0709 12:24:52.461362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.732452 (+1271090us) service_pool.cc:224] Handling call
0709 12:24:54.120204 (+387752us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540014afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:54.120316 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328374016}
I20260709 12:24:54.149880 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:54.086727 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.207912 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:54.154541 10677 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:54.205472 10093 rpcz_store.cc:276] 0709 12:24:51.857051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.204380 (+2347329us) service_pool.cc:224] Handling call
0709 12:24:54.204659 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.204825 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.204849 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:54.204886 (+    37us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:54.204917 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.204966 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.205073 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.209919 10668 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:24:54.211272  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.167424 10019 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:24:54.219550  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 189177us
I20260709 12:24:54.214483 10669 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:54.232199 10020 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.151988 13605 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.233292 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:54.198302  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:54.238731 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:54.239086 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 165860us
W20260709 12:24:54.337905 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.339597 10668 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:54.174871 13306 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:54.412361  9945 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:54.413446  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.415148 10096 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:24:54.415419  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:54.337798 10095 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:24:54.415781 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2844) took 2558 ms. Trace:
I20260709 12:24:54.414767 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:54.416225 10096 rpcz_store.cc:276] 0709 12:24:51.857507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.414617 (+2557110us) service_pool.cc:224] Handling call
0709 12:24:54.414926 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.415558 (+   632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.415732 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.192785 13329 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.418160 10096 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:54.181134 13582 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:54.424168 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2848) took 1978 ms. Trace:
I20260709 12:24:54.424583 10096 rpcz_store.cc:276] 0709 12:24:52.445911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.417709 (+1971798us) service_pool.cc:224] Handling call
0709 12:24:54.418004 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.423944 (+  5940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.424120 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.425678  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.143030 13309 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.410458 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:54.420549 10093 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:54.421083 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2843) took 2563 ms. Trace:
I20260709 12:24:54.204241 13526 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:54.182050 13580 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.235535 13486 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.435413 10095 rpcz_store.cc:276] 0709 12:24:51.857274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.212732 (+2355458us) service_pool.cc:224] Handling call
0709 12:24:54.337457 (+124725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.417053 (+ 79596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.421023 (+  3970us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.339200 13512 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:54.339449 13571 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:54.445461 10092 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:54.446344  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.218843 13367 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:54.438194 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:54.436542 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1745) took 2094 ms. Trace:
I20260709 12:24:54.340579 13614 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:54.236546 13382 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.444048 10097 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:24:54.238420 13587 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:54.509328 10671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.003s	sys 0.024s
I20260709 12:24:54.405915 13569 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:24:54.448860 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 544) took 1972 ms. Trace:
I20260709 12:24:54.426564 13608 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.449829 10093 rpcz_store.cc:276] 0709 12:24:52.341917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.419141 (+2077224us) service_pool.cc:224] Handling call
0709 12:24:54.419847 (+   706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.436209 (+ 16362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.436474 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.515585 10093 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:54.191346 13431 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:54.445403 13658 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 12:24:54.448513  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.437088 13301 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:24:54.512298 13235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.032s	sys 0.029s
W20260709 12:24:54.511252 10671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.004s	sys 0.024s
I20260709 12:24:54.511431 10092 rpcz_store.cc:276] 0709 12:24:52.476273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.421072 (+1944799us) service_pool.cc:224] Handling call
0709 12:24:54.445229 (+ 24157us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.448599 (+  3370us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.448805 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.437565 13526 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.456068 13382 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.516301 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 547) took 1982 ms. Trace:
I20260709 12:24:54.236377 13553 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:54.451598 13589 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.724s	user 0.045s	sys 0.068s
I20260709 12:24:54.551374 10092 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
W20260709 12:24:54.414875 13632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.021s	sys 0.013s
W20260709 12:24:54.516968 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78856.4us
W20260709 12:24:54.454140 13561 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.549s	user 0.007s	sys 0.003s
I20260709 12:24:54.440968 13486 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:54.444831 13512 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.555658  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.559264 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 548) took 1991 ms. Trace:
I20260709 12:24:54.212081 13537 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:54.463972 13329 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:54.510350 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2845) took 2065 ms. Trace:
I20260709 12:24:54.423067 13306 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:54.485505 10024 raft_consensus.cc:2368] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6.
I20260709 12:24:54.515003 13507 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:54.543429 10093 rpcz_store.cc:276] 0709 12:24:52.534321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.515036 (+1980715us) service_pool.cc:224] Handling call
0709 12:24:54.515383 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.515998 (+   615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.516219 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.636745 10093 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:54.633931 10097 rpcz_store.cc:276] 0709 12:24:52.445005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.419656 (+1974651us) service_pool.cc:224] Handling call
0709 12:24:54.443338 (+ 23682us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.510055 (+ 66717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.510286 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.637391 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 549) took 1968 ms. Trace:
I20260709 12:24:54.637756 10093 rpcz_store.cc:276] 0709 12:24:52.668869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.636313 (+1967444us) service_pool.cc:224] Handling call
0709 12:24:54.636579 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.637132 (+   553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.637339 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.509621 13587 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.440495 13582 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:54.535580 13515 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.001s
I20260709 12:24:54.518900 13614 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:54.550366 10099 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:54.540236 13235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.032s	sys 0.029s
I20260709 12:24:54.234721 13499 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:54.571761 10016 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:24:54.542848 13166 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.528s	user 0.006s	sys 0.003s
I20260709 12:24:54.573943 13431 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:54.576561 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1699) took 1038 ms. Trace:
I20260709 12:24:54.581967 13593 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:54.585384 13444 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:54.588081 13493 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:54.589426  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.645874 13587 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:54.591521  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1572) took 1863 ms. Trace:
I20260709 12:24:54.540684 13542 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:54.594453  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:54.647917 13659 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.542011 13301 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:54.549036 13382 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:54.597903  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 417104us
I20260709 12:24:54.543705 13492 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:54.556480 13589 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.823s	user 0.045s	sys 0.069s
W20260709 12:24:54.554364 13632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.022s	sys 0.013s
I20260709 12:24:54.520004 13608 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.531347 13404 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.558983 13486 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.570994 13512 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:54.582321 13503 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.601270 13359 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.615051 13339 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.646s	user 0.009s	sys 0.000s
I20260709 12:24:54.615679 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:54.555824 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:54.625627 10092 rpcz_store.cc:276] 0709 12:24:52.567568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.538897 (+1971329us) service_pool.cc:224] Handling call
0709 12:24:54.551127 (+ 12230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.556623 (+  5496us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.559181 (+  2558us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.635550  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:54.737684 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 182034us
I20260709 12:24:54.738981 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:54.603363 13561 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:24:54.610170 13591 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:54.641976 13329 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:54.634213 13643 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.562s	user 0.003s	sys 0.007s
I20260709 12:24:54.658922 10097 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:54.658334 10018 rpcz_store.cc:276] 0709 12:24:53.537637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.922047 (+384410us) service_pool.cc:224] Handling call
0709 12:24:53.972755 (+ 50708us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:53.973093 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.973121 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:53.988342 (+ 15221us) log.cc:844] Serialized 52 byte log entry
0709 12:24:54.182718 (+194376us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:54.182756 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.182820 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.576186 (+393366us) raft_consensus.cc:1685] finished
0709 12:24:54.576214 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.576469 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1160,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":11214,"thread_start_us":164774,"threads_started":1,"wal-append.queue_time_us":536592}
I20260709 12:24:54.589728 13505 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:54.635459 13537 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:54.801571 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 551) took 1958 ms. Trace:
I20260709 12:24:54.802078 10017 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:54.665169 10093 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:54.696784  9943 rpcz_store.cc:276] 0709 12:24:52.727847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.716469 (+988622us) service_pool.cc:224] Handling call
0709 12:24:53.716790 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.200425 (+483635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.591080 (+390655us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b5400096354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:54.591418 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":950503168}
I20260709 12:24:54.633996 13287 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:24:54.801942 10097 rpcz_store.cc:276] 0709 12:24:52.843008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.643947 (+1800939us) service_pool.cc:224] Handling call
0709 12:24:54.658648 (+ 14701us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.801249 (+142601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.801503 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.661253 13593 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:54.700289 13529 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.560s	user 0.009s	sys 0.001s
I20260709 12:24:54.661419 13166 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.715370  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:54.663868 13309 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:54.722580  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:54.662932 13493 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:54.663487 13499 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:54.649257 13515 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.738045  9946 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
W20260709 12:24:54.610672 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:54.816745 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.818198 10092 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:24:54.819005 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 555) took 1416 ms. Trace:
I20260709 12:24:54.819425 10092 rpcz_store.cc:276] 0709 12:24:53.403178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.737631 (+1334453us) service_pool.cc:224] Handling call
0709 12:24:54.807673 (+ 70042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.818702 (+ 11029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.818935 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.821621 10672 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:54.743181 13492 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.824064  9946 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:54.805310 13431 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:54.741362 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 550) took 2003 ms (client timeout 2146 ms). Trace:
W20260709 12:24:54.828872 10099 rpcz_store.cc:269] 0709 12:24:52.738378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.687049 (+948671us) service_pool.cc:224] Handling call
0709 12:24:54.741075 (+1054026us) spinlock_profiling.cc:243] Waited 832 ms on lock 0x7b54002fc454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:54.741277 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1913963904}
W20260709 12:24:54.739677  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 324349us
I20260709 12:24:54.741407 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:54.743235  9940 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
W20260709 12:24:54.830467 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 89196us
I20260709 12:24:54.804811 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:54.717026  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.831176  9946 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:54.804971 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:54.831471  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.723488 13589 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:54.812049  9941 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:54.831339 10019 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:24:54.806036 13593 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:54.806559 10097 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:54.832573 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:54.834997 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2857) took 1103 ms. Trace:
I20260709 12:24:54.832507  9946 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:54.835423 10097 rpcz_store.cc:276] 0709 12:24:53.731890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.803717 (+1071827us) service_pool.cc:224] Handling call
0709 12:24:54.806254 (+  2537us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.834737 (+ 28483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.834940 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:54.831205 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:54.837167 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:54.838218 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:54.833179 10099 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: [7.4-7.4]
W20260709 12:24:54.838812  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 640549us
I20260709 12:24:54.807325 13505 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:54.743393 13359 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.843726 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2862) took 1020 ms. Trace:
W20260709 12:24:54.829418 13674 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
W20260709 12:24:54.840991  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.849203 10099 rpcz_store.cc:276] 0709 12:24:53.823534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.829952 (+1006418us) service_pool.cc:224] Handling call
0709 12:24:54.830370 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.843472 (+ 13102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.843674 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:54.858767  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.852571 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:54.860581 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:54.801309 13673 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.104s
I20260709 12:24:54.855615  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:54.822108 13580 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.857198 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:54.879509 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:54.880577 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:54.829717 13569 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:24:54.864488 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:54.811300 13529 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.865831 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2773) took 1319 ms. Trace:
I20260709 12:24:54.866159 13444 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:24:54.868584 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 545) took 2366 ms. Trace:
I20260709 12:24:54.865731 13359 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:54.843995 13614 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.883862 10674 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:54.904079 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:54.888447 10023 rpcz_store.cc:276] 0709 12:24:53.546403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.121272 (+574869us) service_pool.cc:224] Handling call
0709 12:24:54.121587 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.121955 (+   368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.121982 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.152963 (+ 30981us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.536699 (+383736us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:24:54.536753 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.536823 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.865605 (+328782us) raft_consensus.cc:1685] finished
0709 12:24:54.865624 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.865767 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":660,"raft.queue_time_us":90,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":16266,"thread_start_us":196760,"threads_started":1,"wal-append.queue_time_us":434907}
I20260709 12:24:54.850816 13339 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:24:54.847437 13166 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.896071 10096 rpcz_store.cc:276] 0709 12:24:52.502623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.425411 (+1922788us) service_pool.cc:224] Handling call
0709 12:24:54.446253 (+ 20842us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.446701 (+   448us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.446741 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.510518 (+ 63777us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.840722 (+330204us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:54.840762 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.840851 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.868318 (+ 27467us) raft_consensus.cc:1685] finished
0709 12:24:54.868343 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.868508 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7416,"thread_start_us":298341,"threads_started":1,"wal-append.queue_time_us":334680}
I20260709 12:24:54.870306 13520 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:54.908768 13670 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:24:54.909154 13520 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:54.911048 13663 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 12:24:54.911926 13520 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:54.840406 13662 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:54.839937 13477 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:54.917495 13526 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.919075 13582 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:54.920207 13503 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:54.921936 13529 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:54.883424 13492 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:24:54.983261 13680 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:24:54.983536 13492 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:54.983716 13680 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:54.983847 13508 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.984007 13492 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:54.890195 12938 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:54.986804 12938 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:54.987289 12938 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.965382 13679 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.988957 13590 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:54.989356 13590 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:54.928519 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:54.990132 13590 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.991942 13590 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:54.929282 13553 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:54.929014 13537 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.995754 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:54.856591 13431 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:54.998256 13431 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.998559 13587 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.000685 13587 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:54.975839 13635 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.087s	user 0.009s	sys 0.003s
I20260709 12:24:54.929883  9944 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:55.001847  9947 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:54.930413 13608 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:54.930483 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 860) took 1167 ms. Trace:
W20260709 12:24:54.931476 13669 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
I20260709 12:24:55.003854 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:55.004047 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:55.004487 10670 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:55.004604 10019 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:55.003149 10676 rpcz_store.cc:276] 0709 12:24:53.762948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.909462 (+146514us) service_pool.cc:224] Handling call
0709 12:24:53.910147 (+   685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.910495 (+   348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.910525 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.133717 (+223192us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b2c000d07e8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:24:54.134087 (+   370us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.642720 (+508633us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:54.642756 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.643779 (+  1023us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.930181 (+286402us) raft_consensus.cc:1685] finished
0709 12:24:54.930201 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.930421 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6789,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"spinlock_wait_cycles":25098112,"thread_start_us":439537,"threads_started":1,"wal-append.queue_time_us":642622}
I20260709 12:24:55.005851 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 869) took 1144 ms. Trace:
I20260709 12:24:55.005980 10019 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:24:54.931712 13404 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:54.932168 13667 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:24:54.853179 13591 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:24:54.939649 13329 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.940335 13515 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:54.940506 13499 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.849205 13672 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.100s
W20260709 12:24:54.944247 13363 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.992s	user 0.009s	sys 0.002s
I20260709 12:24:54.944675 13589 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:54.949535 13523 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.000s
I20260709 12:24:54.950400 13505 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:55.059437  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 203898us
W20260709 12:24:55.027477 13647 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.000s
I20260709 12:24:54.897547 13681 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.067s
I20260709 12:24:54.910195 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2781) took 1091 ms. Trace:
I20260709 12:24:55.075495 10022 rpcz_store.cc:276] 0709 12:24:53.819030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.131269 (+312239us) service_pool.cc:224] Handling call
0709 12:24:54.175875 (+ 44606us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.182370 (+  6495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.182400 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.182855 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.651389 (+468534us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:54.651432 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.651499 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.909961 (+258462us) raft_consensus.cc:1685] finished
0709 12:24:54.909985 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.910131 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6129,"raft.queue_time_us":7099,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":468130,"threads_started":1,"wal-append.queue_time_us":632570}
I20260709 12:24:54.962450 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1752) took 1749 ms. Trace:
I20260709 12:24:55.077813 10093 rpcz_store.cc:276] 0709 12:24:53.213589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.639275 (+1425686us) service_pool.cc:224] Handling call
0709 12:24:54.639656 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.802615 (+162959us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.962156 (+159541us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b54002fc954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:54.962377 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52048256}
I20260709 12:24:54.963091 13287 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.890895 13493 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:54.835002 13301 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:54.963944 13522 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.840605 13571 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:54.908785 10096 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:24:54.969895 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 546) took 2457 ms. Trace:
W20260709 12:24:54.975687  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 253164us
W20260709 12:24:54.978008 10091 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1755) took 1743 ms (client timeout 2220 ms). Trace:
I20260709 12:24:55.083734 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 562) took 1020 ms. Trace:
I20260709 12:24:54.982826 13503 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:54.997169 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:55.085963 10024 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.005309 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:55.087419 10023 raft_consensus.cc:2376] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8. Re-sending same reply.
I20260709 12:24:55.006232 10671 rpcz_store.cc:276] 0709 12:24:53.861886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.920057 (+ 58171us) service_pool.cc:224] Handling call
0709 12:24:53.920368 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:53.944211 (+ 23843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:53.944239 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:53.944734 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.538299 (+593565us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:54.538353 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.538445 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.005658 (+467213us) raft_consensus.cc:1685] finished
0709 12:24:55.005678 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.005795 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1096,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":22134,"thread_start_us":593178,"threads_started":1,"wal-append.queue_time_us":717363}
I20260709 12:24:55.008167 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 871) took 1028 ms. Trace:
I20260709 12:24:55.088250 10670 rpcz_store.cc:276] 0709 12:24:53.977079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.017694 (+ 40615us) service_pool.cc:224] Handling call
0709 12:24:55.005362 (+987668us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54004c9ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.005460 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2234799872}
I20260709 12:24:55.085426 10095 rpcz_store.cc:276] 0709 12:24:52.512767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.439674 (+1926907us) service_pool.cc:224] Handling call
0709 12:24:54.451507 (+ 11833us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.517361 (+ 65854us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.517439 (+    78us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:54.531772 (+ 14333us) log.cc:844] Serialized 26 byte log entry
0709 12:24:54.817564 (+285792us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:54.817604 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:54.817682 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:54.969574 (+151892us) raft_consensus.cc:1685] finished
0709 12:24:54.969604 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:54.969817 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":508,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":55971,"thread_start_us":259999,"threads_started":1,"wal-append.queue_time_us":415435}
I20260709 12:24:55.015506 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:55.016283 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2791) took 1078 ms. Trace:
I20260709 12:24:55.017354 10640 leader_election.cc:400] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6.
W20260709 12:24:55.091952 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 76571.7us
I20260709 12:24:55.022792 10025 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:24:55.025543  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:55.060868  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:55.062803 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.064735 13507 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:55.066262 13561 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:55.019248 13404 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:24:55.067153 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 872) took 1089 ms. Trace:
I20260709 12:24:55.067806 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1711) took 1523 ms. Trace:
I20260709 12:24:55.067971 13614 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:55.100217 10016 rpcz_store.cc:276] 0709 12:24:53.544449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.884767 (+340318us) service_pool.cc:224] Handling call
0709 12:24:55.067620 (+1182853us) spinlock_profiling.cc:243] Waited 686 ms on lock 0x7b54001f7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.067748 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1576816512}
I20260709 12:24:55.068632 13339 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:55.069211 10675 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:55.074896  9945 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:24:54.952615 13605 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.001s
W20260709 12:24:55.029621 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 135762us
I20260709 12:24:55.029282 13635 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.029100 13608 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:55.110863  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1801) took 1007 ms. Trace:
I20260709 12:24:55.111343  9944 rpcz_store.cc:276] 0709 12:24:54.103301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.589138 (+485837us) service_pool.cc:224] Handling call
0709 12:24:55.110648 (+521510us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b5400015ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.110806 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":612706560}
I20260709 12:24:55.112584 10024 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:24:55.113127 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:55.084667  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:55.084121 10096 rpcz_store.cc:276] 0709 12:24:54.063328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.907970 (+844642us) service_pool.cc:224] Handling call
0709 12:24:54.908373 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.083456 (+175083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.083676 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.115087  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:55.116194  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:55.116367 10024 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: [6.9-6.9]
W20260709 12:24:55.083875 10091 rpcz_store.cc:269] 0709 12:24:53.235019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.860230 (+625211us) service_pool.cc:224] Handling call
0709 12:24:54.977724 (+1117494us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400162454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:54.977933 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749786368}
I20260709 12:24:55.092149 10640 leader_election.cc:403] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:55.117419 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 106908us
I20260709 12:24:55.118949 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:24:54.967470 13498 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.710s	user 0.006s	sys 0.001s
I20260709 12:24:55.120518  9943 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:55.091619 10019 rpcz_store.cc:276] 0709 12:24:53.937648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.005374 (+1067726us) service_pool.cc:224] Handling call
0709 12:24:55.005681 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.006445 (+   764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.016210 (+  9765us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.099085 10672 rpcz_store.cc:276] 0709 12:24:53.977319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.080588 (+103269us) service_pool.cc:224] Handling call
0709 12:24:55.066951 (+986363us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b54007d7b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.067089 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1700837248}
W20260709 12:24:55.104611  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79336us
I20260709 12:24:55.123531  9991 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:55.105389  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44593.9us
I20260709 12:24:55.108996 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.124426 10095 raft_consensus.cc:2368] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:24:55.122478 10016 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:55.088657 13687 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.062114 13523 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.094971 13561 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:55.079667 13512 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.094798 13507 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:55.152812 10023 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:24:55.110970 13022 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:55.132973  9948 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:55.156029 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.134562  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:55.090129 13492 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.102979 13339 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:55.094532 13503 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.119192 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.178035 10093 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:55.110888 13493 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.085542 13287 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.168382 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 564) took 1095 ms. Trace:
I20260709 12:24:55.194778 10098 rpcz_store.cc:276] 0709 12:24:54.072979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.144100 (+ 71121us) service_pool.cc:224] Handling call
0709 12:24:55.168223 (+1024123us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54001928d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.168326 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52369280}
I20260709 12:24:55.195981 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.169055  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:55.168413 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:55.169080  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43707.7us
I20260709 12:24:55.197382 10091 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.135191 13580 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:55.154772 13306 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.003s
I20260709 12:24:55.135856 13404 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:55.089771 13522 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.119035 13180 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6.
I20260709 12:24:55.135460 13635 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.108356 13591 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:55.088092 13647 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.151371 13582 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.150394 13329 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:55.104281 13553 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:24:55.150058 13515 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:55.178954 10094 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:55.192152 10668 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:55.198764  9941 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:55.198969  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:55.243476  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44583us
I20260709 12:24:55.200203 13580 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.109853 13605 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.204627 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.169518 13022 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.172423 13673 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.265909  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:55.169436 13431 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.212921 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.155459 13499 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:24:55.152245 13498 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.120280 13608 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:55.229574 13166 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.150696 13486 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.086440 13680 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.238049 10018 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:55.149848 12938 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.164179 13382 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.203526 13444 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:24:55.244855 10677 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:55.199599 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.168506 13593 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.175951 13505 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:24:55.231717 13614 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.277199 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:55.277819 10671 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:24:55.201522 13688 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:24:55.278435 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:55.359041 10098 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:55.278775  9949 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:55.266968  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:55.163115 13687 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:55.288398 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:55.290185  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.362502  9990 leader_election.cc:400] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9.
I20260709 12:24:55.363689  9990 leader_election.cc:400] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:24:55.245414 13515 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:55.248281 13580 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.265650 13022 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.280382 13673 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.226665 13306 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.244684 13404 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:24:55.284811 13526 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.231473 13561 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.279110 13689 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:55.334340 10091 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:55.340796 10096 raft_consensus.cc:2368] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9.
I20260709 12:24:55.328241 10020 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:55.335500 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:55.344147 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 251419us
I20260709 12:24:55.345263 13659 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:55.346012 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1712) took 1801 ms. Trace:
W20260709 12:24:55.346701 13696 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:24:55.346647 13512 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:55.347317 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:55.328164 13706 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.348053 10024 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:55.349148 13522 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:55.349998 13593 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:55.350569 13382 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:55.200681 13571 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.350808 13641 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:55.352579 13523 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.352850 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2795) took 1414 ms. Trace:
I20260709 12:24:55.353276 13647 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:55.355051 12938 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.279443 13431 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:55.339185  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 73370us
I20260709 12:24:55.452404  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:55.352771 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.454955 10095 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:55.363999  9990 leader_election.cc:403] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:24:55.380230 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 267176us
W20260709 12:24:55.456447  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 92924.8us
I20260709 12:24:55.456943 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:55.457068  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:55.377135 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:55.389647  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 183164us
I20260709 12:24:55.346413 13635 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.400445 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:55.401618 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:55.339073 13679 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:24:55.388027 13583 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.855s	user 0.014s	sys 0.022s
I20260709 12:24:55.394747 13662 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:55.490361 10642 leader_election.cc:336] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.547s (SENT)
I20260709 12:24:55.388392 13642 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:55.425592 10092 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:55.352802 13587 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:24:55.402822 13673 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.423810 10021 rpcz_store.cc:276] 0709 12:24:53.544808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.097540 (+552732us) service_pool.cc:224] Handling call
0709 12:24:54.097836 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.104969 (+  7133us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:24:54.448583 (+343614us) log.cc:844] Serialized 14 byte log entry
0709 12:24:54.635881 (+187298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:54.635936 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:24:54.635990 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.345791 (+709801us) spinlock_profiling.cc:243] Waited 6.91 ms on lock 0x7b54001f6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:55.345835 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.345948 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":544138,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":75,"spinlock_wait_cycles":21648512,"thread_start_us":530232,"threads_started":2,"wal-append.queue_time_us":380627}
I20260709 12:24:55.430934 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:55.431663 10673 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.340662 13166 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:55.432092  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 256892us
I20260709 12:24:55.503400  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:55.340818 13537 consensus_queue.cc:1175] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" responder_term: 10 status { last_received { term: 3 index: 3 } last_committed_idx: 3 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:55.441417 10672 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:24:55.447711 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:55.444844 10022 rpcz_store.cc:276] 0709 12:24:53.938536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.094557 (+1156021us) service_pool.cc:224] Handling call
0709 12:24:55.107289 (+ 12732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.107636 (+   347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.107663 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.121929 (+ 14266us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.243449 (+121520us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:55.243487 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.243548 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.352631 (+109083us) raft_consensus.cc:1685] finished
0709 12:24:55.352651 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.352795 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":546,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":22144,"thread_start_us":51288,"threads_started":1,"wal-append.queue_time_us":185460}
I20260709 12:24:55.429358 13659 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.428200 13689 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:55.459348 10671 raft_consensus.cc:2410] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:55.465294  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:55.471184 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2772) took 1925 ms. Trace:
W20260709 12:24:55.471565 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.473155 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:55.492516 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:55.474682 13614 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.495606 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.496847  9941 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:55.444523 13641 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:55.497596 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:55.527884 10095 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:55.459316 13431 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.500746 10091 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:55.454020 13492 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:55.474342 13583 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.014s	sys 0.023s
I20260709 12:24:55.504501 10096 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.459657 13505 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.474886 13522 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.445283 13710 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:24:55.513486  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 683) took 1526 ms. Trace:
I20260709 12:24:55.544987  9948 rpcz_store.cc:276] 0709 12:24:53.987479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.719008 (+731529us) service_pool.cc:224] Handling call
0709 12:24:54.719414 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.166492 (+447078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.513212 (+346720us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b540042e854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:55.513424 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":950668544}
I20260709 12:24:55.493412 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:55.515221  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49992.8us
I20260709 12:24:55.546234  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:55.547686  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:55.549396  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:55.511665 10017 rpcz_store.cc:276] 0709 12:24:53.546083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.107142 (+561059us) service_pool.cc:224] Handling call
0709 12:24:54.107443 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.802512 (+695069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.470964 (+668452us) spinlock_profiling.cc:243] Waited 694 ms on lock 0x7b54002161d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:55.471129 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597108480}
I20260709 12:24:55.470227 13287 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.523847 10098 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 7 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260709 12:24:55.474437 13707 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:55.532833 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59810.3us
I20260709 12:24:55.553277 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:55.555867 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.522141 13647 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.474795 13523 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.047s
I20260709 12:24:55.536129 13301 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.445120 13706 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.577293 13706 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.537042 13505 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.577983 13505 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.578338 13508 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:55.573709 13180 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:24:55.541791 10673 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:55.452822 13515 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.533720 10022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:55.527299 13486 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:55.585090 10022 raft_consensus.cc:606] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:55.528101 13498 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:55.568096 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.522415 13359 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:55.568897  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:55.586222  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.588073 10096 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:55.571002 10091 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:24:55.499706 13642 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.585489 10022 raft_consensus.cc:614] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:24:55.560911 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.591390  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:55.595659 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:55.596259 10098 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.570762 10095 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:24:55.600396 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:55.551821 13553 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:24:55.610939 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 560) took 1561 ms. Trace:
I20260709 12:24:55.613265 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:55.573791 13287 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:55.600163 13647 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:24:55.615752 10099 rpcz_store.cc:276] 0709 12:24:54.049215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.859574 (+810359us) service_pool.cc:224] Handling call
0709 12:24:54.859904 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.863294 (+  3390us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:24:55.021864 (+158570us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.359920 (+338056us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.360154 (+   234us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.360587 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.378450 (+ 17863us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:55.378500 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.378614 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.610719 (+232105us) raft_consensus.cc:1685] finished
0709 12:24:55.610743 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.610880 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48100,"prepare.queue_time_us":240742,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":1419,"raft.queue_time_us":110633,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":39085,"spinlock_wait_cycles":10400512,"thread_start_us":434660,"threads_started":3,"wal-append.queue_time_us":221789}
I20260709 12:24:55.594561 13681 raft_consensus.cc:2764] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:24:55.628155 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.594858 13359 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.625512  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:55.601243  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:55.610316 13707 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.640218 13507 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.583611 13512 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:24:55.623950 13589 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:55.645046 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2774) took 2057 ms. Trace:
I20260709 12:24:55.669972 10025 rpcz_store.cc:276] 0709 12:24:53.587287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.122606 (+535319us) service_pool.cc:224] Handling call
0709 12:24:54.122888 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.096925 (+974037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.644770 (+547845us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b54002170d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:55.644973 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2066135936}
W20260709 12:24:55.670755 13711 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:24:55.611893 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:55.631978 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63950.9us
W20260709 12:24:55.648094  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46978.3us
I20260709 12:24:55.673095  9990 leader_election.cc:400] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9.
I20260709 12:24:55.674072  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:55.669296 13553 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:55.612392 13614 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.689352 13614 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.691895 13614 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.649917 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.663976 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:55.692806 10096 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:55.693468 10099 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:55.693668 10096 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:55.694067 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2866) took 1575 ms. Trace:
I20260709 12:24:55.694371 10099 rpcz_store.cc:276] 0709 12:24:54.118416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.693043 (+1574627us) service_pool.cc:224] Handling call
0709 12:24:55.693319 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.693860 (+   541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.694019 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:55.695770 10099 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:55.697053 10096 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:55.631937 13535 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:24:55.668483 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.719679 10095 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:55.721024 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:55.637079 13339 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.675566 13635 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:55.651963 13359 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:55.679595 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:55.728302 10025 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:55.728768 10025 raft_consensus.cc:614] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:24:55.718112 13553 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:55.731695 13553 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.672438 13306 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.735407 13722 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:55.735862 13722 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:55.736647 13722 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.737279 13022 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.738045 13722 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.738631  9948 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:55.739171 13708 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:24:55.739952 13635 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:55.680612 13166 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:55.686487 13642 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:55.665022 13701 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:24:55.753369 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2863) took 1918 ms. Trace:
I20260709 12:24:55.720432 13614 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.754086 13614 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.753770 10097 rpcz_store.cc:276] 0709 12:24:53.835407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.836558 (+1001151us) service_pool.cc:224] Handling call
0709 12:24:54.860712 (+ 24154us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.078495 (+217783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.078528 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.084838 (+  6310us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.175093 (+ 90255us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:55.175222 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.175410 (+   188us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.753145 (+577735us) raft_consensus.cc:1685] finished
0709 12:24:55.753169 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.753311 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2362,"raft.queue_time_us":8005,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":230928,"threads_started":2,"wal-append.queue_time_us":242027}
I20260709 12:24:55.756563 13614 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.756958 13339 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.758571 13431 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.759164 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 766) took 1803 ms. Trace:
I20260709 12:24:55.698720  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:55.759831 10093 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:55.760462  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61871.1us
I20260709 12:24:55.761564 13431 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:55.761860  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:55.762403 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 554) took 1160 ms. Trace:
I20260709 12:24:55.762523 13723 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:55.721195 10095 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:55.763883 13662 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:55.638388 13505 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.764405 13662 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:55.742889  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:55.765795  9944 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:55.759593 10675 rpcz_store.cc:276] 0709 12:24:53.956092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.090381 (+134289us) service_pool.cc:224] Handling call
0709 12:24:55.069018 (+978637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.103284 (+ 34266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.758949 (+655665us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54007d7b50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.759097 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239286016}
I20260709 12:24:55.726325 13477 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:55.741164 13635 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:55.772957  9946 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:55.773311  9946 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:55.750983 13522 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
W20260709 12:24:55.751385  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 686) took 1749 ms (client timeout 2146 ms). Trace:
W20260709 12:24:55.776017  9940 rpcz_store.cc:269] 0709 12:24:54.002314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.096023 (+ 93709us) service_pool.cc:224] Handling call
0709 12:24:55.751200 (+1655177us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b540003f554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.751323 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486290176}
I20260709 12:24:55.776638 13507 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:55.777350  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:55.777946 10094 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 556) took 1770 ms (client timeout 1959 ms). Trace:
I20260709 12:24:55.762687 13707 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:55.723449 13367 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:55.774000  9946 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.767684  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:55.769856  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:55.770725 13404 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:24:55.771404 13505 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:55.772044 13719 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:24:55.770767 13477 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.762784 10019 rpcz_store.cc:276] 0709 12:24:54.602073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.163729 (+561656us) service_pool.cc:224] Handling call
0709 12:24:55.189315 (+ 25586us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.399059 (+209744us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.399104 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.411870 (+ 12766us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.526391 (+114521us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:55.526446 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.526544 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:55.761058 (+234514us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:55.762099 (+  1041us) raft_consensus.cc:1685] finished
0709 12:24:55.762121 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:55.762347 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":51463,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":5700992,"thread_start_us":279834,"threads_started":2,"wal-append.queue_time_us":224060}
I20260709 12:24:55.778401  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:55.779120  9947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1580) took 1848 ms (client timeout 1565 ms). Trace:
I20260709 12:24:55.706879 10098 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
W20260709 12:24:55.787532 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 202044us
W20260709 12:24:55.778308 10094 rpcz_store.cc:269] 0709 12:24:54.025744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.149612 (+123868us) service_pool.cc:224] Handling call
0709 12:24:55.777773 (+1628161us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b5400305a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.777892 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1934995840}
I20260709 12:24:55.748588 13535 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:55.614671 13689 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.815193  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:55.815994  9940 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:55.776065 13493 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.574s	user 0.008s	sys 0.013s
W20260709 12:24:55.809242  9947 rpcz_store.cc:269] 0709 12:24:53.930416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.930520 (+   104us) service_pool.cc:224] Handling call
0709 12:24:55.778857 (+1848337us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540009bad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:55.779038 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2338026112}
I20260709 12:24:55.780304  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:55.822111 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 141080us
I20260709 12:24:55.786660 13367 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:55.827096 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:55.827361  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:55.830668  9944 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:55.799237 13526 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.790334 13287 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:55.837661 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:55.836804 13526 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:55.839339 13526 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.838501 13642 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:55.841886 13642 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:55.842497 13642 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:55.835597 13367 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:55.838773  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:55.847522  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:55.848039 13477 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:55.809780 13404 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:55.813701 13431 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.802855 13706 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:24:55.857712 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:24:55.967315  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:55.787281 13659 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.834873 13508 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:24:55.880801 13505 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.017423 13505 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.993852  9940 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:24:55.880471 13180 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.001591 13687 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.002821 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 778) took 1377 ms. Trace:
I20260709 12:24:56.039443 10677 rpcz_store.cc:276] 0709 12:24:54.625130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.625248 (+   118us) service_pool.cc:224] Handling call
0709 12:24:54.625561 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.340228 (+714667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.002551 (+662323us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b5400545bd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:56.002760 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423864064}
I20260709 12:24:56.006608 13507 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:55.857944 13431 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:55.978015 13706 consensus_queue.cc:1243] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:24:56.014190 13512 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:24:56.015074 13580 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.613s	user 0.008s	sys 0.001s
I20260709 12:24:56.015501 13404 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.027168  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:55.887593 13728 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.087247  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.087926  9940 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:56.039072 13553 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:55.980473 13679 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.050558 13512 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:55.979202 13306 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:56.052870 13704 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:56.055073 13727 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:24:56.054814 13608 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.020815 13022 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.072404 13659 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:56.073159 13699 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.831s	user 0.004s	sys 0.000s
I20260709 12:24:56.047613 13687 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.034963 13707 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:55.981345 13535 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.074563 13507 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:56.080276 13359 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.040512 13593 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:24:56.082998 10019 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:24:56.129370 10019 raft_consensus.cc:606] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:24:56.088865  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.131286  9940 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:24:56.132416  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.011729 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2860) took 2278 ms. Trace:
I20260709 12:24:56.085676 13522 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.134755 10092 rpcz_store.cc:276] 0709 12:24:53.733172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.820405 (+1087233us) service_pool.cc:224] Handling call
0709 12:24:55.169044 (+348639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.497383 (+328339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.010876 (+513493us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54001928d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:56.011540 (+   664us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b54001928d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:56.011666 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1348078592}
I20260709 12:24:56.135057 13522 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.137082 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:56.102324 13673 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.000s
I20260709 12:24:56.134755  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:24:56.092978 13553 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.086912 13605 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.056255 13526 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.084154 13580 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.151373 13526 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.103993 13306 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.126761 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.128113 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.115770 13608 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.133729  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.117504 13687 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.119656 13707 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.099282 13679 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.154950 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.175132 10017 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:56.171027 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 553) took 1601 ms. Trace:
I20260709 12:24:56.169727 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 780) took 1204 ms. Trace:
I20260709 12:24:56.187934 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:24:56.182264  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 158117us
I20260709 12:24:56.200405 10019 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:56.196496 10016 rpcz_store.cc:276] 0709 12:24:54.570217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.133570 (+563353us) service_pool.cc:224] Handling call
0709 12:24:55.152002 (+ 18432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.158797 (+  6795us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:55.162194 (+  3397us) log.cc:844] Serialized 14 byte log entry
0709 12:24:55.354877 (+192683us) spinlock_profiling.cc:243] Waited 9.75 ms on lock 0x7b1800390a60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:24:55.356420 (+  1543us) spinlock_profiling.cc:243] Waited 7 ms on lock 0x7b1800390a60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:24:56.051014 (+694594us) log.cc:844] Serialized 14 byte log entry
0709 12:24:56.051325 (+   311us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:56.051854 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:24:56.052008 (+   154us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:24:56.052247 (+   239us) log.cc:844] Serialized 14 byte log entry
0709 12:24:56.052418 (+   171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.052617 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.170812 (+118195us) raft_consensus.cc:1685] finished
0709 12:24:56.170833 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.170966 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2280,"prepare.queue_time_us":395180,"prepare.run_cpu_time_us":118,"prepare.run_wall_time_us":4147,"raft.queue_time_us":29357,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":26872,"spinlock_wait_cycles":38526208,"thread_start_us":687881,"threads_started":2,"wal-append.queue_time_us":928411}
I20260709 12:24:56.200795 10019 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:56.202560 10090 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2850) took 2922 ms (client timeout 1813 ms). Trace:
W20260709 12:24:56.203009 10090 rpcz_store.cc:269] 0709 12:24:53.280411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.739862 (+459451us) service_pool.cc:224] Handling call
0709 12:24:56.202390 (+2462528us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:56.202498 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4104488320}
I20260709 12:24:56.203197 10669 rpcz_store.cc:276] 0709 12:24:54.965450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.978395 (+ 12945us) service_pool.cc:224] Handling call
0709 12:24:54.979695 (+  1300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:54.981243 (+  1548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:54.981345 (+   102us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.018476 (+ 37131us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.165037 (+146561us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:55.167677 (+  2640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.169739 (+  2062us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.072059 (+902320us) spinlock_profiling.cc:243] Waited 4.09 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:56.163772 (+ 91713us) raft_consensus.cc:1685] finished
0709 12:24:56.163805 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.169648 (+  5843us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":122,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":2778,"spinlock_wait_cycles":9412352,"thread_start_us":86557,"threads_started":1,"wal-append.queue_time_us":213287}
I20260709 12:24:56.206238 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:56.207157 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:56.209378 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.202103 10019 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.211548 10019 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.205327 13605 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.215569 13605 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.217763 13605 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.217962 13306 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.221032 13367 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.222949 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.223721 10020 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:56.224542 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:56.225962 10020 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:56.138478 13673 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.226943 13662 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.229269 13662 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.229835 13704 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:56.159214 13522 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.143496 13523 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.138844 13512 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.225984 13642 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.232059 13512 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.233127 13642 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.234303 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:56.235888 13707 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.237123 13553 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.151573 13580 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.239611  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.240995 10017 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:56.289173  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.248040  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 695) took 1079 ms. Trace:
I20260709 12:24:56.248558 10025 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:56.239013 10022 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:56.174836 13431 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.151373 13359 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.289247 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.243880 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 857) took 2516 ms. Trace:
I20260709 12:24:56.290089  9944 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.334038 10020 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:24:56.442093  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.398658  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 199550us
W20260709 12:24:56.457676  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 217017us
I20260709 12:24:56.304491 10019 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
W20260709 12:24:56.349778  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.567430 10019 pending_rounds.cc:85] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 7
I20260709 12:24:56.567765 10019 pending_rounds.cc:107] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:24:56.568038 10019 raft_consensus.cc:2889] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:56.355734 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:56.561920 10674 rpcz_store.cc:276] 0709 12:24:53.727186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:53.798250 (+ 71064us) service_pool.cc:224] Handling call
0709 12:24:53.798537 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:54.904134 (+1105597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.243648 (+1339514us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400610dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:56.243816 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2495568256}
W20260709 12:24:56.570045  9945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1802) took 2466 ms (client timeout 2137 ms). Trace:
I20260709 12:24:56.570078 10019 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:24:56.570319 10016 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:56.573385 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.290110  9949 rpcz_store.cc:276] 0709 12:24:55.168697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.175893 (+  7196us) service_pool.cc:224] Handling call
0709 12:24:55.176182 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.359549 (+183367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.247802 (+888253us) spinlock_profiling.cc:243] Waited 71.7 ms on lock 0x7b540021dc54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:56.247979 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":164906240}
I20260709 12:24:56.575205  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.575795  9949 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:56.571767 10017 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:24:56.570757  9945 rpcz_store.cc:269] 0709 12:24:54.103536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.446044 (+342508us) service_pool.cc:224] Handling call
0709 12:24:56.569863 (+2123819us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b5400016dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:56.569981 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1414211968}
I20260709 12:24:56.554023 13306 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.554481 13710 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.591316 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.495007 13662 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.576607  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.600385  9949 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:56.582970  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:56.535482 13673 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.601884 10020 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:56.593979  9944 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:56.579504  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.604784  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.605422  9945 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:56.605844  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1819) took 1086 ms. Trace:
W20260709 12:24:56.606329 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.607831  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.608495  9944 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 5.
I20260709 12:24:56.608978  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1814) took 1091 ms. Trace:
I20260709 12:24:56.609639 13522 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:56.610177 13522 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.611553 13512 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.612072 13512 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.609287  9944 rpcz_store.cc:276] 0709 12:24:55.517622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.607583 (+1089961us) service_pool.cc:224] Handling call
0709 12:24:56.608923 (+  1340us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.614084 13512 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.614362  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.615120  9944 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:56.616281  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.617087  9944 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:24:56.606137  9945 rpcz_store.cc:276] 0709 12:24:55.519150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.604534 (+1085384us) service_pool.cc:224] Handling call
0709 12:24:56.605795 (+  1261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.622413  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.623843  9945 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:56.594285 10016 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:56.583376 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:56.575903 13752 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.577003 13526 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.588660 10017 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:24:56.625957 10019 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:56.658028  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.660902  9945 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:56.661304 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:56.578045 13553 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.662190  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.662683 13477 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.663062 13752 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.666122 13583 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.627686  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.680084 13632 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:56.680235  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 688) took 2654 ms. Trace:
I20260709 12:24:56.681509 13166 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:56.682013 13512 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:56.682199 13728 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.583s	user 0.007s	sys 0.004s
I20260709 12:24:56.586697 13679 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.685258  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.617741 13535 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:56.694901 13715 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.023s	user 0.002s	sys 0.009s
I20260709 12:24:56.695125 13535 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.695712 13535 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:56.696717 13698 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.457s	user 0.003s	sys 0.000s
I20260709 12:24:56.697992 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 556) took 2065 ms. Trace:
I20260709 12:24:56.699450  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.700155  9949 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:56.701872 13339 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:56.687197  9943 rpcz_store.cc:276] 0709 12:24:54.026057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.802945 (+776888us) service_pool.cc:224] Handling call
0709 12:24:55.112591 (+309646us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.120896 (+  8305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.680033 (+1559137us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b5400015ed0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:56.680173 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711416192}
I20260709 12:24:56.702376 13339 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:56.703866  9943 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:24:56.703433 13339 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.704689  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1586) took 1276 ms. Trace:
I20260709 12:24:56.705163  9943 rpcz_store.cc:276] 0709 12:24:55.428771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.703058 (+1274287us) service_pool.cc:224] Handling call
0709 12:24:56.703442 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.704380 (+   938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.704621 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.583083 13431 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.080s
I20260709 12:24:56.690960  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.709438  9944 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
W20260709 12:24:56.710067  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.710306 13713 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.710493 13680 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:56.710734  9944 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:56.710829 13713 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.711400  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1590) took 1276 ms. Trace:
I20260709 12:24:56.711750  9944 rpcz_store.cc:276] 0709 12:24:55.434742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.710149 (+1275407us) service_pool.cc:224] Handling call
0709 12:24:56.710492 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:56.711170 (+   678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.711352 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:24:56.712105 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1759) took 1698 ms (client timeout 1670 ms). Trace:
W20260709 12:24:56.712484 10093 rpcz_store.cc:269] 0709 12:24:55.013416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.178737 (+165321us) service_pool.cc:224] Handling call
0709 12:24:56.711928 (+1533191us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b5400164c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:56.712041 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133423488}
I20260709 12:24:56.712908  9944 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:56.712970 13713 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.627199 13367 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.688318 13477 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.692698 13512 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.704738 13339 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.714159  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1582) took 2783 ms. Trace:
I20260709 12:24:56.725289 13512 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.725569  9942 rpcz_store.cc:276] 0709 12:24:53.931025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.663927 (+732902us) service_pool.cc:224] Handling call
0709 12:24:56.713997 (+2050070us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b5400207e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:56.714105 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":718366592}
I20260709 12:24:56.684777 13022 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:56.726130  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:56.713637  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1817) took 1195 ms. Trace:
I20260709 12:24:56.717603 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:56.698354 10023 rpcz_store.cc:276] 0709 12:24:54.632951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.175942 (+542991us) service_pool.cc:224] Handling call
0709 12:24:55.204947 (+ 29005us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:55.281051 (+ 76104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:55.281084 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:55.290847 (+  9763us) log.cc:844] Serialized 26 byte log entry
0709 12:24:55.506339 (+215492us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:55.506393 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:55.506485 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:56.694276 (+1187791us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:24:56.697674 (+  3398us) raft_consensus.cc:1685] finished
0709 12:24:56.697695 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.697930 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":36112,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"spinlock_wait_cycles":24550784,"thread_start_us":148322,"threads_started":2,"wal-append.queue_time_us":224597}
I20260709 12:24:56.684471 13642 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:56.716249 13728 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.716812 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:56.721503 10092 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:56.721748 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:56.747774 10091 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:56.721974 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:56.716223 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:56.749717 10096 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:56.688555 13632 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:56.600442 13306 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.624296 13687 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.728112 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:56.742671  9944 rpcz_store.cc:276] 0709 12:24:55.518512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.712498 (+1193986us) service_pool.cc:224] Handling call
0709 12:24:56.712763 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.713387 (+   624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.713581 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.785315  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.617350 13662 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.723721 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:56.788770 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60785.4us
I20260709 12:24:56.687628 13553 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.790345 13493 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.790829 10095 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:56.788082 13713 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.791625 13713 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.792070 13553 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.793018 13679 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.793187 13662 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.726711 13022 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.743355  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.722157 13505 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.745798  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:56.723368 10099 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:56.750801 10094 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:56.602877 13710 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.683995 13526 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.781073 10097 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:56.782404  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66381.8us
I20260709 12:24:56.782763  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1589) took 1348 ms. Trace:
I20260709 12:24:56.786532  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:56.795130 10090 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:56.786957  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:24:56.787564  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.793534 13679 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.793888 13306 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:56.777108  9941 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:24:56.726079 13512 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.746824 13704 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:56.728928 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:56.815429  9943 rpcz_store.cc:276] 0709 12:24:55.434460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.706099 (+1271639us) service_pool.cc:224] Handling call
0709 12:24:56.706433 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.782380 (+ 75947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.782409 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:56.782457 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:56.782494 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.782560 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:56.782701 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66509,"raft.queue_time_us":68037,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":8053}
I20260709 12:24:56.713944 13166 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.826786 13498 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:56.821902 13512 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.827733  9943 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:24:56.723111 13382 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:24:56.818687  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:56.750500 13367 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:56.819687  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 74032.6us
I20260709 12:24:56.783708 13678 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.729159 13583 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.085s
I20260709 12:24:56.790421 13477 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.821995 13704 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.840174  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:56.847431  9944 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:24:56.829434 13507 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:56.750873 13642 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.827926 13553 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:56.837343  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:24:56.848865 13367 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:56.865974 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:56.866864 10674 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
W20260709 12:24:56.863008 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 254458us
I20260709 12:24:56.863509  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:56.869128  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:56.843312 13529 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:56.863786 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:56.862056 13553 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.858614 13477 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:56.848357 13678 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.875547 13707 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:56.875607 13673 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:24:56.876169 13707 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:56.876545 13707 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:56.876670 13673 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:56.876987 13678 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.877048 13673 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:56.872959 10670 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:56.842905 13505 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 12:24:56.853556 13520 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:24:56.878149  9941 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:56.879809 10674 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:56.868789 13507 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:24:56.881722  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:56.884759  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.867436 13713 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.891605 13507 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:56.892107 13507 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:56.892237  9944 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:56.879777 13553 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:56.893927 13287 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:24:56.836968 13632 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.895358 13632 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:56.896694 13166 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.892099 13713 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:56.898084 13713 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:56.898924 13713 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.900105 13713 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.901193 13642 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.902951 13526 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.904192 13512 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.905304 13752 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.887575 13679 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:56.881176 13477 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:56.909416  9949 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:56.911727  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1587) took 1477 ms. Trace:
W20260709 12:24:56.936267  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 97541.8us
I20260709 12:24:56.943326 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:56.943408  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:56.873785 13359 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:56.944051 10022 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:56.946347  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:24:56.947939  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.949846  9944 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:56.950691 13309 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:56.951167 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 777) took 2736 ms. Trace:
I20260709 12:24:56.951661 13710 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:56.951385 13553 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.951563 10668 rpcz_store.cc:276] 0709 12:24:54.214297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.413336 (+199039us) service_pool.cc:224] Handling call
0709 12:24:54.413785 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.242773 (+828988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.950938 (+1708165us) spinlock_profiling.cc:243] Waited 778 ms on lock 0x7b54007f4754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:56.951104 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1789421440}
I20260709 12:24:56.867827 13498 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:24:56.905984 13367 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:56.956796 13709 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.451s	user 0.009s	sys 0.003s
I20260709 12:24:56.868520 13537 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:24:56.957988 13707 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:56.958765 10018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 555) took 2350 ms (client timeout 2067 ms). Trace:
W20260709 12:24:56.959276 10018 rpcz_store.cc:269] 0709 12:24:54.608795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:54.928228 (+319433us) service_pool.cc:224] Handling call
0709 12:24:56.958524 (+2030296us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54002d5354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:56.958691 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567936256}
I20260709 12:24:56.963673 13477 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:56.958866  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:24:56.968670  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:56.908756 13580 consensus_queue.cc:1175] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd983415389444e292b2330ef630850a" responder_term: 7 status { last_received { term: 7 index: 7 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [6.7-6.7]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:56.880510 13404 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 12:24:56.912374  9949 rpcz_store.cc:276] 0709 12:24:55.433896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.699152 (+1265256us) service_pool.cc:224] Handling call
0709 12:24:56.911648 (+212496us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:24:56.971910  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:56.995386  9944 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:56.989449 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:56.987627 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:56.996477 10673 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:56.990682 10669 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7. Ops: [5.6-6.7]
I20260709 12:24:56.997380 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:56.998317 10677 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:56.999056 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:56.974260 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 408875us
I20260709 12:24:57.000031 10669 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:24:56.986647 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:56.986753 10668 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:24:56.987974 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:57.005537 10098 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:57.007174 10098 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:24:57.007745 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:56.985157 10022 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:57.008273 10098 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:57.008322 10016 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:57.009958 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:57.010267 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:56.989640  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 128283us
I20260709 12:24:57.013180  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:56.985752 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:57.006034 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:57.016700 10094 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:24:57.027931 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.029597 10668 raft_consensus.cc:2410] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 8 because replica has last-logged OpId of term: 6 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 12:24:57.030514 10677 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:57.031788 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.034130 10668 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:57.033366  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.033428 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:57.035203 10094 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:24:57.035669 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.036278 10668 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:57.037533 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.038937 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:57.041052 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:56.971256  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:57.047396 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.047670 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.048488 10671 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:57.049431 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.049932 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.048149 10091 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:57.050509 10673 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:57.053941 10669 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
W20260709 12:24:57.055891  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 84726.3us
I20260709 12:24:57.058701 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:57.047420 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:57.059125 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:57.059372 10021 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:57.059754 10094 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:57.060616 10093 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 5. Current term is 6. Ops: []
I20260709 12:24:57.060761 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:57.048175 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.065186 10675 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:24:57.065698 10675 pending_rounds.cc:85] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 6
I20260709 12:24:57.066010 10675 pending_rounds.cc:107] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:24:57.066308 10675 raft_consensus.cc:2889] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:56.977221 13306 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:56.974677 13486 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:57.062028 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:57.073853  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:57.084360 13728 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:24:57.084674 13756 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:57.084878  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:57.075531 13752 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.085217 13756 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.085920 13526 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:57.075636 13679 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:57.028473 13309 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:57.085265 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.075187 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.088276 10676 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:57.028878 13687 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:57.087327 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.090917 10097 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:57.091310 10673 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:57.080417 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:57.080719  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.080456 10671 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:57.083467 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:57.047418 13498 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.094987 10091 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:57.081876 10640 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.067s (SENT) [suppressed 27 similar messages]
I20260709 12:24:57.085614 13752 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:56.976851 13672 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:24:57.105566 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:57.077046 13306 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:57.082286 13486 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:57.068830 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.090647 13359 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:57.113699 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:57.120752 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:57.119338 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:24:57.100286 13728 consensus_queue.cc:1243] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:24:57.079753 13642 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:24:56.975941 13641 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.101215 13309 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:57.066911 13758 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:24:57.128710 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:57.126413 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:57.120191 13359 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:57.134878 13693 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.887s	user 0.011s	sys 0.018s
W20260709 12:24:57.147686  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 62892.9us
I20260709 12:24:57.103577 13498 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.126880 13710 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:57.081539 13673 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.126711 13691 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:57.153077  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:57.161782  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.137378 13662 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:24:57.144681 13641 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.149803 13583 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:57.173578  9945 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:57.149621 13359 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:57.172171 10017 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:24:57.139564 13493 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:24:57.173892 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:57.175910 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:57.181450 10019 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:57.182112 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:57.178709  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.178611 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 6 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.179298 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.183418 10673 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:57.183612 10017 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:57.180099 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.184765 10674 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:57.143857 13523 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 12:24:57.173892 13583 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.186316 13583 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:57.188342 13309 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:57.189338 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:57.190544 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:57.167867 13756 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:57.168701 13526 consensus_queue.cc:1175] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 10 status { last_received { term: 7 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:24:57.191783 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:57.162487 13691 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.183091 13673 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:57.159798 13306 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:57.172525 13743 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:57.202988 13503 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:57.190474 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.341239 10676 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 6.
I20260709 12:24:57.211143 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:57.184830 10673 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:24:57.172804 13329 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.144472 13382 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:24:57.171113 13512 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:57.164578 13642 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:57.183844 10672 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:57.182433 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:57.273335 10677 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:57.274904 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:57.275828 10670 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:57.417374 10017 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:57.418243 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:57.394620 13699 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.074s	user 0.000s	sys 0.005s
I20260709 12:24:57.274928  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1594) took 1463 ms. Trace:
I20260709 12:24:57.277809 10024 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:57.419934  9942 rpcz_store.cc:276] 0709 12:24:55.811575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.726522 (+914947us) service_pool.cc:224] Handling call
0709 12:24:56.726935 (+   413us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:56.783887 (+ 56952us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:56.783919 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:56.787292 (+  3373us) log.cc:844] Serialized 52 byte log entry
0709 12:24:56.967511 (+180219us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:56.967552 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:56.967632 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:57.274722 (+307090us) raft_consensus.cc:1685] finished
0709 12:24:57.274740 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:57.274874 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56573,"raft.queue_time_us":60231,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":24591,"thread_start_us":179859,"threads_started":1,"wal-append.queue_time_us":349922}
I20260709 12:24:57.423732  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:57.319994 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 239615us
I20260709 12:24:57.322532 13691 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.333374 13743 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.180718 13166 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.164s
I20260709 12:24:57.392354 10020 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:24:57.383728 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 217628us
W20260709 12:24:57.434494  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1591) took 1999 ms (client timeout 1509 ms). Trace:
I20260709 12:24:57.416625 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:57.435000  9948 rpcz_store.cc:269] 0709 12:24:55.435001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.545994 (+110993us) service_pool.cc:224] Handling call
0709 12:24:57.434308 (+1888314us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400189cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:57.434432 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":437104896}
I20260709 12:24:57.426832 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:57.398763 13512 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:57.426698  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:57.420470 13535 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.724s	user 0.019s	sys 0.003s
I20260709 12:24:57.436594 10019 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:24:57.320396 13710 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:57.449736 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.420154 13306 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:57.435141 13743 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:57.477025 13756 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:24:57.441967 13642 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:57.435590 13758 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:57.491876 13767 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:24:57.429405 13704 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:57.506724 10020 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:24:57.425717 13751 consensus_queue.cc:1175] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aa345bcd43b44088114b7b6f2830b17" responder_term: 8 status { last_received { term: 8 index: 6 } last_committed_idx: 5 last_received_current_leader { term: 8 index: 6 } }
I20260709 12:24:57.402942 13583 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.463058 13529 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:57.399106 13329 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:57.430949 13503 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:24:57.524955 10020 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:57.505952 13758 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:24:57.537202  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.545174  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:57.513170 13529 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:57.476215 13535 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.547340 10018 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:57.483245 13776 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
W20260709 12:24:57.558727 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 378437us
I20260709 12:24:57.558296 10020 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
W20260709 12:24:57.543869 13632 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.637s	user 0.017s	sys 0.006s
W20260709 12:24:57.566215 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 130888us
I20260709 12:24:57.526691 13752 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.569893 10020 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:57.496013 13766 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:24:57.544652 13758 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:57.543113 13498 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.528968 13512 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.557018 13529 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:57.591383 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:24:57.677582 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.570663 13493 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:57.586400 13716 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.977s	user 0.005s	sys 0.020s
W20260709 12:24:57.584870 13507 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.690s	user 0.015s	sys 0.011s
I20260709 12:24:57.575238 13752 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:24:57.682799 13680 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.949s	user 0.026s	sys 0.025s
I20260709 12:24:57.683377 13512 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.565866 13710 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:57.679685 13771 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:24:57.575620 13642 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.717986 13771 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:57.718355 13771 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:57.575484 13589 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:57.708123 13770 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:24:57.738071 13784 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:57.738585 13784 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:57.739647 13784 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:57.729082 13641 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.554s	user 0.016s	sys 0.016s
I20260709 12:24:57.728248 13589 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.743060 13758 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:57.709044 13608 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.746730 13608 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:57.744328 13589 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.695845 13752 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.751276 13752 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.750875 13771 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.787115  9947 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:24:57.788049  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.789088  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.791680  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.794095 10025 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:57.794353  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.795830  9948 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:57.798744 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 207296us
I20260709 12:24:57.799281  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.800390 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:57.803814 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:24:57.805253  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.682199 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:57.691357 13507 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.685708 13493 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:57.725255 13583 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.806555 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:57.840066 13535 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.840184  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.841230 13535 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.843073 13680 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.843981 13680 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.845566 13673 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:57.846268  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.849690 13727 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:57.850196 13727 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:57.846482 13673 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:57.851652 13727 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.853190 13727 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:57.854161 13673 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:57.854306  9943 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 5. (term mismatch)
I20260709 12:24:57.854887  9943 pending_rounds.cc:85] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 4
I20260709 12:24:57.855229  9943 pending_rounds.cc:107] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 10.5
I20260709 12:24:57.855545  9943 raft_consensus.cc:2889] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:24:57.856711  9947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1813) took 2339 ms (client timeout 2065 ms). Trace:
W20260709 12:24:57.857259  9947 rpcz_store.cc:269] 0709 12:24:55.517301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.847180 (+329879us) service_pool.cc:224] Handling call
0709 12:24:57.853327 (+2006147us) spinlock_profiling.cc:243] Waited 1.94 s on lock 0x7b54000159d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:57.856556 (+  3229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4455657728}
I20260709 12:24:57.858001  9949 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:24:57.861377 10021 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:24:57.861888 10021 pending_rounds.cc:85] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 6
I20260709 12:24:57.862252 10021 pending_rounds.cc:107] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:24:57.863571  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1601) took 1514 ms. Trace:
I20260709 12:24:57.864035  9941 rpcz_store.cc:276] 0709 12:24:56.348823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.820690 (+471867us) service_pool.cc:224] Handling call
0709 12:24:56.820978 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.884441 (+ 63463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.863080 (+978639us) spinlock_profiling.cc:243] Waited 56.9 ms on lock 0x7b540003f054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:57.863388 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130767232}
I20260709 12:24:57.865037 13529 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:57.867837  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:57.868480  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:57.819474 10668 raft_consensus.cc:2410] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:24:57.862545 10021 raft_consensus.cc:2889] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:57.842285 13535 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.819501 10670 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:57.912974  9948 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:57.914640 13535 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.916152 13529 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:57.917784 13404 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:57.836163 13589 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.918488 13404 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:57.920357 13404 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.835816 13583 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:57.807065 13793 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.931348 13793 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.910166 13523 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:57.926546  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.822075 13329 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:57.828299 13359 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.633s	user 0.015s	sys 0.013s
I20260709 12:24:57.823891 13507 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:57.936986 13792 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:24:57.937492 13359 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:57.939834 13791 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:24:57.940676 13404 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.935868 13771 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.913026 13022 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.939410 13486 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.959162 13707 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:57.959549 13486 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:57.963788 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 157520us
I20260709 12:24:57.966010 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:57.967702 13166 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.968662 13166 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.969468 13793 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.971236 13166 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:57.953627 13769 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.972272 13680 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:57.956122 13777 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:57.958539 13687 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:57.967767 13642 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:57.829597 13493 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:57.972383 13769 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:57.930198 13758 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:57.977638 13687 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:57.975261 13777 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:57.989359  9949 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:24:57.989879 13692 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:57.991165 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:57.992071 13704 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:57.992494 13710 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:24:57.992981  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.994793 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:57.995422  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.995819  9947 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:57.989913  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:57.996456 13622 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:57.988234  9940 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:58.009611 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:58.012921  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1834) took 1130 ms. Trace:
I20260709 12:24:58.013356  9945 rpcz_store.cc:276] 0709 12:24:56.882158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.883497 (+  1339us) service_pool.cc:224] Handling call
0709 12:24:56.945981 (+ 62484us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:57.173956 (+227975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.012374 (+838418us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400206f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:58.012709 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522753792}
I20260709 12:24:58.010850  9941 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:58.011749  9944 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:58.001959 13692 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:58.017514 13692 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.022863 13493 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.023128  9949 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:58.025678 13769 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.019405  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:58.032001 13493 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.033990  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:58.034902  9944 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.035595 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.036265 10021 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:58.041674 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.044157 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 483658us
I20260709 12:24:58.045428 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:58.048470 10021 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:24:57.985873 13642 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.036288 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.016686 13359 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.054000  9945 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.057232 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2878) took 1345 ms. Trace:
I20260709 12:24:58.057665 10090 rpcz_store.cc:276] 0709 12:24:56.711431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.720426 (+  8995us) service_pool.cc:224] Handling call
0709 12:24:56.720732 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:56.815776 (+ 95044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.056728 (+1240952us) spinlock_profiling.cc:243] Waited 73.9 ms on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:58.057015 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170060032}
W20260709 12:24:58.060652 13778 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.383s	user 0.003s	sys 0.000s
W20260709 12:24:58.066627 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57219.9us
I20260709 12:24:58.069084 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:58.070334 13771 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.062356  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.074802 13642 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:58.076872 10018 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.077991 13553 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.074879 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:58.081957 13771 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:24:58.083899 13493 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.084668 13493 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.087035 13678 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:24:58.087543 13678 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:24:58.088397 13306 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.006s
I20260709 12:24:58.089015 13678 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.091322 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.094547  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.095963 13790 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:58.095930 13505 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:58.096431  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.096601 13553 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.097419 13553 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.098073  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.099061  9946 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.099403 13505 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.099920 13789 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:58.100243 13505 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.101343 13359 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.119141 13359 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.120253 13692 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:58.102388 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.102437  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.103451 13580 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.103560  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:58.104597 13673 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:58.104355 13777 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.090296 10024 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:58.099987 13553 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.107357 13710 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.107478 13754 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.096638 13790 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:58.101758 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54863.2us
I20260709 12:24:58.108502 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.110459 13523 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:24:58.113030 13756 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:58.113938 13743 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.605s	user 0.006s	sys 0.008s
I20260709 12:24:58.114429 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:58.115334 13493 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.115703 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 790) took 1075 ms. Trace:
W20260709 12:24:58.117743 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.106353 13779 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.418s	user 0.002s	sys 0.000s
I20260709 12:24:58.121064 13704 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.121482 13359 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.101749  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.121986 13608 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:58.179296 10668 rpcz_store.cc:276] 0709 12:24:57.040728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.040843 (+   115us) service_pool.cc:224] Handling call
0709 12:24:58.115447 (+1074604us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b540050a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:58.115598 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786782976}
I20260709 12:24:58.122397  9949 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:24:58.123910  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1835) took 1241 ms. Trace:
I20260709 12:24:58.125178 13796 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:58.126222  9946 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:24:58.194973  9943 rpcz_store.cc:276] 0709 12:24:56.882513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:56.882732 (+   219us) service_pool.cc:224] Handling call
0709 12:24:57.853581 (+970849us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:57.856185 (+  2604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.123709 (+267524us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b54000159d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:58.123851 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2231670016}
I20260709 12:24:58.195784  9946 pending_rounds.cc:85] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 7
I20260709 12:24:58.196122  9946 pending_rounds.cc:107] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:24:58.196381  9946 raft_consensus.cc:2889] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:24:58.196533  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.197026 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:58.197796 13444 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:24:58.127882 13583 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.128667 13431 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.129051 13561 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.198477 13235 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.198558 13431 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.199095 13235 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.200852 13431 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:58.201464 13583 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.173969 13553 raft_consensus.cc:2764] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:24:58.131451 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.133004 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.133436 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.134212 13678 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.136386 13687 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.136765 13773 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:58.137977 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 92713us
I20260709 12:24:58.138106 13522 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.139029 13742 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.134518  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.149372 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.150156 10018 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:58.156306  9944 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:58.157832 13580 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.223773 10018 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:24:58.225270 13580 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.159651  9948 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.160126 13673 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.215720 13773 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:58.166095 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.228636 13687 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:58.229800 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.169525  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67354.5us
I20260709 12:24:58.229974 13773 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:58.230141 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:58.194913 13743 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.231885 10023 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:58.233606 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2816) took 1056 ms. Trace:
W20260709 12:24:58.172638 13691 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.735s	user 0.007s	sys 0.007s
I20260709 12:24:58.233928 10023 rpcz_store.cc:276] 0709 12:24:57.177353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.177466 (+   113us) service_pool.cc:224] Handling call
0709 12:24:58.233397 (+1055931us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b540038dbd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:58.233535 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2203369216}
I20260709 12:24:58.174417  9945 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.182693 13526 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:58.185750 13692 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.188362 10671 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:24:58.197211  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 698) took 2865 ms. Trace:
I20260709 12:24:58.126686 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.241871  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:58.252246 13758 raft_consensus.cc:2764] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:24:58.242468 10095 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:58.252799  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.127760  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.157226 13754 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:58.129640 13589 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.198020  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.168439 13790 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:58.206609 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 92311.6us
I20260709 12:24:58.169307 13756 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:58.210625 13306 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:58.210834 10092 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:24:58.215225 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:58.297363 10090 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.212143 10094 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
W20260709 12:24:58.214984 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 106771us
I20260709 12:24:58.213124 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:24:58.220395  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.222143 13784 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.223165  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:24:58.243979  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.197903 10093 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:58.241050 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.253968  9946 rpcz_store.cc:276] 0709 12:24:55.331787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:55.332016 (+   229us) service_pool.cc:224] Handling call
0709 12:24:55.332313 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:55.772864 (+440551us) spinlock_profiling.cc:243] Waited 78.4 ms on lock 0x7b54003de120. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:24:58.082271 (+2309407us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b580042add0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:24:58.196902 (+114631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.197142 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5452174464}
W20260709 12:24:58.301391 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.253508  9945 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.299228  9944 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:24:58.251696 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.300959  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.211505 13523 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.246415 13486 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:24:58.300621 13589 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.306864 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.306712 13522 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:24:58.307341 13512 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.307530  9943 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:24:58.308614 13754 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.324808  9944 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:24:58.325292  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.324174 13512 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.327529  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.328156 13583 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.328229 13526 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.329412  9947 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:58.329807  9943 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:24:58.330583 13561 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:58.331300 13608 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.332422 13754 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:58.309299 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 96300.2us
I20260709 12:24:58.309994 13166 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.324122  9941 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.298429 13580 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.308645 13766 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 12:24:58.312192 10016 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:58.312337  9948 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.312808 13742 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.311358 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:58.315368  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.238855 13777 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.253964 13796 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 12:24:58.317582 10094 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:58.228636 13673 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.319420 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.236867 13743 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.322432 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:58.323382 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.381126 13796 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:58.381341 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.381934 13796 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:24:58.382063 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61144.4us
I20260709 12:24:58.383671 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:58.314347 13306 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:58.383399 13796 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.385857 13306 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.386373 13796 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.382287 10094 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:58.323621  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.238061 13553 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.086s
I20260709 12:24:58.314771 13523 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.318266 13622 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:58.244262 13642 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:24:58.335628 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.392831 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:58.339709  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.341184 13589 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:58.341419 13367 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.342082 13235 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:58.344177 13794 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.442s	user 0.000s	sys 0.004s
I20260709 12:24:58.345870 13756 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.347590 13580 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.348534 13512 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:58.353147 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 892) took 1173 ms. Trace:
I20260709 12:24:58.354146 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 890) took 1159 ms. Trace:
I20260709 12:24:58.354970 10091 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:58.405647 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:58.355273 13526 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.356066 10668 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:58.357115  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:58.357867  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 230277us
I20260709 12:24:58.358325 13776 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:24:58.246675 13789 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.093s
I20260709 12:24:58.393239 13523 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.337666 13166 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:58.364764 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:58.365630 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.365449 13742 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.313886 13773 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:24:58.432438 13580 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.429613 13756 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.366652 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:58.367137 13582 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:58.367561 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:58.367962 13728 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.166s	user 0.005s	sys 0.045s
I20260709 12:24:58.372965 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.367587 13561 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:58.378712 10092 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:24:58.437268 13444 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:58.437716 13444 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.438055 13444 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:58.381520 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.393472 10094 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.234453 13691 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.400887 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:24:58.407550 13235 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:58.404242 10671 rpcz_store.cc:276] 0709 12:24:57.178850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.179728 (+   878us) service_pool.cc:224] Handling call
0709 12:24:57.412620 (+232892us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.296810 (+884190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.333311 (+ 36501us) spinlock_profiling.cc:243] Waited 750 ms on lock 0x7b540058f2d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:58.338223 (+  4912us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1724140928}
I20260709 12:24:58.211901 13710 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:24:58.403882 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1739) took 1394 ms. Trace:
I20260709 12:24:58.391412 13692 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:24:58.420755 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.421612  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:58.444592 10016 rpcz_store.cc:276] 0709 12:24:57.009592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.009722 (+   130us) service_pool.cc:224] Handling call
0709 12:24:58.391192 (+1381470us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:58.403804 (+ 12612us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2993209216}
I20260709 12:24:58.429751 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.431900 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.432219 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:58.446336  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 139395us
I20260709 12:24:58.437886 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.440346 10672 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:24:58.399205  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:58.435750  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78864.2us
I20260709 12:24:58.439585 13561 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 5.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:58.440037 13728 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.276s	user 0.005s	sys 0.045s
I20260709 12:24:58.423406 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2818) took 1245 ms. Trace:
I20260709 12:24:58.428565 13166 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.426153 10095 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:58.442304 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.408190 13367 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.444377  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.451432 10677 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.404170 10675 rpcz_store.cc:276] 0709 12:24:57.179430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.179553 (+   123us) service_pool.cc:224] Handling call
0709 12:24:57.274419 (+ 94866us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:57.276264 (+  1845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.276291 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:57.327108 (+ 50817us) log.cc:844] Serialized 26 byte log entry
0709 12:24:58.111158 (+784050us) spinlock_profiling.cc:243] Waited 4.74 ms on lock 0x7b18003d0960. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:24:58.316217 (+205059us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:58.316282 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.316383 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.352669 (+ 36286us) raft_consensus.cc:1685] finished
0709 12:24:58.352686 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.352952 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":143521,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7743,"spinlock_wait_cycles":10892544,"thread_start_us":92556,"threads_started":1,"wal-append.queue_time_us":937364}
I20260709 12:24:58.447919 10674 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.450028 10670 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.446162 10091 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.452188 10669 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.454377 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:58.468811 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 144328us
W20260709 12:24:58.482643  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.446825 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63319.9us
I20260709 12:24:58.494611 10090 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:24:58.495288 10094 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:58.451151 10093 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:58.449818 13710 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:24:58.501071  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 65001.6us
I20260709 12:24:58.502040 13583 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:58.503471 13583 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.503968 13583 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.504092 13678 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:24:58.507093 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 117300us
I20260709 12:24:58.490515 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.497561 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:58.465010 13742 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:58.491495 10017 rpcz_store.cc:276] 0709 12:24:57.177847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.419140 (+241293us) service_pool.cc:224] Handling call
0709 12:24:57.419471 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:57.431710 (+ 12239us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:57.431742 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:57.432507 (+   765us) log.cc:844] Serialized 52 byte log entry
0709 12:24:57.903472 (+470965us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:57.903587 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:57.903728 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:58.343619 (+439891us) spinlock_profiling.cc:243] Waited 32.5 ms on lock 0x7b24001e8648. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a809d 00007f4e090da10e 00007f4e090f188c 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221
0709 12:24:58.422910 (+ 79291us) raft_consensus.cc:1685] finished
0709 12:24:58.422930 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:58.423196 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13839,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":23686,"spinlock_wait_cycles":74704128,"thread_start_us":399651,"threads_started":1,"wal-append.queue_time_us":437691}
I20260709 12:24:58.500236 13498 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:24:58.451887 13235 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.496726 10092 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:24:58.510380  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 111290us
I20260709 12:24:58.511469  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.444057 13582 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:58.523414 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.523119  9946 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.534478 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:58.534981 10669 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.443737 13535 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:24:58.512017 13789 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:58.533414 10671 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.542188 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:58.542632  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:58.542932 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45501.5us
I20260709 12:24:58.543103 13704 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:24:58.543588 13692 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:24:58.543763 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.544910 13811 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:24:58.546669 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:58.546900 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.551834 13691 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.556386 10674 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:58.546270 13742 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.561182 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.561124 13673 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:24:58.563334 13235 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.564370 13582 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.559265 13784 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 12:24:58.531955 13553 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.575577 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.558924 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.566852 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:24:58.568295 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.574955  9946 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:58.569362 10092 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:58.572609 13367 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:58.572561 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:24:58.574716  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59350.7us
I20260709 12:24:58.557158 13789 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.577625 10020 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:24:58.581566 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:58.582280 10095 raft_consensus.cc:2410] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:24:58.579145 10098 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:58.578325 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:58.587221 10093 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:24:58.573290 13498 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 12:24:58.589775 10093 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:24:58.578052 13580 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.591413 10017 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:24:58.591562 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:58.592289 10092 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:58.583745 13789 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:58.580974 13535 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:24:58.594056 13535 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.594317 13773 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:58.594628 13535 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:58.590490 13580 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:58.595734 13580 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.596212 10099 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:24:58.598067 10099 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:24:58.576113 13553 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.601380 13553 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:58.601574 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59485.2us
I20260709 12:24:58.603507 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.604395 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.599058 10091 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:58.601308 10673 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.596149 13535 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.607980 10093 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:24:58.606618  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:58.608884 10677 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:24:58.576856 13692 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:24:58.620836 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.621483 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.624859 10098 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:24:58.626065  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:58.580590 13766 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:24:58.568305 13710 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
W20260709 12:24:58.628006 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49773.2us
I20260709 12:24:58.631395  9944 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:24:58.631871  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.633625 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.634470 10017 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:24:58.617619 13773 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.617441 13589 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:58.579322 13523 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.638247 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.648499  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:58.612953 13704 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:58.630752 13777 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:24:58.677448 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.623337 13687 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:24:58.613544 13691 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 12:24:58.630928 13535 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.690582  9946 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:24:58.691704 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.643975 13773 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:58.700750 10676 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:58.643790 13589 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.707561  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:58.684195 13404 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.707046  9944 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
W20260709 12:24:58.718299  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100196us
W20260709 12:24:58.719565  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 161903us
I20260709 12:24:58.723055 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.725208 10097 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:24:58.726533  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.729437 10017 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:58.708463 13773 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.732161  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.650470 13766 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.737941 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:58.744257 10092 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:58.715260 13589 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:58.686524 13704 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.763312 10024 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:58.738876  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:58.764999  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.696326 13790 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.783829 10017 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:24:58.730888 13687 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.795104 13756 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:58.800668 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.802194 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.766531 13673 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:58.808251  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:58.804188 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:58.811302 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.811401  9843 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 5 to 7, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:24:58.813416 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.814028 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.815461 13522 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:58.816051 13522 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.816489 13580 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.811218 10018 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:58.819716 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.818315 13522 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.821863 13235 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:24:58.729709 13404 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.784601 13367 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:58.826476 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.836067 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:58.856859 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.858876  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.788412 13687 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:58.862016 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.865255  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:58.866320  9946 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:58.857806 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.818471 13580 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:58.886674  9991 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.885730 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:58.889469  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:58.890034  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.882019 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:58.881903 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.882705 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.828864 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.890836 10672 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.899259 10019 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:58.904014 10024 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:58.786890 13444 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.852312 13367 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:58.907769 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.912889 10675 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:58.887346  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:58.865425 13687 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:58.909690 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:58.913865 10021 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.763944 13692 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.919907 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:24:58.924360 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 196278us
I20260709 12:24:58.898710 13608 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:24:58.825348 13583 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.927541 10022 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:24:58.926465  9940 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:58.933145 10023 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:24:58.931489 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.917487 13367 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:58.937863 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:24:58.937856  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 149127us
W20260709 12:24:58.937961 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45182us
I20260709 12:24:58.939643 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:58.942795 10674 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:58.923434 13580 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:24:58.867558 13678 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:24:58.947515 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:58.926471 13710 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.953223 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:58.937373 13796 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.979074 13796 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:58.981500 13796 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:58.983811 13444 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:58.944978 13522 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:24:58.984848 13522 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:58.985276 13583 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:24:58.989609 13813 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.069s	user 0.001s	sys 0.002s
W20260709 12:24:58.991312  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 114274us
I20260709 12:24:58.992123  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:58.993669  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:58.946046 13728 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:24:58.958793 10675 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:58.999886 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:58.959780 13309 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:58.957686 13710 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:58.964486 13526 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:24:58.965320 13812 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:24:58.967093 10024 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:58.967437 13523 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.969530  9843 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:24:58.971072 13493 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:58.971330 13758 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:58.971711 13404 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:58.947494 13691 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:24:58.975725 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2826) took 1796 ms. Trace:
I20260709 12:24:58.947921 13622 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:24:58.996150 10671 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:24:58.958297 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:59.005452  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51442.7us
I20260709 12:24:58.985287 13522 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:24:59.035650 13776 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:24:59.043206 13404 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.015885 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:58.955209 13580 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:59.059332 13580 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.030395 10021 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:24:59.063391 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:58.999691 13728 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:59.024165 13493 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:59.010067 13526 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:59.029856 13758 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:59.036685 13622 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.043939  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:24:59.104200  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 60366.3us
I20260709 12:24:59.030993 13561 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:24:59.009079 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:59.037744 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:59.046999 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:59.051268 13789 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.053472 13710 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:24:59.043248 13776 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:24:59.055392 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:59.081440  9843 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 4 to 7, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 6 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:24:59.136778  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:59.137023 10024 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:59.033458 10023 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:59.137817  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:59.064035 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:24:59.138774 10021 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:24:59.066843 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.140058 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.084790 10016 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:24:59.035001 13796 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:24:59.035260 10020 rpcz_store.cc:276] 0709 12:24:57.179688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:57.574016 (+394328us) service_pool.cc:224] Handling call
0709 12:24:58.423929 (+849913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.580529 (+156600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:58.935464 (+354935us) spinlock_profiling.cc:243] Waited 98.8 ms on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:24:58.975520 (+ 40056us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b54002d5850. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:24:58.975645 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2122418176}
W20260709 12:24:59.117838 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.131474 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:59.047266 13790 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:59.103446 13622 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.025441 13309 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.064102 13704 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:59.147504 10673 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
W20260709 12:24:59.141963 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 104317us
I20260709 12:24:59.155887 10677 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:24:59.160495 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:59.138538 13678 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:59.167169  9843 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a reported cstate change: term changed from 4 to 9, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 9 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
W20260709 12:24:59.168254 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 157883us
W20260709 12:24:59.172504  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 177129us
I20260709 12:24:59.173243  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:59.174134  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:59.174575 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:24:59.175554 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.148468 13742 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:24:59.171320 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:24:59.183586 10024 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 12:24:59.163940 13553 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.562s	user 0.004s	sys 0.008s
I20260709 12:24:59.182530 10020 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [7.5-9.6]
W20260709 12:24:59.194058 13821 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:24:59.195665 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:59.166692 13608 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:59.144840 13444 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:24:59.202745 10021 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
W20260709 12:24:59.183614  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 322040us
I20260709 12:24:59.169694 13678 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:59.180536 13561 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:24:59.226019  9945 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:24:59.220705  9843 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 4 to 6, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 6 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:24:59.227811  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:59.228438  9946 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.184906 13742 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.234031  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:59.225903  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:59.227335  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:59.232559  9944 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 9. Current term is 10. Ops: []
I20260709 12:24:59.230937  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:59.244626  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:59.245234  9945 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:24:59.240159  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:59.257166 10019 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:24:59.197088 13766 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:59.226805 13678 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.298779  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:24:59.242573 13742 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:24:59.235167 13561 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:59.191644 13367 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.295411  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:59.303956  9947 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:24:59.199015 13553 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:59.313524  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59321.5us
I20260709 12:24:59.314939  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:24:59.321810  9945 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:24:59.323046 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:59.338791 10672 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:24:59.339493 10094 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:59.323115 13704 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:24:59.302091 13306 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:59.333515 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:24:59.335825 10090 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:24:59.334059 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79761.1us
W20260709 12:24:59.332099  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 338523us
I20260709 12:24:59.326335 13309 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:59.377174  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:59.341748 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:59.256866 13498 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.071s
I20260709 12:24:59.346398 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:59.347267 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:24:59.382490 10098 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:24:59.306223 13561 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:24:59.348959 10025 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:24:59.351493  9947 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:59.225049 13493 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:59.348508 13773 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.003s
I20260709 12:24:59.235015 13728 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.395053  9944 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:24:59.381821 10097 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:24:59.357940 13306 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:24:59.256352 13758 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:59.458822  9946 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:24:59.255893 13526 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:59.360523 13404 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:59.466066  9943 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:24:59.410905 13673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.027s	sys 0.054s
I20260709 12:24:59.511469 13523 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:24:59.535732 13523 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:59.519443 13776 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.456575 13306 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:24:59.468063 13804 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 12:24:59.532794 13673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.027s	sys 0.054s
I20260709 12:24:59.530014 13826 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:59.402846 13309 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:24:59.545872 10016 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 6. Current term is 7. Ops: []
I20260709 12:24:59.524304 13580 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.529068  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:24:59.540453  9843 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a reported cstate change: config changed from index 3 to 6, term changed from 4 to 5, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to <none>, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New cstate: current_term: 5 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:24:59.468940 13493 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:24:59.558491 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.595407 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.559620 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.467550 13834 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:59.414881 13773 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:59.534803 13789 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:24:59.592398 13836 consensus_queue.cc:1175] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" responder_term: 7 status { last_received { term: 4 index: 3 } last_committed_idx: 3 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: [6.4-6.4]" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:24:59.553122 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.635612 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.597014  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68067.9us
W20260709 12:24:59.600409  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 231806us
I20260709 12:24:59.636792  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:24:59.532217 13756 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 12:24:59.641175 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 255930us
I20260709 12:24:59.643643 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:59.637754  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:59.580924 13235 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:24:59.645412  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 330559us
W20260709 12:24:59.666038 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.666123  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:59.581669 13673 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:24:59.590876 13535 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.881s	user 0.011s	sys 0.001s
I20260709 12:24:59.663561  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:59.551440 13830 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:59.595889 13526 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.584619 13309 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:59.671669 10091 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:24:59.673413 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 226825us
I20260709 12:24:59.681910 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:24:59.586133 13608 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:59.641314 13704 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:24:59.693209 10675 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.687s	user 0.006s	sys 0.006s
I20260709 12:24:59.702476 10675 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:24:59.664456 13776 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:24:59.714793 13512 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:24:59.734757  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 97112.7us
W20260709 12:24:59.737891  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62509.1us
W20260709 12:24:59.739074 10640 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.067s (SENT)
I20260709 12:24:59.739722 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:24:59.743527  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 80080.6us
I20260709 12:24:59.744542  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:59.754231  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:24:59.673620 13498 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:24:59.758342 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:24:59.758661 10677 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:24:59.669175 13404 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:59.688249 13589 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.897s	user 0.010s	sys 0.002s
I20260709 12:24:59.634953 13580 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.691687 13756 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.778122  9942 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
W20260709 12:24:59.783365  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44597us
I20260709 12:24:59.774463 13404 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:59.785748 13851 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:59.786477 13851 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:24:59.787564 13404 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:24:59.787457 13851 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.774524 13642 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:24:59.785492 13431 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:59.789853 13673 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.785431 13704 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:24:59.788158 13404 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:24:59.762288 13367 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:59.692787 13754 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 12:24:59.694067 13235 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:59.634056 13834 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.772604 13553 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:59.605845 13826 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:59.822032  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:24:59.791482 13622 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.618s	user 0.009s	sys 0.002s
I20260709 12:24:59.792217 13743 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:24:59.792743 13727 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:24:59.851809  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 97659.1us
I20260709 12:24:59.878229  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:24:59.779093 13691 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
W20260709 12:24:59.907357 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.922873 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:59.924966 10672 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:24:59.856689 13404 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:24:59.774729 13589 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.891474  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:24:59.898564 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:24:59.817641 13824 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:59.713651 13830 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.962730 13850 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:24:59.792102 13756 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.901134 13367 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:24:59.940999 13522 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.879s	user 0.008s	sys 0.012s
I20260709 12:24:59.942004 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 575) took 1140 ms. Trace:
I20260709 12:24:59.990234 10017 rpcz_store.cc:276] 0709 12:24:58.801089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.886278 (+ 85189us) service_pool.cc:224] Handling call
0709 12:24:58.886573 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:58.908586 (+ 22013us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:24:59.390773 (+482187us) log.cc:844] Serialized 14 byte log entry
0709 12:24:59.939807 (+549034us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:24:59.939861 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:59.939915 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.939984 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.941931 (+  1947us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":687597,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":165631,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":856334,"threads_started":3}
I20260709 12:24:59.991396 13831 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.089s
I20260709 12:24:59.882229 13754 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.992388 13754 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:24:59.992894 13825 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.196s	user 0.000s	sys 0.000s
I20260709 12:24:59.924602 13727 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.994288 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 805) took 1138 ms. Trace:
I20260709 12:24:59.963438 13850 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:24:59.798094 13431 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:24:59.945730 10677 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:24:59.952252  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60853.7us
I20260709 12:24:59.900029 13582 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:24:59.959461 13743 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:24:59.900485 13553 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:24:59.960103 13589 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:24:59.960817 13404 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:59.888291 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.051416 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.984021 13367 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:24:59.989588 10675 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:24:59.940245 13704 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:24:59.997565  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.112999  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:24:59.946090 13847 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.040539 13535 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:00.113298  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:24:59.950989 13773 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:00.015738 13823 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.445s	user 0.000s	sys 0.002s
I20260709 12:25:00.012358 10669 rpcz_store.cc:276] 0709 12:24:58.856350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.856481 (+   131us) service_pool.cc:224] Handling call
0709 12:24:58.856807 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:58.886918 (+ 30111us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.886953 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.911930 (+ 24977us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.376014 (+464084us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:59.376069 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.376169 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:24:59.994053 (+617884us) raft_consensus.cc:1685] finished
0709 12:24:59.994077 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:24:59.994230 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":477,"raft.queue_time_us":25399,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":16060,"thread_start_us":384994,"threads_started":1,"wal-append.queue_time_us":741564}
I20260709 12:25:00.045019  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:24:59.941879  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.128582  9942 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:00.127458  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 232153us
W20260709 12:25:00.019043 13838 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.412s	user 0.001s	sys 0.000s
I20260709 12:24:59.994508 13754 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.013831 13522 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:00.106825 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:24:59.852516 13309 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:24:59.877803 13622 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.040220 13743 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:00.056641 13166 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.547s	user 0.013s	sys 0.001s
I20260709 12:25:00.119874  9943 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:00.111421 13367 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:24:59.939713 13608 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:24:59.948666 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 266874us
I20260709 12:24:59.995010 13824 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:00.159611 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:00.160949 13856 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
W20260709 12:25:00.161561 13840 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.374s	user 0.001s	sys 0.000s
I20260709 12:25:00.161893 13431 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:00.161612 13850 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.163092 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 763) took 1294 ms. Trace:
W20260709 12:25:00.163642 13839 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.455s	user 0.001s	sys 0.000s
I20260709 12:25:00.163439 10670 rpcz_store.cc:276] 0709 12:24:58.869130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.869371 (+   241us) service_pool.cc:224] Handling call
0709 12:24:58.911380 (+ 42009us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:58.954072 (+ 42692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:58.954093 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.981710 (+ 27617us) spinlock_profiling.cc:243] Waited 4.54 ms on lock 0x7b480002ea58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:24:59.004647 (+ 22937us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.305390 (+300743us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:24:59.305442 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.305530 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.162796 (+857266us) raft_consensus.cc:1685] finished
0709 12:25:00.162822 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.163029 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3114,"raft.queue_time_us":16934,"raft.run_cpu_time_us":2134,"raft.run_wall_time_us":28893,"spinlock_wait_cycles":10447616,"thread_start_us":257241,"threads_started":1,"wal-append.queue_time_us":519662}
I20260709 12:25:00.157646 13166 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.166936 13880 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:00.167624 13880 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:00.168730 13166 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:00.168565 13880 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:00.050717 13582 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.171940 13235 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:00.173205 13743 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.173967 13235 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:00.122740 13535 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.157886  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:00.157900 13847 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:00.121147 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 426928us
I20260709 12:25:00.175786 13523 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:00.111850 13553 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:00.127334 13830 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:00.345139  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.393905 10091 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.692s	user 0.008s	sys 0.004s
I20260709 12:25:00.200223 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1751) took 1271 ms. Trace:
I20260709 12:25:00.422086 10091 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:00.422670 10016 rpcz_store.cc:276] 0709 12:24:58.929232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.145938 (+216706us) service_pool.cc:224] Handling call
0709 12:24:59.146342 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:24:59.584751 (+438409us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.199918 (+615167us) spinlock_profiling.cc:243] Waited 399 ms on lock 0x7b54001c5054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:00.200153 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":918250752}
W20260709 12:25:00.254721 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 95242.7us
W20260709 12:25:00.245472 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.424640 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:00.425441  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:00.198951 10677 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:00.427312  9940 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:00.429216  9942 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:00.429759 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 764) took 1545 ms. Trace:
W20260709 12:25:00.432339  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.122969 13866 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:00.450757  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:00.457095 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:00.228334 13561 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.759s	user 0.012s	sys 0.000s
I20260709 12:25:00.461341 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:00.406582 13880 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:00.477303 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.229866 13493 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.624s	user 0.006s	sys 0.002s
I20260709 12:25:00.482761 13861 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:00.483121 13826 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.484954 13826 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:00.485491 13826 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:00.466941  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:00.441267 13834 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:00.229502 13687 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.303s	user 0.010s	sys 0.002s
I20260709 12:25:00.403995 13859 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:25:00.487681 13824 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:00.477890 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.404711 13849 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:00.355959 13580 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.489534 13523 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.489852 13235 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:25:00.491989 13235 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:00.433545 10668 rpcz_store.cc:276] 0709 12:24:58.884473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.884634 (+   161us) service_pool.cc:224] Handling call
0709 12:24:58.915377 (+ 30743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:58.962249 (+ 46872us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:24:58.962295 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:58.962911 (+   616us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.189565 (+226654us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:24:59.189617 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.189691 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.429485 (+1239794us) raft_consensus.cc:1685] finished
0709 12:25:00.429514 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.429681 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3057,"raft.queue_time_us":6611,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2420,"thread_start_us":226227,"threads_started":1,"wal-append.queue_time_us":412721}
I20260709 12:25:00.430393 13535 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:00.480333 13835 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.463s	user 0.001s	sys 0.001s
I20260709 12:25:00.493567 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2915) took 1165 ms. Trace:
I20260709 12:25:00.480706 13867 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:25:00.483983 13857 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.364s	user 0.003s	sys 0.000s
I20260709 12:25:00.487437 13687 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:00.488435 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.177500 13589 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.345432 13858 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.479660 13756 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:00.429577 13851 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.479313 13561 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:00.450222  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2465425189998314s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19340000000000002s for a response
I20260709 12:25:00.475479 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:00.537734 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1759) took 1548 ms. Trace:
W20260709 12:25:00.708055 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 232660us
W20260709 12:25:00.529250 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.550753 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.498849  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:00.603120  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 490171us
W20260709 12:25:00.710539 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.532840  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1850) took 1338 ms. Trace:
W20260709 12:25:00.535770 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 111216us
W20260709 12:25:00.709826  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2096425189997717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1975s for a response
I20260709 12:25:00.708395 10022 rpcz_store.cc:276] 0709 12:24:58.989253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.081090 (+ 91837us) service_pool.cc:224] Handling call
0709 12:25:00.537422 (+1456332us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54001f7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:00.537652 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3043778432}
I20260709 12:25:00.715200 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:00.709592 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.529125 13844 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.020s	sys 0.012s
W20260709 12:25:00.723572 13844 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.021s	sys 0.031s
I20260709 12:25:00.723328 13851 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.724407 13848 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.725997  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:00.726716 13756 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.727514  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:00.727370 13756 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.728174 13498 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:00.728847 10023 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:00.728716 13498 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.730142 13846 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:00.730248 13687 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.731915 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1757) took 1747 ms. Trace:
W20260709 12:25:00.732782 10640 leader_election.cc:336] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.584s (SENT)
I20260709 12:25:00.732632 10023 rpcz_store.cc:276] 0709 12:24:58.984509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.146627 (+162118us) service_pool.cc:224] Handling call
0709 12:25:00.731718 (+1585091us) spinlock_profiling.cc:243] Waited 1.55 s on lock 0x7b5400137ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:00.731855 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3572408576}
I20260709 12:25:00.733494 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:00.736020 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2909) took 1424 ms. Trace:
I20260709 12:25:00.726166 13851 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.487206 13834 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.483325 13861 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.449716 13878 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:00.706329 13894 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:25:00.705340 13553 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:00.511063 10095 rpcz_store.cc:276] 0709 12:24:59.328617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.332392 (+  3775us) service_pool.cc:224] Handling call
0709 12:24:59.391547 (+ 59155us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:59.730651 (+339104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.730693 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:59.731467 (+   774us) log.cc:844] Serialized 52 byte log entry
0709 12:24:59.899348 (+167881us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:00.035802 (+136454us) log.cc:844] Serialized 14 byte log entry
0709 12:25:00.120526 (+ 84724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.121103 (+   577us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.493277 (+372174us) raft_consensus.cc:1685] finished
0709 12:25:00.493300 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.493490 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":227743,"prepare.queue_time_us":213981,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":384349,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":24167,"thread_start_us":528050,"threads_started":4,"wal-append.queue_time_us":749750}
I20260709 12:25:00.510485 13692 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:00.523326 13306 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.928s	user 0.012s	sys 0.008s
I20260709 12:25:00.736578 10092 rpcz_store.cc:276] 0709 12:24:59.311904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.332311 (+ 20407us) service_pool.cc:224] Handling call
0709 12:24:59.359122 (+ 26811us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:24:59.697603 (+338481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.697635 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:24:59.776590 (+ 78955us) log.cc:844] Serialized 52 byte log entry
0709 12:25:00.019318 (+242728us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:00.019366 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.019436 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.735708 (+716272us) raft_consensus.cc:1685] finished
0709 12:25:00.735741 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.735937 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":183547,"raft.queue_time_us":287202,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":17895,"thread_start_us":477791,"threads_started":2,"wal-append.queue_time_us":664901}
W20260709 12:25:00.525143 13742 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.188s	user 0.010s	sys 0.002s
I20260709 12:25:00.794978  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:00.538864 13431 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:00.492429 13235 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:00.667671 13890 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.521814 13727 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.538189 13743 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.591660 13166 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:00.797722 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:00.713196 13728 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:00.792773 13878 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:00.604429 13678 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.298s	user 0.013s	sys 0.000s
I20260709 12:25:00.534618 13885 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:00.801225 13754 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.015s	sys 0.007s
I20260709 12:25:00.725036 13848 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:00.486505 13826 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.526175 13622 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.731274 13830 raft_consensus.cc:2764] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:25:00.521770 13881 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:00.730749 13846 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.814007 13444 raft_consensus.cc:2764] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:25:00.545557 13582 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:00.807503 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74751.5us
I20260709 12:25:00.712211  9949 rpcz_store.cc:276] 0709 12:24:59.194369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.221503 (+ 27134us) service_pool.cc:224] Handling call
0709 12:24:59.301217 (+ 79714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.682164 (+380947us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.682562 (+   398us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.762215 (+ 79653us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.932751 (+170536us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b18003f4780. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:25:00.115788 (+183037us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:00.115847 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.115912 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.532547 (+416635us) raft_consensus.cc:1685] finished
0709 12:25:00.532566 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.532768 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118866,"raft.queue_time_us":420464,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":26836,"spinlock_wait_cycles":8030080,"thread_start_us":486988,"threads_started":2,"wal-append.queue_time_us":752851}
I20260709 12:25:00.812089 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:00.812446  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:00.836917 10640 leader_election.cc:336] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.766s (SENT)
W20260709 12:25:00.813180  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 346432us
I20260709 12:25:00.837394 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:00.838975 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:00.811721  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:00.498375 13673 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.303s
I20260709 12:25:00.839596 10017 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.841850 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 768) took 1891 ms. Trace:
W20260709 12:25:00.494884 13526 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.781s	user 0.007s	sys 0.004s
W20260709 12:25:00.798271 13704 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.026s	sys 0.046s
I20260709 12:25:00.792480 13861 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:00.788515 13832 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:00.750658 13777 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.003s	sys 0.022s
I20260709 12:25:00.801108 13864 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.802922 13851 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.731340 13833 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:00.812664 13235 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.815979 13867 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:00.802356 13692 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:00.794843 13691 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 12:25:00.825788 13874 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:00.828042  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 329268us
I20260709 12:25:00.808776 13493 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.800757 13889 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:00.806326 13498 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.810910 13890 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:00.795538 13834 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:00.820441 13754 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.015s	sys 0.007s
I20260709 12:25:00.526932 13522 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.751060 13589 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.889683 13873 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.819855 13878 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:00.842201 10674 rpcz_store.cc:276] 0709 12:24:58.950753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:58.950942 (+   189us) service_pool.cc:224] Handling call
0709 12:24:58.996082 (+ 45140us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:24:59.001741 (+  5659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:24:59.001769 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:24:59.008946 (+  7177us) log.cc:844] Serialized 26 byte log entry
0709 12:24:59.302548 (+293602us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:24:59.302603 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:24:59.302697 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:00.841636 (+1538939us) raft_consensus.cc:1685] finished
0709 12:25:00.841657 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:00.841802 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13399,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"spinlock_wait_cycles":69888,"thread_start_us":287133,"threads_started":1,"wal-append.queue_time_us":541261}
I20260709 12:25:00.837100 13885 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.849421  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:00.851236 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:00.851917 10016 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:00.973560  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 124240us
I20260709 12:25:00.808640 13306 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:00.852404 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:00.889528 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:00.821444 13166 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.895430 13851 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:00.897538  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:00.875340 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.810485 13742 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.900449 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:00.994814 10669 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:00.855722 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:00.903427 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:00.903836 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:00.903419 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:00.824631 13678 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:00.907195 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.833444 13881 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:00.847841 13832 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:00.911629 13833 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:00.912637 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:00.913077 13758 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:00.913169 10675 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:25:00.847280 13704 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.027s	sys 0.047s
I20260709 12:25:00.924729 10673 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:25:00.851091 13777 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.004s	sys 0.022s
I20260709 12:25:00.895593 13444 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.843628 13831 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 12:25:00.930059 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:00.890058 13830 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.037011 10676 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:00.903505 13692 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:00.948655  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:00.904378 13849 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:00.853829 13847 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:00.860427 13872 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:00.731213 13756 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:00.855319 13535 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.750893 13858 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:00.989532  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:00.858049 13887 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.131s
W20260709 12:25:01.020644 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 183756us
I20260709 12:25:00.972024 13874 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.022948 10093 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:25:01.023838 10671 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:00.930616 13493 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.026242 10670 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:01.026896 10668 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:01.030512 10099 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:00.994105 13522 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:01.027454 10672 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:00.994839 13903 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:01.027846 13498 raft_consensus.cc:2764] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:00.854578 13523 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:01.041466 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:00.891181 13864 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:00.996199 13561 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.019968 13881 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:01.023804 13832 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:01.028065 13704 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:01.105520 13879 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:00.969781 13873 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:01.074580  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.058247 13431 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.173710 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:01.174170  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 323985us
W20260709 12:25:01.081202 10640 leader_election.cc:336] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.827s (SENT)
I20260709 12:25:01.036203 13444 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.058624 13872 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.057371 13789 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:01.087949  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.106273 13892 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 12:25:01.145933  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 219186us
I20260709 12:25:01.077174 13553 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:01.050375 13830 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.147322 10092 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
W20260709 12:25:01.166898 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.141748 13522 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:01.060845 13907 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:01.141583 13367 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.942s	user 0.011s	sys 0.002s
I20260709 12:25:01.183362  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:01.169400 13899 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:25:01.187407 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:01.165403 13523 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:01.203033 10091 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2917) took 1870 ms (client timeout 1531 ms). Trace:
I20260709 12:25:01.105278 13874 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.206027  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:01.207005  9940 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:25:01.207782  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.219231  9942 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:01.175151 13848 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.208218  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:01.210052  9945 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:01.212378  9948 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:25:01.178777 13431 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.191874 13872 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:01.186023 13846 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.214063 13742 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:01.215803 10091 rpcz_store.cc:269] 0709 12:24:59.332999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.333156 (+   157us) service_pool.cc:224] Handling call
0709 12:25:01.202755 (+1869599us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b5400093154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:01.202913 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719782656}
I20260709 12:25:01.218833 13847 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:01.235181 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:01.235126 13847 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:01.237259 13905 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:25:01.237751 13847 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:01.238230 13893 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 12:25:01.239203 13902 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:25:01.242244 10668 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:01.242956 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2918) took 1101 ms. Trace:
I20260709 12:25:01.243366 10098 rpcz_store.cc:276] 0709 12:25:00.141998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.142175 (+   177us) service_pool.cc:224] Handling call
0709 12:25:00.163167 (+ 20992us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.288077 (+124910us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.288125 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.441428 (+153303us) log.cc:844] Serialized 52 byte log entry
0709 12:25:00.831697 (+390269us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:00.831738 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.831792 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.242678 (+410886us) raft_consensus.cc:1685] finished
0709 12:25:01.242707 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.242889 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119433,"raft.queue_time_us":124926,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":46458,"thread_start_us":205866,"threads_started":1,"wal-append.queue_time_us":474607}
I20260709 12:25:01.245334 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:01.247329 13523 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.248528 13873 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.250830 13834 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.251421 13864 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.251978 13851 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.252357 13885 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.215703 13367 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.254737 13367 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.219645  9943 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:01.200460 13789 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.256243  9943 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:01.256431 13874 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.058992 13849 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.256961 13789 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.225570  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.200953 13678 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.257979  9948 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:01.201540 13831 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:25:01.258688 13849 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.259382 10097 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:01.262115  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.193598 13879 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.262697 13704 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.264696 13879 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:01.205058 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 123929us
I20260709 12:25:01.231397  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.231263 13887 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:01.269946 10670 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:01.221612  9949 raft_consensus.cc:2368] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
W20260709 12:25:01.279016 10640 leader_election.cc:336] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.596s (SENT)
I20260709 12:25:01.281384  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.282236  9948 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:01.285235  9943 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:01.285450 13512 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.286315 13512 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.281378 13789 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.288691 13789 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.288968 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:01.219175 13589 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.273375 10017 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:01.302858 13789 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.303068 13830 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.305843 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:01.228503 13166 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.216935 13844 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.344170 13858 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.285241 13728 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.346431  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:01.267920 13890 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.162117 13833 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:01.374318 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 145219us
W20260709 12:25:01.351704  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 104867us
I20260709 12:25:01.352448 10099 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:01.367952  9943 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:01.373265 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 94284.2us
I20260709 12:25:01.266443 13522 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.380296  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.383723 13493 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.371641 13589 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.371313 13728 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.388679 13728 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.389688  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:01.390450  9948 raft_consensus.cc:2368] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11.
I20260709 12:25:01.391362  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.391942  9948 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:01.392757  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.393844  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.395643  9942 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:01.399327 13444 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:01.399739 13880 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.901s	user 0.011s	sys 0.000s
I20260709 12:25:01.402868 13535 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.394385  9948 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:01.415273 10671 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:01.398082 13861 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:01.419786 10640 leader_election.cc:336] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.916s (SENT)
W20260709 12:25:01.427109  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 255595us
W20260709 12:25:01.422034 13892 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:25:01.519389 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 144295us
W20260709 12:25:01.519850  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94825us
I20260709 12:25:01.523169 10643 leader_election.cc:400] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11.
I20260709 12:25:01.523523 10643 leader_election.cc:403] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 12:25:01.529580 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 105174us
W20260709 12:25:01.531080  9912 connection.cc:256] RPC call timeout handler was delayed by -1.1315940229997068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28940000000000005s for a response
W20260709 12:25:01.531723  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0590634069999396s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2767s for a response
I20260709 12:25:01.372656 13830 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.145s
I20260709 12:25:01.533743 10018 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:25:01.537093 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2921) took 1111 ms. Trace:
I20260709 12:25:01.539216 10090 rpcz_store.cc:276] 0709 12:25:00.425600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.425757 (+   157us) service_pool.cc:224] Handling call
0709 12:25:00.429201 (+  3444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.493489 (+ 64288us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:00.493527 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:00.493573 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:00.493612 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.536850 (+1043238us) spinlock_profiling.cc:243] Waited 64 ms on lock 0x7b54002fd854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:01.536896 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.537030 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":147263872}
I20260709 12:25:01.535655 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:01.546352 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:01.546932 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.022s	sys 0.048s
W20260709 12:25:01.548772 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.022s	sys 0.048s
I20260709 12:25:01.522526  9832 maintenance_manager.cc:419] P 2121bfede85b447c9f51ef23233606cc: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033376
I20260709 12:25:01.551299 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:01.551667 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 172764us
W20260709 12:25:01.557852 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.557880 10671 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:25:01.558363 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.558854 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.559981 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.374528 13844 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.536765 13728 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11.
I20260709 12:25:01.537057 13836 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.560370 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.542184 13535 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.562021 10670 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:01.561169  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1860) took 1559 ms. Trace:
I20260709 12:25:01.553354 13890 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:01.561120 13166 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.568387 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.520781 13903 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:01.564607 10671 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:01.521399 13881 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.573820  9943 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:25:01.574651 10672 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:01.575673 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:01.576709 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.580904 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:01.581166 10640 leader_election.cc:336] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.006s (SENT)
I20260709 12:25:01.581718 10670 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:01.581588 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:01.571177 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.567535  9941 rpcz_store.cc:276] 0709 12:25:00.001330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.019623 (+ 18293us) service_pool.cc:224] Handling call
0709 12:25:00.051202 (+ 31579us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.114704 (+ 63502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.114750 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:00.153040 (+ 38290us) log.cc:844] Serialized 52 byte log entry
0709 12:25:00.719839 (+566799us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:00.797667 (+ 77828us) log.cc:844] Serialized 14 byte log entry
0709 12:25:00.797884 (+   217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:00.797952 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:01.347501 (+549549us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b180033cd60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:01.555082 (+207581us) raft_consensus.cc:1685] finished
0709 12:25:01.560869 (+  5787us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.561080 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22652,"prepare.queue_time_us":80754,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":64993,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":211,"spinlock_wait_cycles":9205120,"thread_start_us":452700,"threads_started":1,"wal-append.queue_time_us":599999}
I20260709 12:25:01.583719 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.584439 10668 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:01.552982 13858 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:25:01.585842 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.579049 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.571240 13887 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:01.571628 13890 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.629983 10674 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 6.
I20260709 12:25:01.630731  9942 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:01.635905 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.631495 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.637187 10672 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:25:01.638712 10668 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:01.586371 13728 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.572201 13535 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.642570 10676 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:01.643333 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:01.568954 13836 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.655318 10670 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
W20260709 12:25:01.647866  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.579490 13166 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.636497 13890 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:01.662021 10671 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:01.666342 10669 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:01.668771 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:01.669646 10670 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:01.670578 10674 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:01.654793 13756 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:01.674166  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:01.676507 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 98944.9us
W20260709 12:25:01.677119 10640 leader_election.cc:336] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.099s (SENT)
I20260709 12:25:01.677424 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:01.675653 13833 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:01.680490 13878 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.002s
I20260709 12:25:01.680294 13833 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.681151 13878 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.659611 13728 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.682932  9945 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:01.684978 13728 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.679217 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.686404 10676 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:01.636143 13887 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:01.720897  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.723881 13756 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:01.737998 13897 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.801s	user 0.001s	sys 0.010s
I20260709 12:25:01.739639 13833 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.675784 13907 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:01.750718 13907 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.758069 13881 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:01.750767 13919 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
W20260709 12:25:01.765230 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 88155.3us
I20260709 12:25:01.754313 13861 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.775552 13589 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:01.793951 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.783850 13907 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.802405 13878 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.789260 13166 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.792868 13535 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:01.800204 13692 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.000s
W20260709 12:25:01.812335 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44308.9us
I20260709 12:25:01.812463 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.815716 13166 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:01.816323 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.816663 13166 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:01.817044 13166 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:01.817703 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.818069 13166 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.818897 10674 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
W20260709 12:25:01.819075 13921 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:25:01.819455 13166 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:01.819871 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.826826 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:01.827715 10095 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:01.827898 10674 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:01.828620 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:01.829196 10095 raft_consensus.cc:2376] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 11. Re-sending same reply.
I20260709 12:25:01.830063 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:01.824401 13887 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.832744 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:01.833390 10097 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:01.835309 13235 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.925s	user 0.012s	sys 0.000s
I20260709 12:25:01.835973 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:01.836803 10675 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:01.837620 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.838212 10675 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:01.839375 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.839979 10673 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:01.840873 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.841429 10673 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:01.842216 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.845783 13431 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:01.846151 13832 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.002s
I20260709 12:25:01.793462 13589 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.847939 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:01.848388 13589 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.848874 10091 raft_consensus.cc:2410] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:01.849745 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:01.850311 10091 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:01.851629 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:01.853360  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.835781 13235 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.858000 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:01.857882 13832 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.858774 10091 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:01.812945 13692 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.863087 13692 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.855885 13907 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:01.855768  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.857604 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:01.856388  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:01.856601  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.857230  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.857311 13306 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:01.857347 13872 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.599s	user 0.005s	sys 0.005s
I20260709 12:25:01.871482 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:01.871495 13872 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.860146 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:01.860726 13431 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:01.879818 13431 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.880298 13431 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:01.879487 13535 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:01.883335 13535 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:01.883844 10672 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260709 12:25:01.884402 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:01.883406 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.886582 10675 raft_consensus.cc:2410] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:25:01.859229 13589 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
W20260709 12:25:01.819453 13904 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.714s	user 0.006s	sys 0.004s
I20260709 12:25:01.855404  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.888216  9945 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:01.865869  9946 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:01.885368 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:01.854735 10642 connection.cc:256] RPC call timeout handler was delayed by -1.1076740939997762s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
I20260709 12:25:01.863978 13907 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.872344 10676 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:01.842290 13890 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.874101  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:01.877454 10091 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:01.865250  9949 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:01.885160 10671 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:01.860865 13832 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.874027  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:01.890049 10642 connection.cc:256] RPC call timeout handler was delayed by -1.0491740939996816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:25:01.902424  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:01.890081 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:01.903110 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:01.895848 13925 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:25:01.899173 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:01.884289 13535 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:01.859665 13235 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.925045 10090 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:01.872860 13166 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:01.904538 13878 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:01.904843 13692 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:01.905217 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:01.905453 13583 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:01.908098 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:01.908739 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:01.909246 10676 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:01.910815 10095 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:01.900425 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:01.938508 10091 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:01.915908 10097 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:01.898910 13887 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:01.904352 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:01.925786 13847 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.933246 13848 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:01.934062 13890 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:01.935004 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 772) took 1069 ms. Trace:
I20260709 12:25:01.935487 13589 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:01.951392 10677 rpcz_store.cc:276] 0709 12:25:00.865475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.865596 (+   121us) service_pool.cc:224] Handling call
0709 12:25:00.866043 (+   447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.028749 (+162706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:01.028792 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:01.028854 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:01.028904 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:01.934761 (+905857us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b54004d49d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:01.934814 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:01.934941 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":373594880}
I20260709 12:25:01.887699 13306 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:01.943037 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:01.914290 13756 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:01.940027 13583 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:01.946951 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:01.947568  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:01.948217 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:01.948797 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44553.4us
W20260709 12:25:01.969784 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.952440 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:01.939294 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:01.959626 13907 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:01.954535 13166 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.968361  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:01.972666 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:01.949465  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 75529.2us
I20260709 12:25:01.975780  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:01.938558 13878 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.978879 13878 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:01.986058 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:01.986449  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:01.987532 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:01.994980 13166 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.966367 13773 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:01.947129 13917 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:01.995760 13773 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:01.999037  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:01.950940 13692 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:01.998440 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:01.953653 13887 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:02.008041 13878 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.008895 13887 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.010630 13878 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.966162 13848 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:02.012490 13848 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:02.013748 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.014806 13872 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:01.996882 13881 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:01.998487 13907 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:02.016372 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.016546 13907 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.016943 13907 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:01.966493 13903 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:25:02.002722 13692 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:01.998503 13880 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.020743 13880 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.021229 13880 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.005748 13583 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.022333 13583 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:01.968217 13589 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.000089 13900 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:01.998564 13890 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:01.985625 13235 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:01.996114 13917 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:01.984061 13847 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:02.000361 13858 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:01.998471 13773 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.026691 13900 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.027729 13890 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.031400 13728 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:02.034154 13306 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.030171 13917 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.058784 13431 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.061103 13431 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.062026 10097 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:02.030035 13847 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:02.062947 13847 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.033169 13235 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:02.031499 13858 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:02.039528 13900 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.058498 13306 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:02.077791 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.071473 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45884.7us
I20260709 12:25:02.041946 13890 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:02.081730  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.074275 13431 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 12:25:02.131652 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 183510us
I20260709 12:25:02.198201  9912 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.116459 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:02.200055  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:02.112602 10640 leader_election.cc:336] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.681s (SENT)
I20260709 12:25:02.201304 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:02.202243 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 89703us
I20260709 12:25:02.105394 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2914) took 2777 ms. Trace:
I20260709 12:25:02.117889 10677 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:02.131563 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:02.203960  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.204698 10099 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:02.200371 10676 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:02.163318 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.202997 10093 rpcz_store.cc:276] 0709 12:24:59.328316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:59.328459 (+   143us) service_pool.cc:224] Handling call
0709 12:24:59.348157 (+ 19698us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.106117 (+1757960us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.105066 (+998949us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:02.105287 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3851130752}
I20260709 12:25:02.072218 13858 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:02.211943  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 1.36252e+06us
I20260709 12:25:02.213017  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:02.101704 13938 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:25:02.225003 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:02.228497 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.229362 10677 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.081089 13166 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.229537 10093 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:02.080897 13306 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:02.223573 13940 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:25:02.266235 10640 leader_election.cc:336] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.550s (SENT)
I20260709 12:25:02.297681 13309 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:25:02.300228 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.289994 13850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.385s	user 0.027s	sys 0.027s
I20260709 12:25:02.233115 13878 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:25:02.222426 13881 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.214293 13942 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:02.287822 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.099s (SENT) [suppressed 26 similar messages]
I20260709 12:25:02.288579 10676 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:02.207058 13889 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.213975 13535 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.289361  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:02.292094  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.267376 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:02.216161 13849 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:25:02.310220 10099 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:02.235631 13166 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:02.306951 13850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.409s	user 0.028s	sys 0.028s
I20260709 12:25:00.111148  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:02.378612 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.223336 13235 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.337884 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:02.391724 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53961.9us
I20260709 12:25:02.390065  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:02.392592 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.394098 10640 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.870s (SENT)
W20260709 12:25:02.351109  9839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 212) took 18592 ms (client timeout 15000 ms). Trace:
I20260709 12:25:02.346002 13831 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.350306  9843 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.328619 10677 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:02.310956 13881 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.397257 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:02.294687 13863 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
W20260709 12:25:02.388512  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 99209.4us
I20260709 12:25:02.394717 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:02.387975 10093 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.402143  9843 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 6 to 7, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.355270 13589 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.369241 13535 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.368721 13878 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.382129 13309 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.409101  9843 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.398813 13881 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.330653 13306 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.412036 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.400374 13942 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:02.391112 13848 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:02.421608 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.356284 13678 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:02.438264 10099 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:25:02.406364 13756 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:02.422135 13850 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:25:02.494482  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 281568us
I20260709 12:25:02.420992 13535 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.402066 13831 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.415009 13773 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:02.394948  9839 rpcz_store.cc:269] 0709 12:24:43.758485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:43.758649 (+   164us) service_pool.cc:224] Handling call
0709 12:25:02.350914 (+18592265us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:24:52.605713 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:24:52.854075 (+248362us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:24:52.854129 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:24:52.854147 (+    18us) tablet.cc:662] Decoding operations
0709 12:24:52.855931 (+  1784us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:24:52.856030 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:24:52.856056 (+    26us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:24:52.857892 (+  1836us) tablet.cc:701] Row locks acquired
0709 12:24:52.857939 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:24:52.858107 (+   168us) write_op.cc:270] Start()
0709 12:24:52.858252 (+   145us) write_op.cc:276] Timestamp: P: 1783599892858044 usec, L: 0
0709 12:24:52.858273 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:24:53.021598 (+163325us) log.cc:844] Serialized 9084 byte log entry
0709 12:24:53.661604 (+640006us) op_driver.cc:464] REPLICATION: finished
0709 12:24:54.066566 (+404962us) write_op.cc:301] APPLY: starting
0709 12:24:54.066712 (+   146us) tablet.cc:1367] starting BulkCheckPresence
0709 12:24:54.067356 (+   644us) tablet.cc:1370] finished BulkCheckPresence
0709 12:24:54.067535 (+   179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:24:54.573199 (+505664us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:24:54.573903 (+   704us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=23
0709 12:24:54.573958 (+    55us) write_op.cc:312] APPLY: finished
0709 12:24:54.574515 (+   557us) log.cc:844] Serialized 158 byte log entry
0709 12:24:54.863793 (+289278us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:54.905029 (+ 41236us) write_op.cc:454] Released schema lock
0709 12:24:54.964323 (+ 59294us) write_op.cc:341] FINISH: Updating metrics
0709 12:24:54.991016 (+ 26693us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:24:54.991046 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2533431,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":401679,"apply.run_cpu_time_us":30167,"apply.run_wall_time_us":924418,"mutex_wait_us":15287,"num_ops":23,"prepare.queue_time_us":775072,"prepare.run_cpu_time_us":23489,"prepare.run_wall_time_us":661135,"raft.queue_time_us":266901,"raft.run_cpu_time_us":49185,"raft.run_wall_time_us":209874,"replication_time_us":759222,"thread_start_us":1176805,"threads_started":5,"wal-append.queue_time_us":501035,"wal-append.run_cpu_time_us":58340,"wal-append.run_wall_time_us":2308889}]]}
I20260709 12:25:02.427065 13878 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.459826 13848 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:02.461351  9843 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 2 to 7, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.464012 13880 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.440623 13942 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.366989 13889 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.444733 13678 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:02.444643 13589 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:02.468269 13847 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.469280 10093 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:02.469878 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.545779 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.549081 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.549719 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:02.550665 13847 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.550873 10099 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260709 12:25:02.507827 13850 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:02.472775 13869 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:02.473027 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2849) took 1633 ms. Trace:
W20260709 12:25:02.557497 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85648.5us
W20260709 12:25:02.473449  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.27189e+06us
I20260709 12:25:02.559159  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:02.473894  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.26824e+06us
I20260709 12:25:02.559631  9843 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a reported cstate change: term changed from 4 to 7, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 7 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:02.474225 13742 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:02.561904 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.474524 13561 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:02.530017 13942 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.478101 13166 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:02.479278 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 584) took 1547 ms. Trace:
I20260709 12:25:02.481067  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:02.574355 13846 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.939s	user 0.005s	sys 0.011s
W20260709 12:25:02.574785 13846 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.006s	sys 0.011s
W20260709 12:25:02.481648 13932 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:02.574283 10094 rpcz_store.cc:276] 0709 12:25:00.932345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.932545 (+   200us) service_pool.cc:224] Handling call
0709 12:25:00.932888 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.934658 (+  1770us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:00.934690 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.205711 (+1271021us) spinlock_profiling.cc:243] Waited 17.4 ms on lock 0x7b2c00115cd8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:25:02.226953 (+ 21242us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.381613 (+154660us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:02.381678 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.381785 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.479001 (+ 97216us) raft_consensus.cc:1685] finished
0709 12:25:02.479020 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.479216 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1366,"raft.queue_time_us":1485,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":30458,"spinlock_wait_cycles":39981952,"thread_start_us":82465,"threads_started":1,"wal-append.queue_time_us":204716}
I20260709 12:25:02.575562  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.484619 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2931) took 1043 ms. Trace:
I20260709 12:25:02.576330 10090 rpcz_store.cc:276] 0709 12:25:01.441665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.545151 (+103486us) service_pool.cc:224] Handling call
0709 12:25:02.484423 (+939272us) spinlock_profiling.cc:243] Waited 377 ms on lock 0x7b54001928d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:02.484559 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":868158720}
I20260709 12:25:02.484827  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.485857  9946 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:02.486922 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.486985  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.487650  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.578176 10091 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:02.488533 13582 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.489203 13743 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:02.490353 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.437976 13309 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.491717  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1640) took 1144 ms. Trace:
I20260709 12:25:02.492336 13907 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.586031  9943 rpcz_store.cc:276] 0709 12:25:01.347124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.379717 (+ 32593us) service_pool.cc:224] Handling call
0709 12:25:02.491560 (+1111843us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400222ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:02.491660 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442578304}
I20260709 12:25:02.493134 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.493611 13789 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:02.497130 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.589428 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.498906 13874 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:25:02.499485 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.501572  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:02.503567  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 105248us
I20260709 12:25:02.535338 10672 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:02.590445 10095 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:02.590860 10672 raft_consensus.cc:606] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:25:02.536335 13306 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.537281 13844 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.539085 13889 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:02.538264 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2922) took 1698 ms (client timeout 2206 ms). Trace:
I20260709 12:25:02.542940  9940 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:25:02.592280 13954 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.592311 13864 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:02.552484 13943 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.382s	user 0.000s	sys 0.003s
I20260709 12:25:02.554561  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.592882 13954 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.594012 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2923) took 1753 ms. Trace:
I20260709 12:25:02.555356 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.594774 13830 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:02.595016  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:02.595183 10093 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:02.551468 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 588) took 1504 ms. Trace:
I20260709 12:25:02.585027 13704 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.472404 13900 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.567191  9843 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.603223 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.476189 13917 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.509356 13872 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:25:02.510773 13890 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.482965  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.612246  9949 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:02.579124  9941 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:02.578248  9944 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:02.571417 13728 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:02.582733 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.562351 13742 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:02.583782 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:02.587118 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:02.590176 13891 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.624279  9843 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 4 to 6, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 6 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:02.588459 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2929) took 1142 ms. Trace:
I20260709 12:25:02.626957 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.626559 10094 rpcz_store.cc:276] 0709 12:25:01.440774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.582130 (+1141356us) service_pool.cc:224] Handling call
0709 12:25:02.582396 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.582566 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.582592 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.582650 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:02.582691 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.582748 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.582824 (+    76us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.591660 13789 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:02.592276 10092 rpcz_store.cc:269] 0709 12:25:00.839481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.840012 (+   531us) service_pool.cc:224] Handling call
0709 12:25:02.538016 (+1698004us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540014f0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:02.538174 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276939008}
W20260709 12:25:02.595284 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.597326 10093 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:02.597326  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 95810.1us
I20260709 12:25:02.597678 10095 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:02.636554  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:02.637643  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:02.596082 13704 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.638876 13836 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.639204 10092 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:02.638953 13789 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.639959 13704 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.640625 10092 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:02.620160 13728 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.554622 13589 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.595749 10099 rpcz_store.cc:276] 0709 12:25:01.047378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.550290 (+1502912us) service_pool.cc:224] Handling call
0709 12:25:02.550603 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.551225 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.551416 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:02.567332 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:02.646950 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 79694.4us
I20260709 12:25:02.580720 13673 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:02.611486  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.594383 10096 rpcz_store.cc:276] 0709 12:25:00.840827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.840973 (+   146us) service_pool.cc:224] Handling call
0709 12:25:00.841330 (+   357us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:00.935355 (+ 94025us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:01.017503 (+ 82148us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.862445 (+844942us) spinlock_profiling.cc:243] Waited 8.16 ms on lock 0x7b180035b480. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:25:01.926955 (+ 64510us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.927357 (+   402us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.927570 (+   213us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:01.946464 (+ 18894us) log.cc:844] Serialized 52 byte log entry
0709 12:25:01.951038 (+  4574us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:01.951370 (+   332us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.955203 (+  3833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.593719 (+638516us) spinlock_profiling.cc:243] Waited 84.6 ms on lock 0x7b5400341f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:02.593764 (+    45us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.593806 (+    42us) raft_consensus.cc:1685] finished
0709 12:25:02.593819 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.593952 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6656,"prepare.queue_time_us":149206,"prepare.run_cpu_time_us":135,"prepare.run_wall_time_us":3586,"raft.queue_time_us":11014,"raft.run_cpu_time_us":490,"raft.run_wall_time_us":25134,"spinlock_wait_cycles":213403136,"thread_start_us":58414,"threads_started":2,"wal-append.queue_time_us":896428}
I20260709 12:25:02.617071  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.647634 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:02.649971  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.650614  9941 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:02.653270  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:02.618032 10065 leader_election.cc:400] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:02.583328 13834 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:02.557631 10024 rpcz_store.cc:276] 0709 12:25:00.839784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.839935 (+   151us) service_pool.cc:224] Handling call
0709 12:25:00.856796 (+ 16861us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:00.857047 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:01.020420 (+163373us) log.cc:844] Serialized 14 byte log entry
0709 12:25:01.195061 (+174641us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.472488 (+1277427us) spinlock_profiling.cc:243] Waited 6.59 ms on lock 0x7b180034f5a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:02.472651 (+   163us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.472702 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:02.472772 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.472837 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.472966 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1308561,"prepare.queue_time_us":332170,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":79,"raft.queue_time_us":1283341,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":10249,"spinlock_wait_cycles":15159680,"thread_start_us":264911,"threads_started":2,"wal-append.queue_time_us":218316,"wal-append.run_cpu_time_us":1454,"wal-append.run_wall_time_us":1026121}
I20260709 12:25:02.595367 13830 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:02.624390 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:02.665019 10670 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:02.629804 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2848) took 1790 ms. Trace:
I20260709 12:25:02.631009 13944 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:02.631095  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.632110 13756 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:02.596815 13874 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:02.640578 13704 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:25:02.595750  9947 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.641420 10094 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:02.605118 13849 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:02.598434  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:02.641974  9843 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 7, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.605695 13861 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:02.649518 13879 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:02.616405 13522 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:25:02.654870 10065 leader_election.cc:403] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:02.655413 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:02.655548 13673 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:02.655977 13833 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:02.584823 13582 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.640408 13789 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.625378 13889 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:02.590974 13743 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:02.665967  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48975.2us
I20260709 12:25:02.701982  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.593025 13864 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.595099 13858 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.668807  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.672435  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.669637 13836 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:02.669669 13944 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:02.677708 13874 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:25:02.620996 13404 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.046s
I20260709 12:25:02.649641 13728 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.642223 13891 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:02.684029 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68203.6us
I20260709 12:25:02.683673  9843 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 5 to 8, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 8 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.687150 13561 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:02.725768  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 139883us
I20260709 12:25:02.665478 10019 rpcz_store.cc:276] 0709 12:25:00.839498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.844818 (+  5320us) service_pool.cc:224] Handling call
0709 12:25:00.845124 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:00.853657 (+  8533us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:01.283672 (+430015us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b18001c7560. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469557 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:02.215807 (+932135us) spinlock_profiling.cc:243] Waited 5.73 ms on lock 0x7b18001c7560. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:25:02.362676 (+146869us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.493536 (+130860us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.494249 (+   713us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.494410 (+   161us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:02.494441 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.494504 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:02.629570 (+135066us) raft_consensus.cc:1685] finished
0709 12:25:02.629588 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.629746 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1277256,"prepare.queue_time_us":220321,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":73875,"raft.run_cpu_time_us":316,"raft.run_wall_time_us":13837,"spinlock_wait_cycles":55697024,"thread_start_us":204225,"threads_started":2,"wal-append.queue_time_us":155124}
I20260709 12:25:02.739715 10098 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:02.633514 13875 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 12:25:02.692044  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.601089 13848 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.676803 13704 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:02.704550 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.704696  9949 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:02.674080 13756 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:02.680698 13861 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.742183 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:02.742818 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.591444 13678 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:02.721001 13309 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.721323 13917 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.711980 13836 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:02.708015 13773 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.718303 13944 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.816324 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.817442 10098 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:02.723248 13589 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.722898 13727 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:02.724416 13890 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.684067 13849 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.687391 13844 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:02.629186 13306 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:25:02.817061 13836 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:02.692034 13523 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:02.697650 13879 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:02.699890 13833 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:02.696910 13673 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:02.740375 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.831668 13678 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:02.696175 13889 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:02.741319 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 112732us
I20260709 12:25:02.741743 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:02.608542 13850 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.711128 13876 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:02.787288 10094 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:02.799222  9843 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 5, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 5 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.866827 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:02.799376 10670 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:02.867547 10670 raft_consensus.cc:606] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:25:02.868085 10670 raft_consensus.cc:614] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:02.867743 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.785732 10099 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:02.868835 10098 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:02.728327 13830 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.873243  9843 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:02.724612 13742 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.727120 13512 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.727402 13582 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.785560 13704 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:02.728091 13900 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.803130 13756 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:02.832829  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:02.833920  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 196349us
I20260709 12:25:02.789997 13878 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.809970 13861 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.712291 13847 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 12:25:02.958292  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:02.843631 13166 raft_consensus.cc:993] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: : Instructing follower cd983415389444e292b2330ef630850a to start an election
I20260709 12:25:02.845012 13743 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.846175 13849 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:02.728410  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1873) took 1384 ms. Trace:
I20260709 12:25:02.846948  9947 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:02.978925  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1638) took 1794 ms. Trace:
I20260709 12:25:02.979243  9947 rpcz_store.cc:276] 0709 12:25:01.184476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.184640 (+   164us) service_pool.cc:224] Handling call
0709 12:25:02.594934 (+1410294us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431f8c 00007f4e06431ef6 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:02.978844 (+383910us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3317760}
W20260709 12:25:02.836974  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 169933us
I20260709 12:25:02.943472 13880 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:02.851094 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 110802us
I20260709 12:25:02.675369 13834 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:02.856729 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:02.993278 10019 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:02.857502 10096 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:02.857493 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:02.994169 10017 raft_consensus.cc:2410] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:25:02.859874 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:02.994740 10022 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:25:02.860160 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:02.860873  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:02.863306 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:02.868988 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:02.996666 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.996692 10021 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:02.997754 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:02.799268 10092 raft_consensus.cc:2368] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11.
I20260709 12:25:02.867575 10094 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:02.866824 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:02.808898  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105667us
I20260709 12:25:02.879518 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:02.999657 10016 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:02.880183 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:02.880963 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:02.881989 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.884280 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:02.884974  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1865) took 1667 ms (client timeout 1773 ms). Trace:
I20260709 12:25:02.887027 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:02.861701 13889 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:02.994457 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:02.861423 13890 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:02.891219 13742 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:02.896900 13945 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.570s	user 0.001s	sys 0.002s
I20260709 12:25:02.860543 13907 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:02.903406  9843 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 1 to 5, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 5 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:02.914425 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:02.848340 13844 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:03.051470 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:03.053282 10671 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:02.920989 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 906) took 1160 ms. Trace:
I20260709 12:25:02.866197 13876 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:02.922565 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2846) took 2083 ms. Trace:
I20260709 12:25:02.925493 13831 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:02.936523 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.060201 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:02.918628 13917 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
W20260709 12:25:02.939327 13948 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.574s	user 0.002s	sys 0.000s
I20260709 12:25:02.943301 13858 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:02.945201  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:02.955430  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 216703us
I20260709 12:25:02.951079 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.080050 10093 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:02.972455 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2935) took 1307 ms. Trace:
I20260709 12:25:02.977849  9942 rpcz_store.cc:276] 0709 12:25:01.344147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.396453 (+ 52306us) service_pool.cc:224] Handling call
0709 12:25:01.396750 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.634933 (+238183us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.728170 (+1093237us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54000159d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:02.728349 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":537499008}
I20260709 12:25:02.988828 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:02.991029 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:03.081416 10670 raft_consensus.cc:2376] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 9. Re-sending same reply.
I20260709 12:25:03.081740 10673 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:02.998989 10676 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.082628 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:02.983227 13512 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.004782  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:03.005707  9990 leader_election.cc:400] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:03.006528  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:03.091365  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84918.2us
W20260709 12:25:03.001546  9940 rpcz_store.cc:269] 0709 12:25:01.217734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.224769 (+  7035us) service_pool.cc:224] Handling call
0709 12:25:02.884792 (+1660023us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:02.884910 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2943594752}
I20260709 12:25:03.010835 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:03.011472  9945 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:03.011538  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:02.983430 13961 raft_consensus.cc:1081] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Signalling peer cd983415389444e292b2330ef630850a to start an election
W20260709 12:25:03.100533  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.101195 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:03.102214 10671 raft_consensus.cc:2368] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10.
I20260709 12:25:02.995790 10025 raft_consensus.cc:2376] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8. Re-sending same reply.
I20260709 12:25:03.013975 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:03.104254 10092 raft_consensus.cc:2368] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9.
I20260709 12:25:03.104486 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.105351 10025 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:03.105695 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.106159 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:02.966107 13754 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.106809 10025 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:03.106948 13754 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.107698 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.014535 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:03.108752 10094 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:03.108812 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.109787 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.109921 10025 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:03.037286 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.110776 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.110975 10022 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:03.111429 10025 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:03.111821 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:03.112112 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.112527 10022 raft_consensus.cc:2368] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:03.113266 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.112653 10025 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:03.114230 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.120072  9941 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.048264 13834 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:03.054615 10674 rpcz_store.cc:276] 0709 12:25:01.760306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.819596 (+ 59290us) service_pool.cc:224] Handling call
0709 12:25:02.920719 (+1101123us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b5400610dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:02.920903 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16418048}
I20260709 12:25:03.056569 10018 rpcz_store.cc:276] 0709 12:25:00.838869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:00.838978 (+   109us) service_pool.cc:224] Handling call
0709 12:25:00.855355 (+ 16377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:01.538574 (+683219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.922311 (+1383737us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54001c5554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:02.922501 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1559566080}
I20260709 12:25:03.063926 10675 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:02.994669 13826 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.009960 13833 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.013837 13444 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:25:03.010869 13890 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.059384 13876 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:03.073812 13881 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.627s	user 0.008s	sys 0.000s
I20260709 12:25:03.073702 13966 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:03.074919  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:03.198381  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 123547us
I20260709 12:25:03.075531 13861 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.011866 13900 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:25:03.080961 10090 rpcz_store.cc:276] 0709 12:25:01.664691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.579626 (+914935us) service_pool.cc:224] Handling call
0709 12:25:02.581611 (+  1985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.581820 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:02.809034 (+227214us) log.cc:844] Serialized 14 byte log entry
0709 12:25:02.971967 (+162933us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:02.972040 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:02.972117 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.972209 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:02.972381 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11437,"prepare.queue_time_us":352617,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":5773,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":21991,"thread_start_us":331298,"threads_started":2}
I20260709 12:25:03.078086  9949 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:03.088701 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:03.090698  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 90871.4us
I20260709 12:25:03.204437 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:03.205112 10674 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:03.103416 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:02.992761 13727 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.106309 10092 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:03.206197 10670 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.207006 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.123705 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:03.208294 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84673.3us
I20260709 12:25:03.209154 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:02.966905 13728 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.014159 13965 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:03.051899 13306 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:03.115263 10099 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 7: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:03.114660 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.115974 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:03.120477  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:03.245339 10094 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:03.245569  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 125174us
W20260709 12:25:03.231967 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.125808  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:03.115705 13589 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:03.124598 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:03.126081 13874 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.280259 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.150024 10676 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:03.148650 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.125655 13512 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.336860 13512 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:03.173449 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.336944 13954 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:03.337730 10019 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:25:03.338769 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.212553 13881 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.339349 10019 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:03.179073 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2933) took 1536 ms. Trace:
I20260709 12:25:03.180536 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.340348 10017 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:03.050628 13879 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.186460 10023 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:03.186770 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:03.181761  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:03.191720 10022 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 6.
I20260709 12:25:03.191459  9843 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:03.361045  9942 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:03.120173 13758 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:03.120798 13864 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:03.201484 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:03.392249 13758 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:03.392554 10018 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:03.393498 13758 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:03.144878 13831 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:03.211663  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:03.394922 13831 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:03.395439  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 183847us
I20260709 12:25:03.396432  9990 leader_election.cc:400] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11.
I20260709 12:25:03.397154 13864 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:03.397583 13831 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:03.340669 13966 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:03.400898 13966 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:03.401944 13966 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.340507 13589 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:03.405854 13589 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:03.253391 13891 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:03.251883 13942 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.000s
I20260709 12:25:03.407583 13942 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.216022 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.222070 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:03.194892 13890 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:03.255172 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 130629us
I20260709 12:25:03.420711 10092 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:25:03.141285 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:03.459615 10671 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:03.219769 13727 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:03.183043 13944 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.251616 13235 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:03.213099 13907 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.059520 13673 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.303391  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:03.304672 13874 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:03.297677 13754 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:03.213320 13832 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:03.309170 13952 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.947s	user 0.005s	sys 0.000s
I20260709 12:25:03.309665  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:03.312464 13728 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:03.312564 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.522266 10095 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:03.319470  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:03.320109 13836 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:03.524302 10668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.017s	sys 0.016s
I20260709 12:25:03.524596  9947 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:03.524691 10668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.017s	sys 0.016s
I20260709 12:25:03.323143 10094 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:03.322145 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:03.299744 13844 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:03.324595 10096 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
W20260709 12:25:03.327366  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 86121.3us
W20260709 12:25:03.327854  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78925.5us
I20260709 12:25:03.333103 13861 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:03.333724  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:03.341943 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:03.340075 10091 rpcz_store.cc:276] 0709 12:25:01.642283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.578969 (+936686us) service_pool.cc:224] Handling call
0709 12:25:02.579274 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.579487 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:02.852511 (+273024us) log.cc:844] Serialized 14 byte log entry
0709 12:25:03.178368 (+325857us) log.cc:844] Serialized 14 byte log entry
0709 12:25:03.178577 (+   209us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:03.178650 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:03.178731 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.178831 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.178990 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21335,"prepare.queue_time_us":354878,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":21555,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":75286528,"thread_start_us":561459,"threads_started":2}
I20260709 12:25:03.287436 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:03.293879 10670 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:03.282785 13872 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:03.385778  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:03.340021 13404 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.233s
I20260709 12:25:03.421370 10674 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:25:03.334403 13535 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.778s	user 0.009s	sys 0.000s
I20260709 12:25:03.249385 13965 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.250711 13789 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:03.498451 10672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.007s	sys 0.025s
I20260709 12:25:03.326314 13889 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:03.359679 13917 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:03.214655 13704 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:03.391577 13970 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
W20260709 12:25:03.662096 10641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.030s	sys 0.045s
I20260709 12:25:03.609884 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.527307  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:03.758723 10097 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.014s	sys 0.013s
I20260709 12:25:03.525593 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 784) took 1864 ms. Trace:
I20260709 12:25:03.496120 13235 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:03.538540 13756 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.000s
W20260709 12:25:03.759131 13773 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.018s	sys 0.016s
W20260709 12:25:03.670977 13850 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.018s	sys 0.022s
W20260709 12:25:03.671309 13869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.034s	sys 0.038s
I20260709 12:25:03.514861 13907 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.771957 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:03.836910 13830 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.014s	sys 0.042s
W20260709 12:25:03.647413 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.033s	sys 0.041s
W20260709 12:25:03.725802 13848 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.014s	sys 0.024s
W20260709 12:25:03.781852 13972 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:25:03.802489 13881 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:03.953946  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:03.954094  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.954746 10673 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.742s	user 0.013s	sys 0.001s
I20260709 12:25:03.955320  9945 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
W20260709 12:25:03.955758 10097 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.014s	sys 0.022s
I20260709 12:25:03.983615 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2925) took 2811 ms. Trace:
I20260709 12:25:03.983822  9949 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:03.956143  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 559825us
I20260709 12:25:03.985121  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:03.986030  9944 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:03.986189  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:03.950510  9941 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.804s	user 0.007s	sys 0.000s
I20260709 12:25:03.954895 10095 raft_consensus.cc:2368] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
W20260709 12:25:03.725944 13949 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.369s	user 0.000s	sys 0.005s
I20260709 12:25:03.989552  9945 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:03.959435 13235 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:03.959826 13756 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:03.592875 13878 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.016s	sys 0.042s
W20260709 12:25:03.993217 13878 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.020s	sys 0.063s
W20260709 12:25:03.961005 13869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.263s	user 0.037s	sys 0.040s
W20260709 12:25:03.960079 13773 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.025s	sys 0.018s
I20260709 12:25:03.958263 13889 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:03.957122 13858 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:03.996176  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:03.997579  9990 leader_election.cc:400] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9.
I20260709 12:25:03.997912  9990 leader_election.cc:403] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:25:03.983992 10097 rpcz_store.cc:276] 0709 12:25:01.172410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.552834 (+1380424us) service_pool.cc:224] Handling call
0709 12:25:02.553130 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:02.553356 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:02.893387 (+340031us) log.cc:844] Serialized 14 byte log entry
0709 12:25:03.180873 (+287486us) log.cc:844] Serialized 14 byte log entry
0709 12:25:03.983214 (+802341us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:03.983274 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:03.983349 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.983424 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.983554 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":355076,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":100,"raft.queue_time_us":71337,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":91497,"spinlock_wait_cycles":103150720,"thread_start_us":1428555,"threads_started":3,"wal-append.queue_time_us":700234}
I20260709 12:25:03.999907 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1806) took 1210 ms. Trace:
I20260709 12:25:04.000566 10095 rpcz_store.cc:276] 0709 12:25:02.789802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.609273 (+819471us) service_pool.cc:224] Handling call
0709 12:25:03.999823 (+390550us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:03.960762 13850 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.021s	sys 0.029s
I20260709 12:25:03.944468 13944 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:03.955530 13826 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.769s	user 0.009s	sys 0.001s
I20260709 12:25:03.987744  9941 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:03.956315 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.003031  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 740) took 1444 ms. Trace:
I20260709 12:25:03.525723 13836 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:03.951527 13977 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:25:04.007488 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:03.994391 13756 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:03.986785 13881 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:04.020491 10097 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:04.021090  9945 rpcz_store.cc:276] 0709 12:25:02.545641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.988959 (+1443318us) service_pool.cc:224] Handling call
0709 12:25:03.989334 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:03.990039 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.990265 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:03.958150 10676 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:04.022037 10093 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:03.960706 13966 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:03.965118 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1804) took 1175 ms. Trace:
W20260709 12:25:03.963182 10672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.016s	sys 0.029s
W20260709 12:25:03.965955 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.039s	sys 0.049s
I20260709 12:25:03.966923 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:04.029559 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.03513e+06us
I20260709 12:25:03.963104 10675 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.029838 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 600) took 1139 ms. Trace:
I20260709 12:25:04.030524 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1822) took 1100 ms. Trace:
I20260709 12:25:04.002336 13874 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:03.963873  9940 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:25:04.033612 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:03.958431 10668 rpcz_store.cc:276] 0709 12:25:01.661483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:01.661634 (+   151us) service_pool.cc:224] Handling call
0709 12:25:01.671688 (+ 10054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:01.671904 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:02.945945 (+1274041us) spinlock_profiling.cc:243] Waited 1.6 ms on lock 0x7b1800321be0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf
0709 12:25:02.946090 (+   145us) log.cc:844] Serialized 14 byte log entry
0709 12:25:03.525178 (+579088us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:03.525234 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:03.525294 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.525366 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.525519 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":108449,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1106,"raft.queue_time_us":101,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":82,"spinlock_wait_cycles":3676032,"thread_start_us":678273,"threads_started":2}
I20260709 12:25:04.034222 10676 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:04.030211  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:04.034608  9944 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:03.963086 10098 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:03.975948 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.031993 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:04.030376 10019 rpcz_store.cc:276] 0709 12:25:02.890192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.966692 (+1076500us) service_pool.cc:224] Handling call
0709 12:25:04.029779 (+ 63087us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:03.970502 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.031085 10093 rpcz_store.cc:276] 0709 12:25:02.929817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.956049 (+1026232us) service_pool.cc:224] Handling call
0709 12:25:04.030466 (+ 74417us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.037063 10021 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:04.037318 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.037571 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1770) took 1060 ms. Trace:
W20260709 12:25:03.954397 10641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.030s	sys 0.060s
I20260709 12:25:04.038406  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1653) took 1535 ms. Trace:
I20260709 12:25:03.954303 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 797) took 1098 ms. Trace:
I20260709 12:25:04.039397 10677 rpcz_store.cc:276] 0709 12:25:02.856058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.858021 (+  1963us) service_pool.cc:224] Handling call
0709 12:25:02.858321 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.866909 (+  8588us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.866940 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.867362 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 12:25:03.258550 (+391188us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:03.258633 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:03.258735 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.702987 (+444252us) raft_consensus.cc:1685] finished
0709 12:25:03.703021 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:03.954209 (+251188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8980,"raft.queue_time_us":403,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":7577,"thread_start_us":340960,"threads_started":1,"wal-append.queue_time_us":800432}
I20260709 12:25:04.028615 10096 rpcz_store.cc:276] 0709 12:25:02.789227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.010554 (+221327us) service_pool.cc:224] Handling call
0709 12:25:03.965038 (+954484us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:03.957123 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1816) took 1029 ms. Trace:
I20260709 12:25:03.973496 10020 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 7: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:03.962289  9942 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:04.043354 10094 rpcz_store.cc:276] 0709 12:25:02.927965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.115735 (+187770us) service_pool.cc:224] Handling call
0709 12:25:03.957066 (+841331us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.043741  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1648) took 1562 ms. Trace:
I20260709 12:25:04.044341  9942 rpcz_store.cc:276] 0709 12:25:02.481650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.961806 (+1480156us) service_pool.cc:224] Handling call
0709 12:25:03.962121 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.043396 (+ 81275us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.043667 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.020853 13826 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:04.038877  9944 rpcz_store.cc:276] 0709 12:25:02.503180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.523874 (+1020694us) service_pool.cc:224] Handling call
0709 12:25:04.038329 (+514455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.037895 10021 rpcz_store.cc:276] 0709 12:25:02.976759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.970143 (+993384us) service_pool.cc:224] Handling call
0709 12:25:04.037516 (+ 67373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.039036 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:03.981667 13861 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.056540 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.057874 10097 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:04.059355 10096 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.062732 10019 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:04.064013 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.065019 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1771) took 1079 ms. Trace:
I20260709 12:25:04.065390 10019 rpcz_store.cc:276] 0709 12:25:02.985279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.063699 (+1078420us) service_pool.cc:224] Handling call
0709 12:25:04.064953 (+  1254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:03.956593 13872 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:03.971653 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.046767  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1882) took 1376 ms. Trace:
I20260709 12:25:04.047736 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:04.048516 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.075816 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:03.965328 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.077724 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1825) took 1098 ms. Trace:
W20260709 12:25:03.966001 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 678666us
W20260709 12:25:04.081683 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.972468 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.044956 10094 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.026111 10673 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:04.089087 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 925) took 1100 ms. Trace:
I20260709 12:25:04.067118 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:04.090644 10677 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:03.957670  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.091727  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:03.966382 13848 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.014s	sys 0.031s
I20260709 12:25:04.093886 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.094781 10676 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:25:04.098735  9940 rpcz_store.cc:276] 0709 12:25:02.670153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.333356 (+663203us) service_pool.cc:224] Handling call
0709 12:25:04.046681 (+713325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.078171 10091 rpcz_store.cc:276] 0709 12:25:02.979244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.965077 (+985833us) service_pool.cc:224] Handling call
0709 12:25:04.077638 (+112561us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:03.964300 13876 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.753s	user 0.008s	sys 0.000s
I20260709 12:25:04.104532 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:03.965631 13830 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.888s	user 0.014s	sys 0.046s
I20260709 12:25:04.110473 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:04.102873 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1762) took 1128 ms. Trace:
I20260709 12:25:04.110471 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.111543 10024 rpcz_store.cc:276] 0709 12:25:02.974630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.971386 (+996756us) service_pool.cc:224] Handling call
0709 12:25:04.102800 (+131414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.112510 10021 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:04.105834 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.101560 10675 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:04.113047 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 598) took 1223 ms. Trace:
I20260709 12:25:04.103564 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 929) took 1113 ms. Trace:
I20260709 12:25:03.976130 13789 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:04.124776 10668 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:04.129415 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.131584 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.112318 10097 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [7.4-11.5]
I20260709 12:25:04.132817 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1821) took 1203 ms. Trace:
I20260709 12:25:04.133249 10097 rpcz_store.cc:276] 0709 12:25:02.929526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.059420 (+1129894us) service_pool.cc:224] Handling call
0709 12:25:04.111956 (+ 52536us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:04.132436 (+ 20480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.132746 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.113111 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.031224 13404 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:04.113598 10021 rpcz_store.cc:276] 0709 12:25:02.889512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.046749 (+1157237us) service_pool.cc:224] Handling call
0709 12:25:04.112987 (+ 66238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.113937 10677 rpcz_store.cc:276] 0709 12:25:02.990255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.066346 (+1076091us) service_pool.cc:224] Handling call
0709 12:25:04.103492 (+ 37146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.139148 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:04.140393 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:04.089552 10673 rpcz_store.cc:276] 0709 12:25:02.989052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.088432 (+ 99380us) service_pool.cc:224] Handling call
0709 12:25:04.089003 (+1000571us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.107846 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:04.107568 10098 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:04.083048 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.011s	sys 0.020s
I20260709 12:25:03.970290 13965 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.119069 10096 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:25:04.108361 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 590) took 1244 ms. Trace:
I20260709 12:25:04.055190 13756 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:03.974738 13704 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:04.103267 13743 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.001s	sys 0.024s
I20260709 12:25:04.141453 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:04.141571 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.011s	sys 0.020s
I20260709 12:25:04.119671 10094 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:04.157366 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1819) took 1228 ms. Trace:
I20260709 12:25:04.068768 13773 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:04.046638 13961 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:04.105031 13977 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:04.148589 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 790) took 2033 ms. Trace:
I20260709 12:25:04.209025 10669 rpcz_store.cc:276] 0709 12:25:02.115176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.116461 (+  1285us) service_pool.cc:224] Handling call
0709 12:25:02.116837 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.133202 (+ 16365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.133237 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.133794 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 12:25:02.283987 (+150193us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:02.284028 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:02.284093 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:03.511740 (+1227647us) spinlock_profiling.cc:243] Waited 59.4 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:04.114349 (+602609us) raft_consensus.cc:1685] finished
0709 12:25:04.114379 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.148526 (+ 34147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7809,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":53736,"spinlock_wait_cycles":136574848,"thread_start_us":149814,"threads_started":1,"wal-append.queue_time_us":222195}
I20260709 12:25:03.971212 13942 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:04.140913 13826 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.211118 13942 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:04.211645 13942 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.213989 13942 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:04.152108 10019 rpcz_store.cc:276] 0709 12:25:02.864289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.093562 (+1229273us) service_pool.cc:224] Handling call
0709 12:25:04.108282 (+ 14720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.168618 13166 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:04.205255 13773 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.215050 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.215571 13773 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:04.215826 13961 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.218036 13961 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:04.135551 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 593) took 1270 ms. Trace:
I20260709 12:25:04.219933 10024 rpcz_store.cc:276] 0709 12:25:02.865349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.112882 (+1247533us) service_pool.cc:224] Handling call
0709 12:25:04.135466 (+ 22584us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.220867 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.221387 10024 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.074802 13889 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:04.153090 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1809) took 1337 ms. Trace:
I20260709 12:25:04.139503 13789 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:03.963420 13535 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.154969 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 930) took 1164 ms. Trace:
I20260709 12:25:04.236749 10677 rpcz_store.cc:276] 0709 12:25:02.990566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.140135 (+1149569us) service_pool.cc:224] Handling call
0709 12:25:04.154907 (+ 14772us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.238019 10677 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:04.238652 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 934) took 1229 ms. Trace:
I20260709 12:25:04.238960 10677 rpcz_store.cc:276] 0709 12:25:03.009077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.237497 (+1228420us) service_pool.cc:224] Handling call
0709 12:25:04.237807 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.238377 (+   570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.238576 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.240077 10677 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:04.240608 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 935) took 1208 ms. Trace:
I20260709 12:25:04.240907 10677 rpcz_store.cc:276] 0709 12:25:03.032120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.239669 (+1207549us) service_pool.cc:224] Handling call
0709 12:25:04.239921 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.240403 (+   482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.240562 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.242084 10677 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:03.964690 13907 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:04.157294 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 948164us
I20260709 12:25:04.147260 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1815) took 1219 ms. Trace:
I20260709 12:25:04.246315 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 799) took 1180 ms. Trace:
W20260709 12:25:04.158129 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82403.8us
I20260709 12:25:04.247139 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:04.248853 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:04.157873 10094 rpcz_store.cc:276] 0709 12:25:02.928953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.044405 (+1115452us) service_pool.cc:224] Handling call
0709 12:25:04.044798 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.157088 (+112290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.157306 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.157295 10673 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:04.250363 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 931) took 1255 ms. Trace:
I20260709 12:25:04.250757 10673 rpcz_store.cc:276] 0709 12:25:02.995431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.141226 (+1145795us) service_pool.cc:224] Handling call
0709 12:25:04.250286 (+109060us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.246869 10098 rpcz_store.cc:276] 0709 12:25:02.927630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.035954 (+1108324us) service_pool.cc:224] Handling call
0709 12:25:04.147190 (+111236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.252287 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:04.074010 13956 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.003s	sys 0.016s
I20260709 12:25:04.122308 13876 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:04.166347  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:04.175400 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.176703  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:04.178013 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.179250  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:04.182279 13875 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:04.183697 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:04.186009  9946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1649) took 1704 ms (client timeout 1805 ms). Trace:
I20260709 12:25:04.186872  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1881) took 1534 ms. Trace:
I20260709 12:25:04.188930 13917 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.158352  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:04.030992 13881 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.110340 13728 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:04.193444 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.167507 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 800) took 1075 ms. Trace:
I20260709 12:25:04.199610 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:04.120074 13493 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:04.208007 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:04.204197 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.122047 13754 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.226006 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.226275 10096 rpcz_store.cc:276] 0709 12:25:02.815331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.058833 (+1243502us) service_pool.cc:224] Handling call
0709 12:25:04.059211 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.152791 (+ 93580us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.153028 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.298225 10018 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.074209 13872 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.301823  9941 rpcz_store.cc:276] 0709 12:25:02.651955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.812293 (+160338us) service_pool.cc:224] Handling call
0709 12:25:04.186672 (+1374379us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b5400189cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.186811 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460584704}
I20260709 12:25:04.075133 13861 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.138268 13404 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:04.210625 13743 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.810s	user 0.002s	sys 0.025s
I20260709 12:25:04.311839 10671 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:04.160363 10097 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:04.253047 10094 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:04.165268 13523 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:04.299108 10021 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 6.
I20260709 12:25:04.392040 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1777) took 1275 ms. Trace:
I20260709 12:25:04.392524 10021 rpcz_store.cc:276] 0709 12:25:03.117069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.138484 (+1021415us) service_pool.cc:224] Handling call
0709 12:25:04.391959 (+253475us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:04.299957  9946 rpcz_store.cc:269] 0709 12:25:02.481891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.596531 (+114640us) service_pool.cc:224] Handling call
0709 12:25:04.185774 (+1589243us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400206f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.185941 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3452304640}
I20260709 12:25:04.298177  9940 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:04.300947  9944 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.271643  9948 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:04.301126  9942 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:04.258972  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1883) took 1527 ms. Trace:
I20260709 12:25:04.394997  9947 rpcz_store.cc:276] 0709 12:25:02.731194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.980310 (+249116us) service_pool.cc:224] Handling call
0709 12:25:04.258671 (+1278361us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b5400222ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.258884 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990525184}
I20260709 12:25:04.230412 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 933) took 1221 ms. Trace:
I20260709 12:25:04.299680 10675 rpcz_store.cc:276] 0709 12:25:03.091700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.183354 (+ 91654us) service_pool.cc:224] Handling call
0709 12:25:04.167301 (+983947us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54007602d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.167437 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1385467264}
I20260709 12:25:04.229242 13889 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:04.304250 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 344415us
W20260709 12:25:04.373660 13742 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.316s	user 0.009s	sys 0.003s
I20260709 12:25:04.305317 10670 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:04.246978 10677 rpcz_store.cc:276] 0709 12:25:03.062212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.241564 (+1179352us) service_pool.cc:224] Handling call
0709 12:25:04.241798 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.242388 (+   590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.242552 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.301247 13875 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:04.252456 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 803) took 1158 ms. Trace:
I20260709 12:25:04.413576 10673 rpcz_store.cc:276] 0709 12:25:03.094202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.251592 (+1157390us) service_pool.cc:224] Handling call
0709 12:25:04.251901 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.252121 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.252147 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.252188 (+    41us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:04.252223 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.252279 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.252399 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.414309 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 805) took 1313 ms. Trace:
I20260709 12:25:04.414952 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:04.344302 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 795) took 1488 ms. Trace:
I20260709 12:25:04.422149 10673 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:04.421981 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:04.427322 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:04.205093 13756 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.299979 13493 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.317188 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 927) took 1327 ms. Trace:
I20260709 12:25:04.487174 10671 rpcz_store.cc:276] 0709 12:25:02.989672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.207680 (+1218008us) service_pool.cc:224] Handling call
0709 12:25:04.317092 (+109412us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.488086 10671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:04.488472 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.341415 10023 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:04.489069 10095 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.489228 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1781) took 1371 ms. Trace:
I20260709 12:25:04.489576 10023 rpcz_store.cc:276] 0709 12:25:03.118112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.193143 (+1075031us) service_pool.cc:224] Handling call
0709 12:25:04.489169 (+296026us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.365880 10024 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:04.490325 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1782) took 1372 ms. Trace:
I20260709 12:25:04.492774 10024 rpcz_store.cc:276] 0709 12:25:03.118493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.220631 (+1102138us) service_pool.cc:224] Handling call
0709 12:25:04.490277 (+269646us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.215059 13166 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.352535 13535 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:04.360524 13985 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:25:04.377364 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.386212  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:04.370579 10018 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:04.506992 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1783) took 1301 ms. Trace:
I20260709 12:25:04.507511 10018 rpcz_store.cc:276] 0709 12:25:03.205935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.192799 (+986864us) service_pool.cc:224] Handling call
0709 12:25:04.506904 (+314105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:04.386135  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 388723us
I20260709 12:25:04.395848 10669 rpcz_store.cc:276] 0709 12:25:03.008764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.210546 (+1201782us) service_pool.cc:224] Handling call
0709 12:25:04.229730 (+ 19184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.229965 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.230004 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.230059 (+    55us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:04.230112 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.230182 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.230341 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.397030 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1823) took 1467 ms. Trace:
I20260709 12:25:04.409118 13889 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.541425 10024 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:25:04.541946 10024 pending_rounds.cc:85] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 8
I20260709 12:25:04.542224 10024 pending_rounds.cc:107] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:25:04.400260 10016 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:04.510406  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:04.543334 10676 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:04.227041 13986 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:04.408296 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:04.386826 13983 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:25:04.415001 10670 rpcz_store.cc:276] 0709 12:25:03.100484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.199220 (+1098736us) service_pool.cc:224] Handling call
0709 12:25:04.414226 (+215006us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.425267  9944 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:04.311151 13404 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:04.372795 13987 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:04.310073 13754 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.372308 13907 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:04.485905  9912 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.024s	sys 0.052s
W20260709 12:25:04.486378 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 589) took 1622 ms (client timeout 1870 ms). Trace:
I20260709 12:25:04.310704 13917 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.416044 10672 rpcz_store.cc:276] 0709 12:25:02.855506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.865468 (+  9962us) service_pool.cc:224] Handling call
0709 12:25:02.865865 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:02.866256 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:02.866282 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:02.879510 (+ 13228us) log.cc:844] Serialized 26 byte log entry
0709 12:25:04.039249 (+1159739us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:04.039289 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.039355 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.344081 (+304726us) raft_consensus.cc:1685] finished
0709 12:25:04.344100 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.344238 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20509,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":64459,"thread_start_us":1108310,"threads_started":1,"wal-append.queue_time_us":1314933}
I20260709 12:25:04.570338  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1884) took 1490 ms. Trace:
I20260709 12:25:04.553479 13704 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:04.490417 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.585871 10023 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 11 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:25:04.394806 13979 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.491158 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.252151 13961 raft_consensus.cc:2764] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:25:04.254410 13956 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.762s	user 0.004s	sys 0.016s
I20260709 12:25:04.495060 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:04.486232 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:04.298409 13980 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.116s
I20260709 12:25:04.511237 10097 rpcz_store.cc:276] 0709 12:25:02.930097 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.134126 (+1204029us) service_pool.cc:224] Handling call
0709 12:25:04.160122 (+ 25996us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.384291 (+224169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.396977 (+ 12686us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.507391 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.650048 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.651351 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.652160 10094 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:04.542498 10024 raft_consensus.cc:2889] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.544395  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:04.653249  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108981us
I20260709 12:25:04.654068  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:04.373741 13773 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:04.583420 13942 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.657997 13942 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.659125 13773 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:04.549938  9941 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:04.550689  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 744) took 1346 ms. Trace:
W20260709 12:25:04.551704  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 173841us
I20260709 12:25:04.491566 13523 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:04.552620 10668 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:04.495728 13493 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.561538 10670 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:04.566529  9912 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.024s	sys 0.053s
I20260709 12:25:04.504755 13826 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9.
I20260709 12:25:04.573339 13907 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.675380 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:04.688642  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 718908us
I20260709 12:25:04.488596 10671 raft_consensus.cc:606] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:04.566617 10020 rpcz_store.cc:269] 0709 12:25:02.863990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.864113 (+   123us) service_pool.cc:224] Handling call
0709 12:25:04.486168 (+1622055us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54002d5d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.486318 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2236461440}
I20260709 12:25:04.689354 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 821) took 1567 ms. Trace:
I20260709 12:25:04.586340 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1785) took 1293 ms. Trace:
I20260709 12:25:04.577531  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1887) took 1457 ms. Trace:
W20260709 12:25:04.568256 13890 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.072s	user 0.013s	sys 0.000s
I20260709 12:25:04.558008 13833 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:04.590272 10021 raft_consensus.cc:2376] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11. Re-sending same reply.
I20260709 12:25:04.687500  9990 leader_election.cc:400] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:04.594678 10675 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.596143 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.690567 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1763) took 1715 ms. Trace:
I20260709 12:25:04.691865  9990 leader_election.cc:400] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:25:04.691974 10021 rpcz_store.cc:276] 0709 12:25:02.975063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.395822 (+1420759us) service_pool.cc:224] Handling call
0709 12:25:04.690506 (+294684us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.692049 10025 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:25:04.598403 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.607018 13944 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:04.611174 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2953) took 1507 ms. Trace:
I20260709 12:25:04.618849 13589 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:04.624711 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.703488 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 599) took 1813 ms. Trace:
I20260709 12:25:04.703960 10017 rpcz_store.cc:276] 0709 12:25:02.889830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.624444 (+1734614us) service_pool.cc:224] Handling call
0709 12:25:04.703405 (+ 78961us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.705183 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.625339  9947 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:04.630895 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2954) took 1527 ms. Trace:
I20260709 12:25:04.630785  9940 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a"
dest_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:38784
W20260709 12:25:04.631829  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1654) took 2123 ms (client timeout 2666 ms). Trace:
I20260709 12:25:04.707165 10092 rpcz_store.cc:276] 0709 12:25:03.103675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.304342 (+1200667us) service_pool.cc:224] Handling call
0709 12:25:04.304690 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.411846 (+107156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.411878 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.411934 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:04.411985 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.630654 (+218669us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b5400163354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:04.630698 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.630836 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245838080}
I20260709 12:25:04.709756 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.710501 10092 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.702507 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:04.708287  9948 rpcz_store.cc:269] 0709 12:25:02.508296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.309321 (+801025us) service_pool.cc:224] Handling call
0709 12:25:04.631650 (+1322329us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b54001a36d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.631774 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1741064448}
W20260709 12:25:04.712610  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:04.713505  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.635124 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:04.635536 13728 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:04.660320 13983 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:25:04.639213 13727 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:04.654520 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:04.688930  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.562387 13844 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:04.661625 10091 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:04.693029 10025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1774) took 1576 ms (client timeout 2054 ms). Trace:
I20260709 12:25:04.692889  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:04.690073 10023 rpcz_store.cc:276] 0709 12:25:03.293438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.490189 (+1196751us) service_pool.cc:224] Handling call
0709 12:25:04.586280 (+ 96091us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.660917 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 594) took 1795 ms. Trace:
I20260709 12:25:04.739681 10024 rpcz_store.cc:276] 0709 12:25:02.865591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.505031 (+1639440us) service_pool.cc:224] Handling call
0709 12:25:04.505473 (+   442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.660604 (+155131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.660850 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.687984  9944 rpcz_store.cc:276] 0709 12:25:03.204022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.549813 (+1345791us) service_pool.cc:224] Handling call
0709 12:25:04.550065 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.550242 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.550279 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.550332 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:04.550386 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.550456 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.550613 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.689951 10670 rpcz_store.cc:276] 0709 12:25:03.121670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.549604 (+1427934us) service_pool.cc:224] Handling call
0709 12:25:04.549879 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.689065 (+139186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.689280 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.742249 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:04.742961 10670 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:25:04.689512 10671 raft_consensus.cc:614] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:04.659791 13995 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:04.738391 10025 rpcz_store.cc:269] 0709 12:25:03.116289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.595794 (+1479505us) service_pool.cc:224] Handling call
0709 12:25:04.692970 (+ 97176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:04.744979 10016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 601) took 1833 ms (client timeout 2117 ms). Trace:
I20260709 12:25:04.696202 10096 rpcz_store.cc:276] 0709 12:25:03.103140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.298474 (+1195334us) service_pool.cc:224] Handling call
0709 12:25:04.298854 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.299060 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:04.600473 (+301413us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.608873 (+  8400us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.609294 (+   421us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:04.609441 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.609958 (+   517us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.610847 (+   889us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":322456,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":19702,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":304370,"threads_started":2}
I20260709 12:25:04.671970 13673 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:04.690733  9943 rpcz_store.cc:276] 0709 12:25:03.119965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.329470 (+1209505us) service_pool.cc:224] Handling call
0709 12:25:04.335983 (+  6513us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.336214 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:04.336572 (+   358us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.576812 (+240240us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.576861 (+    49us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:04.576910 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.576969 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.577104 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5282,"prepare.queue_time_us":178,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":96,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":11224,"spinlock_wait_cycles":169856,"thread_start_us":239723,"threads_started":1}
I20260709 12:25:04.708320  9940 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:04.715538 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.722539 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 823) took 1522 ms. Trace:
I20260709 12:25:04.719959 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:04.671758 13989 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:25:04.565483 13986 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.732820 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.578537  9949 rpcz_store.cc:276] 0709 12:25:03.079581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.073420 (+993839us) service_pool.cc:224] Handling call
0709 12:25:04.073782 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.074002 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:04.074252 (+   250us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.570003 (+495751us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.570053 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:04.570098 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.570163 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.570281 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29428,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":745,"thread_start_us":495420,"threads_started":1}
I20260709 12:25:04.743446 10675 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:04.671386 13861 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:04.670334 13980 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:04.669863 13881 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:04.785734  9943 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:04.723223 13523 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:04.745612 10016 rpcz_store.cc:269] 0709 12:25:02.890539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.175072 (+1284533us) service_pool.cc:224] Handling call
0709 12:25:04.723240 (+548168us) spinlock_profiling.cc:243] Waited 98.3 ms on lock 0x7b54002d4e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.723406 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":226157824}
W20260709 12:25:04.747829 10640 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.016s	sys 0.015s
W20260709 12:25:04.794911 10640 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.016s	sys 0.015s
W20260709 12:25:04.795243 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 668522us
I20260709 12:25:04.748487 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.752076  9940 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:04.749449  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56621.4us
I20260709 12:25:04.797166 10021 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:04.797729  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:04.748245 10025 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:04.756330 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.016s	sys 0.022s
I20260709 12:25:04.655301 13858 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.767197 13756 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:04.812661 13756 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:04.812412 13858 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:04.813390 13756 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.768458 13754 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:04.765054 13673 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.770287 10097 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:04.773303 13826 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:04.776908 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 58370us
I20260709 12:25:04.777487 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1805) took 1988 ms. Trace:
W20260709 12:25:04.778326 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 351104us
I20260709 12:25:04.780035 10668 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:04.778234 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.730763 13728 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.793010  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1885) took 1710 ms. Trace:
I20260709 12:25:04.774744 13942 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:04.785363 13980 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:04.798934 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.017s	sys 0.022s
W20260709 12:25:04.843148 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 594362us
I20260709 12:25:04.845322 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.846040 10023 raft_consensus.cc:2368] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:04.752723 10676 rpcz_store.cc:276] 0709 12:25:03.200451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.107472 (+907021us) service_pool.cc:224] Handling call
0709 12:25:04.722296 (+614824us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.722460 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924440320}
I20260709 12:25:04.848145 10676 pending_rounds.cc:85] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 7
I20260709 12:25:04.848476 10676 pending_rounds.cc:107] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:25:04.848796 10676 raft_consensus.cc:2889] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:04.567559 13987 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:04.746909 13874 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.059s	sys 0.076s
I20260709 12:25:04.733844 13833 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:04.852913 13987 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:04.853106 13833 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.744943 13844 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:04.809716 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1765) took 1834 ms. Trace:
I20260709 12:25:04.829532 13826 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:04.593009 13875 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:04.769222  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1880) took 2117 ms. Trace:
I20260709 12:25:04.861126  9945 rpcz_store.cc:276] 0709 12:25:02.651709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.025055 (+1373346us) service_pool.cc:224] Handling call
0709 12:25:04.025384 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.047654 (+ 22270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.047701 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.068853 (+ 21152us) log.cc:844] Serialized 26 byte log entry
0709 12:25:04.570829 (+501976us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:04.570870 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.570939 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:04.769026 (+198087us) raft_consensus.cc:1685] finished
0709 12:25:04.769045 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.769160 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5846,"raft.queue_time_us":22469,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":13432,"thread_start_us":440244,"threads_started":1,"wal-append.queue_time_us":600989}
I20260709 12:25:04.643749 13979 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.820192  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:04.867729  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47639.3us
I20260709 12:25:04.869426  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:04.702843 13589 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:04.704398 13890 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:04.828055 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:04.876799 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48820us
I20260709 12:25:04.829766  9941 rpcz_store.cc:276] 0709 12:25:03.082528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.329733 (+1247205us) service_pool.cc:224] Handling call
0709 12:25:04.549613 (+219880us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.660091 (+110478us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.792844 (+132753us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b54000163d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:04.792959 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":487616256}
I20260709 12:25:04.793278 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.712903 13944 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:04.855786 13826 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:04.793828  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1889) took 1605 ms. Trace:
I20260709 12:25:04.785202 13961 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:04.841290  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:04.811619 10021 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:25:04.855283 10025 rpcz_store.cc:276] 0709 12:25:02.975600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.745217 (+1769617us) service_pool.cc:224] Handling call
0709 12:25:04.745494 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.809401 (+ 63907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.809638 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.858748 10023 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:04.860620 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:04.825165 13673 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:04.831040 13984 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.436s	user 0.000s	sys 0.001s
I20260709 12:25:04.825404 13553 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:04.816722 13876 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:04.854869 13844 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:04.938273 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1772) took 1822 ms. Trace:
I20260709 12:25:04.938664 10023 rpcz_store.cc:276] 0709 12:25:03.115594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.846505 (+1730911us) service_pool.cc:224] Handling call
0709 12:25:04.858475 (+ 11970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.937982 (+ 79507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.938206 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:04.940436 13861 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:25:04.941406 10672 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:04.827749 10093 rpcz_store.cc:276] 0709 12:25:02.789523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.059845 (+1270322us) service_pool.cc:224] Handling call
0709 12:25:04.060189 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.060463 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:04.581521 (+521058us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.776543 (+195022us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.776823 (+   280us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.776954 (+   131us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:04.777075 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.777173 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.777403 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21745,"prepare.queue_time_us":550963,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":82,"raft.queue_time_us":106408,"thread_start_us":534664,"threads_started":2}
W20260709 12:25:04.942394  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 67876.3us
I20260709 12:25:04.943701 10093 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 12:25:04.865129 13994 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:25:04.883347  9947 rpcz_store.cc:276] 0709 12:25:03.188061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.412150 (+1224089us) service_pool.cc:224] Handling call
0709 12:25:04.412514 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.706843 (+294329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.793602 (+ 86759us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54000b6154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:04.793768 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":488959360}
I20260709 12:25:04.918692 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:04.978971 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:04.876809 13995 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.134s
I20260709 12:25:05.008427  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.003024 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.000918  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1888) took 1851 ms. Trace:
I20260709 12:25:05.029338  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1891) took 1787 ms. Trace:
I20260709 12:25:05.007812  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:04.986439 10099 raft_consensus.cc:2368] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:05.007023  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:04.876557 14005 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:04.876062 13589 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:05.019991 10677 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:05.012939 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 826) took 1009 ms. Trace:
I20260709 12:25:05.036368 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:05.024339 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 806) took 1903 ms. Trace:
I20260709 12:25:05.143473 10668 rpcz_store.cc:276] 0709 12:25:04.003698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.183366 (+179668us) service_pool.cc:224] Handling call
0709 12:25:05.012719 (+829353us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54007396d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:05.012863 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581308672}
I20260709 12:25:05.143944 10669 rpcz_store.cc:276] 0709 12:25:03.121385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.548065 (+1426680us) service_pool.cc:224] Handling call
0709 12:25:04.564144 (+ 16079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.853757 (+289613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.853802 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.853860 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:04.853912 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.024061 (+170149us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b54007e4fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:05.024127 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.024255 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":665425152}
I20260709 12:25:04.981737 10095 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:05.146222 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:04.986480 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1764) took 2011 ms. Trace:
I20260709 12:25:05.045050  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:04.876363 13743 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:05.050768  9940 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:05.148248 10024 rpcz_store.cc:276] 0709 12:25:02.975373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.740655 (+1765282us) service_pool.cc:224] Handling call
0709 12:25:04.740992 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.741230 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:04.770798 (+ 29568us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.985995 (+215197us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:04.986049 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:04.986108 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.986173 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:04.986308 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36463,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":7503,"raft.queue_time_us":84,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":99,"thread_start_us":214741,"threads_started":1}
W20260709 12:25:05.004140 13789 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.750s	user 0.005s	sys 0.005s
I20260709 12:25:05.153095 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.139071 10019 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
W20260709 12:25:04.968976 13872 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.570s	user 0.007s	sys 0.003s
W20260709 12:25:05.011407 13966 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.979s	user 0.011s	sys 0.000s
W20260709 12:25:05.009881 14015 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:25:04.996237 14010 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
W20260709 12:25:05.009325 14012 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:25:05.196419 10024 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:05.135214 13861 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.153844  9942 rpcz_store.cc:276] 0709 12:25:03.242382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.615895 (+1373513us) service_pool.cc:224] Handling call
0709 12:25:04.616432 (+   537us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.616895 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.616931 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.617564 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.028583 (+411019us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b1800318b20. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1
0709 12:25:05.028807 (+   224us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:05.028860 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.028977 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.029037 (+    60us) raft_consensus.cc:1685] finished
0709 12:25:05.029056 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.029274 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1986,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":102,"spinlock_wait_cycles":39686912,"thread_start_us":10767,"threads_started":1,"wal-append.queue_time_us":248600}
W20260709 12:25:05.016062 13998 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:25:05.025184 13875 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:05.018030 13235 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:05.157897 10025 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:05.169549 10091 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:25:05.157336  9945 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:05.264050 10091 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:05.210191  9940 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:04.968642 13166 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:05.158385  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.227623 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:05.161803  9949 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:05.020190 13992 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.431s	user 0.001s	sys 0.000s
I20260709 12:25:05.004256 13553 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.212s
I20260709 12:25:05.140769 13944 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:05.153649 13789 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:05.140578 13869 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:05.434263  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:05.263768 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:05.158540 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 280358us
I20260709 12:25:05.154629  9948 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:05.156769  9946 rpcz_store.cc:276] 0709 12:25:03.149592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.407097 (+1257505us) service_pool.cc:224] Handling call
0709 12:25:04.408236 (+  1139us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:04.408875 (+   639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.408912 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:04.415767 (+  6855us) log.cc:844] Serialized 52 byte log entry
0709 12:25:04.647868 (+232101us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:04.648087 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.648190 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.000551 (+352361us) raft_consensus.cc:1685] finished
0709 12:25:05.000587 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.000830 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6326,"raft.queue_time_us":1305,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":78359,"thread_start_us":85059,"threads_started":1,"wal-append.queue_time_us":293376}
I20260709 12:25:05.444414 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:05.213568  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.445338  9941 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:25:05.136525 13993 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.549s	user 0.000s	sys 0.001s
I20260709 12:25:05.024536 14011 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:05.268872  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.216640 13850 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.271837  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 136102us
I20260709 12:25:05.448776  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.217700 13917 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:05.244067 13889 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.000s
I20260709 12:25:05.378559 13986 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:05.281243 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1766) took 2305 ms. Trace:
I20260709 12:25:05.290684 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:25:05.369196 10025 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:05.499253 10024 raft_consensus.cc:606] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:05.499555 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1784) took 2241 ms. Trace:
I20260709 12:25:05.499764 10024 raft_consensus.cc:614] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:05.500077 10025 rpcz_store.cc:276] 0709 12:25:03.258087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.279897 (+2021810us) service_pool.cc:224] Handling call
0709 12:25:05.280266 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.499192 (+218926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.499481 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.027874 13523 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:05.271822 13965 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.069s	user 0.013s	sys 0.000s
I20260709 12:25:05.279558 13861 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:05.365554 10092 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.654s	user 0.009s	sys 0.002s
I20260709 12:25:05.531695 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:05.532328 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 608) took 1125 ms. Trace:
I20260709 12:25:05.532682 10092 rpcz_store.cc:276] 0709 12:25:04.406490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.709446 (+302956us) service_pool.cc:224] Handling call
0709 12:25:05.532256 (+822810us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:05.348752  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:05.241070 13906 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.403452  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48177.4us
I20260709 12:25:05.377225  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:05.208806 13872 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:05.021981 13980 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:05.446437  9942 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:05.380832 13890 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:05.457240  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.474632  9945 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:05.453662  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:05.280344  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:38784 (request call id 748) took 1183 ms. Trace:
I20260709 12:25:05.573028  9948 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:05.572979  9940 rpcz_store.cc:276] 0709 12:25:04.135928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.630622 (+494694us) service_pool.cc:224] Handling call
0709 12:25:05.264725 (+634103us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:05.493541  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.497205 10020 rpcz_store.cc:276] 0709 12:25:02.975850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.746498 (+1770648us) service_pool.cc:224] Handling call
0709 12:25:04.776484 (+ 29986us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:04.776697 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:04.780553 (+  3856us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.280882 (+500329us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:05.280940 (+    58us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:05.281005 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.281066 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.281182 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68662,"prepare.queue_time_us":56952,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":30,"raft.queue_time_us":66408,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"thread_start_us":434714,"threads_started":2,"wal-append.queue_time_us":239549}
I20260709 12:25:05.504295  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.501485 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:05.506318 10019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 606) took 2569 ms (client timeout 3276 ms). Trace:
W20260709 12:25:05.575519 10019 rpcz_store.cc:269] 0709 12:25:02.936747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.214791 (+1278044us) service_pool.cc:224] Handling call
0709 12:25:05.506103 (+1291312us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54001f6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:05.506248 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146926848}
I20260709 12:25:05.577112 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:05.380983 13844 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:05.537575 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:05.538688 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:05.540343 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:05.588248 10675 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:05.588608  9942 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:05.589308 10018 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:05.446463 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 182830us
I20260709 12:25:05.590298 10023 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:05.590346 10677 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:05.590741  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 213742us
I20260709 12:25:05.591205  9941 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:25:05.552151 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.437601  9947 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:05.487273 13987 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:05.573871  9945 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:05.578504 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.438457 13874 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.095s
I20260709 12:25:05.584033 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.591991  9912 leader_election.cc:400] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10.
I20260709 12:25:05.610013  9912 leader_election.cc:403] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:05.591919 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:05.501696 13996 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.612212 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:05.608578 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.613283 10020 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:05.617766  9948 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:05.440936 13869 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:05.624511  9944 raft_consensus.cc:2368] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:05.607779 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.625465 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.626294 10025 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:05.626466 10019 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:05.626921  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.627331 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:05.627889 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:05.632320 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 187868us
W20260709 12:25:05.633263 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.634647  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.494823 13889 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:05.633687 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.494956 13850 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:05.715922 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.716837 10020 raft_consensus.cc:2410] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 10 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:25:05.605094 14020 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:05.709532  9940 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:05.603384 13987 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:05.784461  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:05.645442  9942 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:05.711457 10019 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:05.709160 14005 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:05.719691 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 92443.7us
I20260709 12:25:05.789086 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:05.790462 10025 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 12:25:05.787665 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:05.804948 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 84073us
I20260709 12:25:05.536195 13965 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:05.631259 13944 raft_consensus.cc:2764] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:25:05.712182 13917 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:25:05.719139 13673 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.787s	user 0.004s	sys 0.008s
I20260709 12:25:05.810143 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:05.829928 10020 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:05.834901 10090 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:05.718161 13995 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:05.824820 14019 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 12:25:05.825870 10019 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:05.635885 13874 consensus_queue.cc:1243] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:05.571959 13906 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:05.857506  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:05.886901  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:05.899520 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2955) took 2651 ms. Trace:
I20260709 12:25:05.899251  9942 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:05.798950 13987 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:05.880867 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1818) took 2952 ms (client timeout 2237 ms). Trace:
I20260709 12:25:05.900095 10094 rpcz_store.cc:276] 0709 12:25:03.248575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.736118 (+1487543us) service_pool.cc:224] Handling call
0709 12:25:04.736588 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.746658 (+ 10070us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:04.746692 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.747435 (+   743us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.234373 (+486938us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:05.234411 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.234483 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:05.899330 (+664847us) raft_consensus.cc:1685] finished
0709 12:25:05.899349 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:05.899463 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9522,"raft.queue_time_us":22781,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":14895,"thread_start_us":406902,"threads_started":1,"wal-append.queue_time_us":825518}
I20260709 12:25:05.904736  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.823282 14021 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10.
I20260709 12:25:05.935555 10673 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:25:05.859453 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53507.6us
W20260709 12:25:05.909687 10099 rpcz_store.cc:269] 0709 12:25:02.928628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:03.321793 (+393165us) service_pool.cc:224] Handling call
0709 12:25:05.880558 (+2558765us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540028fd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:05.880740 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2364987776}
I20260709 12:25:05.932858 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:05.956307 10019 raft_consensus.cc:2376] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 11. Re-sending same reply.
I20260709 12:25:05.957470 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:05.960058 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:05.961097 10019 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:05.870115 14039 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 12:25:05.710372 13166 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:05.877391 13704 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.284s	user 0.008s	sys 0.000s
W20260709 12:25:05.873116 13404 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.306s	user 0.009s	sys 0.000s
I20260709 12:25:05.965297  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.812304 14040 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:05.972501 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:05.974118  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 380086us
I20260709 12:25:05.874670 13861 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:05.878580 13858 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:06.022059 13872 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.000542 13789 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:05.994630 13833 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 12:25:05.872175 14031 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:25:05.878000 13979 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:05.995090  9940 raft_consensus.cc:2410] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:25:05.840335 13917 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:06.025084 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 939) took 1635 ms. Trace:
I20260709 12:25:06.025929  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.030045 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.030336 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:06.030943 10019 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:06.031149  9948 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:06.025579 10676 rpcz_store.cc:276] 0709 12:25:04.389189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.847455 (+458266us) service_pool.cc:224] Handling call
0709 12:25:04.847912 (+   457us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:04.849483 (+  1571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.849524 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:04.850110 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:25:05.198347 (+348237us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:05.198404 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.198492 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.024901 (+826409us) raft_consensus.cc:1685] finished
0709 12:25:06.024922 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.025028 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":347774,"threads_started":1,"wal-append.queue_time_us":721043}
I20260709 12:25:05.856905 13673 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:05.906883 13944 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:05.630811 13996 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:05.871431 14023 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.487s	user 0.000s	sys 0.001s
I20260709 12:25:05.982502 13986 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:06.021294 13166 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.044477 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:06.034046 14033 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.035069  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.099412 14033 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.100365  9946 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:06.101708 14033 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.102905 10671 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:06.103030 13844 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.105624 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 809) took 1387 ms. Trace:
I20260709 12:25:06.112165 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2851) took 1236 ms. Trace:
I20260709 12:25:06.112614 10023 rpcz_store.cc:276] 0709 12:25:04.875832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.939444 (+ 63612us) service_pool.cc:224] Handling call
0709 12:25:06.111922 (+1172478us) spinlock_profiling.cc:243] Waited 454 ms on lock 0x7b54002657d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:06.112081 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1043327232}
I20260709 12:25:06.113685 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:06.115392  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1656) took 1109 ms. Trace:
I20260709 12:25:06.115775  9947 rpcz_store.cc:276] 0709 12:25:05.005669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.006720 (+  1051us) service_pool.cc:224] Handling call
0709 12:25:06.115224 (+1108504us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540003e154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:06.115333 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255473408}
I20260709 12:25:06.116567 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2952) took 3013 ms. Trace:
I20260709 12:25:06.117713 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1761) took 3154 ms. Trace:
W20260709 12:25:06.118278 13493 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.405s	user 0.007s	sys 0.000s
I20260709 12:25:06.118129 10018 rpcz_store.cc:276] 0709 12:25:02.973990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.542273 (+1568283us) service_pool.cc:224] Handling call
0709 12:25:04.542691 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.589793 (+1047102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.117476 (+527683us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001f6154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:06.117650 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2406521088}
I20260709 12:25:06.121922 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:06.122159 13589 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.923s	user 0.006s	sys 0.000s
W20260709 12:25:06.123875 13826 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.143s	user 0.011s	sys 0.000s
I20260709 12:25:06.006673 14038 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.121s
W20260709 12:25:06.050443 10062 connection.cc:256] RPC call timeout handler was delayed by -1.5578469400002177s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.39780000000000004s for a response
W20260709 12:25:06.125571 10062 connection.cc:256] RPC call timeout handler was delayed by -1.5449481219998233s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1995s for a response
W20260709 12:25:06.125902 10062 connection.cc:256] RPC call timeout handler was delayed by -1.4720481220001602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2076s for a response
W20260709 12:25:06.126369 10062 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.612s (SENT)
I20260709 12:25:05.984972 14011 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:06.069446 10676 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:06.050752 14020 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:05.875068 14005 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.025111 13917 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:06.013833 13907 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:05.990862 14030 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 12:25:06.092478 10097 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:25:06.054442 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.161976 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.163210 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.092675  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:05.954142 13995 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:05.989305 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 182513us
W20260709 12:25:06.095938 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.080344 13861 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 12:25:06.091104 14027 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.209s	user 0.002s	sys 0.000s
I20260709 12:25:06.013595 14040 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.082382 13944 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.124497 13826 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.060206  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:06.126987 10673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 936) took 1738 ms (client timeout 1835 ms). Trace:
I20260709 12:25:06.127135 13869 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.128432 13965 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:06.038296 13966 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.091s
I20260709 12:25:05.997061 13704 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:25:06.074702 14025 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.591s	user 0.000s	sys 0.002s
I20260709 12:25:06.227164  9943 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:06.227895 10017 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:25:06.051738 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:06.228828 10094 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:06.229702 14040 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.082106 13743 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:06.133425 10018 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:06.233666 13944 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.106149 10671 rpcz_store.cc:276] 0709 12:25:04.718180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.027217 (+309037us) service_pool.cc:224] Handling call
0709 12:25:05.027576 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.103556 (+1075980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.105282 (+  1726us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54004d49d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:06.105541 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2472378240}
W20260709 12:25:06.137849  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.065s	sys 0.093s
W20260709 12:25:06.234925  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.065s	sys 0.093s
W20260709 12:25:06.235508  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 643609us
I20260709 12:25:06.236239  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:06.237529 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2956) took 2154 ms. Trace:
I20260709 12:25:06.237933  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:06.117095 10098 rpcz_store.cc:276] 0709 12:25:03.102852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.251890 (+1149038us) service_pool.cc:224] Handling call
0709 12:25:04.299733 (+ 47843us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:04.573187 (+273454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.573217 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:04.574058 (+   841us) log.cc:844] Serialized 78 byte log entry
0709 12:25:04.790286 (+216228us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:04.829628 (+ 39342us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.856698 (+ 27070us) log.cc:844] Serialized 14 byte log entry
0709 12:25:04.868096 (+ 11398us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:04.868197 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.116242 (+1248045us) raft_consensus.cc:1685] finished
0709 12:25:06.116276 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.116482 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15561,"prepare.queue_time_us":191854,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"raft.queue_time_us":337612,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":36136,"spinlock_wait_cycles":82760448,"thread_start_us":336736,"threads_started":2,"wal-append.queue_time_us":823040}
I20260709 12:25:06.076848 13872 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:06.239022 13881 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.239126 13872 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.240058 13826 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.240841 10021 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:06.241364 13872 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.006255 13404 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.243371 13404 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.168720 10676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:06.243700 13673 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.244086 10676 raft_consensus.cc:606] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
W20260709 12:25:06.244257 13875 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.974s	user 0.007s	sys 0.005s
I20260709 12:25:06.244469 10676 raft_consensus.cc:614] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:06.237886 10096 rpcz_store.cc:276] 0709 12:25:04.083342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.746034 (+662692us) service_pool.cc:224] Handling call
0709 12:25:04.746390 (+   356us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:04.776892 (+ 30502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:04.776926 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:04.795788 (+ 18862us) log.cc:844] Serialized 52 byte log entry
0709 12:25:04.998059 (+202271us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:05.167263 (+169204us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:05.246121 (+ 78858us) log.cc:844] Serialized 14 byte log entry
0709 12:25:05.246414 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:05.246523 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.237243 (+990720us) raft_consensus.cc:1685] finished
0709 12:25:06.237343 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.237475 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":136571,"prepare.queue_time_us":73616,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":69,"raft.queue_time_us":415040,"raft.run_cpu_time_us":2341,"raft.run_wall_time_us":231117,"spinlock_wait_cycles":16303872,"thread_start_us":292093,"threads_started":1,"wal-append.queue_time_us":1048200}
I20260709 12:25:06.245396 10022 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:06.245992 14051 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:06.246536 14051 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:06.247396 14051 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.168964 10020 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
W20260709 12:25:06.169724  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:06.149224 13728 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.289s	user 0.008s	sys 0.004s
I20260709 12:25:06.169831 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:06.271822 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 102082us
I20260709 12:25:06.272783 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:06.171274 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 382236us
I20260709 12:25:06.177237 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:06.216892 10673 rpcz_store.cc:269] 0709 12:25:04.388316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.494788 (+106472us) service_pool.cc:224] Handling call
0709 12:25:06.126710 (+1631922us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:06.126905 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2970552192}
I20260709 12:25:06.139036 14032 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:06.216500  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.148635 13858 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:06.137897 13493 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.277082 13756 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.291924 10099 raft_consensus.cc:2368] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:25:06.177987 14038 consensus_queue.cc:1243] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:06.293330 13493 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:06.266963 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.293941 13858 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.295238 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:06.296206 10020 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:06.135687 13589 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:06.269480  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 480358us
I20260709 12:25:06.297158  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:06.296957 14028 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 12:25:06.274679 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:06.277590  9942 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:06.286792 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.299201 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.300271 14032 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.298152 13589 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.304774 13995 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.307284 13906 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.309099 13850 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.212380 13754 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:06.310531 13754 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.312193 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:06.312413 13889 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:06.311440 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.312894 13754 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.313292 10019 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:06.312939 10018 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:06.315325 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.151252 14050 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.130s
I20260709 12:25:06.323630 14028 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:06.323794 14050 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:06.250079 14049 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:06.324710 14049 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:06.325441 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:06.326236 14028 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.322264  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:06.326855 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52300.7us
I20260709 12:25:06.157415 14020 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.324290 13980 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.348412  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:06.348940 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:06.349416  9948 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.350374 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:06.325670 14049 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.326678 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:06.322847 13861 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.322845 13858 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:06.333123 10673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:06.332862 10023 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:06.353615 13907 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.363319 13961 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:06.354722 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:06.390010 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:06.390997 10676 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:06.391426 10675 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:06.327313  9942 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:06.391803 14020 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.392478 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:06.358525 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:06.393337 10672 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:06.360374 10019 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:25:06.394234 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:06.393337 13987 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.390899 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:06.395709 10019 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:06.395944 10669 raft_consensus.cc:2368] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:25:06.396777 13987 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:06.396836 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:06.397202 10676 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:06.393332 10677 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:06.397562 10675 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:06.403566 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:06.404560  9942 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:06.404742 13881 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:06.404556 10677 raft_consensus.cc:2410] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 8 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 8.
I20260709 12:25:06.318884 14046 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.405831  9943 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:06.406116 14046 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.407003 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.407238 14032 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.407648 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.408296 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:06.408435 14046 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:06.343652 14028 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.408731 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:06.408857 10093 raft_consensus.cc:2376] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10. Re-sending same reply.
I20260709 12:25:06.409390 10098 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.409209 14028 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.410044 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:06.326696 14000 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.044s	user 0.000s	sys 0.004s
I20260709 12:25:06.355011 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:06.290076 13944 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:06.390834 13789 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:06.412349 13889 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.412892 13889 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.411499 14028 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:06.408574 13235 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.125s	user 0.013s	sys 0.000s
I20260709 12:25:06.397578 13987 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:06.415462 13987 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.414939 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:06.411568 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.385998 13907 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.365005 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.418623 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:06.343218  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 105334us
I20260709 12:25:06.420280 10675 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:25:06.415930 13987 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.358454 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:06.348515 13756 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.363621 13166 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:06.417188 10097 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:06.435985 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:06.318732 13965 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:06.426492 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:06.411355  9948 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:06.428522  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:06.431062 10018 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:06.363539 13979 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:06.411682 13944 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.327543 14050 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:06.435362 10024 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.004s	sys 0.015s
W20260709 12:25:06.463979 10024 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.005s	sys 0.015s
I20260709 12:25:06.289580 13728 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:06.435737 10062 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.853s (SENT)
W20260709 12:25:06.422922  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 351994us
I20260709 12:25:06.416260 13961 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:06.433868  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 469044us
I20260709 12:25:06.450956  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:06.452594 10098 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
W20260709 12:25:06.450911  9987 connection.cc:256] RPC call timeout handler was delayed by -1.7941075140001885s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
I20260709 12:25:06.472813 10671 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:06.482229  9987 connection.cc:256] RPC call timeout handler was delayed by -1.7590075140001318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1814s for a response
I20260709 12:25:06.434846 14049 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.476004 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:06.446336 13979 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:06.512117  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61239.6us
W20260709 12:25:06.479204  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.512898  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:06.482774  9987 connection.cc:256] RPC call timeout handler was delayed by -1.5691075139998247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
I20260709 12:25:06.445397 14028 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:06.526168  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.447585 13944 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:06.513679  9987 connection.cc:256] RPC call timeout handler was delayed by -1.4638075140001092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 12:25:06.530475  9987 connection.cc:256] RPC call timeout handler was delayed by -1.4134075140000277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2198s for a response
W20260709 12:25:06.530885  9987 connection.cc:256] RPC call timeout handler was delayed by -1.3702075139999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22460000000000002s for a response
I20260709 12:25:06.435842 13756 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.526955 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.543892 10098 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
W20260709 12:25:06.506742  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.471266 13235 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:06.532100 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:06.531596  9987 connection.cc:256] RPC call timeout handler was delayed by -1.0885075139999572s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2559s for a response
I20260709 12:25:06.556327  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:06.485467 13966 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:06.542999 10674 raft_consensus.cc:2368] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:06.447180 13889 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:06.360836 14011 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:06.559067  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46246.2us
I20260709 12:25:06.548962  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.511471 14046 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:06.530787 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 136604us
I20260709 12:25:06.477460 13965 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:06.561046  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:06.521355 13979 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:06.565804  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:06.566243 10673 raft_consensus.cc:606] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:06.579141 10673 raft_consensus.cc:614] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:06.567408  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:06.579890 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:06.578430  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.554615 14028 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:06.554646 13906 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:06.582847  9945 raft_consensus.cc:2368] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:06.523852 14032 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:25:06.587610  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:06.588274  9945 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:06.589622  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.571066  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:06.562999 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:06.583190  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.592202 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:06.547183 14060 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.583374 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:06.592408 14068 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:06.593616 14055 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:06.593720  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:06.594094 14055 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.594489 14055 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.593214 14060 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:25:06.596258 14060 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:06.594508  9940 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:06.528733 13523 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:25:06.608428  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:06.609558  9948 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:06.563990 13166 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:06.542089 14013 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
I20260709 12:25:06.542281 13589 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 12:25:06.542466 14044 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 12:25:06.600989 14048 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.332s	user 0.001s	sys 0.000s
I20260709 12:25:06.600499  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.627131  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.602090  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.610208 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:06.612633  9941 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:25:06.582322 14038 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:06.624217  9945 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
W20260709 12:25:06.602322  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46088.7us
I20260709 12:25:06.630373  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:06.628026  9949 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:25:06.632741  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:06.637439 14013 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:06.638341 13907 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.639041 13907 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:06.642056 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.647497  9945 raft_consensus.cc:1217] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Deduplicated request from leader. Original: 7.6->[8.7-8.7]   Dedup: 8.7->[]
I20260709 12:25:06.648218  9945 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:06.610399 14051 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:06.616483 13523 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:06.605604 13906 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:06.665449 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55379.2us
I20260709 12:25:06.666901  9949 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:06.618656 13166 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:06.570887 13966 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.667709 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:06.657852  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 68825.3us
I20260709 12:25:06.654862 13858 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:06.675524 10065 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:06.677518 14057 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:25:06.724643 13889 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:06.684137 14035 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:06.663092 13907 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.673573 13166 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:06.684450  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:06.686534 13996 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.718168 13858 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:06.687721 14045 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:06.674351 14013 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:06.688707 14066 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:06.680239 13917 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.658908 14051 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:06.692811 13965 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.563994 14050 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.698554 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:06.701061 10674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 804) took 3606 ms (client timeout 1995 ms). Trace:
W20260709 12:25:06.821005 10674 rpcz_store.cc:269] 0709 12:25:03.094815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.634829 (+1540014us) service_pool.cc:224] Handling call
0709 12:25:06.700775 (+2065946us) spinlock_profiling.cc:243] Waited 1.83 s on lock 0x7b540073a354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:06.700966 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4203543296}
I20260709 12:25:06.674302 14028 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:06.705528  9944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1668) took 1465 ms (client timeout 1858 ms). Trace:
I20260709 12:25:06.708652 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1792) took 1939 ms. Trace:
I20260709 12:25:06.606799 13961 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.719097 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:06.827639 10017 rpcz_store.cc:276] 0709 12:25:04.769377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.792960 (+ 23583us) service_pool.cc:224] Handling call
0709 12:25:06.708429 (+1915469us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54001c5f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:06.708576 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3101586304}
W20260709 12:25:06.731284  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46922.3us
I20260709 12:25:06.829037  9912 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:06.829713 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 248848us
I20260709 12:25:06.741984 10096 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:06.823942 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:06.832176 10092 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:06.826624  9944 rpcz_store.cc:269] 0709 12:25:05.239979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.267309 (+ 27330us) service_pool.cc:224] Handling call
0709 12:25:06.705259 (+1437950us) spinlock_profiling.cc:243] Waited 90.4 ms on lock 0x7b540042e354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:06.705444 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208007040}
I20260709 12:25:06.735244 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.744933 14066 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:06.854269 14066 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:06.835337 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 116301us
I20260709 12:25:06.857666 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1832) took 1821 ms. Trace:
I20260709 12:25:06.733589 13756 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:06.730774 14071 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.093s	user 0.002s	sys 0.001s
I20260709 12:25:06.834180 13523 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.726500 13907 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.876111 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:06.876729 10674 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:06.877034 13907 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.877307 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:06.842453 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.868372 13773 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:25:06.878659 13773 consensus_queue.cc:1243] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:06.845448 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.878662 10098 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:06.883512 10097 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:06.834726 14021 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:06.725975 14035 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:06.899832 10019 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:06.848212 13917 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.849246 14051 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.834558 14050 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.851521 13728 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:06.732096 13850 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:06.866662 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:06.862560 13980 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:06.912608 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:06.913479 10097 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:06.915514 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:06.873684  9912 connection.cc:256] RPC call timeout handler was delayed by -1.9130624809999972s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21589999999999998s for a response
I20260709 12:25:06.734301 14045 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:06.841734 13966 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:06.877028 10674 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:06.858094 10091 rpcz_store.cc:276] 0709 12:25:05.035872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.417360 (+381488us) service_pool.cc:224] Handling call
0709 12:25:05.430687 (+ 13327us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:05.770895 (+340208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:05.804588 (+ 33693us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:05.855383 (+ 50795us) log.cc:844] Serialized 52 byte log entry
0709 12:25:06.311486 (+456103us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:06.704453 (+392967us) spinlock_profiling.cc:243] Waited 4.13 ms on lock 0x7b180038a460. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1
0709 12:25:06.704698 (+   245us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:06.726756 (+ 22058us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.727111 (+   355us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.727305 (+   194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:06.727397 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:06.857461 (+130064us) raft_consensus.cc:1685] finished
0709 12:25:06.857480 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:06.857608 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42109,"prepare.queue_time_us":23340,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":60,"raft.queue_time_us":511408,"raft.run_cpu_time_us":3378,"raft.run_wall_time_us":356956,"spinlock_wait_cycles":36598912,"thread_start_us":562930,"threads_started":2,"wal-append.queue_time_us":528525}
I20260709 12:25:06.884895 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:06.916987  9912 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.531s (SENT)
I20260709 12:25:06.892649 10092 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:06.898392 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2865) took 1022 ms. Trace:
I20260709 12:25:06.929682 10023 rpcz_store.cc:276] 0709 12:25:05.875694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.113396 (+237702us) service_pool.cc:224] Handling call
0709 12:25:06.898185 (+784789us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b5400216bd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:06.898322 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300966784}
I20260709 12:25:06.725744 14055 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.909720 13987 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.733332 13944 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:06.917105 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1794) took 2050 ms (client timeout 1980 ms). Trace:
W20260709 12:25:06.939883 10021 rpcz_store.cc:269] 0709 12:25:04.866642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.918306 (+ 51664us) service_pool.cc:224] Handling call
0709 12:25:06.916872 (+1998566us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001c6454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:06.917041 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2492286208}
I20260709 12:25:06.917876 10020 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:06.842176 13979 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.734453 13858 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.902161 13756 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:06.953588 10021 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:06.954255 10021 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:06.955960 10023 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:06.926891 13850 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:06.961179 10023 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:06.732897 13906 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.981633 10670 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:06.900827 13523 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:06.927662 13917 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:25:06.955549 10021 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:06.996407 10021 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.950445 14068 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:06.980832 13850 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:06.950461 14051 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:06.999917  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:06.982213 13858 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.007187  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:06.992491 13166 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.014989 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:07.016530 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:07.016678 13166 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.017084 10097 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.021121 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:07.022534 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:07.023229 10092 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:06.997743 13789 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:06.961928 13756 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:07.033764 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 176035us
I20260709 12:25:07.034605 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:07.036604 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:06.959677 14035 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:07.051898  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:07.052562  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:07.053215  9946 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:07.057783  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:07.058657 10016 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:07.059127  9943 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:07.060662  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:07.062953  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:07.063402 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1776) took 3941 ms (client timeout 2207 ms). Trace:
I20260709 12:25:07.064234  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:07.065034  9944 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:07.065582 10673 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:07.047051 10021 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:07.047982 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 2967) took 1017 ms. Trace:
I20260709 12:25:07.066844 10094 rpcz_store.cc:276] 0709 12:25:06.030272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.030390 (+   118us) service_pool.cc:224] Handling call
0709 12:25:06.030724 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.229331 (+198607us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.047646 (+818315us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b5400305a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:07.047899 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":454509440}
I20260709 12:25:07.067560 10668 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:07.068356 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:07.064234  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:07.069706  9949 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 12:25:07.063966 10022 rpcz_store.cc:269] 0709 12:25:03.116781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.642627 (+1525846us) service_pool.cc:224] Handling call
0709 12:25:07.058067 (+2415440us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b54002666d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:07.058232 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3579605376}
I20260709 12:25:07.044291 10092 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:07.072769 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:07.051520 13961 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:07.074388  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:07.075248 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:07.078262 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:07.054867 14080 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:07.000272 14045 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.061471 14035 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.099633 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:07.084398 13979 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.111438 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:07.085285 14073 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.445s	user 0.000s	sys 0.002s
W20260709 12:25:07.087791 10090 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 2950) took 3985 ms (client timeout 5030 ms). Trace:
I20260709 12:25:07.091708 10097 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:07.103888 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:07.124276 10671 raft_consensus.cc:2368] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10.
W20260709 12:25:07.123768 10090 rpcz_store.cc:269] 0709 12:25:03.102225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.036195 (+933970us) service_pool.cc:224] Handling call
0709 12:25:07.087514 (+3051319us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:07.087705 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4040838528}
I20260709 12:25:07.127640 10094 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: [5.5-9.7]
I20260709 12:25:07.079176 13754 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:07.130232 10094 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:07.093808 13961 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.093286 13907 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:07.136857  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 207170us
I20260709 12:25:07.144337 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:07.146210 10671 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:07.123438 14055 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.069530 14068 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.130821 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:07.165556 10097 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:07.130949 14045 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.132512 14080 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:07.139322 13961 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.172518 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:07.176544 10671 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:07.129969 13754 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:07.179265 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:07.186100 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1767) took 4207 ms. Trace:
I20260709 12:25:07.186530 10016 rpcz_store.cc:276] 0709 12:25:02.976067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:04.794513 (+1818446us) service_pool.cc:224] Handling call
0709 12:25:07.058385 (+2263872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.059240 (+   855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.182960 (+123720us) spinlock_profiling.cc:243] Waited 2.23 s on lock 0x7b54002666d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:07.183113 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5133809024}
I20260709 12:25:07.191063 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:07.150377 13986 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.985s	user 0.021s	sys 0.021s
I20260709 12:25:07.191879 10669 raft_consensus.cc:2368] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
W20260709 12:25:07.198629  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.200744  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.146770 13907 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:07.220031  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4895168630000626s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
W20260709 12:25:07.220714  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4463168629999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1582s for a response
W20260709 12:25:07.221400  9912 connection.cc:256] RPC call timeout handler was delayed by -1.3176168630002394s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
I20260709 12:25:07.188383 13965 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:07.226212 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2877) took 1346 ms. Trace:
I20260709 12:25:07.226914 10019 rpcz_store.cc:276] 0709 12:25:05.879477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.396416 (+516939us) service_pool.cc:224] Handling call
0709 12:25:06.899389 (+502973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.900329 (+   940us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.225343 (+325014us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b5400216bd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:07.225543 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156026624}
W20260709 12:25:07.200352 13986 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.025s	sys 0.021s
I20260709 12:25:07.198292 14052 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:07.236518 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 351003us
I20260709 12:25:07.204299 14055 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:07.229067 13756 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.241596 14055 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.237530 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:07.249298 10096 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:07.230566 13965 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.248703 14055 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:07.260612 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61349.4us
I20260709 12:25:07.236006 13986 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:07.266742 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:07.253695 13965 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.240761 14052 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.242698 13879 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:25:07.239636 13754 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:07.287771  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:07.254701 14021 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.023s
I20260709 12:25:07.290107 10096 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:07.295287  9914 leader_election.cc:400] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:25:07.295146  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:07.295779  9914 leader_election.cc:403] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:07.249467 14080 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.271181 14066 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:07.294648 14086 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:25:07.292724 13754 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.283053 13966 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:25:07.316335 10096 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:07.317181  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:07.300706 14060 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:07.292912 14036 consensus_queue.cc:1175] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd983415389444e292b2330ef630850a" responder_term: 10 status { last_received { term: 9 index: 7 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:25:07.312287 14092 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:25:07.299949 13850 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:07.349947  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50134.2us
W20260709 12:25:07.329488 14087 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:25:07.352257  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:07.353690  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:07.347030 14084 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:25:07.340095 14101 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.346904 14021 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
W20260709 12:25:07.363307 14097 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:25:07.363819 14021 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:07.364332 14021 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.364678 14021 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.365993 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2852) took 2386 ms. Trace:
I20260709 12:25:07.310623 13986 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:07.368356  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 738046us
I20260709 12:25:07.369077  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:07.369664  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 721007us
I20260709 12:25:07.370076  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:07.355649 13906 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:07.378288  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:07.379208  9943 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.384003  9991 leader_election.cc:400] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:07.384372  9991 leader_election.cc:403] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:25:07.385417  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:07.386102  9942 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.371465 13966 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:07.388409 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.391016 10019 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:07.391686  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:07.392004 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:07.392325  9946 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:07.389276 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.372655 14038 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:07.393481  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:07.393996  9944 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.397004 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:07.397233  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:07.373764 14049 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:07.401207 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2881) took 1520 ms. Trace:
I20260709 12:25:07.401678 10020 rpcz_store.cc:276] 0709 12:25:05.880956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.411633 (+530677us) service_pool.cc:224] Handling call
0709 12:25:06.917643 (+506010us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.941247 (+ 23604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.401047 (+459800us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54001c6450. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:07.401158 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1162964224}
I20260709 12:25:07.402762 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.403069 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:07.405022  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:07.405486  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.406278 13553 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.406450 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.406870 13553 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:07.407330 13850 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:07.408968 13553 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.410810 14054 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:07.404398 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:07.412218  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:07.414239 10020 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:07.416805 14066 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:07.407128 10016 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:07.417598 14037 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:07.418028 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:07.419097  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45077.3us
I20260709 12:25:07.419656 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:07.420282 14047 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.420665 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.421361 10019 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:07.422124 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.422636 10016 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:07.423949  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:07.379835 13987 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:07.426584  9944 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:07.429175  9946 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:25:07.432893  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.433719 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.436704 10023 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:07.437181 10095 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1829) took 2421 ms (client timeout 2216 ms). Trace:
I20260709 12:25:07.429621  9943 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:07.432003  9942 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
W20260709 12:25:07.437628 10095 rpcz_store.cc:269] 0709 12:25:05.015302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.145901 (+130599us) service_pool.cc:224] Handling call
0709 12:25:07.436961 (+2291060us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b540014a0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:07.437107 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4341663488}
I20260709 12:25:07.439271  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:07.387850 14052 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:07.449136 10020 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:07.395071 14033 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:07.433131 13756 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:07.475548 13756 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:07.475981 13756 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:07.476765 14090 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.244s	user 0.001s	sys 0.001s
I20260709 12:25:07.468523 13987 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.478276 10098 raft_consensus.cc:2410] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:25:07.478229 13987 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.473505  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:07.468541 14030 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:07.479007 14050 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.001s
I20260709 12:25:07.452126 13789 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:07.481524 13789 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.482336 13987 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.483526 13235 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:07.483930 13235 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:07.484771 13235 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.485786 13235 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:07.485924 14021 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:07.479738 10097 raft_consensus.cc:1217] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Deduplicated request from leader. Original: 7.6->[8.7-8.7]   Dedup: 8.7->[]
I20260709 12:25:07.491422 10097 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:07.493667  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:07.452677 13966 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:07.469523 13861 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.434720 14035 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:25:07.502882 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2882) took 1621 ms. Trace:
I20260709 12:25:07.505244  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:07.503499 10018 rpcz_store.cc:276] 0709 12:25:05.881236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.444499 (+563263us) service_pool.cc:224] Handling call
0709 12:25:06.449326 (+  4827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:06.449551 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:06.579082 (+129531us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.491621 (+912539us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.502255 (+ 10634us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:07.502328 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:07.502408 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.502500 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.502798 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":746712,"prepare.queue_time_us":199184,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":84,"raft.queue_time_us":715466,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":9492,"thread_start_us":295726,"threads_started":3,"wal-append.queue_time_us":439876}
I20260709 12:25:07.450619 13961 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.514460 14035 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:07.515127 14035 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:07.516021 14035 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:07.489517 14049 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:07.521770 14030 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:07.524555  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 207461us
I20260709 12:25:07.507122 13996 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:07.527648  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:07.474361 14033 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.521253 14049 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:07.543242  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 119648us
I20260709 12:25:07.544471  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:07.547499  9987 leader_election.cc:400] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:25:07.523913 13523 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:07.524746 14030 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:07.548118  9987 leader_election.cc:403] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:25:07.558787 10676 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:07.560241 10673 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:07.560309  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 3.207s (SENT) [suppressed 51 similar messages]
I20260709 12:25:07.561684 10022 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:07.564436 10672 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:07.568715 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:07.565346 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:07.569867 10673 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:07.535627 14102 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:07.548254 14049 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.528174 13789 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.574508 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:07.550622 14038 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:07.601425  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 96300.5us
I20260709 12:25:07.552567 14055 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.598495 13961 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.600157 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:07.595496 10671 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:07.554992 13965 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.595467 10674 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.605568 10674 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.567152 14050 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:07.616948  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64062us
I20260709 12:25:07.617908  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:07.607481 14094 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.315s	user 0.001s	sys 0.001s
W20260709 12:25:07.611887 14085 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.439s	user 0.003s	sys 0.001s
I20260709 12:25:07.598800  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:07.599476 14110 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:07.628167  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:07.551118 13966 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:07.633724  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:07.638020 10674 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:07.601236 14052 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:25:07.640483 10092 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:07.642480  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:07.604897 14105 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:25:07.653630 13965 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:07.655268  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 301656us
I20260709 12:25:07.634552 14050 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 6. Current term is 7.
I20260709 12:25:07.663131  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:07.666343  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 244206us
I20260709 12:25:07.642335 13850 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:25:07.671191  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.612396 14030 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.666792 14102 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:07.685858  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:07.640573 14117 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:25:07.690068  9946 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:07.671016 13850 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.675274 14105 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.700567 13235 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:07.672405 14035 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.670575 13906 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 12:25:07.705180  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62783us
I20260709 12:25:07.692620 13553 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:07.689466 14102 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:07.702126 13850 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:07.718664  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 85026.5us
I20260709 12:25:07.691219 14120 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:07.721722 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:07.723275 10094 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:07.704947 14030 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.725443  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:07.664013 14038 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.730193 10094 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:07.708410 14050 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:07.716825 13789 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.740011 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:07.742357 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:07.745836 13944 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.746824 14123 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:25:07.748018 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 607) took 2479 ms. Trace:
I20260709 12:25:07.749575  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:07.692304 14112 consensus_queue.cc:1175] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 11 status { last_received { term: 8 index: 8 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:07.760254 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:07.767179 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:07.755237 10025 rpcz_store.cc:276] 0709 12:25:05.268856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.855538 (+586682us) service_pool.cc:224] Handling call
0709 12:25:05.919285 (+ 63747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:06.314309 (+395024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:06.314359 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:06.328528 (+ 14169us) log.cc:844] Serialized 26 byte log entry
0709 12:25:06.566848 (+238320us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:06.917446 (+350598us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.390890 (+473444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.747455 (+356565us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400138dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:07.747527 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:07.747608 (+    81us) raft_consensus.cc:1685] finished
0709 12:25:07.747733 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.747936 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":552766,"prepare.queue_time_us":70684,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":534951,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":48252,"spinlock_wait_cycles":700860672,"thread_start_us":487777,"threads_started":2,"wal-append.queue_time_us":365159}
W20260709 12:25:07.755959 14109 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:25:07.773066 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:07.738821 14055 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:07.725448 14035 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:25:07.772127 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.775169 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.755765 14005 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.781445 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:07.781841 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.782697 10020 raft_consensus.cc:2376] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9. Re-sending same reply.
I20260709 12:25:07.783850 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.727138 14121 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:25:07.786106 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:07.770426 13789 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:07.794241 10025 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:07.794480 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.794672 10677 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:07.767279 13944 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:07.810107 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:07.792280 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:07.768945 13961 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:07.817056 10099 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:07.801199 14005 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:07.816784 10669 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:25:07.821635  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.811333 10672 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
W20260709 12:25:07.826211  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 76732.1us
I20260709 12:25:07.774029 13756 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.827628 14123 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:07.808615 13789 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:07.848678 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:07.849581 10023 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:07.777175 14117 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:25:07.808926 13906 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:25:07.769791 14102 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.872123 10674 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:07.834002 14126 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:07.852926 13789 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:07.823135 14130 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.062s
W20260709 12:25:07.884500 13235 consensus_queue.cc:1175] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aa345bcd43b44088114b7b6f2830b17" responder_term: 9 status { last_received { term: 6 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 6 index: 7 } }
I20260709 12:25:07.895479 14123 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:07.932969 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2886) took 1946 ms. Trace:
I20260709 12:25:07.933490 10017 rpcz_store.cc:276] 0709 12:25:05.987456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.834199 (+846743us) service_pool.cc:224] Handling call
0709 12:25:06.834492 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.403582 (+569090us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:07.403925 (+   343us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.521188 (+117263us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:07.521243 (+    55us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:07.521292 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.932697 (+411405us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b54001f7554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:07.932754 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.932896 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81453,"prepare.queue_time_us":1497,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":77293,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":30681,"spinlock_wait_cycles":1219952512,"thread_start_us":105881,"threads_started":1,"wal-append.queue_time_us":160196}
I20260709 12:25:07.936930 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:07.882512 14127 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:07.883010 14126 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:07.864851 14128 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:25:07.894055 14102 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:07.961709 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.873601 14120 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:25:07.977895 10677 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:07.938534 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.011718 10017 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:08.015839 10023 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
W20260709 12:25:07.977701 14133 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:25:08.016788 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1837) took 1159 ms. Trace:
I20260709 12:25:08.018306 10098 rpcz_store.cc:276] 0709 12:25:06.857116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:06.884579 (+ 27463us) service_pool.cc:224] Handling call
0709 12:25:08.016590 (+1132011us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b5400162454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:08.016726 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1262184320}
I20260709 12:25:08.016954 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.019325 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.020026 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.021253 10017 raft_consensus.cc:2376] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 12. Re-sending same reply.
I20260709 12:25:08.023514 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.023944 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.024499 10017 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:08.025604 10023 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:08.026126 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.027678 10017 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:08.029026 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.029474 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.029827 10017 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:07.953423 14126 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:08.032999 10023 raft_consensus.cc:2376] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9. Re-sending same reply.
W20260709 12:25:08.037259  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 159883us
W20260709 12:25:08.038031  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.039391 10642 leader_election.cc:336] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.677s (SENT)
I20260709 12:25:08.039865 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:08.040655 13965 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:08.041056 14035 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.041581 14035 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.043452 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.045307 10023 raft_consensus.cc:2410] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:08.049480 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:07.963908 14127 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.051652 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.047958 10017 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:08.097976 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.099860 10017 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:08.098963 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.100811 10023 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:08.102545 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.103363 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.105240 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.106093 10017 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:08.107725 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.108372 10017 raft_consensus.cc:2376] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7. Re-sending same reply.
I20260709 12:25:08.109627 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.115825 14104 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:08.116463 14104 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 7 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:08.117656 14104 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.120968 14033 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:08.125368 14021 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:08.129034 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.129578 14033 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.130025 14033 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:08.131987 14033 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:08.132956 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:08.133199 14033 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.133646 14033 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.134088 13961 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:08.134613 13961 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:08.135076 13961 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.135713 14119 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:08.136161 14119 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:08.136951 14119 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:08.146369 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1845) took 1268 ms. Trace:
W20260709 12:25:08.508263  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:08.522079 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 1.46258e+06us
W20260709 12:25:08.544513 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:07.366415 10024 rpcz_store.cc:276] 0709 12:25:04.979911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:05.631822 (+651911us) service_pool.cc:224] Handling call
0709 12:25:05.632134 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:05.632380 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:06.464566 (+832186us) log.cc:844] Serialized 14 byte log entry
0709 12:25:06.586778 (+122212us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.084740 (+497962us) spinlock_profiling.cc:243] Waited 9.48 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:07.365481 (+280741us) spinlock_profiling.cc:243] Waited 8.3 ms on lock 0x7b180034f4e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:07.365640 (+   159us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:07.365691 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:07.365747 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:07.365809 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:07.365932 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":772355,"prepare.queue_time_us":1046355,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":73,"raft.queue_time_us":801057,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":1617,"spinlock_wait_cycles":40879744,"thread_start_us":770416,"threads_started":2,"wal-append.queue_time_us":175461}
I20260709 12:25:08.525725 10097 rpcz_store.cc:276] 0709 12:25:06.877658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.167696 (+290038us) service_pool.cc:224] Handling call
0709 12:25:07.479609 (+311913us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:07.492580 (+ 12971us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.146109 (+653529us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b5400161f50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:08.146293 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":716512512}
W20260709 12:25:08.551126 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.545044 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:08.533385 14104 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:08.554555 13879 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:08.557286 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:08.557705 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:08.142974 13756 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:08.240813 14013 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.321s
I20260709 12:25:08.561878 13756 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:08.562528 13756 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:08.562987 13756 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:08.563575 13789 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.002s
I20260709 12:25:08.564024 13756 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.565238 13965 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:08.564038 13789 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:08.134088 14049 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.002s
I20260709 12:25:08.566999 14049 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:08.569110 14049 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:08.569303 14102 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.591s	user 0.002s	sys 0.005s
I20260709 12:25:08.569921 10020 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:08.552577 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.31504e+06us
W20260709 12:25:08.570287 14030 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.000s
I20260709 12:25:08.152184 14037 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.152710 14005 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:08.570778 14037 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:08.189951 14091 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.016s	user 0.010s	sys 0.011s
W20260709 12:25:08.158656 14047 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.738s	user 0.012s	sys 0.001s
I20260709 12:25:08.571969 14047 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:08.174813 13850 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:08.573899 14126 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.001s
I20260709 12:25:08.575495 13850 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:08.160575 13861 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.001s
I20260709 12:25:08.576642 13861 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:08.579113 10019 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:08.580191 14033 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:08.582049 13789 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:08.585896 10018 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:08.587363 10023 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:25:08.588994 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.156123 14110 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:08.161646 14035 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:08.522048 14119 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:08.525313 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 392412us
I20260709 12:25:08.400039 14034 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:08.141805 14127 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:08.153236 14125 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:08.574343 14126 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:08.555269 13879 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:08.160687 13907 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:08.603350 10640 leader_election.cc:336] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.958s (SENT)
I20260709 12:25:08.610369 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:08.622210 13907 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:25:08.592939 13961 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.601500 14034 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.638091 14034 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:08.646343 14148 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.647696 14148 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.654417 14126 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:08.602171 14125 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:08.651823 14148 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:07.617188  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:08.141999 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:08.695048  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.697679 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:08.185458  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 366893us
W20260709 12:25:08.699484  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:08.525020 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:08.702982  9949 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.007s	sys 0.007s
W20260709 12:25:08.703708  9949 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.007s	sys 0.007s
I20260709 12:25:08.551290 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.704116 10017 raft_consensus.cc:2368] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10.
I20260709 12:25:08.704161 10022 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 7: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:08.705322 10024 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:08.705516  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1913) took 1037 ms. Trace:
I20260709 12:25:08.706959 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.707134 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.707576  9945 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:08.709004 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.710487 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.711441 10024 raft_consensus.cc:2376] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10. Re-sending same reply.
I20260709 12:25:08.712759 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.715682  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:08.716944  9945 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.723196  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:08.724978  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:08.726395  9946 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:08.705948  9949 rpcz_store.cc:276] 0709 12:25:07.667947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.668089 (+   142us) service_pool.cc:224] Handling call
0709 12:25:07.668420 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.668658 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:07.754257 (+ 85599us) log.cc:844] Serialized 14 byte log entry
0709 12:25:07.827061 (+ 72804us) log.cc:844] Serialized 14 byte log entry
0709 12:25:08.184287 (+357226us) spinlock_profiling.cc:243] Waited 6.95 ms on lock 0x7b18001b2b00. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:08.704406 (+520119us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:08.704551 (+   145us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:08.704753 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.704934 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:08.705382 (+   448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":349333,"prepare.queue_time_us":34167,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":499013,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":19347,"spinlock_wait_cycles":15982464,"thread_start_us":642221,"threads_started":3,"wal-append.queue_time_us":60711}
W20260709 12:25:08.735292 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0145763789996636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1694s for a response
I20260709 12:25:08.736847  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:08.738221  9942 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.745915 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:08.747511 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 168420us
I20260709 12:25:08.752238 10097 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:08.752511 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:08.755088 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:08.759263 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:08.762763 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 205453us
I20260709 12:25:08.763361 10668 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:08.765588 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:08.732677  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:08.809388 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48170.9us
I20260709 12:25:08.812834 10016 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:08.813515 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2915) took 1013 ms. Trace:
I20260709 12:25:08.813668 14148 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:08.813951 10016 rpcz_store.cc:276] 0709 12:25:07.800510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.696621 (+896111us) service_pool.cc:224] Handling call
0709 12:25:08.813405 (+116784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.814323 14148 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:08.814404 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:08.814961 14148 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:08.815791 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:08.816782 13789 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.816702 10016 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:08.817513 13789 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:08.813697 14080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.021s	sys 0.024s
I20260709 12:25:08.819410 13907 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:08.819855 14080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.021s	sys 0.025s
I20260709 12:25:08.820299 13907 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.821081 14080 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:08.604115 13917 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.823518 13917 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.745915  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:08.813728 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:08.593958 14110 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:08.855710 13907 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:08.844576 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:08.863684 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:08.864967  9949 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:08.867487 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 626) took 1009 ms. Trace:
I20260709 12:25:08.868007 10016 rpcz_store.cc:276] 0709 12:25:07.855466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.815360 (+959894us) service_pool.cc:224] Handling call
0709 12:25:08.864311 (+ 48951us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:08.871519 14127 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.871726 13754 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.872519 14035 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.872579 13754 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.873570 14035 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:08.875197 13773 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:08.875963 13773 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:08.876394 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:08.877249 10091 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:08.877350 14101 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:08.877851  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:08.876178 14079 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:08.871757 10673 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 12:25:08.873991 13523 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:08.914288 13589 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:08.914656 14102 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:08.919876 14148 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:08.877398 13773 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:08.870319 13917 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:08.937474 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:08.865240 14126 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.023705 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 160020us
I20260709 12:25:08.952817  9946 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:09.026456  9945 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:08.949647 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1846) took 2071 ms. Trace:
I20260709 12:25:08.984983 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1810) took 1207 ms. Trace:
W20260709 12:25:08.938064 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1822) took 1158 ms (client timeout 1543 ms). Trace:
W20260709 12:25:08.936362 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 616) took 1547 ms (client timeout 1958 ms). Trace:
W20260709 12:25:09.040823 10020 rpcz_store.cc:269] 0709 12:25:07.779758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.783563 (+  3805us) service_pool.cc:224] Handling call
0709 12:25:08.937561 (+1153998us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b54001383d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:08.937852 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1805726080}
I20260709 12:25:08.949242 14037 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.047808 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.048911 10097 raft_consensus.cc:2410] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 10 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:25:09.050112 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.050355 10022 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:09.051219 14154 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:09.051837 14037 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.051925 14154 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.054273  9942 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:09.055646 14035 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.055774 13789 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:09.058804  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 313137us
I20260709 12:25:08.978778 13879 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.060348  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:09.062027 14079 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:08.935469 13906 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.064982 13906 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.067350 14128 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.068264 14128 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.069600 14034 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:09.070093 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 324433us
I20260709 12:25:08.872439 14127 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.070380 14034 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.071149 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:09.071488 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1815) took 1293 ms. Trace:
I20260709 12:25:09.071935 10017 rpcz_store.cc:276] 0709 12:25:07.778103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.109391 (+331288us) service_pool.cc:224] Handling call
0709 12:25:09.071270 (+961879us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:09.071426 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1362883840}
I20260709 12:25:09.072597 14127 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.072897 14034 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.073053 14030 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:09.073312 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.074231 10017 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:09.073905 14030 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.075229 14066 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:09.075501 10019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1797) took 1303 ms (client timeout 1694 ms). Trace:
I20260709 12:25:09.076045 14066 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:09.076413 14030 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.077626 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:09.078423 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.078640 10671 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:08.935725 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:09.072628 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:09.083292 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:09.075973 10019 rpcz_store.cc:269] 0709 12:25:07.771728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.771861 (+   133us) service_pool.cc:224] Handling call
0709 12:25:09.075129 (+1303268us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54002657d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:09.075315 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840769408}
I20260709 12:25:09.039991 10099 rpcz_store.cc:276] 0709 12:25:06.877914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.228192 (+350278us) service_pool.cc:224] Handling call
0709 12:25:07.228560 (+   368us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:07.824668 (+596108us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.948994 (+1124326us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b5400305f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:08.949425 (+   431us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1353013632}
I20260709 12:25:08.922412 14106 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:08.911900 13850 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:08.949417 13961 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:09.042850 10021 rpcz_store.cc:269] 0709 12:25:07.388915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.389032 (+   117us) service_pool.cc:224] Handling call
0709 12:25:08.935831 (+1546799us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54001f7a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:08.936115 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2664567552}
I20260709 12:25:09.087668 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2897) took 1286 ms. Trace:
I20260709 12:25:09.040369 10018 rpcz_store.cc:276] 0709 12:25:07.777049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.779113 (+  2064us) service_pool.cc:224] Handling call
0709 12:25:08.984623 (+1205510us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b54002666d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:08.984809 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1756472192}
I20260709 12:25:09.092780  9949 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
W20260709 12:25:09.093564 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.087028 13589 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:09.094376 13906 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.093294 10017 rpcz_store.cc:276] 0709 12:25:07.794874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.072979 (+1278105us) service_pool.cc:224] Handling call
0709 12:25:09.081032 (+  8053us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.107116 10097 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:08.937292 14102 raft_consensus.cc:2764] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:09.054432 14154 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.109922 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.116039 10016 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:25:09.117731 10675 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:08.983757 14110 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.119583 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.120085 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1805) took 1345 ms. Trace:
I20260709 12:25:09.122399 10016 rpcz_store.cc:276] 0709 12:25:07.774213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.936984 (+1162771us) service_pool.cc:224] Handling call
0709 12:25:09.120011 (+183027us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.113517 14106 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:09.133162 10091 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:09.112903 13754 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.141904 14035 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:09.142731 14035 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.142843 13589 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.144098 14054 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:09.144778 14054 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.145290 14035 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:08.946871 14125 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.147313 14125 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:08.990996 14146 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.139s
I20260709 12:25:09.150713 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.137699 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.152082 10099 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.152134 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.142462 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2913) took 1342 ms. Trace:
W20260709 12:25:09.158798 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 393458us
I20260709 12:25:09.160414 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.156055 10023 rpcz_store.cc:276] 0709 12:25:07.799926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.103129 (+303203us) service_pool.cc:224] Handling call
0709 12:25:09.142038 (+1038909us) spinlock_profiling.cc:243] Waited 482 ms on lock 0x7b540012c854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:09.142285 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1108565760}
I20260709 12:25:09.162390 10065 leader_election.cc:400] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:09.162802 10065 leader_election.cc:403] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:09.162693 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:09.163532  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 103406us
I20260709 12:25:09.163990 10016 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:09.159719 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
W20260709 12:25:09.164896 14152 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:25:09.165182 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.165800 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.166437 10677 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:09.167402  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:09.167641 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.168363 14066 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:09.168939 14066 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:09.166109 10025 raft_consensus.cc:2368] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:25:09.169426 14066 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:09.169783 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2898) took 1374 ms. Trace:
I20260709 12:25:09.170153 10025 rpcz_store.cc:276] 0709 12:25:07.795182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.092068 (+1296886us) service_pool.cc:224] Handling call
0709 12:25:09.169721 (+ 77653us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.170915 14110 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.172163  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:09.172392  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50851.8us
I20260709 12:25:09.174006  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:09.174247 10643 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:09.127985 13850 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.174813 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.174893 13966 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:09.175140 13961 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:25:09.175668 10098 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:09.150372 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.177201 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2904) took 1380 ms. Trace:
I20260709 12:25:09.177704 10020 rpcz_store.cc:276] 0709 12:25:07.796949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.109563 (+1312614us) service_pool.cc:224] Handling call
0709 12:25:09.177113 (+ 67550us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.180455 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1818) took 1401 ms. Trace:
I20260709 12:25:09.180950 10024 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:09.180953 10019 rpcz_store.cc:276] 0709 12:25:07.778828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.106041 (+1327213us) service_pool.cc:224] Handling call
0709 12:25:09.180369 (+ 74328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.183252 13906 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:09.183382 14037 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:09.183912 13906 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:09.185218 13906 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.169107 10092 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:09.198699 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 126191us
I20260709 12:25:09.202143 10017 raft_consensus.cc:2376] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10. Re-sending same reply.
I20260709 12:25:09.202950 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1816) took 1424 ms. Trace:
I20260709 12:25:09.153374 14128 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.203285 10017 rpcz_store.cc:276] 0709 12:25:07.778353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.113887 (+1335534us) service_pool.cc:224] Handling call
0709 12:25:09.202884 (+ 88997us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.205650 10092 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:09.206632 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:09.190249 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2919) took 1388 ms. Trace:
W20260709 12:25:09.225404  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.216835 10016 rpcz_store.cc:276] 0709 12:25:07.801714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.127649 (+1325935us) service_pool.cc:224] Handling call
0709 12:25:09.189978 (+ 62329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.240492  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:09.247143 14119 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.645s	user 0.019s	sys 0.000s
W20260709 12:25:09.250860 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 76768.1us
I20260709 12:25:09.248396 14106 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:09.254261  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47937.6us
I20260709 12:25:09.253289  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:09.256270 10091 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:09.134354 14126 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.258872  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:09.257452 13850 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.258980 14111 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:09.254022 14062 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:09.263852 10098 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:09.269495  9949 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:09.270416 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:09.276590  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.268510 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:09.278487 10023 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.278645 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:09.281397 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 165883us
I20260709 12:25:09.282480 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.287955 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.266645 14148 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.291011 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2905) took 1493 ms. Trace:
I20260709 12:25:09.292203 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2903) took 1487 ms. Trace:
I20260709 12:25:09.293573 10016 rpcz_store.cc:276] 0709 12:25:07.796652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.258052 (+1461400us) service_pool.cc:224] Handling call
0709 12:25:09.283589 (+ 25537us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.293453 10024 rpcz_store.cc:276] 0709 12:25:07.797272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.712258 (+914986us) service_pool.cc:224] Handling call
0709 12:25:09.290821 (+578563us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:09.290947 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1075079552}
W20260709 12:25:09.297546 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 696673us
I20260709 12:25:09.297871 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:09.298931 10677 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:09.300124 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.284883 13773 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:09.302896 14128 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:09.307721 14128 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.308423 14128 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.308889 14111 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.291072 14102 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:09.311497 14111 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.312249  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:09.313632 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:09.316646 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:09.311316 14102 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:25:09.328922 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77432.3us
W20260709 12:25:09.345655 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 627) took 1489 ms (client timeout 1915 ms). Trace:
W20260709 12:25:09.346194 10022 rpcz_store.cc:269] 0709 12:25:07.855822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.706147 (+850325us) service_pool.cc:224] Handling call
0709 12:25:09.345433 (+639286us) spinlock_profiling.cc:243] Waited 342 ms on lock 0x7b5400265cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:09.345578 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":787214464}
I20260709 12:25:09.347508 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.348141 10022 raft_consensus.cc:2376] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7. Re-sending same reply.
I20260709 12:25:09.348667 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1820) took 1569 ms. Trace:
I20260709 12:25:09.349012 10022 rpcz_store.cc:276] 0709 12:25:07.779328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.347127 (+1567799us) service_pool.cc:224] Handling call
0709 12:25:09.348607 (+  1480us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.350023 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.351719  9949 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:09.353397  9944 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:09.354035  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.354368 10091 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:09.354997 10022 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:09.356422 10025 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:09.357772 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2918) took 1556 ms. Trace:
I20260709 12:25:09.365334 10022 rpcz_store.cc:276] 0709 12:25:07.801415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.349640 (+1548225us) service_pool.cc:224] Handling call
0709 12:25:09.357706 (+  8066us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.328827 13944 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:09.366722 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.368425  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:09.369767  9940 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:25:09.373230 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:09.373948 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:09.374732 13996 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:25:09.375591  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:09.375865  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:09.376806  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.377480  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:09.379369 10024 raft_consensus.cc:2410] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 12:25:09.383778 10096 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:09.335850 14037 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.384397  9942 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:09.387424 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:09.391352  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105431us
I20260709 12:25:09.390682 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.365562 14125 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.362891 13850 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:09.397200 13966 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.401340 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:09.404728 10022 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:09.404345 13966 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.405761 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:09.406374 10022 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.387226 14079 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:09.413089 10099 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
W20260709 12:25:09.416603 14164 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:09.417121 14125 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.417656 14125 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.418045 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:09.420171 10016 raft_consensus.cc:2376] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 9. Re-sending same reply.
I20260709 12:25:09.420454 10091 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:09.420800 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2900) took 1625 ms. Trace:
I20260709 12:25:09.421132 10016 rpcz_store.cc:276] 0709 12:25:07.795767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.297147 (+1501380us) service_pool.cc:224] Handling call
0709 12:25:09.420740 (+123593us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.425673  9947 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:09.426368  9948 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:09.430759 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:09.393777 14062 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:09.431190 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.401881 13944 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:09.434944  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:09.435201 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62088.4us
I20260709 12:25:09.435722 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.437273 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.441136 10016 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:09.442184 10095 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:25:09.437372 14148 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:09.410825 14037 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.451153  9941 raft_consensus.cc:1217] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-10.8]   Dedup: 5.6->[8.7-10.8]
I20260709 12:25:09.455235  9941 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: [2.5-10.8]
I20260709 12:25:09.452646  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:09.456979 14159 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:25:09.458547 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:09.460397 10025 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:09.460251  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 84837.3us
I20260709 12:25:09.448211 13961 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:09.462813  9948 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:09.463133  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:09.463835 13917 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.464028  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.464481  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.465094 13756 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:09.444806 14154 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:09.434279 13850 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.468036  9945 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:09.469774 13917 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:09.474362 14104 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.919s	user 0.008s	sys 0.011s
I20260709 12:25:09.457741 14148 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:09.475834 14163 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:25:09.476166 13917 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:09.476564 14118 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:09.476109 14148 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.476984 14118 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:09.480065 14150 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.528s	user 0.000s	sys 0.004s
I20260709 12:25:09.483894  9940 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:09.471513 14154 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.458745 13966 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.489706 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1844) took 2608 ms. Trace:
I20260709 12:25:09.491184 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.491937 10016 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:09.489996 13756 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:09.489739 13961 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:09.493899 14126 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.494272 13756 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.495915 13961 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.497603  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:09.491043 10090 rpcz_store.cc:276] 0709 12:25:06.876861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.132170 (+255309us) service_pool.cc:224] Handling call
0709 12:25:07.135560 (+  3390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.522353 (+1386793us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.522396 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:08.522973 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 12:25:08.923876 (+400903us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:08.934999 (+ 11123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:08.975032 (+ 40033us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:09.479100 (+504068us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:09.483143 (+  4043us) raft_consensus.cc:1685] finished
0709 12:25:09.483168 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.485525 (+  2357us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1900977,"raft.queue_time_us":1387288,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":15099,"spinlock_wait_cycles":70679424,"thread_start_us":331466,"threads_started":1,"wal-append.queue_time_us":387280}
W20260709 12:25:09.502002 10642 leader_election.cc:336] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.665s (SENT)
I20260709 12:25:09.474144 14066 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.503139  9949 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:09.504609 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:09.506417 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45185.4us
I20260709 12:25:09.507957  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.509661  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.468446 13523 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:09.514469 13996 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:09.514883 14169 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:09.515142 13523 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.514530  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:09.517472  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:09.517750 13523 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:09.522019 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.515240 13996 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.515784 13944 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.524922 13996 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.525787 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:09.527324 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:09.528314 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:09.529147 13789 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.529588 13789 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:09.530014 13789 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.530224 13850 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.533252 14110 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:09.531315 14158 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:09.533015  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.541074 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:09.541949 14151 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.590s	user 0.000s	sys 0.003s
I20260709 12:25:09.542347  9942 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:09.559537 10023 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:09.542748 14037 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:09.543190 14162 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.286s	user 0.000s	sys 0.001s
I20260709 12:25:09.478195 14118 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.565456 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:09.566099 14118 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.562028 13996 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.520759  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:09.571460 13996 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.572281  9940 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
W20260709 12:25:09.544045  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46559.3us
I20260709 12:25:09.572831 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1853) took 1694 ms. Trace:
I20260709 12:25:09.573223 10094 rpcz_store.cc:276] 0709 12:25:07.878309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:07.878447 (+   138us) service_pool.cc:224] Handling call
0709 12:25:07.878894 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:08.589098 (+710204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:08.589133 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:08.593227 (+  4094us) log.cc:844] Serialized 26 byte log entry
0709 12:25:08.947144 (+353917us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:08.947755 (+   611us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.077693 (+129938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.077812 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:09.541071 (+463259us) spinlock_profiling.cc:243] Waited 31.2 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:09.572417 (+ 31346us) raft_consensus.cc:1685] finished
0709 12:25:09.572513 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:09.572658 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":863108,"prepare.queue_time_us":4480,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":841927,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":6683,"spinlock_wait_cycles":71715072,"thread_start_us":306684,"threads_started":1,"wal-append.queue_time_us":357766}
I20260709 12:25:09.575670 14135 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 LEADER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:09.543596 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:09.545249 14154 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:09.545629 10062 leader_election.cc:400] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10.
I20260709 12:25:09.579290 10062 leader_election.cc:403] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:25:09.579425 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74689.8us
I20260709 12:25:09.555160 14066 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:09.562119 10022 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:09.563875  9949 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:09.560516 14158 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.566182 10016 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:09.544451  9948 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:09.586659 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:09.588512 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:09.590108  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 72727.8us
I20260709 12:25:09.592876 10094 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:09.559723 14126 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:09.587954 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:09.595759 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50246.3us
I20260709 12:25:09.563827 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.558332 13523 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:09.564584 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2902) took 1763 ms (client timeout 2236 ms). Trace:
W20260709 12:25:09.603924 10023 rpcz_store.cc:269] 0709 12:25:07.796370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.216236 (+1419866us) service_pool.cc:224] Handling call
0709 12:25:09.560011 (+343775us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.607287 10099 raft_consensus.cc:2410] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:09.606381 10022 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.579923 13996 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.590718 14158 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.589468 13589 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10.
I20260709 12:25:09.617291 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:09.618530 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.621318 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:09.621907 10677 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:09.628834  9912 connection.cc:256] RPC call timeout handler was delayed by -1.3691251560003366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 12:25:09.629896 14149 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:09.630417 14171 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:25:09.630838 10672 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:09.631060 14053 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:09.632058 14062 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:09.632676 14062 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:09.623903 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:09.630235 13996 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:09.636595 14062 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.580859 13773 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:25:09.636014 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:09.622376 13944 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:09.616655 13523 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:09.656535 13523 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.657011 13523 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.659678 14146 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:09.661027 14169 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:25:09.661417 14169 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 6 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:09.662349 14169 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.650380 10099 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:09.655153 13773 consensus_queue.cc:1243] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:09.655961 14051 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.666292 14051 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.666909 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:09.667805 10671 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:09.668685 14051 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.647246 10675 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
W20260709 12:25:09.674841 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 69004.6us
I20260709 12:25:09.629606 14110 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:09.688627 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.689821 10018 raft_consensus.cc:2368] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:25:09.640578 14066 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
W20260709 12:25:09.690266 10018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2899) took 1894 ms (client timeout 2206 ms). Trace:
W20260709 12:25:09.690996 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 70073.5us
I20260709 12:25:09.693581 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:09.680215 13944 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:09.697103 14155 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.760s	user 0.002s	sys 0.000s
I20260709 12:25:09.697609 13944 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.644102 14149 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:09.701678 14149 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.705583  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:09.707008 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:09.632012 14125 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:09.691044 10018 rpcz_store.cc:269] 0709 12:25:07.795470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.688287 (+1892817us) service_pool.cc:224] Handling call
0709 12:25:09.690205 (+  1918us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.703922 10023 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:09.709550 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:09.708639 10022 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:09.702916 13589 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:09.715195 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.715629 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.731178  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:09.732920 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.706776 14154 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:09.712018 13523 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.738618 13906 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.738730 13961 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:09.740592 13996 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:09.767161  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.736666 10022 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:09.779517 10023 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:09.778828 10025 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
W20260709 12:25:09.774749  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:09.726318 14035 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:09.775043  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:09.712248 14066 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:25:09.782639  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:09.794752 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 101350us
I20260709 12:25:09.774194 14062 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.801591 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:09.804694  9912 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.835s (SENT)
I20260709 12:25:09.805703  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:09.797348 13917 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:09.815064  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55960us
I20260709 12:25:09.779006 13996 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:09.738181 13754 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:25:09.816330 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.776592 14154 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:09.819309 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:09.821448 13235 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:09.822194 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:09.794085 13961 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:09.823489  9991 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:09.812472 14037 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:09.819845 13833 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:09.807813 14035 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:09.833174 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.820750 13996 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.835637 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.836149 13965 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:09.836874 10018 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 8: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:09.831818 13235 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.791164 13906 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.823853 13966 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:09.844334 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1832) took 1294 ms. Trace:
I20260709 12:25:09.845641 10018 rpcz_store.cc:276] 0709 12:25:08.549499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.796357 (+1246858us) service_pool.cc:224] Handling call
0709 12:25:09.844262 (+ 47905us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.823971 13917 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:09.847039 14175 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
I20260709 12:25:09.847339 14035 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:09.833138 13756 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:09.856956  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:09.833595 14111 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:09.854398 14110 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:09.862743 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.863085 14110 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:09.863514 10093 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:09.865743 10016 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:09.863538 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.863513 14110 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:09.868675 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:09.872239 14153 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.821s	user 0.002s	sys 0.002s
I20260709 12:25:09.907424 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:09.874918 13523 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.834308 13789 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.881070 14053 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:09.882129 14037 raft_consensus.cc:2764] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
W20260709 12:25:09.883464 14128 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.574s	user 0.013s	sys 0.000s
I20260709 12:25:09.863847 14111 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:09.818899 13589 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:25:09.909243 10093 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
W20260709 12:25:09.912788 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2906) took 2115 ms (client timeout 2196 ms). Trace:
W20260709 12:25:09.932816 10021 rpcz_store.cc:269] 0709 12:25:07.797715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.863200 (+2065485us) service_pool.cc:224] Handling call
0709 12:25:09.912726 (+ 49526us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.908119 10018 raft_consensus.cc:2376] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11. Re-sending same reply.
I20260709 12:25:09.933751 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1828) took 1385 ms. Trace:
I20260709 12:25:09.934151 10018 rpcz_store.cc:276] 0709 12:25:08.548437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.848375 (+1299938us) service_pool.cc:224] Handling call
0709 12:25:09.933692 (+ 85317us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.914816 10674 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:09.916191 13789 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:09.912356 14053 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:09.918145 10095 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:09.918852 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.918177 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.920688 10020 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
W20260709 12:25:09.922880  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 138086us
W20260709 12:25:09.941732 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 119613us
I20260709 12:25:09.857221 13756 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:09.942987 13756 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:09.929373 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:09.947130 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.947374 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.947229 10016 raft_consensus.cc:2376] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11. Re-sending same reply.
I20260709 12:25:09.947050 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.948325 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1829) took 1399 ms. Trace:
I20260709 12:25:09.836439 13961 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.951131 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1803) took 2167 ms. Trace:
I20260709 12:25:09.952517 10020 rpcz_store.cc:276] 0709 12:25:07.773614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.919557 (+2145943us) service_pool.cc:224] Handling call
0709 12:25:09.920144 (+   587us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.940846 (+ 20702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.941044 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.949168 10016 rpcz_store.cc:276] 0709 12:25:08.548718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.867123 (+1318405us) service_pool.cc:224] Handling call
0709 12:25:09.948269 (+ 81146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.948992 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.950963 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.956573 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:09.959261  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 247037us
I20260709 12:25:09.939854 14125 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:09.965548 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:09.969002 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.969076 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.971958 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.971565 13789 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:09.976446 13789 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:09.976123 14037 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:09.978852  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 155410us
I20260709 12:25:09.867942 14154 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.939142 14128 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:09.983203 10021 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:09.984032 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:09.992586 10670 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:09.988478 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1811) took 2211 ms. Trace:
I20260709 12:25:09.994284 10021 rpcz_store.cc:276] 0709 12:25:07.777273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.936406 (+2159133us) service_pool.cc:224] Handling call
0709 12:25:09.972010 (+ 35604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:09.988070 (+ 16060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.988388 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:09.998689 10090 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.004128 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:10.011401 10099 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:10.012928 10677 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:10.016589 14182 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.017239 14182 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.019547 14182 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:10.023677 10098 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:10.023366 10093 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:09.977888 14053 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.031507 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:10.033684  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 91346.2us
I20260709 12:25:09.989753 14037 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:25:10.041801 10021 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 12:25:10.026252 13965 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:10.044269 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1825) took 1496 ms. Trace:
I20260709 12:25:10.044819 10021 rpcz_store.cc:276] 0709 12:25:08.547487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.003586 (+1456099us) service_pool.cc:224] Handling call
0709 12:25:10.038126 (+ 34540us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:10.043999 (+  5873us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.044219 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.052704 10670 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:10.053825 10019 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:10.053898 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:10.056821 10025 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:10.057814 10099 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:10.058672 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1831) took 1505 ms. Trace:
I20260709 12:25:10.058969 10670 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:10.059079 10019 rpcz_store.cc:276] 0709 12:25:08.549265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.053291 (+1504026us) service_pool.cc:224] Handling call
0709 12:25:10.053647 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.054353 (+   706us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.054521 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.067219 13756 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.079034  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.080605  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.082947 14052 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.083420 14130 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.084259  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.037863 13589 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:25:10.086030  9945 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:10.086227 14053 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.037714 14111 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:10.180116  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108660us
I20260709 12:25:10.035298 13789 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.091799 10021 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:25:10.093330 13523 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:10.093798 10019 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:10.094275  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 69126.3us
I20260709 12:25:10.147035  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.152729  9947 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:10.153893  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.151690 14128 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:10.142288 10090 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:10.094744 13917 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:10.152386 13906 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:10.153493  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:10.128541  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81455.2us
I20260709 12:25:10.143474  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:10.142373 14095 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:10.150897 14158 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.506s	user 0.002s	sys 0.011s
I20260709 12:25:10.175491  9943 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:10.111902 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:10.095944  9948 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:10.088727  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.185606 10094 raft_consensus.cc:2410] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:10.191547 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1830) took 1633 ms. Trace:
I20260709 12:25:10.187067 10672 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260709 12:25:10.184847  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:10.182942 10025 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:10.184307  9944 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
W20260709 12:25:10.427544 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.327698 13756 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:10.283809 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:10.182191 14188 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.202414 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:10.185732 14052 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.430400 10095 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.430207 14188 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.432183 14052 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:10.324220 14172 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.257s	user 0.003s	sys 0.000s
I20260709 12:25:10.432596 14188 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:10.194370 14111 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.293730 14128 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:10.434073 14118 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:10.183168 13235 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:10.296865 13965 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.434670 10021 rpcz_store.cc:276] 0709 12:25:08.549005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.091096 (+1542091us) service_pool.cc:224] Handling call
0709 12:25:10.091546 (+   450us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.182526 (+ 90980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.182740 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.438647 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1860) took 1451 ms. Trace:
W20260709 12:25:10.318488  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 165155us
I20260709 12:25:10.197525 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:10.445458 10090 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.296269  9940 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:10.283798 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:10.367813 10019 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
W20260709 12:25:10.437106 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 153365us
I20260709 12:25:10.435407 14158 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.192446 14130 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.437994 13756 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:10.182399 13523 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:10.438644 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.306517 10016 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:10.434693  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 249925us
I20260709 12:25:10.463052  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:10.301905  9912 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.181s (SENT)
I20260709 12:25:10.452594 10095 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:10.447738 10019 pending_rounds.cc:85] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 8
W20260709 12:25:10.331653 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1819) took 2552 ms (client timeout 2686 ms). Trace:
I20260709 12:25:10.451339 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:10.444998 10092 rpcz_store.cc:276] 0709 12:25:08.987549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.206734 (+219185us) service_pool.cc:224] Handling call
0709 12:25:09.222190 (+ 15456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:09.243535 (+ 21345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:09.243590 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:09.285211 (+ 41621us) log.cc:844] Serialized 52 byte log entry
0709 12:25:09.360761 (+ 75550us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:09.394922 (+ 34161us) log.cc:844] Serialized 14 byte log entry
0709 12:25:09.395167 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:09.395253 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:09.591743 (+196490us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b1800389860. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1
0709 12:25:09.693214 (+101471us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b1800389860. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1
0709 12:25:09.693690 (+   476us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b1800389860. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:10.067003 (+373313us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b1800389860. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:10.323285 (+256282us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:10.438422 (+115137us) raft_consensus.cc:1685] finished
0709 12:25:10.438443 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.438562 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13768,"prepare.queue_time_us":36135,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":67,"raft.queue_time_us":1094,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":18729,"spinlock_wait_cycles":90097536,"thread_start_us":34276,"threads_started":1,"wal-append.queue_time_us":91993}
I20260709 12:25:10.467249  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:10.464951 10024 rpcz_store.cc:269] 0709 12:25:07.779033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.297045 (+1518012us) service_pool.cc:224] Handling call
0709 12:25:10.331322 (+1034277us) spinlock_profiling.cc:243] Waited 78.2 ms on lock 0x7b54001392d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:10.331545 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":179939072}
I20260709 12:25:10.472440  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:10.475932  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:10.464505 10019 pending_rounds.cc:107] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:25:10.457052 13756 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.477169 13523 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.461041 14128 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:25:10.478809 14193 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:10.479893 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:10.480659 14154 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:10.473839 14037 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:10.487319 10019 raft_consensus.cc:2889] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:10.477438  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 188742us
I20260709 12:25:10.458681 14054 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:10.398772 13996 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.548s	user 0.010s	sys 0.002s
I20260709 12:25:10.497057  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.507737  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.442097 14110 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.504052  9941 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:25:10.504096  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.505744 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:10.506744 10668 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 11. Ops: []
I20260709 12:25:10.515566  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:10.516525 14130 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:10.518234 10669 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 9. Ops: []
I20260709 12:25:10.474184 14118 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:10.511812  9943 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:10.508394  9940 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:10.494385 13756 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.490634 14053 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:10.536027 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 644) took 1011 ms. Trace:
I20260709 12:25:10.536482 10019 rpcz_store.cc:276] 0709 12:25:09.524655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.282654 (+757999us) service_pool.cc:224] Handling call
0709 12:25:10.302483 (+ 19829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.535745 (+233262us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.535965 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:10.536567 14053 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.537686 14035 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:10.536094 14189 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:10.538724 14189 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:10.489624 14181 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:10.524765 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.524588 10090 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:10.528503 14169 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:25:10.511832 14149 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:10.522303 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 327627us
I20260709 12:25:10.505059 13589 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.580598 10023 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:10.568606 10062 leader_election.cc:400] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10.
I20260709 12:25:10.582399 10062 leader_election.cc:403] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
W20260709 12:25:10.608256 14148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.024s	user 0.013s	sys 0.004s
W20260709 12:25:10.608982 14148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.013s	sys 0.005s
W20260709 12:25:10.612699 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71877.7us
I20260709 12:25:10.521206 14054 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:10.615718  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69447.3us
W20260709 12:25:10.619278  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 78835.5us
I20260709 12:25:10.519960 14154 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:10.557246 14149 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:10.627754 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.631192  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:10.639074 10677 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:10.534884 13996 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.642282 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:10.645030 10097 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:10.646816 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:10.647464 10093 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:10.648942  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:10.633522  9912 leader_election.cc:336] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.206s (SENT)
I20260709 12:25:10.654798 10091 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:10.672484 10098 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:10.624580 14158 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:10.633267 14149 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:10.614988 13589 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.680445 13235 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:25:10.650687 14125 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:10.652556 13523 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:10.626255 13961 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:25:10.703895 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:10.680258 13906 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:10.710721  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61827us
I20260709 12:25:10.680577 14053 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:10.652038 14054 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.725726 14054 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:10.729586 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 847) took 1961 ms. Trace:
I20260709 12:25:10.683740 14126 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:10.713972 14052 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:10.716132 13589 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:10.613827 14189 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.718957 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1871) took 1192 ms. Trace:
I20260709 12:25:10.755100 10096 rpcz_store.cc:276] 0709 12:25:09.527044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.435203 (+908159us) service_pool.cc:224] Handling call
0709 12:25:10.435555 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.435939 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:10.435965 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:10.436387 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 12:25:10.507987 (+ 71600us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:10.508025 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.508096 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:10.718744 (+210648us) raft_consensus.cc:1685] finished
0709 12:25:10.718765 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:10.718897 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6692,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":104,"thread_start_us":71217,"threads_started":1,"wal-append.queue_time_us":241526}
I20260709 12:25:10.720561 13996 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:10.722986 14062 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10.
I20260709 12:25:10.687158 14128 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:10.730996  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 107534us
W20260709 12:25:10.732547 14197 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:25:10.684440 14130 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.730024 10669 rpcz_store.cc:276] 0709 12:25:08.748350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:08.828252 (+ 79902us) service_pool.cc:224] Handling call
0709 12:25:09.884306 (+1056054us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431eee 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:25:09.884501 (+   195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.518662 (+634161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.707653 (+188991us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b54004c9ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:10.709809 (+  2156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1352235136}
I20260709 12:25:10.652663 13917 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:25:10.744674  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:10.749207 14125 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:10.749843  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:10.750808  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:10.752365 14110 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:10.684803 14104 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:25:10.744165 13906 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:10.753839 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 185440us
I20260709 12:25:10.771720 13235 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:10.743330 14052 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:10.736137 14126 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:10.735657 14118 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 12:25:10.788384 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:10.774317 13850 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:10.800591  9912 leader_election.cc:336] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.555s (SENT)
I20260709 12:25:10.792243 13917 consensus_queue.cc:1243] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 9.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:10.792001 13996 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:10.791108 14204 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:10.746126 14154 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.794277 13850 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:10.810827 14203 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:10.811249 14199 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:25:10.811365 14181 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:10.811800 13906 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:10.812673 10062 leader_election.cc:400] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:10.827246 10062 leader_election.cc:403] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:25:10.828145 14181 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11.
I20260709 12:25:10.776154 14125 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.830451 14125 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:10.807077 14130 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.830709 14052 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.832075 14052 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.833665 13833 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:10.791668 14054 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:25:10.822695 10020 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 6. Current term is 7. Ops: []
I20260709 12:25:10.825505 14158 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:10.838215  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.840312 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:10.840924 10092 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:10.843403 13906 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:10.848132 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:10.848452 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:10.848950 10674 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:25:10.849905 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:10.851145 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2917) took 3038 ms. Trace:
I20260709 12:25:10.851194 10668 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:25:10.851541 10020 rpcz_store.cc:276] 0709 12:25:07.801117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.981887 (+2180770us) service_pool.cc:224] Handling call
0709 12:25:10.027146 (+ 45259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.822503 (+795357us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b540024a520. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:25:10.839583 (+ 17080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.839794 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42387840}
I20260709 12:25:10.813884 14204 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:10.817029 13850 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:10.855611 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:10.864878 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:10.871479 14179 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:10.872053 14179 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:10.873242 13833 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:25:10.875401 14202 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:25:10.873100 14179 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.839896 14158 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:10.877317  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:10.866853 14208 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:10.888932 14128 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:10.892516 14158 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.894107 14158 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.904167 14179 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:10.908737 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 824) took 1002 ms. Trace:
I20260709 12:25:10.909363 10672 rpcz_store.cc:276] 0709 12:25:09.926894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.935665 (+  8771us) service_pool.cc:224] Handling call
0709 12:25:09.954872 (+ 19207us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:10.437653 (+482781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:10.908296 (+470643us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54007f3ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:10.908642 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":533110656}
I20260709 12:25:10.913578 14203 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:10.861157 14204 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:10.924369 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:10.919807  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 65481.7us
I20260709 12:25:10.925102 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:10.926349 10670 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:10.883466 14104 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:10.933306  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.936303 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:10.937075 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:10.938310 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:10.939011 10671 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:10.939975 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:10.943310 10676 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:10.946350 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:10.946338 10672 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:10.956580 14118 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.957417 13523 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:10.957851 13523 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:10.958746 13523 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.960050 13523 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:10.961035  9942 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 7.
I20260709 12:25:10.957182 14118 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.963107 13773 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:10.963487 13773 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:10.964443 14118 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:10.964375 13773 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.966135 13773 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:10.965278 14128 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:10.967089 14128 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:10.978214  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:10.979363  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:10.984828 10642 connection.cc:256] RPC call timeout handler was delayed by -1.2006145639998067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2948s for a response
I20260709 12:25:10.988408 14104 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:25:10.995517 14128 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:10.995841 14221 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:25:10.992751  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:10.996937 14118 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.014950 14118 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.017601 14118 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.018834 14104 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:11.019444 14104 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:11.020192 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.020344 14104 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.023919 13756 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:10.997196 14052 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:10.987754 14054 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.026847 14054 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.026994 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:11.028928 14054 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:10.988690 14169 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:25:11.031181 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.032018  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:11.036417  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:11.043716 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:11.034984 14169 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:11.070015 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 115011us
I20260709 12:25:11.061764 14052 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:11.044893 14218 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:11.082521 10018 pending_rounds.cc:85] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 4
I20260709 12:25:11.088338 10018 pending_rounds.cc:107] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:25:11.088708 10018 raft_consensus.cc:2889] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:11.084429 10062 leader_election.cc:336] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.207s (SENT)
W20260709 12:25:11.085511 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.086465 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.074204 14169 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:11.081950 14035 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.037488 14119 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:25:11.087798 14052 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:11.111781 14052 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:11.109622  9843 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 310) took 22586 ms (client timeout 15000 ms). Trace:
I20260709 12:25:11.113643 13235 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:11.114174 13235 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:11.097934 13917 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 12:25:11.115211 13235 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.091876 14218 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:11.088191 13756 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:11.119395 10673 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:11.123706 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:11.128168  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:11.103003 14035 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:11.140255 13235 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:11.124300 14181 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.159375 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:11.160382 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 75973.4us
W20260709 12:25:11.161105 10062 connection.cc:256] RPC call timeout handler was delayed by -2.0299279440000646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2686s for a response
I20260709 12:25:11.124095 14218 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.178769 14158 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:11.188053 10062 connection.cc:256] RPC call timeout handler was delayed by -1.6699279439999373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3086s for a response
I20260709 12:25:11.179546 13756 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:11.180464 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:11.135658 14119 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:11.181409  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:11.133246 14188 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:11.141069 13789 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:11.190246 10093 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:11.197137 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:11.123340 13850 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.204738  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:11.199362 14119 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:11.113864  9843 rpcz_store.cc:269] 0709 12:24:48.543357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:48.553644 (+ 10287us) service_pool.cc:224] Handling call
0709 12:25:11.083223 (+22529579us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:04.731554 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:04.875716 (+144162us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:04.875775 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:25:04.875787 (+    12us) tablet.cc:662] Decoding operations
0709 12:25:05.025774 (+149987us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:05.025861 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:05.026014 (+   153us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:25:05.602934 (+576920us) tablet.cc:701] Row locks acquired
0709 12:25:05.623548 (+ 20614us) write_op.cc:260] PREPARE: finished
0709 12:25:05.783733 (+160185us) write_op.cc:270] Start()
0709 12:25:05.839406 (+ 55673us) write_op.cc:276] Timestamp: P: 1783599905709672 usec, L: 0
0709 12:25:05.859215 (+ 19809us) op_driver.cc:348] REPLICATION: starting
0709 12:25:06.098481 (+239266us) log.cc:844] Serialized 7559 byte log entry
0709 12:25:07.036056 (+937575us) op_driver.cc:464] REPLICATION: finished
0709 12:25:07.318196 (+282140us) write_op.cc:301] APPLY: starting
0709 12:25:07.325016 (+  6820us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:07.328367 (+  3351us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:07.328520 (+   153us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:07.331753 (+  3233us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:07.332100 (+   347us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=38
0709 12:25:07.332155 (+    55us) write_op.cc:312] APPLY: finished
0709 12:25:07.332613 (+   458us) log.cc:844] Serialized 132 byte log entry
0709 12:25:07.355615 (+ 23002us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:07.368388 (+ 12773us) write_op.cc:454] Released schema lock
0709 12:25:07.373452 (+  5064us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:07.391571 (+ 18119us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:07.399787 (+  8216us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4614221,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":244945,"apply.run_cpu_time_us":5560,"apply.run_wall_time_us":74956,"num_ops":19,"prepare.queue_time_us":687109,"prepare.run_cpu_time_us":27370,"prepare.run_wall_time_us":1742016,"raft.queue_time_us":394665,"raft.run_cpu_time_us":70728,"raft.run_wall_time_us":264828,"replication_time_us":1167989,"thread_start_us":1056506,"threads_started":4,"wal-append.queue_time_us":459297,"wal-append.run_cpu_time_us":36478,"wal-append.run_wall_time_us":1815874}]]}
I20260709 12:25:11.198859 14062 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:11.235096 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:11.203624 13996 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:11.234789 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84579.3us
I20260709 12:25:11.223628 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:11.247957 14110 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.313730 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:11.251499  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 82645.3us
W20260709 12:25:11.270660  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:11.294581 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.299789 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 646) took 1007 ms. Trace:
W20260709 12:25:11.303062 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 80697.4us
I20260709 12:25:11.390355 10017 rpcz_store.cc:276] 0709 12:25:10.292726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.154960 (+862234us) service_pool.cc:224] Handling call
0709 12:25:11.299702 (+144742us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:11.406672  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 203585us
I20260709 12:25:11.201629 14181 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:25:11.268026 14119 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:11.247334 14095 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:11.199527 14149 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:11.451570 14227 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:25:11.441278 14214 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:11.448086 14193 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:11.406731 14110 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.411697 14189 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
W20260709 12:25:11.511432 13773 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.545s	user 0.013s	sys 0.002s
I20260709 12:25:11.411600 14062 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.480374 14158 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 12:25:11.503552 14095 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:11.537671 14234 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:25:11.514257 14214 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.543928 13850 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.558949 14214 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.562312 14149 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:11.565254 14181 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:11.566268 14179 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.661s	user 0.008s	sys 0.007s
W20260709 12:25:11.561673 13523 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.001s
I20260709 12:25:11.567334 13996 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:11.583071 13996 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.583336 14169 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.588661 14169 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:11.574566 14204 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.639s	user 0.011s	sys 0.002s
I20260709 12:25:11.573764 14062 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.594794 14204 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.597759 14204 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.599864 14110 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.601080 14095 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:11.603240 14035 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.607131 14035 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.604954 14052 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.609139 14052 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.598454 14218 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.613497 14218 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:11.611346 14119 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:11.616508 14119 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:11.720904 14130 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:11.761134 10094 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 638) took 1842 ms (client timeout 1922 ms). Trace:
W20260709 12:25:11.761936 10094 rpcz_store.cc:269] 0709 12:25:09.918438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.918560 (+   122us) service_pool.cc:224] Handling call
0709 12:25:11.408531 (+1489971us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540022ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:11.761017 (+352486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556738304}
I20260709 12:25:11.763325 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:11.764783 10094 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:25:11.825119 13965 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:11.825624 13965 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:11.826483 13965 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.845396 14126 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:11.846678 14126 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.855265 10024 raft_consensus.cc:2410] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:11.856324 10096 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:11.864954 10096 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:11.414731  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:11.878979 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:11.879640 10668 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:11.899003 14111 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:11.899533 14111 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:11.900305 14111 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:11.901160 14111 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:11.906800 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.910033 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.911367 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.912142 10017 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:11.913334 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.914089 10017 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:11.914969 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:11.915616 10017 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:11.923920 10017 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:11.924955 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.925578 10017 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:11.926404 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.927004 10017 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:11.927825 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.928395 10017 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:11.929209 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.930207 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.931284 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:11.932564 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 828) took 1907 ms. Trace:
W20260709 12:25:11.932854  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:11.933004 10677 rpcz_store.cc:276] 0709 12:25:10.024680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.028109 (+  3429us) service_pool.cc:224] Handling call
0709 12:25:10.028398 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.641893 (+613495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.932296 (+1290403us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540050a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:11.932489 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403876224}
I20260709 12:25:11.931900 10017 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:11.943637 10017 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:11.944978 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.945806 10017 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:11.946743 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.949594 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.950495 10024 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:11.955044 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.010s	sys 0.021s
W20260709 12:25:11.955534 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.010s	sys 0.021s
I20260709 12:25:11.954312 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:11.956188 10024 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:11.958526 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 519094us
I20260709 12:25:11.970258 10674 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
W20260709 12:25:11.979794 10025 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.024s	sys 0.028s
W20260709 12:25:11.980322 10025 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.024s	sys 0.028s
I20260709 12:25:11.981403 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:11.982023 10017 raft_consensus.cc:2376] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 11. Re-sending same reply.
I20260709 12:25:11.988694 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:11.989805 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.990389 10025 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:11.991063 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:11.991312 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:11.991664 10677 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:11.991919 10025 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:11.993141 10025 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:11.994870 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:11.994410 10025 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:12.001809 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:11.999677 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:12.002853 10024 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.002482 10672 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.005679 10677 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:11.987901 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:12.011546 10017 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:12.036026 10017 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:12.038302 10017 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:12.044535 10025 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:12.062937 10025 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:12.292392 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 653) took 1138 ms. Trace:
I20260709 12:25:12.292981 10025 rpcz_store.cc:276] 0709 12:25:11.154055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.045319 (+891264us) service_pool.cc:224] Handling call
0709 12:25:12.062690 (+ 17371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.291996 (+229306us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.292299 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:11.421041 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:12.364478 10672 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:12.364849 10025 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:12.375059 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 654) took 1220 ms. Trace:
I20260709 12:25:12.375480 10025 rpcz_store.cc:276] 0709 12:25:11.154352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.294173 (+1139821us) service_pool.cc:224] Handling call
0709 12:25:12.364387 (+ 70214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.365337 (+   950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.374971 (+  9634us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:12.380151 10025 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:11.424005 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2901) took 3611 ms. Trace:
I20260709 12:25:12.380771 10016 rpcz_store.cc:276] 0709 12:25:07.796063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.954156 (+2158093us) service_pool.cc:224] Handling call
0709 12:25:09.985787 (+ 31631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.462281 (+476494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.407333 (+945052us) spinlock_profiling.cc:243] Waited 320 ms on lock 0x7b54001383d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:11.407751 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":736993536}
I20260709 12:25:12.382807 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2943) took 1200 ms. Trace:
I20260709 12:25:12.383296 10025 rpcz_store.cc:276] 0709 12:25:11.182496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.376192 (+1193696us) service_pool.cc:224] Handling call
0709 12:25:12.379166 (+  2974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.382354 (+  3188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.382700 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:12.399886 10669 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:12.403400 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 982431us
W20260709 12:25:12.409689 10018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.295s	user 0.009s	sys 0.021s
W20260709 12:25:12.410086 10018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.295s	user 0.009s	sys 0.021s
I20260709 12:25:12.410823 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2926) took 1348 ms. Trace:
I20260709 12:25:12.411259 14254 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:12.411201 10018 rpcz_store.cc:276] 0709 12:25:11.062739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.062862 (+   123us) service_pool.cc:224] Handling call
0709 12:25:11.071684 (+  8822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.099149 (+ 27465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.099180 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.114763 (+ 15583us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.410455 (+1295692us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:12.410497 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.410568 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.410627 (+    59us) raft_consensus.cc:1685] finished
0709 12:25:12.410638 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.410769 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1295334,"threads_started":1,"wal-append.queue_time_us":435972}
I20260709 12:25:12.412990 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:12.413789 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 825) took 2463 ms. Trace:
I20260709 12:25:12.413836 14254 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:12.415627 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:12.416309 10093 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:25:12.416774 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1884) took 1011 ms. Trace:
I20260709 12:25:12.414319 10673 rpcz_store.cc:276] 0709 12:25:09.949985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:09.950121 (+   136us) service_pool.cc:224] Handling call
0709 12:25:09.962927 (+ 12806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:11.121037 (+1158110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.413530 (+1292493us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54007e4fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:12.413705 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2659015936}
I20260709 12:25:12.416973 10093 rpcz_store.cc:276] 0709 12:25:11.405557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.415328 (+1009771us) service_pool.cc:224] Handling call
0709 12:25:12.416731 (+  1403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:12.423348 14254 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:12.443969 10024 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:12.447381 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 652) took 1293 ms. Trace:
I20260709 12:25:12.451608 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:12.455586 10024 rpcz_store.cc:276] 0709 12:25:11.153768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.987608 (+833840us) service_pool.cc:224] Handling call
0709 12:25:12.447310 (+459702us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:12.460098 13965 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:12.461534 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.462173 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.461259 14254 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:12.471900  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 785) took 2396 ms. Trace:
I20260709 12:25:12.472560  9946 rpcz_store.cc:276] 0709 12:25:10.075226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.079618 (+  4392us) service_pool.cc:224] Handling call
0709 12:25:10.079935 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.749536 (+669601us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:10.804500 (+ 54964us) log.cc:844] Serialized 14 byte log entry
0709 12:25:10.993677 (+189177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:10.993730 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:10.993778 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.471480 (+1477702us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b5400207954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:12.471553 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.471758 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":58814,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":31,"raft.queue_time_us":1347,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":177,"spinlock_wait_cycles":1539020672,"thread_start_us":204172,"threads_started":2,"wal-append.queue_time_us":177965,"wal-append.run_cpu_time_us":1528,"wal-append.run_wall_time_us":1005179}
I20260709 12:25:12.481307 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 638) took 2958 ms. Trace:
I20260709 12:25:12.481936 10023 rpcz_store.cc:276] 0709 12:25:09.522861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.068150 (+545289us) service_pool.cc:224] Handling call
0709 12:25:10.068511 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:10.581721 (+513210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.480892 (+1899171us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b54001f7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:12.481175 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177279232}
I20260709 12:25:12.485491 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2920) took 1494 ms. Trace:
I20260709 12:25:12.486123 10020 rpcz_store.cc:276] 0709 12:25:10.991569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.996137 (+  4568us) service_pool.cc:224] Handling call
0709 12:25:10.996547 (+   410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.009515 (+ 12968us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:25:11.123758 (+114243us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.191113 (+ 67355us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.246529 (+ 55416us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.246755 (+   226us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:11.250549 (+  3794us) log.cc:844] Serialized 14 byte log entry
0709 12:25:11.272078 (+ 21529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.272172 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.485102 (+1212930us) raft_consensus.cc:1685] finished
0709 12:25:12.485134 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.485391 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22170,"prepare.queue_time_us":382871,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":63890,"raft.run_cpu_time_us":426,"raft.run_wall_time_us":113218,"spinlock_wait_cycles":17492352,"thread_start_us":113969,"threads_started":2,"wal-append.queue_time_us":174120}
I20260709 12:25:12.494972 14259 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:12.495677 14259 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.501833 14259 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:12.502473 14259 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:12.502578 10022 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.210s	user 0.021s	sys 0.049s
W20260709 12:25:12.509500 14253 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.123s	user 0.000s	sys 0.004s
I20260709 12:25:12.516247 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 969) took 1603 ms. Trace:
I20260709 12:25:12.528849 10669 rpcz_store.cc:276] 0709 12:25:10.912776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:10.924865 (+ 12089us) service_pool.cc:224] Handling call
0709 12:25:10.937274 (+ 12409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.400383 (+1463109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.515871 (+115488us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54007f4754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:12.516154 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3363265408}
W20260709 12:25:12.540486 10022 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.248s	user 0.024s	sys 0.049s
W20260709 12:25:12.563372  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.14949e+06us
I20260709 12:25:12.567346 14260 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:12.567978 14260 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.568403 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:12.569476 13965 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:12.569640 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:12.570020 13965 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:12.570222 14260 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:12.574407 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 150828us
I20260709 12:25:12.545282 14128 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.520661 14118 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.539840 14125 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.573398 13756 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:12.574704 14260 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:12.545269 14193 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:12.586129 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:12.586774 10098 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:12.587826 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 650) took 1434 ms. Trace:
I20260709 12:25:12.588621 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2921) took 1587 ms. Trace:
W20260709 12:25:12.590298  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.590660 14054 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.590941 14110 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.591195 14054 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.591425 14110 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.593326 14110 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:12.583649 13756 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:12.594498 13756 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.588222 10017 rpcz_store.cc:276] 0709 12:25:11.153167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.038983 (+885816us) service_pool.cc:224] Handling call
0709 12:25:12.039220 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.587428 (+548208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.587463 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:12.587510 (+    47us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:12.587556 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.587633 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.587767 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":547361}
W20260709 12:25:12.601099 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87809.9us
I20260709 12:25:12.602625 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:12.589226 10019 rpcz_store.cc:276] 0709 12:25:11.000957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.011208 (+ 10251us) service_pool.cc:224] Handling call
0709 12:25:11.040741 (+ 29533us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:11.041306 (+   565us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.041351 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:11.042149 (+   798us) log.cc:844] Serialized 52 byte log entry
0709 12:25:12.387707 (+1345558us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b18002b9f60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1
0709 12:25:12.387920 (+   213us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:12.411087 (+ 23167us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.588057 (+176970us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.588233 (+   176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.588301 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.588425 (+   124us) raft_consensus.cc:1685] finished
0709 12:25:12.588439 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.588563 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128741,"prepare.queue_time_us":56257,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":89,"raft.queue_time_us":148824,"raft.run_cpu_time_us":249,"raft.run_wall_time_us":3136,"spinlock_wait_cycles":4664576,"thread_start_us":90730,"threads_started":3,"wal-append.queue_time_us":93048,"wal-append.run_cpu_time_us":3141,"wal-append.run_wall_time_us":1242479}
I20260709 12:25:12.607546  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:12.613816 10018 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:25:12.613993 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2925) took 1551 ms. Trace:
I20260709 12:25:12.545312 14259 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.579108 14128 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.593333 14054 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:12.618525 10020 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 12:25:12.581543 14118 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.616353 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:12.623098 10023 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.619282 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:12.614338 10022 rpcz_store.cc:276] 0709 12:25:11.062373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.064744 (+  2371us) service_pool.cc:224] Handling call
0709 12:25:11.081521 (+ 16777us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.094584 (+ 13063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.094656 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.120116 (+ 25460us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.267980 (+147864us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:12.541097 (+1273117us) log.cc:844] Serialized 14 byte log entry
0709 12:25:12.613676 (+ 72579us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.613755 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:12.613793 (+    38us) raft_consensus.cc:1685] finished
0709 12:25:12.613820 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:12.613937 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61961,"prepare.queue_time_us":149024,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":7617,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":11844,"thread_start_us":1362626,"threads_started":3,"wal-append.queue_time_us":318375,"wal-append.run_cpu_time_us":1703,"wal-append.run_wall_time_us":1003777}
W20260709 12:25:12.620604 10642 leader_election.cc:336] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.678s (SENT)
I20260709 12:25:12.621140 10020 pending_rounds.cc:85] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 7
I20260709 12:25:12.622071 10025 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:12.624938 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:12.583503 14260 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.626148 10020 pending_rounds.cc:107] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 12:25:12.626479 14037 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:12.606005 14193 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.626832 14126 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:12.606906 13965 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.626510 10020 raft_consensus.cc:2889] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:12.583340 14125 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.629676 14260 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:12.631282 10024 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:12.615958 14204 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.635506 10016 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:12.615680 14035 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:12.638334 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.639042 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.644728 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:12.644614 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50402.9us
I20260709 12:25:12.621928 14257 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:12.646054 14035 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.646080 14260 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:12.646888 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:12.647003 14260 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.647514 10018 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:12.648404 14035 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:12.648500 14204 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.649219 14260 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:12.649775 14260 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.650209 14260 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.650717 14204 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:12.652202  9942 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:12.653968  9949 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:12.654264 14236 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.654918 14236 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:12.657081 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.657351 14236 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:12.657435 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:12.657908  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.2525e+06us
W20260709 12:25:12.658579  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.667066 10020 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:25:12.666374 10670 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:12.668272 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:12.676787  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:12.672652  9991 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.596s (SENT) [suppressed 15 similar messages]
I20260709 12:25:12.683050  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:12.688423 10025 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:12.689155  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:12.689572  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:12.690155  9946 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:12.692111  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:12.692867  9943 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:12.694727 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:12.696893 10020 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:12.695474  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:12.699761  9949 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:12.701290 10018 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:12.652223 14260 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:12.676285 14111 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:12.704311  9944 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:25:12.705668  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.706629 10023 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:12.675679 14052 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:12.707859 13235 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:12.692073 14035 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:12.691870 14125 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:12.721833  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:12.675450 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:12.713372  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:12.694684 13850 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.713024 13235 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.714726  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 96300.6us
I20260709 12:25:12.723212 10022 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:12.723387 13235 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.702689 14236 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.716255 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:12.732172 10096 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:12.717018 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:12.733121 10097 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:12.717126 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:12.733970 10090 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.722685  9949 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.709642 14111 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.701759 13965 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:12.702419 14238 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.706496 10016 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:12.713693 14264 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:12.743645  9946 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:12.749531 13235 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:12.767577 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 80392.5us
I20260709 12:25:12.770112 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:12.756539 14274 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.744009  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:12.771644  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:12.773726 14272 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:12.774289 14272 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.774780 14271 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:12.775101 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:12.700947 13944 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:25:12.715502 14035 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.776440 14274 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.761762 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:12.777619 10093 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.777782 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:12.710844 14214 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:12.763430  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 77411.9us
W20260709 12:25:12.754369  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81744.6us
I20260709 12:25:12.780061 14214 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.768959  9944 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:12.775209 14271 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:12.777072 14035 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:12.732745 14257 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:25:12.758396 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.757392 13965 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:12.757846 14111 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.757612 14130 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:12.791877  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:12.784190 13235 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:12.789314 10093 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:12.799933 10090 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:12.710767 14266 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:25:12.811426 14266 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:12.811770 14266 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:12.743898 13850 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.762650 14236 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.718694 14052 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.813411 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:12.792410 14169 raft_consensus.cc:476] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 LEADER]: Not starting pre-election -- already a leader
I20260709 12:25:12.808334 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:12.822330 13235 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.831835 10090 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:12.801807 13965 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.786033 14271 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.800218 13944 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:12.828817  9949 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
W20260709 12:25:12.829586  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.831832 13235 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.795243 14257 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.829686 14265 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:25:12.815366 13996 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:12.788462 14275 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:25:12.781451 14238 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:12.849092 14268 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
W20260709 12:25:12.852058 10642 leader_election.cc:336] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.910s (SENT)
I20260709 12:25:12.833402 14244 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.814392 14130 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.888854  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:12.830785 14277 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:12.832728 14169 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:12.902931 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 132562us
I20260709 12:25:12.848662 14257 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:12.919034 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 67012.7us
I20260709 12:25:12.922585  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:12.923148 10642 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.922s (SENT)
I20260709 12:25:12.923312  9943 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:12.918934  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:12.923918  9947 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.839007 14128 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:12.850795 13996 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:12.922246  9831 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 1.700s	user 0.008s	sys 0.015s
I20260709 12:25:12.894501 14275 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:12.850052 13944 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:12.920349 14110 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:12.938735  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:12.939960  9949 raft_consensus.cc:2410] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:25:12.918982 14273 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.945905 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:12.946177 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:12.943908  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:12.946964 10023 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.933359 13996 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.946377 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:12.960794 10018 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.946481 10095 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:12.932291 14279 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.967294 14286 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.967417 14279 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.967871 14286 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:12.969648 14279 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:12.969981  9947 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:12.970115 14286 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:12.954013 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:12.957214 10023 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:12.963008 14037 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:12.978106 14130 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:12.963310 14270 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:25:12.964272 14257 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:12.964663  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:12.980059 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:12.980741 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:12.971043  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:12.972061 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:12.956645 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:12.994812 10025 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.952085  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:12.976509 14259 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:12.998607 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.004346 10095 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:13.008690 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:13.012852  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:13.014737  9840 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:13.017714  9840 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 9, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 9 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:13.020682  9840 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 3 to 9. New cstate: current_term: 9 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:13.023186 14118 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:13.023589 10021 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.722s	user 0.019s	sys 0.036s
W20260709 12:25:13.024057 10021 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.723s	user 0.019s	sys 0.036s
I20260709 12:25:13.023149  9840 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:13.027022  9947 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:12.966849 14288 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.932806 14169 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.055681 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.058395 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.058874 10095 raft_consensus.cc:2410] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:25:12.970173 14279 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.951401 14281 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:12.925326 14244 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.931342 14277 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.951547 14273 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:12.943881 13756 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.110s
I20260709 12:25:13.270151 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2923) took 2231 ms. Trace:
I20260709 12:25:13.093211 13944 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.256947 13850 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.286072  9947 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:13.110970 14293 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:13.285339 10021 rpcz_store.cc:276] 0709 12:25:11.038493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.038671 (+   178us) service_pool.cc:224] Handling call
0709 12:25:11.043116 (+  4445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.043479 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.043505 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.044029 (+   524us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.249598 (+205569us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:11.888740 (+639142us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b4800002958. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:13.024452 (+1135712us) log.cc:844] Serialized 14 byte log entry
0709 12:25:13.179874 (+155422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.179985 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.180040 (+    55us) raft_consensus.cc:1685] finished
0709 12:25:13.269868 (+ 89828us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.270071 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":694620,"prepare.queue_time_us":167151,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":15468,"raft.run_cpu_time_us":4230,"raft.run_wall_time_us":845879,"spinlock_wait_cycles":4790144,"thread_start_us":2082716,"threads_started":3,"wal-append.queue_time_us":498338,"wal-append.run_cpu_time_us":2040,"wal-append.run_wall_time_us":1058112}
I20260709 12:25:13.287273  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.287936  9947 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:13.043789 14286 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.288132 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.288820 10021 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:13.288636 14286 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.101415 14279 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:13.289714 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.289345  9840 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 9, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 9 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:13.061321 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.287395 14293 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.277408 10025 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:13.278573 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:13.374259 10095 raft_consensus.cc:2376] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10. Re-sending same reply.
I20260709 12:25:13.375533 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.376770 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.097595  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:13.177572  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 323924us
I20260709 12:25:13.116407 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.105938 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:13.288909  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:13.121335 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.101315 10018 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:13.290319 10021 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:25:13.254523 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 333281us
I20260709 12:25:12.945096 14266 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.459661  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.460317  9946 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:13.066992 14257 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.098362 14284 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.053340 14130 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:13.511119 14244 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.373481 14118 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.511442  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.512734  9940 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:25:13.454350 14193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.015s	sys 0.020s
I20260709 12:25:13.509387 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.105497 14238 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.527642 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.514523 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.098066 14259 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:13.531189  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.531508 14259 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:13.521061 14193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.023s	sys 0.027s
W20260709 12:25:13.515585 14291 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.410s	user 0.000s	sys 0.000s
I20260709 12:25:13.542553  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:13.291070 14119 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.516085 10020 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:13.376948 14281 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:13.524466 10671 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.506062  9840 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 4 to 9, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 9 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:13.529471  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.514829 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:13.551484 10095 raft_consensus.cc:2410] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 11 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:13.293782 14275 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:13.530965 14282 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:25:13.530849 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.530412  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.553027  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.060431 14288 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.543385 10017 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:13.514935 14257 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.492236 13756 consensus_queue.cc:1243] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:13.286006 14277 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.377727 14274 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.553802  9946 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:13.556974  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.558976 10020 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:25:13.561787 10022 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: [6.9-8.10]
W20260709 12:25:13.563062 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.536468 14284 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.542578 14266 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.534260 14169 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.565405 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.569057 10017 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 12:25:13.569793 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3015) took 2439 ms. Trace:
I20260709 12:25:13.569203 10020 pending_rounds.cc:85] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 7
I20260709 12:25:13.573427 10020 pending_rounds.cc:107] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 7.8
W20260709 12:25:13.573515 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116913us
I20260709 12:25:13.574018 10020 raft_consensus.cc:2889] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:13.570990  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 277179us
W20260709 12:25:13.575578 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56366.6us
I20260709 12:25:13.576233 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.565764  9840 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 9, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 9 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:13.572893 10022 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:25:13.579501 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.577399 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.537866 14118 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.573105 10091 rpcz_store.cc:276] 0709 12:25:11.130831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.131014 (+   183us) service_pool.cc:224] Handling call
0709 12:25:11.131480 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:11.132009 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:11.132052 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:11.140469 (+  8417us) log.cc:844] Serialized 26 byte log entry
0709 12:25:11.240530 (+100061us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:11.240585 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.240682 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.569596 (+2328914us) raft_consensus.cc:1685] finished
0709 12:25:13.569615 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.569743 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":627,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":16647,"thread_start_us":38348,"threads_started":1,"wal-append.queue_time_us":224133,"wal-append.run_cpu_time_us":2215,"wal-append.run_wall_time_us":1125026}
I20260709 12:25:13.527933 14130 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:13.569582 14285 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.092s	user 0.001s	sys 0.001s
I20260709 12:25:13.564272 14275 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.555559 14119 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.565464 14286 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:13.708820 10642 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.024s	sys 0.038s
W20260709 12:25:13.843055 10642 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.024s	sys 0.047s
W20260709 12:25:13.650007  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63448.8us
W20260709 12:25:13.843554 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 706667us
I20260709 12:25:13.630220 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:13.616329  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.844754  9947 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:25:13.823486 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3017) took 2691 ms. Trace:
W20260709 12:25:13.797755  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 213040us
I20260709 12:25:13.586087  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.710729  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 795) took 1030 ms. Trace:
I20260709 12:25:13.845851 10092 rpcz_store.cc:276] 0709 12:25:11.131571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.131694 (+   123us) service_pool.cc:224] Handling call
0709 12:25:11.145186 (+ 13492us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:11.153532 (+  8346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:11.153575 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:11.161130 (+  7555us) log.cc:844] Serialized 52 byte log entry
0709 12:25:11.423589 (+262459us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:11.423647 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:11.423743 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.823185 (+2399442us) raft_consensus.cc:1685] finished
0709 12:25:13.823210 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.823412 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149,"raft.queue_time_us":100,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":33596,"thread_start_us":200489,"threads_started":1,"wal-append.queue_time_us":332825,"wal-append.run_cpu_time_us":2118,"wal-append.run_wall_time_us":1036109}
W20260709 12:25:13.817677 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 240505us
I20260709 12:25:13.613107  9840 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:13.856154 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.856959 10091 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:13.853838  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:13.858027  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:13.618445 14281 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:13.614073 14271 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.001s
I20260709 12:25:13.859421 14271 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:13.859889 14281 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:13.860517 14281 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:13.861888 14271 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:13.861542 14281 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.862784 14300 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:13.863368 14300 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:13.864645 14275 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:13.864367 14300 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.865257 14275 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:13.865764 14275 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:13.850239 14277 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.618736 14283 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.248s
I20260709 12:25:13.867063 14272 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:13.836179 14259 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.856055 14284 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.590404 14287 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.278s
I20260709 12:25:13.835685 14288 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.869771 10671 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:13.624008 14274 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.246s
I20260709 12:25:13.871088 10016 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:13.872752 14275 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.858130 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:13.870440 10671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 977) took 1883 ms (client timeout 1743 ms). Trace:
I20260709 12:25:13.873493  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.879233 10091 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:13.871126  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.866742 14277 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:13.879177 10671 rpcz_store.cc:269] 0709 12:25:11.987446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.055101 (+1067655us) service_pool.cc:224] Handling call
0709 12:25:13.870312 (+815211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.848395  9941 rpcz_store.cc:276] 0709 12:25:12.680671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.680832 (+   161us) service_pool.cc:224] Handling call
0709 12:25:12.702982 (+ 22150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:13.249600 (+546618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.249649 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:13.265824 (+ 16175us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:13.265885 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.710338 (+444453us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:13.710409 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.710643 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6797,"raft.queue_time_us":118727,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":10359,"spinlock_wait_cycles":521610496,"thread_start_us":306216,"threads_started":1}
I20260709 12:25:13.883437  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.884334  9943 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:13.885087 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:13.875234  9840 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 4 to 6, leader changed from <none> to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 6 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:13.880280 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.623737 14293 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:13.887025 10091 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:13.621523 14257 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.618906 14279 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:13.622738 14273 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.793195 14306 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:13.893795 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 308065us
I20260709 12:25:13.894075 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.892045  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:13.582965 14294 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.319s
I20260709 12:25:13.898865 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:13.897310 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 326218us
I20260709 12:25:13.900852 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.897663 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.901690 10091 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
W20260709 12:25:13.900068 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48019.1us
I20260709 12:25:13.895676  9840 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 8, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 8 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:13.902508  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.903107  9943 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:13.901197 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:13.879022 14286 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.024s
I20260709 12:25:13.904502 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.906263 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:13.906263 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.908455 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.909652 14287 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:13.905515 14257 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:25:13.909765 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:13.917379 10670 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:13.884385 14299 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:25:13.911093 14277 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:13.908691 14283 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:13.907357 14288 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:25:13.914399 10091 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:13.917379  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:13.907192 14273 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:13.919111 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:13.941586 14288 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:13.941802 14273 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:13.942081 14288 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:13.942293 14273 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.942795 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.943147 14288 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.944368 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.944464 14288 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:13.584324 13756 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.361s
I20260709 12:25:13.945170 14272 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:13.945911 14272 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:13.946295 14272 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:13.946687 14272 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:13.923005  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 796) took 1227 ms. Trace:
I20260709 12:25:13.939428 14283 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:13.947644 14283 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:13.947510  9945 rpcz_store.cc:276] 0709 12:25:12.692548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.692747 (+   199us) service_pool.cc:224] Handling call
0709 12:25:12.707413 (+ 14666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:12.707636 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:12.707660 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:12.726419 (+ 18759us) log.cc:844] Serialized 26 byte log entry
0709 12:25:12.935770 (+209351us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:12.937110 (+  1340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:12.937220 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:13.919816 (+982596us) raft_consensus.cc:1685] finished
0709 12:25:13.919837 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:13.919964 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":147282,"threads_started":1,"wal-append.queue_time_us":363148}
I20260709 12:25:13.949313  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.949906  9945 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:13.950373  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1957) took 1057 ms. Trace:
I20260709 12:25:13.950980  9945 rpcz_store.cc:276] 0709 12:25:12.892706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.949021 (+1056315us) service_pool.cc:224] Handling call
0709 12:25:13.950315 (+  1294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:13.923143 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:13.924048 14299 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:13.952797 14299 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:13.953083 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:13.953754 10674 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:25:13.916970 14287 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:13.957748 14284 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:13.957976 13944 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:13.958453 13944 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:13.958822 14259 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:25:13.959949 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:13.960598  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:13.924724 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:13.961206 14262 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.961705 14262 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:13.962900 14290 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:13.962229 14272 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:13.963384 14290 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:13.964408 14262 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:13.964802 14272 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:13.965328 14272 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:13.966004  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:13.743052 14111 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.935s	user 0.011s	sys 0.000s
I20260709 12:25:13.929839 10668 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:13.930150 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:13.933756  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.934039 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.975626 10092 raft_consensus.cc:2410] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:13.976929 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.935858 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1836) took 1338 ms. Trace:
I20260709 12:25:13.904397 14306 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:13.982476 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:13.983084  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.937361  9840 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:25:13.940546  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57709.2us
I20260709 12:25:13.940084  9943 raft_consensus.cc:2376] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11. Re-sending same reply.
I20260709 12:25:13.954788  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.913868 14279 raft_consensus.cc:2764] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:25:13.955749 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:13.953660 14299 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.959378 13944 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:13.918887 14254 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:13.957849 14287 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:13.967607  9949 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:13.967732 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:13.966131 14272 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:25:13.973454 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:13.977567 10018 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 12:25:13.958321 14284 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:13.977883 10024 rpcz_store.cc:276] 0709 12:25:12.597550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.603445 (+  5895us) service_pool.cc:224] Handling call
0709 12:25:12.603828 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:12.631641 (+ 27813us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:13.935503 (+1303862us) spinlock_profiling.cc:243] Waited 27.3 ms on lock 0x7b54001388d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:13.935766 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62710016}
I20260709 12:25:13.978106 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:14.007162 10025 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:13.976872 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:13.977684 10092 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:14.008359 10021 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:13.966334 14290 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:13.985713  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:13.982185 14264 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:13.986927  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:13.987172 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:13.712862 14035 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.917s	user 0.007s	sys 0.005s
I20260709 12:25:14.000763 14254 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:13.990615  9945 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:13.990993  9840 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 1 to 7, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 7 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.016695 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:13.938759 14314 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:13.996353 10091 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:13.915074 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:14.004822 10676 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:13.988473 13756 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:14.006029 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:14.000427 14262 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:14.010447 10017 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:14.008471 14284 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:13.980113 14111 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.001891 14310 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:14.012473 13944 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:14.036450  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:14.035575  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:14.014945  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:14.038715  9949 raft_consensus.cc:2410] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:25:14.009056 14299 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:14.010921  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.017640 14254 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:14.007215 14118 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:14.010432  9943 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:14.055246  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1950) took 1166 ms. Trace:
I20260709 12:25:14.024627  9840 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 8 to 9, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 9 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:14.007968 14037 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.025s
W20260709 12:25:14.042642 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.043185 14320 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:25:14.069669  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:14.021018 13756 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:14.024329 14262 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.012132 14279 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:14.057461 10642 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:14.105890  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:14.106503 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49123.8us
I20260709 12:25:14.107209  9949 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:14.023128 14264 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:14.059463  9940 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:14.057870 14273 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:14.064035 14037 consensus_queue.cc:1243] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:25:14.065518  9840 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:14.105891 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 190909us
I20260709 12:25:14.030462 14310 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:14.020402 14259 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:25:14.055728  9943 rpcz_store.cc:276] 0709 12:25:12.889336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.985106 (+1095770us) service_pool.cc:224] Handling call
0709 12:25:14.055176 (+ 70070us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:14.115940  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:14.018070 14035 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.023825 14314 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:14.127435 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.120033  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 83650.7us
I20260709 12:25:14.153383 10017 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:14.159106  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:14.052359 14281 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:14.161967  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:14.109257 14262 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:14.171614  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:14.172197 10023 raft_consensus.cc:2368] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8.
I20260709 12:25:14.117200 14272 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:14.117120 14313 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:14.167260 14264 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.117378  9831 maintenance_manager.cc:643] P 2121bfede85b447c9f51ef23233606cc: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 12.459s	user 0.190s	sys 0.095s Metrics: {"bytes_written":30357,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":286821,"dirs.run_cpu_time_us":461,"dirs.run_wall_time_us":4665,"drs_written":1,"lbm_read_time_us":17544,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":2,"lbm_write_time_us":227500,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":6,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"spinlock_wait_cycles":4864896,"thread_start_us":157462,"threads_started":2,"update_count":19,"wal-append.queue_time_us":316843,"wal-append.run_cpu_time_us":742,"wal-append.run_wall_time_us":1049856}
W20260709 12:25:14.176136 10643 connection.cc:256] RPC call timeout handler was delayed by -2.8767142180004157s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:25:14.179854 10643 connection.cc:256] RPC call timeout handler was delayed by -2.5956057060002422s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
W20260709 12:25:14.180213 10643 connection.cc:256] RPC call timeout handler was delayed by -2.543714218000332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 12:25:14.180471 10643 connection.cc:256] RPC call timeout handler was delayed by -2.5434057060001578s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
W20260709 12:25:14.180714 10643 connection.cc:256] RPC call timeout handler was delayed by -2.429414218000602s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 12:25:14.180948 10643 connection.cc:256] RPC call timeout handler was delayed by -2.3943142180005452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2082s for a response
W20260709 12:25:14.181209 10643 connection.cc:256] RPC call timeout handler was delayed by -1.2276057060003041s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.309s for a response
W20260709 12:25:14.181458 10643 connection.cc:256] RPC call timeout handler was delayed by -1.165438787000312s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2657s for a response
I20260709 12:25:14.104228 14320 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:14.176651  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:14.188584  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:14.202170  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:14.207525 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:14.209861  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1953) took 1319 ms. Trace:
I20260709 12:25:14.211557  9943 rpcz_store.cc:276] 0709 12:25:12.890345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.162682 (+1272337us) service_pool.cc:224] Handling call
0709 12:25:14.209780 (+ 47098us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:14.211201  9840 catalog_manager.cc:5697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 2 to 8. New cstate: current_term: 8 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:14.183665 14111 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:14.215112 14293 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:14.215667 14111 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:14.216200 14111 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:14.216600 14111 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:14.208870 14281 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.007s
W20260709 12:25:14.187536 14318 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 12:25:14.222734 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.220481 14324 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:14.178103 14272 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:14.231850  9840 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 8 to 10, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 10 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:14.179420 14279 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:14.245783 14279 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:14.246999 14279 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:14.177793 14275 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:14.179687 13756 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.058696 14306 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.244103  9832 maintenance_manager.cc:419] P 2121bfede85b447c9f51ef23233606cc: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 9939 bytes on disk
I20260709 12:25:14.240221 14324 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:15.765384 14324 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:15.767030  9942 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
W20260709 12:25:14.245030 14322 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:25:14.316388 14313 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:15.770896 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1862) took 1793 ms. Trace:
I20260709 12:25:15.771469 14313 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:15.772239 14313 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:14.184799 14310 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:15.771281 10020 rpcz_store.cc:276] 0709 12:25:13.977597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.977732 (+   135us) service_pool.cc:224] Handling call
0709 12:25:13.978037 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:13.978392 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:13.978419 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:13.978920 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:25:13.997962 (+ 19042us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:13.999368 (+  1406us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.000460 (+  1092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.000531 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:15.770678 (+1770147us) raft_consensus.cc:1685] finished
0709 12:25:15.770698 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.770835 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":825,"prepare.queue_time_us":56409,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":79,"raft.queue_time_us":2922,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":3583,"thread_start_us":10477,"threads_started":2,"wal-append.queue_time_us":38877}
I20260709 12:25:14.277050 14314 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:14.337438  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.299427  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60160.6us
I20260709 12:25:14.315748  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:15.963284  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1726) took 1723 ms (client timeout 1611 ms). Trace:
W20260709 12:25:15.963711  9940 rpcz_store.cc:269] 0709 12:25:14.239693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.306215 (+ 66522us) service_pool.cc:224] Handling call
0709 12:25:15.963159 (+1656944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.964957  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:15.965590  9940 raft_consensus.cc:2376] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10. Re-sending same reply.
W20260709 12:25:15.966082  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1729) took 1501 ms (client timeout 1514 ms). Trace:
I20260709 12:25:14.237792  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:15.967602  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1723) took 1764 ms (client timeout 1533 ms). Trace:
W20260709 12:25:15.967978  9943 rpcz_store.cc:269] 0709 12:25:14.203892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.212366 (+  8474us) service_pool.cc:224] Handling call
0709 12:25:15.967524 (+1755158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.969019  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:15.969646  9943 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:25:15.970112  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1724) took 1748 ms (client timeout 1892 ms). Trace:
I20260709 12:25:14.444037  9945 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:15.524765  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:16.016151  9946 raft_consensus.cc:2410] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 9 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
W20260709 12:25:16.016604  9946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1980) took 1644 ms (client timeout 1519 ms). Trace:
W20260709 12:25:16.016932  9946 rpcz_store.cc:269] 0709 12:25:14.372094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.524416 (+1152322us) service_pool.cc:224] Handling call
0709 12:25:16.016537 (+492121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.017884  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:16.018327  9946 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:14.433516  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:14.221401  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:16.020473  9949 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:25:16.020988  9949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1722) took 1845 ms (client timeout 2219 ms). Trace:
W20260709 12:25:16.021337  9949 rpcz_store.cc:269] 0709 12:25:14.175679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.221183 (+ 45504us) service_pool.cc:224] Handling call
0709 12:25:16.020926 (+1799743us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:14.323275  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:16.053236  9941 raft_consensus.cc:2376] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11. Re-sending same reply.
W20260709 12:25:16.053829  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1977) took 1835 ms (client timeout 1650 ms). Trace:
W20260709 12:25:16.054188  9941 rpcz_store.cc:269] 0709 12:25:14.218132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.322970 (+104838us) service_pool.cc:224] Handling call
0709 12:25:16.053748 (+1730778us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:14.348049  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1706) took 1694 ms. Trace:
I20260709 12:25:16.289881  9948 rpcz_store.cc:276] 0709 12:25:12.653471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.222873 (+1569402us) service_pool.cc:224] Handling call
0709 12:25:14.225869 (+  2996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.226161 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:14.226581 (+   420us) log.cc:844] Serialized 14 byte log entry
0709 12:25:14.347465 (+120884us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:14.347557 (+    92us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:14.347633 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.347733 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.347957 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6080,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":362,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":122,"thread_start_us":119550,"threads_started":1,"wal-append.queue_time_us":202906,"wal-append.run_cpu_time_us":1286,"wal-append.run_wall_time_us":1011712}
W20260709 12:25:14.313180 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 296526us
W20260709 12:25:14.329594 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.224332 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 658) took 1125 ms. Trace:
I20260709 12:25:14.443804 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1898) took 1376 ms. Trace:
I20260709 12:25:14.372382 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:14.399220 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:16.449544 10095 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:15.775394 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:15.813246  9831 maintenance_manager.cc:643] P 2121bfede85b447c9f51ef23233606cc: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.002s	user 0.000s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":353,"lbm_reads_lt_1ms":4}
I20260709 12:25:16.014772  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:15.864238  9914 connection.cc:256] RPC call timeout handler was delayed by -1.3485907550002594s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18480000000000002s for a response
W20260709 12:25:16.560722  9914 connection.cc:256] RPC call timeout handler was delayed by -1.2725924790001955s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2098s for a response
W20260709 12:25:15.769172  9942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1708) took 2494 ms (client timeout 1828 ms). Trace:
I20260709 12:25:16.563912 14353 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:25:16.564237  9942 rpcz_store.cc:269] 0709 12:25:13.275491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.293501 (+ 18010us) service_pool.cc:224] Handling call
0709 12:25:15.768893 (+2475392us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b54001ba154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:15.769033 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5187400064}
I20260709 12:25:16.565421  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:16.033314  9946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1725) took 1810 ms (client timeout 2182 ms). Trace:
W20260709 12:25:16.567160  9946 rpcz_store.cc:269] 0709 12:25:14.223477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.017621 (+1794144us) service_pool.cc:224] Handling call
0709 12:25:16.018585 (+   964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.022316  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:15.966445  9940 rpcz_store.cc:269] 0709 12:25:14.464561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.964667 (+1500106us) service_pool.cc:224] Handling call
0709 12:25:15.966027 (+  1360us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:15.970434  9943 rpcz_store.cc:269] 0709 12:25:14.221684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.968715 (+1747031us) service_pool.cc:224] Handling call
0709 12:25:15.970059 (+  1344us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.582000  9943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:16.582792  9943 raft_consensus.cc:606] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:16.583112  9943 raft_consensus.cc:614] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:16.583549  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34882 (request call id 24) took 2684 ms. Trace:
I20260709 12:25:16.583848  9943 rpcz_store.cc:276] 0709 12:25:13.901997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.581582 (+2679585us) service_pool.cc:224] Handling call
0709 12:25:16.583460 (+  1878us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.790127  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:16.585667  9944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1979) took 2267 ms (client timeout 1639 ms). Trace:
W20260709 12:25:16.586037  9944 rpcz_store.cc:269] 0709 12:25:14.321672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.789898 (+1468226us) service_pool.cc:224] Handling call
0709 12:25:16.585587 (+795689us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.587330  9944 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:25:16.587920  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1954) took 3697 ms. Trace:
I20260709 12:25:16.588207  9944 rpcz_store.cc:276] 0709 12:25:12.891799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.586778 (+3694979us) service_pool.cc:224] Handling call
0709 12:25:16.587104 (+   326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.587692 (+   588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.587870 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:15.973053  9945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 802) took 1891 ms (client timeout 1895 ms). Trace:
W20260709 12:25:16.590363  9945 rpcz_store.cc:269] 0709 12:25:14.082125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.115673 (+ 33548us) service_pool.cc:224] Handling call
0709 12:25:15.972833 (+1857160us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54001a31d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:15.972989 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":657991040}
I20260709 12:25:16.591717  9945 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:16.592347  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1959) took 3699 ms. Trace:
I20260709 12:25:16.592657  9945 rpcz_store.cc:276] 0709 12:25:12.893283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.591259 (+3697976us) service_pool.cc:224] Handling call
0709 12:25:16.591540 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.592105 (+   565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.592294 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.594030  9945 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:25:15.725107  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:15.669214  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:16.019850  9947 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.595715  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1961) took 3701 ms. Trace:
I20260709 12:25:16.598470  9943 rpcz_store.cc:276] 0709 12:25:12.893901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.594787 (+3700886us) service_pool.cc:224] Handling call
0709 12:25:16.595105 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.595305 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.595335 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.595390 (+    55us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:16.595431 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.595492 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.595629 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.594552  9945 pending_rounds.cc:85] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 5
I20260709 12:25:16.601843  9945 pending_rounds.cc:107] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 10.6
W20260709 12:25:16.051667  9840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 321) took 13568 ms (client timeout 15000 ms). Trace:
I20260709 12:25:14.296398 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 2954) took 1337 ms. Trace:
I20260709 12:25:16.605306 10016 rpcz_store.cc:276] 0709 12:25:12.960230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.008465 (+ 48235us) service_pool.cc:224] Handling call
0709 12:25:14.296083 (+1287618us) spinlock_profiling.cc:243] Waited 861 ms on lock 0x7b54001c5054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:14.296302 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1981379456}
I20260709 12:25:16.607555 10016 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:14.377556 10019 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:14.368257 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:16.611809 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 664) took 2243 ms. Trace:
W20260709 12:25:16.604447  9840 rpcz_store.cc:269] 0709 12:25:02.483112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:02.485152 (+  2040us) service_pool.cc:224] Handling call
0709 12:25:16.051578 (+13566426us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:15.830687 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:15.831258 (+   571us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:15.831313 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:25:15.831331 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:15.832700 (+  1369us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:15.832948 (+   248us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:15.832978 (+    30us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:25:15.834254 (+  1276us) tablet.cc:701] Row locks acquired
0709 12:25:15.834315 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:25:15.834491 (+   176us) write_op.cc:270] Start()
0709 12:25:15.834649 (+   158us) write_op.cc:276] Timestamp: P: 1783599915834420 usec, L: 0
0709 12:25:15.834672 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:25:15.835414 (+   742us) log.cc:844] Serialized 6367 byte log entry
0709 12:25:15.850651 (+ 15237us) op_driver.cc:464] REPLICATION: finished
0709 12:25:15.852245 (+  1594us) write_op.cc:301] APPLY: starting
0709 12:25:15.852348 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:15.865622 (+ 13274us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:15.865778 (+   156us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:15.875140 (+  9362us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:15.875493 (+   353us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:25:15.875526 (+    33us) write_op.cc:312] APPLY: finished
0709 12:25:15.875949 (+   423us) log.cc:844] Serialized 148 byte log entry
0709 12:25:15.876152 (+   203us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:15.876335 (+   183us) write_op.cc:454] Released schema lock
0709 12:25:15.876415 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:15.878951 (+  2536us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:15.878985 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":1319,"apply.run_cpu_time_us":22109,"apply.run_wall_time_us":26647,"cfile_cache_hit":62,"cfile_cache_hit_bytes":64821,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":1364,"lbm_reads_lt_1ms":12,"num_ops":16,"prepare.queue_time_us":1200,"prepare.run_cpu_time_us":5852,"prepare.run_wall_time_us":5982,"raft.queue_time_us":10902,"raft.run_cpu_time_us":1414,"raft.run_wall_time_us":1696,"replication_time_us":15773,"spinlock_wait_cycles":13696,"thread_start_us":14598,"threads_started":4,"wal-append.queue_time_us":1179}]]}
I20260709 12:25:16.612425 10018 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:16.612150 10019 rpcz_store.cc:276] 0709 12:25:14.368003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.376930 (+  8927us) service_pool.cc:224] Handling call
0709 12:25:14.377250 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.611486 (+2234236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.611726 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.613003 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 663) took 2245 ms. Trace:
I20260709 12:25:16.613349 10018 rpcz_store.cc:276] 0709 12:25:14.367602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.367829 (+   227us) service_pool.cc:224] Handling call
0709 12:25:16.612913 (+2245084us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.773981 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:15.678018 10021 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:15.670411 10025 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:25:14.321054 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1846) took 1356 ms. Trace:
I20260709 12:25:16.618408 10023 rpcz_store.cc:276] 0709 12:25:12.964370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.998288 (+ 33918us) service_pool.cc:224] Handling call
0709 12:25:14.320813 (+1322525us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b540038e0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:14.320982 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2696804096}
I20260709 12:25:14.374130 10024 pending_rounds.cc:85] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 6
I20260709 12:25:16.620232  9947 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:16.620482  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1958) took 3727 ms. Trace:
I20260709 12:25:16.620534 10024 pending_rounds.cc:107] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 6.7
W20260709 12:25:16.620800  9947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1727) took 2309 ms (client timeout 1565 ms). Trace:
I20260709 12:25:16.621058 10024 raft_consensus.cc:2889] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:16.621347  9947 rpcz_store.cc:269] 0709 12:25:14.311578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.433184 (+121606us) service_pool.cc:224] Handling call
0709 12:25:16.620734 (+2187550us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.293118  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:16.380026  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08002ec9e0 after lost signal to thread 8743
W20260709 12:25:16.371632 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.042742 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.626003 14359 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:16.626675 14359 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:16.627076 14359 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:15.582652 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.444428 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:16.440991 10092 rpcz_store.cc:276] 0709 12:25:13.067624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.017947 (+950323us) service_pool.cc:224] Handling call
0709 12:25:14.031300 (+ 13353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:14.164646 (+133346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.164677 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:14.176244 (+ 11567us) log.cc:844] Serialized 26 byte log entry
0709 12:25:14.429845 (+253601us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:25:14.429887 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.429955 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:14.440738 (+ 10783us) raft_consensus.cc:1685] finished
0709 12:25:14.443551 (+  2813us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.443743 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":193105,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":7896,"thread_start_us":323186,"threads_started":2,"wal-append.queue_time_us":213632,"wal-append.run_cpu_time_us":2309,"wal-append.run_wall_time_us":1002326}
I20260709 12:25:16.631136 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:16.631918 10092 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:16.632324 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1900) took 2217 ms (client timeout 2342 ms). Trace:
W20260709 12:25:16.632670 10092 rpcz_store.cc:269] 0709 12:25:14.415494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.630872 (+2215378us) service_pool.cc:224] Handling call
0709 12:25:16.632271 (+  1399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.633658 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:16.634347 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1903) took 2217 ms (client timeout 2539 ms). Trace:
I20260709 12:25:15.788992 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.636078  9947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:16.636412 10098 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:25:16.636682  9947 raft_consensus.cc:606] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:16.636947 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1901) took 2221 ms (client timeout 2331 ms). Trace:
I20260709 12:25:16.635540 14355 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:16.637347 10098 rpcz_store.cc:269] 0709 12:25:14.416034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.788647 (+1372613us) service_pool.cc:224] Handling call
0709 12:25:16.636892 (+848245us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:15.595734 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.639567 10096 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:25:16.640014 10096 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3040) took 2196 ms (client timeout 1696 ms). Trace:
W20260709 12:25:16.640420 10096 rpcz_store.cc:269] 0709 12:25:14.443466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.595494 (+1152028us) service_pool.cc:224] Handling call
0709 12:25:16.639947 (+1044453us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.641306 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:15.665906 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:16.643322 10097 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:25:11.415696 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:15.521764 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.648839 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:14.339560 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.238052 10642 leader_election.cc:336] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.128s (SENT)
W20260709 12:25:16.652077 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 414073us
W20260709 12:25:16.372550 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:15.644368 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 879) took 1716 ms. Trace:
I20260709 12:25:16.656507 10677 rpcz_store.cc:276] 0709 12:25:13.927508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:13.929017 (+  1509us) service_pool.cc:224] Handling call
0709 12:25:15.628605 (+1699588us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:25:15.628858 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:15.629058 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:15.629998 (+   940us) log.cc:844] Serialized 14 byte log entry
0709 12:25:15.644066 (+ 14068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:15.644107 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:15.644150 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:15.644198 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:15.644300 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":935,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":42,"raft.queue_time_us":51610,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":125,"spinlock_wait_cycles":3166592,"thread_start_us":14440,"threads_started":2,"wal-append.queue_time_us":12979,"wal-append.run_cpu_time_us":902,"wal-append.run_wall_time_us":1000926}
I20260709 12:25:15.946322 13235 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:14.294236 13756 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:14.439900 13965 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.600s	user 0.005s	sys 0.010s
I20260709 12:25:16.668720 13965 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.293646 14035 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:13.615923 13996 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.000s
W20260709 12:25:15.626150 10699 debug-util.cc:398] Leaking SignalData structure 0x7b0800278940 after lost signal to thread 8746
I20260709 12:25:14.450377 14110 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:16.676566 10699 debug-util.cc:398] Leaking SignalData structure 0x7b08002d7f80 after lost signal to thread 9879
I20260709 12:25:16.676841 14110 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:16.678090 10699 debug-util.cc:398] Leaking SignalData structure 0x7b08002d84c0 after lost signal to thread 9973
I20260709 12:25:16.642452 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.678717 13965 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:16.679242 10096 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:16.680202 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.680480 13965 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:16.680956 10096 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
W20260709 12:25:16.681901 10699 debug-util.cc:398] Leaking SignalData structure 0x7b080035cde0 after lost signal to thread 10049
I20260709 12:25:14.443634 14111 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:16.683204 10092 rpcz_store.cc:269] 0709 12:25:14.416779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.633407 (+2216628us) service_pool.cc:224] Handling call
0709 12:25:16.634294 (+   887us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.683523  9943 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:16.684229 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.684489 14111 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:14.232324 14118 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:15.695819 14119 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:16.687630 14118 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.326148 14130 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:15.665096 14193 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.726667 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 672) took 1057 ms. Trace:
I20260709 12:25:16.727185 14193 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.727321 10019 rpcz_store.cc:276] 0709 12:25:15.668862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.613256 (+944394us) service_pool.cc:224] Handling call
0709 12:25:16.613811 (+   555us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.614044 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:16.708633 (+ 94589us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.726136 (+ 17503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.726233 (+    97us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:16.726316 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.726436 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.726586 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93206,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":411,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":110985,"threads_started":2,"wal-append.queue_time_us":17104}
I20260709 12:25:16.729532 14193 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:15.816459 14214 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:15.809790 14236 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:15.722720 14246 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.733757 10677 pending_rounds.cc:85] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 9
I20260709 12:25:16.734184 10677 pending_rounds.cc:107] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:25:16.734495 10677 raft_consensus.cc:2889] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:14.247501 14260 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:14.427465 14262 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.743974 14262 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:14.285429 14273 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:14.270612 14272 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.747710 14118 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:14.337258 14284 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:14.440285 14287 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.752732 14284 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:16.752867 14287 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:16.753674 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:14.186779 14296 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 12:25:16.000222 14310 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:15.780632 14313 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.771407 14313 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:14.294850 14275 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.247455 14279 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:16.773204 14110 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:14.320111 14283 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.405609 14277 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.778016 14277 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.778553 14283 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:16.774135 14279 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.780169 14277 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:15.984431 14297 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:16.782130  9942 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:16.784340  9942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1976) took 2566 ms (client timeout 3048 ms). Trace:
W20260709 12:25:16.784703  9942 rpcz_store.cc:269] 0709 12:25:14.217639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.565165 (+2347526us) service_pool.cc:224] Handling call
0709 12:25:16.784130 (+218965us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540003fa54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:16.784243 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496970496}
I20260709 12:25:15.545337 14309 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:16.787307 14309 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:16.788120 14110 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:15.772670 14324 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.791240 14324 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:15.575814 14327 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:16.792906 14327 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:16.491657  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:16.795084  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1730) took 2330 ms (client timeout 2128 ms). Trace:
W20260709 12:25:16.549847  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.562352  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 547627us
I20260709 12:25:16.568105  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:16.795661 14359 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.796267  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:16.796792  9946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 805) took 2493 ms (client timeout 3193 ms). Trace:
W20260709 12:25:16.797127  9946 rpcz_store.cc:269] 0709 12:25:14.303837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.567873 (+2264036us) service_pool.cc:224] Handling call
0709 12:25:16.796734 (+228861us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.797855  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:16.798303 14359 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:16.571626  9914 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.501s (SENT)
I20260709 12:25:16.798792 14296 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:16.799170  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 227581us
I20260709 12:25:16.800091 14296 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:16.799494 14272 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.788187 14309 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 8.10, Last appended by leader: 10, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.800784 14296 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:16.801152 14309 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:16.801546 10094 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:16.801779 14296 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.603165  9945 raft_consensus.cc:2889] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:16.622689  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08001cd260 after lost signal to thread 8746
I20260709 12:25:16.622628  9948 raft_consensus.cc:2376] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11. Re-sending same reply.
W20260709 12:25:16.805167  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1731) took 2338 ms (client timeout 2154 ms). Trace:
W20260709 12:25:16.805812  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800064b20 after lost signal to thread 9879
I20260709 12:25:16.805564  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1960) took 3912 ms. Trace:
W20260709 12:25:16.805760  9948 rpcz_store.cc:269] 0709 12:25:14.466638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.292812 (+1826174us) service_pool.cc:224] Handling call
0709 12:25:16.805062 (+512250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.806766  9945 rpcz_store.cc:276] 0709 12:25:12.893551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.593403 (+3699852us) service_pool.cc:224] Handling call
0709 12:25:16.593703 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.805200 (+211497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.805473 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.807868  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800307b40 after lost signal to thread 9973
W20260709 12:25:16.808264  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08003199c0 after lost signal to thread 9974
I20260709 12:25:16.808770  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1973) took 2917 ms. Trace:
W20260709 12:25:16.629455 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.448997 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1892) took 3674 ms (client timeout 1809 ms). Trace:
W20260709 12:25:16.821497 10093 rpcz_store.cc:269] 0709 12:25:12.774124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.808069 (+ 33945us) service_pool.cc:224] Handling call
0709 12:25:14.372206 (+1564137us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:16.448903 (+2076697us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4080384}
I20260709 12:25:16.629483 10090 raft_consensus.cc:2376] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 9. Re-sending same reply.
I20260709 12:25:16.637058  9947 raft_consensus.cc:614] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:16.647541 10099 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:16.822631 10090 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3039) took 2565 ms (client timeout 1501 ms). Trace:
W20260709 12:25:16.823241 10090 rpcz_store.cc:269] 0709 12:25:14.257273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.443876 (+186603us) service_pool.cc:224] Handling call
0709 12:25:16.822522 (+2378646us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:14.305042 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 97615us
I20260709 12:25:16.827831 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.828394 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:16.828680 10093 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:16.638396 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.828969  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:16.673795 13996 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.829458 10098 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:16.829901 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1906) took 1158 ms (client timeout 1519 ms). Trace:
W20260709 12:25:16.830250 10098 rpcz_store.cc:269] 0709 12:25:15.671818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.638149 (+966331us) service_pool.cc:224] Handling call
0709 12:25:16.829848 (+191699us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.681800 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.830893 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:16.831486 10098 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:25:16.831740 10090 raft_consensus.cc:2410] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 12 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 12:25:16.831812 10098 pending_rounds.cc:85] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 8
I20260709 12:25:15.726689 14052 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:16.684859 10092 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
W20260709 12:25:16.832542 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1881) took 5578 ms (client timeout 3048 ms). Trace:
W20260709 12:25:16.686185 10699 debug-util.cc:398] Leaking SignalData structure 0x7b08002a9700 after lost signal to thread 10123
I20260709 12:25:16.689178 10097 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
W20260709 12:25:16.832926 10099 rpcz_store.cc:269] 0709 12:25:11.247242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:11.314841 (+ 67599us) service_pool.cc:224] Handling call
0709 12:25:16.825141 (+5510300us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.833428 10097 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1904) took 2416 ms (client timeout 2117 ms). Trace:
W20260709 12:25:16.833796 10097 rpcz_store.cc:269] 0709 12:25:14.417173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.665579 (+1248406us) service_pool.cc:224] Handling call
0709 12:25:16.833363 (+1167784us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.835040  9947 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:16.835343 10097 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: [5.5-10.7]
I20260709 12:25:16.835683  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 801) took 2773 ms. Trace:
I20260709 12:25:16.835891 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 668) took 3323 ms. Trace:
I20260709 12:25:16.836226 10097 rpcz_store.cc:276] 0709 12:25:13.512480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.834414 (+3321934us) service_pool.cc:224] Handling call
0709 12:25:16.834968 (+   554us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:16.835667 (+   699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.835853 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.836908  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:48678 (request call id 0) took 2985 ms. Trace:
I20260709 12:25:16.837038 13965 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:16.837288  9946 rpcz_store.cc:276] 0709 12:25:13.851136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.797871 (+2946735us) service_pool.cc:224] Handling call
0709 12:25:16.836627 (+ 38756us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955c735 00007f4e095549fa 00007f4e0e51c4c9 00007f4e07a98c54 00007f4e07a98be2 00007f4e07a98b11 00007f4e07a98a8c 00007f4e07a97d42 00007f4e0789148c 00007f4e078908c6
0709 12:25:16.836830 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59928960}
I20260709 12:25:16.689991  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1715) took 3165 ms. Trace:
W20260709 12:25:16.691313  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 966192us
I20260709 12:25:16.838375  9943 rpcz_store.cc:276] 0709 12:25:13.524917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.599447 (+3074530us) service_pool.cc:224] Handling call
0709 12:25:16.599745 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.683987 (+ 84242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.689725 (+  5738us) spinlock_profiling.cc:243] Waited 83.5 ms on lock 0x7b5400096354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:16.689926 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":192012672}
I20260709 12:25:16.836072  9947 rpcz_store.cc:276] 0709 12:25:14.062826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.834581 (+2771755us) service_pool.cc:224] Handling call
0709 12:25:16.834880 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.835440 (+   560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.835628 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.620925  9944 rpcz_store.cc:276] 0709 12:25:12.893009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.588869 (+3695860us) service_pool.cc:224] Handling call
0709 12:25:16.589125 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.589304 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:16.602281 (+ 12977us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.620145 (+ 17864us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.620202 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:16.620245 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.620313 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.620430 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7151,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":41,"raft.queue_time_us":241,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"thread_start_us":29729,"threads_started":2,"wal-append.queue_time_us":16162}
I20260709 12:25:16.831895 10096 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:16.840083 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 663) took 3567 ms. Trace:
I20260709 12:25:16.840369 10096 rpcz_store.cc:276] 0709 12:25:13.272559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.831472 (+3558913us) service_pool.cc:224] Handling call
0709 12:25:16.831743 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.839867 (+  8124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.840023 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.841621 10096 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:25:16.841382 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:16.841962 10096 pending_rounds.cc:85] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 9
I20260709 12:25:16.842221 10096 pending_rounds.cc:107] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:16.842274  9946 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:25:16.833061 10699 debug-util.cc:398] Leaking SignalData structure 0x7b0800212020 after lost signal to thread 10124
W20260709 12:25:16.708681 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55236.2us
I20260709 12:25:16.842499 10096 raft_consensus.cc:2889] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.845170 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 670) took 3331 ms. Trace:
I20260709 12:25:16.720242 10670 raft_consensus.cc:1217] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Deduplicated request from leader. Original: 5.7->[6.8-11.11]   Dedup: 8.9->[9.10-11.11]
I20260709 12:25:16.731213 10016 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:16.733444 14246 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.846987 14361 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.847528 14361 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.848064 14246 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:14.303457 14254 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.850507 14254 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:16.851836  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003885c0 after lost signal to thread 8743
W20260709 12:25:16.852216  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800389140 after lost signal to thread 8746
I20260709 12:25:16.738188 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:16.858767 10672 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:16.738087 14295 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.741258 14260 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:16.862944 14295 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.837445 13965 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:16.864442 13965 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.863817 14260 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.865578 14260 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:16.830940 13996 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:16.866788  9943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:16.867465  9943 raft_consensus.cc:606] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:16.867846  9943 raft_consensus.cc:614] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:16.749737 10062 connection.cc:256] RPC call timeout handler was delayed by -1.9836377569999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:25:16.870864 10062 connection.cc:256] RPC call timeout handler was delayed by -1.9836377569999968s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18280000000000002s for a response
W20260709 12:25:16.871665 10062 connection.cc:256] RPC call timeout handler was delayed by -1.7343377570000484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21050000000000002s for a response
W20260709 12:25:16.872524 10062 connection.cc:256] RPC call timeout handler was delayed by -1.1626633799996853s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18810000000000002s for a response
W20260709 12:25:16.873018 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0551215679997767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 12:25:16.873288 14272 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:16.873530 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0380215679997491s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1533s for a response
I20260709 12:25:16.754954 14273 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.873965 14272 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.874544 14273 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.876233 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 671) took 1207 ms. Trace:
I20260709 12:25:16.876655 10016 rpcz_store.cc:276] 0709 12:25:15.668585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.608553 (+939968us) service_pool.cc:224] Handling call
0709 12:25:16.608879 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.845846 (+236967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.875981 (+ 30135us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:16.876169 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280216448}
I20260709 12:25:16.877763 14273 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:16.878201 14254 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:16.767522  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.09831e+06us
I20260709 12:25:16.774117 14275 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:16.776770 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 665) took 2408 ms. Trace:
I20260709 12:25:16.780560 14279 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.775399 14363 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.569264  9949 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:16.570469  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:16.897794  9940 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:16.577329 10095 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
W20260709 12:25:16.898458 10095 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3037) took 2714 ms (client timeout 1527 ms). Trace:
W20260709 12:25:16.898862 10095 rpcz_store.cc:269] 0709 12:25:14.184383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.377544 (+193161us) service_pool.cc:224] Handling call
0709 12:25:16.898359 (+2520815us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.900156 10095 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 12:25:16.900463 10095 pending_rounds.cc:85] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 9
I20260709 12:25:16.900687 10095 pending_rounds.cc:107] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:25:16.900914 10095 raft_consensus.cc:2889] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.901582 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 671) took 3388 ms. Trace:
I20260709 12:25:16.901885 10095 rpcz_store.cc:276] 0709 12:25:13.513609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.899526 (+3385917us) service_pool.cc:224] Handling call
0709 12:25:16.899837 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.901352 (+  1515us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.901529 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:15.664034  9991 connection.cc:256] RPC call timeout handler was delayed by -1.1350104200000715s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22410000000000002s for a response
I20260709 12:25:16.802834  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:16.903995  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 75105.3us
W20260709 12:25:16.904292  9991 connection.cc:256] RPC call timeout handler was delayed by -1.0025706950000313s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260709 12:25:16.803083 10094 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1902) took 2386 ms (client timeout 2233 ms). Trace:
W20260709 12:25:16.904811 10094 rpcz_store.cc:269] 0709 12:25:14.416413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:15.775124 (+1358711us) service_pool.cc:224] Handling call
0709 12:25:16.802779 (+1027655us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:16.802982 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568329472}
I20260709 12:25:16.905196 14297 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:16.905936 14297 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:16.795456  9941 rpcz_store.cc:269] 0709 12:25:14.464913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.056314 (+1591401us) service_pool.cc:224] Handling call
0709 12:25:16.795003 (+738689us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.786422  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 1971) took 2949 ms. Trace:
I20260709 12:25:16.906406 14297 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:16.904464  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 101741us
I20260709 12:25:16.793833 14327 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.906878 10092 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:16.907497 14327 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.907438 14297 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.908360 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 665) took 3544 ms. Trace:
I20260709 12:25:16.908707 10092 rpcz_store.cc:276] 0709 12:25:13.363872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.833129 (+3469257us) service_pool.cc:224] Handling call
0709 12:25:16.906715 (+ 73586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.907393 (+   678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.908185 (+   792us) spinlock_profiling.cc:243] Waited 73.3 ms on lock 0x7b5400304650. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:16.908310 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168596736}
I20260709 12:25:16.910177 14359 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:16.626173  9842 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 5 to 3, term changed from 6 to 7, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from VOTER to NON_VOTER, now has a pending config: VOTER cd983415389444e292b2330ef630850a (127.8.137.193), VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194), VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195), VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 7 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:16.809350  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800064aa0 after lost signal to thread 10049
I20260709 12:25:16.911685  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1707) took 3637 ms. Trace:
W20260709 12:25:16.912256  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800376480 after lost signal to thread 10123
I20260709 12:25:16.912202  9940 rpcz_store.cc:276] 0709 12:25:13.274253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.570228 (+3295975us) service_pool.cc:224] Handling call
0709 12:25:16.911593 (+341365us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.809883 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.813971  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.819619  9948 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:16.440498 10091 rpcz_store.cc:276] 0709 12:25:13.098483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.018544 (+920061us) service_pool.cc:224] Handling call
0709 12:25:14.018863 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:14.223822 (+204959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:14.223856 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:14.223902 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:14.223947 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:14.224025 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:14.224237 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":205568,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":68707,"thread_start_us":203553,"threads_started":1}
I20260709 12:25:16.829110 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 1908) took 1156 ms. Trace:
I20260709 12:25:16.832260 10098 pending_rounds.cc:107] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:25:16.673820 14320 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:16.922211 10093 rpcz_store.cc:276] 0709 12:25:15.672423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.827525 (+1155102us) service_pool.cc:224] Handling call
0709 12:25:16.829053 (+  1528us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.923043  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 800) took 2861 ms. Trace:
I20260709 12:25:16.922854 14320 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.923872 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3038) took 2667 ms. Trace:
I20260709 12:25:16.707859 14349 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:16.708091 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 662) took 2341 ms. Trace:
I20260709 12:25:16.923453  9948 rpcz_store.cc:276] 0709 12:25:14.062121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.818968 (+2756847us) service_pool.cc:224] Handling call
0709 12:25:16.819293 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.915992 (+ 96699us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.922978 (+  6986us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.922379 10098 raft_consensus.cc:2889] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.922868  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:16.844166 10699 debug-util.cc:398] Leaking SignalData structure 0x7b08003dfa20 after lost signal to thread 10702
I20260709 12:25:16.845836 10670 pending_rounds.cc:85] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 9
I20260709 12:25:16.806366 14324 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:16.853407  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08002f05a0 after lost signal to thread 9879
W20260709 12:25:16.930301  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800315700 after lost signal to thread 9912
W20260709 12:25:16.930958  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003886e0 after lost signal to thread 9940
W20260709 12:25:16.931233  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800388700 after lost signal to thread 9941
W20260709 12:25:16.931499  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800388b20 after lost signal to thread 9943
W20260709 12:25:16.931600 10699 debug-util.cc:398] Leaking SignalData structure 0x7b08003f9e40 after lost signal to thread 14204
I20260709 12:25:16.853595 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:16.932134 10699 debug-util.cc:398] Leaking SignalData structure 0x7b080039fa00 after lost signal to thread 14230
I20260709 12:25:16.931252 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:16.932420 10699 debug-util.cc:398] Leaking SignalData structure 0x7b08003f9e20 after lost signal to thread 14233
I20260709 12:25:16.854351 14361 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:16.854837 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:16.925132 10024 rpcz_store.cc:276] 0709 12:25:14.367020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.373450 (+  6430us) service_pool.cc:224] Handling call
0709 12:25:14.373876 (+   426us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:16.625749 (+2251873us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.625781 (+    32us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:16.626461 (+   680us) log.cc:844] Serialized 78 byte log entry
0709 12:25:16.657715 (+ 31254us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:16.702507 (+ 44792us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.704811 (+  2304us) log.cc:844] Serialized 14 byte log entry
0709 12:25:16.707686 (+  2875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.707812 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.707852 (+    40us) raft_consensus.cc:1685] finished
0709 12:25:16.707863 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.708029 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2480,"prepare.queue_time_us":52929,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":19114,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":6785,"spinlock_wait_cycles":141312,"thread_start_us":78920,"threads_started":3,"wal-append.queue_time_us":28711}
I20260709 12:25:16.855767 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:16.938148 10671 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:16.855942 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:16.856590 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:16.939492 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3028) took 3411 ms. Trace:
I20260709 12:25:16.939854 10675 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:16.857235 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:16.939956 10094 rpcz_store.cc:276] 0709 12:25:13.528482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.905619 (+3377137us) service_pool.cc:224] Handling call
0709 12:25:16.905921 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.927927 (+ 22006us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.927960 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.928005 (+    45us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:16.928045 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.939286 (+ 11241us) spinlock_profiling.cc:243] Waited 21.8 ms on lock 0x7b5400094554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:16.939327 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.939440 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":50130816}
I20260709 12:25:16.944192 10023 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: [6.9-8.10]
W20260709 12:25:16.945250  9912 connection.cc:256] RPC call timeout handler was delayed by -3.4208412210000461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
I20260709 12:25:16.874003 14295 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:16.774391 10017 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:16.845479 10096 rpcz_store.cc:276] 0709 12:25:13.513328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.841061 (+3327733us) service_pool.cc:224] Handling call
0709 12:25:16.841314 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.844952 (+  3638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.845119 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:16.874039 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0353215679997447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
I20260709 12:25:16.959338 10672 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
W20260709 12:25:16.959714 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0353215679997447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 12:25:16.960330 10062 connection.cc:256] RPC call timeout handler was delayed by -1.0092215679997025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
W20260709 12:25:16.960871 10062 connection.cc:256] RPC call timeout handler was delayed by -1.002463379999881s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20590000000000003s for a response
I20260709 12:25:16.963284 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:16.963961 10019 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:16.965513 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:16.879189 10022 rpcz_store.cc:276] 0709 12:25:14.368340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.418505 (+ 50165us) service_pool.cc:224] Handling call
0709 12:25:14.419024 (+   519us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.711627 (+2292603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.711655 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:16.712142 (+   487us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.749018 (+ 36876us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:16.749065 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.759814 (+ 10749us) spinlock_profiling.cc:243] Waited 2.29 s on lock 0x7b54001c5f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:16.759864 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:16.774751 (+ 14887us) raft_consensus.cc:1685] finished
0709 12:25:16.774773 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.776708 (+  1935us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5271295744,"thread_start_us":36490,"threads_started":1,"wal-append.queue_time_us":57889}
I20260709 12:25:16.843065  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 803) took 2738 ms. Trace:
I20260709 12:25:16.967995  9946 rpcz_store.cc:276] 0709 12:25:14.105112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.838365 (+2733253us) service_pool.cc:224] Handling call
0709 12:25:16.842020 (+  3655us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.842744 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.842996 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.879796  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:16.879506 14272 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.878821 14254 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.891916 14363 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.984841  9949 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:25:16.890911 14275 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:16.894122  9943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:16.990254  9943 raft_consensus.cc:606] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:16.912724  9876 debug-util.cc:398] Leaking SignalData structure 0x7b080036f1c0 after lost signal to thread 10124
I20260709 12:25:16.992646  9943 raft_consensus.cc:614] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:16.948971 14356 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:16.906455  9942 rpcz_store.cc:276] 0709 12:25:13.837133 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.785590 (+2948457us) service_pool.cc:224] Handling call
0709 12:25:16.785942 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.786105 (+   163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.786129 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.786169 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:16.786199 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.786250 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.786367 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.920686  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:16.924254 10093 rpcz_store.cc:276] 0709 12:25:14.256908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.923069 (+2666161us) service_pool.cc:224] Handling call
0709 12:25:16.923363 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.923538 (+   175us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.923581 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.923622 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:16.923657 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.923710 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.923816 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.928476 10670 pending_rounds.cc:107] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:25:16.997339 10670 raft_consensus.cc:2889] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:16.926674 10091 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:16.929296  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:16.931833  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003891a0 after lost signal to thread 9945
I20260709 12:25:16.932758 10025 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:16.998627  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003888e0 after lost signal to thread 9947
W20260709 12:25:16.998922  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800388960 after lost signal to thread 9948
W20260709 12:25:16.999143  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800389260 after lost signal to thread 9949
W20260709 12:25:16.999643  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800401140 after lost signal to thread 9973
W20260709 12:25:16.999868  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800401280 after lost signal to thread 9974
W20260709 12:25:17.000445  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080037da00 after lost signal to thread 10016
W20260709 12:25:17.000699  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080037dcc0 after lost signal to thread 10018
W20260709 12:25:17.000902  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac880 after lost signal to thread 10019
W20260709 12:25:17.001977 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:16.936210  9991 connection.cc:256] RPC call timeout handler was delayed by -1.2320342329999221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.165s for a response
I20260709 12:25:16.927258  9941 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
W20260709 12:25:17.002678  9991 connection.cc:256] RPC call timeout handler was delayed by -1.1667476990000978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 12:25:16.857856 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:16.948482 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:16.947023 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51393.1us
I20260709 12:25:16.949515 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 662) took 3677 ms. Trace:
I20260709 12:25:17.003708 10098 rpcz_store.cc:276] 0709 12:25:13.272302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.830896 (+3558594us) service_pool.cc:224] Handling call
0709 12:25:16.831188 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.949268 (+118080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.949460 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.910760 14359 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:16.946990  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:16.952018 10065 connection.cc:256] RPC call timeout handler was delayed by -1.2309903099999246s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260709 12:25:16.957027 10095 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:25:17.006242  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080037dea0 after lost signal to thread 10023
I20260709 12:25:16.974431 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.007458 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:16.977008 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 666) took 2608 ms. Trace:
I20260709 12:25:17.008147 10017 rpcz_store.cc:276] 0709 12:25:14.368644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.450696 (+ 82052us) service_pool.cc:224] Handling call
0709 12:25:14.451101 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.957814 (+2506713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.976634 (+ 18820us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b5400021054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:16.976917 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5342654720}
I20260709 12:25:16.922232 14277 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.931402 14273 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.980186 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:16.931607 14310 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.981324 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:16.971220 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:16.945538 14283 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:16.985708  9949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1978) took 2692 ms (client timeout 2117 ms). Trace:
I20260709 12:25:16.986148  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:16.984678 14110 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.941979 14349 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:16.987140 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.024721 10672 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:17.024900 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:16.943086 14284 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.894804 13756 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:16.990513 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:16.991361 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:16.948645 14119 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:16.943322 14287 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:16.993568  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08003ca040 after lost signal to thread 10702
I20260709 12:25:16.809209  9945 rpcz_store.cc:276] 0709 12:25:13.891126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.807909 (+2916783us) service_pool.cc:224] Handling call
0709 12:25:16.808248 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:16.808435 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.808460 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:16.808500 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:16.808526 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.808574 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:16.808699 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:16.926043 14052 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 12:25:16.953579 13996 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:16.994076  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 114388us
I20260709 12:25:16.994546 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:17.055056 10023 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:17.052105  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:16.995484 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:16.966041 14320 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:16.996035 10096 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:16.960130 14309 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:17.002985  9991 connection.cc:256] RPC call timeout handler was delayed by -1.1120912740002495s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16390000000000002s for a response
I20260709 12:25:17.022428 14296 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:16.969129 14350 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:17.006184 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 672) took 3492 ms. Trace:
W20260709 12:25:17.061455  9991 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.596s (SENT)
I20260709 12:25:16.971199  9940 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:25:17.009995  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080037dbe0 after lost signal to thread 10024
I20260709 12:25:17.080044  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:17.080567  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080033c020 after lost signal to thread 10049
I20260709 12:25:17.079653 10095 rpcz_store.cc:276] 0709 12:25:13.513882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.902566 (+3388684us) service_pool.cc:224] Handling call
0709 12:25:16.956590 (+ 54024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.005956 (+ 49366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.006135 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:17.081295  9991 leader_election.cc:336] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.716s (SENT)
I20260709 12:25:16.941836 14324 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:17.012259 13235 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:17.010411 10025 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:25:17.013043  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66142.3us
I20260709 12:25:17.013773 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:17.015826 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:17.017918  9949 rpcz_store.cc:269] 0709 12:25:14.293755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.022046 (+1728291us) service_pool.cc:224] Handling call
0709 12:25:16.985633 (+963587us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.018630 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3033) took 3442 ms. Trace:
I20260709 12:25:17.019062 10669 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:17.018985 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.019555 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:17.023355  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:17.034008  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:17.048293 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:16.998420 14246 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:17.035759 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:17.052793  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08003df2e0 after lost signal to thread 10703
I20260709 12:25:17.053486 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:17.011404 14353 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:17.003351 10673 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:17.081208  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800329940 after lost signal to thread 10062
I20260709 12:25:17.010978 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3036) took 3118 ms. Trace:
W20260709 12:25:17.082211  9991 leader_election.cc:336] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.889s (SENT)
I20260709 12:25:17.015218 14361 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:25:17.024763 14310 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:17.024549 14283 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:17.017455 14277 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:17.174670  8748 debug-util.cc:398] Leaking SignalData structure 0x7b080000f620 after lost signal to thread 9987
W20260709 12:25:17.060885 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55421.5us
I20260709 12:25:17.180944 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 669) took 3668 ms. Trace:
I20260709 12:25:17.230000 10097 rpcz_store.cc:276] 0709 12:25:13.512977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.836787 (+3323810us) service_pool.cc:224] Handling call
0709 12:25:16.837064 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.837364 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.837388 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:16.837868 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:25:16.986755 (+148887us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:16.986812 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:16.996926 (+ 10114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:17.180700 (+183774us) raft_consensus.cc:1685] finished
0709 12:25:17.180727 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.180879 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":239,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"thread_start_us":148427,"threads_started":1,"wal-append.queue_time_us":194168}
I20260709 12:25:17.174554  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:17.155206 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 75925.6us
I20260709 12:25:17.209046  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:17.191707 10090 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:25:17.127749 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.210666 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:17.211959 10017 raft_consensus.cc:2376] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 11. Re-sending same reply.
I20260709 12:25:17.212430 10091 rpcz_store.cc:276] 0709 12:25:13.892388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.922327 (+3029939us) service_pool.cc:224] Handling call
0709 12:25:16.926392 (+  4065us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.010635 (+ 84243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.010901 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:17.234766 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:17.173364 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:17.020792 14273 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:17.236567 10092 rpcz_store.cc:276] 0709 12:25:13.576579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.909574 (+3332995us) service_pool.cc:224] Handling call
0709 12:25:16.909877 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.003938 (+ 94061us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.003968 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:17.004008 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:17.004040 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.018385 (+ 14345us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b5400163d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:17.018427 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:17.018544 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215903616}
I20260709 12:25:17.079962  9842 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 5 to 7, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 7 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:17.214254 13235 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:17.165472 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.218559  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.202497 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.238399  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:17.214552  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080039bbe0 after lost signal to thread 10063
W20260709 12:25:17.239063 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.241068  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08001f4440 after lost signal to thread 10090
I20260709 12:25:17.241379  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:17.241768 10022 raft_consensus.cc:2368] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9.
I20260709 12:25:17.245700 10019 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
W20260709 12:25:17.241428  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800311f60 after lost signal to thread 10091
W20260709 12:25:17.251422  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003b0140 after lost signal to thread 10092
W20260709 12:25:17.251711  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080040e920 after lost signal to thread 10093
W20260709 12:25:17.252013  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0340 after lost signal to thread 10095
W20260709 12:25:17.252326  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08002ce1c0 after lost signal to thread 10099
W20260709 12:25:17.252995  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003d1840 after lost signal to thread 10123
W20260709 12:25:17.253266  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080035a940 after lost signal to thread 10124
W20260709 12:25:17.253674  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003a2b60 after lost signal to thread 10641
W20260709 12:25:17.253975  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08004143c0 after lost signal to thread 10643
I20260709 12:25:17.213510  9991 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:17.259333 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:17.260037  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:17.032393 14110 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:17.266575 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 664) took 3993 ms. Trace:
I20260709 12:25:17.267112 10090 rpcz_store.cc:276] 0709 12:25:13.272852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.832341 (+3559489us) service_pool.cc:224] Handling call
0709 12:25:16.832641 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.232731 (+400090us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.266343 (+ 33612us) spinlock_profiling.cc:243] Waited 359 ms on lock 0x7b5400304154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:17.266519 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":824693504}
I20260709 12:25:17.269030  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:17.260586  9948 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.261369  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:17.271090  9943 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:17.261847 10675 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
W20260709 12:25:17.265074  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 182869us
I20260709 12:25:17.272949 10677 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:17.265698 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:17.005683 14356 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:17.270120  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:17.266144 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1905) took 2848 ms. Trace:
I20260709 12:25:17.281123 10096 rpcz_store.cc:276] 0709 12:25:14.417576 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.958731 (+2541155us) service_pool.cc:224] Handling call
0709 12:25:16.959209 (+   478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.058002 (+ 98793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.265899 (+207897us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b5400340b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:17.266083 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83642624}
I20260709 12:25:17.273545 10025 raft_consensus.cc:2376] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 9. Re-sending same reply.
I20260709 12:25:17.274747 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.275807 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.284880 10675 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:25:17.277287  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800392b00 after lost signal to thread 10702
W20260709 12:25:17.279460 10065 connection.cc:256] RPC call timeout handler was delayed by -1.1556812299995727s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2828s for a response
I20260709 12:25:17.280153 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:17.336839 10091 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:17.337386 10099 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 12:25:17.287282 10025 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 12:25:17.337965 10099 pending_rounds.cc:85] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 7
I20260709 12:25:17.338232 10099 pending_rounds.cc:107] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:25:17.338615 10099 raft_consensus.cc:2889] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:17.339452 10097 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:17.340417  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080034bd40 after lost signal to thread 10703
I20260709 12:25:17.340344 10094 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 12:25:17.340971  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800309dc0 after lost signal to thread 13756
W20260709 12:25:17.341236  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800362f00 after lost signal to thread 13965
W20260709 12:25:17.341436  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08001ae960 after lost signal to thread 14035
I20260709 12:25:17.338148 10090 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
W20260709 12:25:17.341661  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08000ee920 after lost signal to thread 14110
I20260709 12:25:17.288694 10095 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
W20260709 12:25:17.342018  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0e80 after lost signal to thread 14111
W20260709 12:25:17.342466  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08000db9a0 after lost signal to thread 14118
I20260709 12:25:17.342658 10092 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 10. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
W20260709 12:25:17.342815  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003dbc00 after lost signal to thread 14130
W20260709 12:25:17.343268  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800235420 after lost signal to thread 14254
I20260709 12:25:17.341902 10090 pending_rounds.cc:85] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 8
W20260709 12:25:17.343683  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800235260 after lost signal to thread 14260
I20260709 12:25:17.343828 10090 pending_rounds.cc:107] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 9.9
W20260709 12:25:17.344331  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08000855e0 after lost signal to thread 14262
I20260709 12:25:17.344372 10090 raft_consensus.cc:2889] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:17.051079 14284 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:17.055369 14119 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:17.214893 14246 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:17.345623 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:17.360337 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80427.9us
I20260709 12:25:17.360515 10672 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:17.050814 14295 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.223s
I20260709 12:25:17.346247 10674 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:17.346827  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080035f6c0 after lost signal to thread 14272
I20260709 12:25:17.348097 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.349201 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:17.365192  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08002a8920 after lost signal to thread 14273
I20260709 12:25:17.350796 13235 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:17.350984 10017 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:17.365567  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003ebd40 after lost signal to thread 14275
I20260709 12:25:17.285396  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:17.365880  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800353fe0 after lost signal to thread 14279
W20260709 12:25:17.366405  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08003959e0 after lost signal to thread 14283
W20260709 12:25:17.366641  9970 debug-util.cc:398] Leaking SignalData structure 0x7b08000eb420 after lost signal to thread 14284
W20260709 12:25:17.366870  9970 debug-util.cc:398] Leaking SignalData structure 0x7b0800342980 after lost signal to thread 14287
I20260709 12:25:17.274781 14052 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:17.312350 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:17.368135 10019 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:17.143447  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:17.265635 14296 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:17.278149 14297 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:17.278169 14118 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:17.354696 14272 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:17.363813 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:17.302995 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:17.367276  9970 debug-util.cc:398] Leaking SignalData structure 0x7b080010c7a0 after lost signal to thread 14296
I20260709 12:25:17.187397 14355 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:17.375730  9842 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 3 to 6, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 6 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:17.266651 14353 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:17.059104 13996 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:17.369530 14295 consensus_queue.cc:1243] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:17.368656 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:17.382684  9948 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:17.383700 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:17.446065 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62464.9us
I20260709 12:25:17.446781 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:17.384693 10065 connection.cc:256] RPC call timeout handler was delayed by -1.0986116399999446s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:25:17.397895 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 391715us
I20260709 12:25:17.057292 14287 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:17.083803 14309 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:17.414790 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.364986 14130 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:17.420179 10022 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:17.427264  9944 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:17.364373 14246 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:17.380360  9830 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 410732us
W20260709 12:25:17.441306  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 168423us
I20260709 12:25:17.450958 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1899) took 3053 ms. Trace:
I20260709 12:25:17.380271 14296 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:17.612939 10094 rpcz_store.cc:276] 0709 12:25:14.397800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.950658 (+2552858us) service_pool.cc:224] Handling call
0709 12:25:16.951023 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.341049 (+390026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.450750 (+109701us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b5400340654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:17.450903 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":894458240}
W20260709 12:25:17.461238  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 404182us
I20260709 12:25:17.124939 14350 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:17.517231 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 419725us
I20260709 12:25:17.520057 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:17.649310 10024 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:17.460690 14260 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:17.554554 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:17.454638 14272 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:17.558967 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 667) took 4047 ms. Trace:
I20260709 12:25:17.656504 10099 rpcz_store.cc:276] 0709 12:25:13.511983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.833705 (+3321722us) service_pool.cc:224] Handling call
0709 12:25:16.833965 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:17.339057 (+505092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.521421 (+182364us) spinlock_profiling.cc:243] Waited 503 ms on lock 0x7b540014a0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:17.558881 (+ 37460us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1156991744}
I20260709 12:25:17.516462 14359 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:17.519380 13965 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.654s	user 0.011s	sys 0.000s
I20260709 12:25:17.572788  9842 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 5 to 6, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 6 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:17.577868  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.471223 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08000965c0 after lost signal to thread 8743
I20260709 12:25:17.599437 14349 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:17.676880 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.564523 14254 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:17.461442 10065 connection.cc:256] RPC call timeout handler was delayed by -1.0966980269995474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
I20260709 12:25:17.615700  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:17.612596  9991 leader_election.cc:336] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 2.057s (SENT)
I20260709 12:25:17.683197  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:17.649432 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:17.684983 10640 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.634s (SENT) [suppressed 50 similar messages]
W20260709 12:25:17.685993 10642 leader_election.cc:336] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.634s (SENT)
I20260709 12:25:17.686060 10024 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:17.659885 10675 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:17.689085 10675 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:17.693616 10675 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:17.667490 10020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.004s	sys 0.003s
W20260709 12:25:17.674527 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800097840 after lost signal to thread 8746
I20260709 12:25:17.676203 10021 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:25:17.456761 14361 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:17.648571 14130 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:17.753095 14130 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:17.753360 10672 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:17.697165  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81561.4us
W20260709 12:25:17.697782  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 85218.9us
W20260709 12:25:17.708477 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.697813 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.648869 14296 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:17.754781 10675 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
W20260709 12:25:17.762835 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800096820 after lost signal to thread 9879
I20260709 12:25:17.696979 10677 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:17.778483  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:17.655417 14363 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:17.656315 14272 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:17.654996 14277 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:17.755350  9842 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:17.790295 10675 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:17.831591 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.792394  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:17.697448 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:17.698688  9948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:17.833865  9948 raft_consensus.cc:606] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:17.766939 10020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.005s	sys 0.003s
I20260709 12:25:17.834306  9948 raft_consensus.cc:614] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:17.806387 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080016af20 after lost signal to thread 9973
W20260709 12:25:17.835889 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080016af40 after lost signal to thread 9974
W20260709 12:25:17.836229 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080016afc0 after lost signal to thread 9987
W20260709 12:25:17.836489 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800252ba0 after lost signal to thread 9990
W20260709 12:25:17.836710 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003af380 after lost signal to thread 9991
W20260709 12:25:17.811601 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 363770us
W20260709 12:25:17.827000 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.837437 10062 leader_election.cc:400] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8.
I20260709 12:25:17.838182 10062 leader_election.cc:403] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:17.839221 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:17.767959 14193 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:17.826150  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:17.847672  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.704607 14283 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:17.859778 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080021afc0 after lost signal to thread 10016
W20260709 12:25:17.862111 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800098340 after lost signal to thread 10018
W20260709 12:25:17.862571 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080039b9a0 after lost signal to thread 10019
W20260709 12:25:17.863240 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800049740 after lost signal to thread 10020
W20260709 12:25:17.863623 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08002e5b00 after lost signal to thread 10021
W20260709 12:25:17.864053 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5f00 after lost signal to thread 10023
W20260709 12:25:17.864475 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08001004e0 after lost signal to thread 10024
W20260709 12:25:17.864893 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080038a400 after lost signal to thread 10025
W20260709 12:25:17.865706 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080037a6a0 after lost signal to thread 10049
W20260709 12:25:17.866410 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080037a660 after lost signal to thread 10065
W20260709 12:25:17.867265 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08001a9be0 after lost signal to thread 10090
W20260709 12:25:17.867753 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003b4560 after lost signal to thread 10096
W20260709 12:25:17.868141 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080036bfa0 after lost signal to thread 10097
W20260709 12:25:17.874186  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81854.4us
I20260709 12:25:17.675752 14362 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.168s
I20260709 12:25:17.809531 14273 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:17.869375 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.893306 10677 raft_consensus.cc:2368] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8.
I20260709 12:25:17.847652 14310 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:17.833369 14272 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:17.845361 14193 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:17.889349 14377 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.369s	user 0.001s	sys 0.000s
I20260709 12:25:17.682843 13965 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:17.991688 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.846902 14349 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:17.959086  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78062.9us
I20260709 12:25:17.875414 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:17.754052 14380 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.199s
I20260709 12:25:17.780198 14361 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:18.029024 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 153700us
W20260709 12:25:17.887300 14279 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.998s	user 0.013s	sys 0.000s
W20260709 12:25:17.881659 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800149380 after lost signal to thread 10098
I20260709 12:25:17.960300 14110 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:17.880407 14119 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:17.983040 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:17.990208  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:17.971153 14353 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:18.037271  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47132.7us
I20260709 12:25:18.038285  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:17.834395 14359 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:17.967489 14275 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.968s	user 0.008s	sys 0.004s
W20260709 12:25:18.026000 10673 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.767s	user 0.010s	sys 0.002s
I20260709 12:25:17.972206 14273 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:17.965391 14402 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:17.960631 14254 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8.
W20260709 12:25:18.032112 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003b4620 after lost signal to thread 10099
I20260709 12:25:18.033370 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:17.966434 14260 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:17.991459 14193 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:17.966840 13996 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.040562 10668 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:18.040676  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:18.042132  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 194805us
I20260709 12:25:18.041816 14110 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:18.078823 14359 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:18.035616 14310 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:18.080701 14402 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:18.081332  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:18.081460 14309 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:18.044278 14035 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 12:25:18.082720 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003a6ae0 after lost signal to thread 10123
I20260709 12:25:18.043387 10673 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:18.079689 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:17.975450 14118 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
W20260709 12:25:18.263597 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080030ea60 after lost signal to thread 10124
W20260709 12:25:18.264595 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003ffee0 after lost signal to thread 10640
W20260709 12:25:18.264875 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800326de0 after lost signal to thread 10641
W20260709 12:25:18.264861  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 224256us
W20260709 12:25:18.265370 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003b08a0 after lost signal to thread 10643
W20260709 12:25:18.266077 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800364e20 after lost signal to thread 10677
I20260709 12:25:18.045624 10674 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:25:18.266621 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08002702e0 after lost signal to thread 10699
W20260709 12:25:18.266927 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003d0b80 after lost signal to thread 10702
W20260709 12:25:18.267284 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08004151e0 after lost signal to thread 10703
I20260709 12:25:18.159312  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1983) took 1241 ms. Trace:
I20260709 12:25:18.175376 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1869) took 1245 ms. Trace:
W20260709 12:25:18.215363 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 376186us
W20260709 12:25:18.236488 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:18.267536 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800269680 after lost signal to thread 13756
W20260709 12:25:18.146445  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.863s	user 0.015s	sys 0.007s
W20260709 12:25:18.143746 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.143489 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1870) took 1212 ms. Trace:
I20260709 12:25:18.221853  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:18.276095 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 897) took 1402 ms. Trace:
I20260709 12:25:18.272125 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.296116 10673 rpcz_store.cc:276] 0709 12:25:16.854787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.857620 (+  2833us) service_pool.cc:224] Handling call
0709 12:25:18.257041 (+1399421us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:18.143527  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1728) took 3681 ms. Trace:
W20260709 12:25:18.269220 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080023b5e0 after lost signal to thread 13965
W20260709 12:25:18.297322 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003a06c0 after lost signal to thread 13996
W20260709 12:25:18.054204  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.270115 10016 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 12:25:18.297739 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08002fa200 after lost signal to thread 14035
W20260709 12:25:18.298053 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08001e5860 after lost signal to thread 14052
W20260709 12:25:18.298257 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800222900 after lost signal to thread 14110
W20260709 12:25:18.298504 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003f8300 after lost signal to thread 14111
W20260709 12:25:18.298930 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08002a87e0 after lost signal to thread 14118
I20260709 12:25:18.270174 10023 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:25:18.299590  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:18.295701 10018 rpcz_store.cc:276] 0709 12:25:16.934784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.942294 (+  7510us) service_pool.cc:224] Handling call
0709 12:25:16.942790 (+   496us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.011178 (+ 68388us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:25:17.011209 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:17.025234 (+ 14025us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.381047 (+355813us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:25:17.381087 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.381148 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.143258 (+762110us) raft_consensus.cc:1685] finished
0709 12:25:18.143286 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.143432 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49103,"raft.queue_time_us":56974,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":5437,"thread_start_us":297689,"threads_started":1,"wal-append.queue_time_us":988179}
W20260709 12:25:18.239945 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.257308 10676 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:18.267879  9944 rpcz_store.cc:276] 0709 12:25:16.918052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.920396 (+  2344us) service_pool.cc:224] Handling call
0709 12:25:18.159064 (+1238668us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b5400208354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:18.159223 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":989494144}
I20260709 12:25:18.303334  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:18.304066  9944 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
W20260709 12:25:18.267546 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 234238us
I20260709 12:25:18.297108  9947 rpcz_store.cc:276] 0709 12:25:14.461946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.882947 (+2421001us) service_pool.cc:224] Handling call
0709 12:25:16.903344 (+ 20397us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:16.911100 (+  7756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:16.911147 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:16.924540 (+ 13393us) log.cc:844] Serialized 52 byte log entry
0709 12:25:17.272436 (+347896us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:17.272491 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.276547 (+  4056us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.143356 (+866809us) raft_consensus.cc:1685] finished
0709 12:25:18.143373 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.143478 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1389,"raft.queue_time_us":2491,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":347486,"threads_started":1,"wal-append.queue_time_us":458036}
I20260709 12:25:18.306427  9912 leader_election.cc:400] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9.
I20260709 12:25:18.267997 10020 rpcz_store.cc:276] 0709 12:25:16.930284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.930700 (+   416us) service_pool.cc:224] Handling call
0709 12:25:16.931030 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:16.963374 (+ 32344us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:16.963407 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:16.988753 (+ 25346us) spinlock_profiling.cc:243] Waited 1.51 ms on lock 0x7b2c00046bc8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:25:16.995033 (+  6280us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.834842 (+839809us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:17.834889 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.834961 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.174958 (+339997us) raft_consensus.cc:1685] finished
0709 12:25:18.174989 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.175293 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23046,"raft.queue_time_us":33223,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":15925,"spinlock_wait_cycles":3483520,"thread_start_us":809069,"threads_started":1,"wal-append.queue_time_us":1158871}
I20260709 12:25:18.296514 10025 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:18.307420  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:18.308090  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.018s	sys 0.009s
W20260709 12:25:18.308492  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.02471e+06us
W20260709 12:25:18.308424  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 227153us
W20260709 12:25:18.309752 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08004150c0 after lost signal to thread 14119
W20260709 12:25:18.310092 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800311d20 after lost signal to thread 14130
I20260709 12:25:18.309894  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:18.310403 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003a47a0 after lost signal to thread 14193
I20260709 12:25:18.270684 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.141607 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 681) took 1155 ms. Trace:
I20260709 12:25:18.311014 10022 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
W20260709 12:25:18.310700 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08001205e0 after lost signal to thread 14214
I20260709 12:25:18.298312  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:25:18.313093 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800386ea0 after lost signal to thread 14236
I20260709 12:25:18.312956 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:18.313423 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08002c0920 after lost signal to thread 14246
W20260709 12:25:18.313745 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08002be820 after lost signal to thread 14254
I20260709 12:25:18.313614  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:18.314180 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003599c0 after lost signal to thread 14260
W20260709 12:25:18.120540 14397 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:25:18.151929 14394 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 12:25:18.078336 14275 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:18.314530 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003621a0 after lost signal to thread 14262
I20260709 12:25:18.160979 13965 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:18.182461 14246 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.505s	user 0.005s	sys 0.002s
I20260709 12:25:18.151412 14356 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.304337 14111 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:18.318813  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:18.319464  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:18.319512  9948 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:25:18.319818  9942 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:18.320216  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:18.366021  9948 pending_rounds.cc:85] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 9
I20260709 12:25:18.366300  9948 pending_rounds.cc:107] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:18.366496  9940 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:18.320461 14119 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:18.321250 14355 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.321719 14353 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:18.322340 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.318845 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.388823 10019 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:18.208303 14296 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.322845  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:18.140434 14361 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.209642 14284 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:18.323413  9945 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:18.148350 14130 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.329164 13756 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.314800 14110 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:18.304833 14309 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:18.330842 14396 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:18.333518 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:18.194824 14260 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.336383 14407 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:18.397806  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:18.339377 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.268998 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:18.334904  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:18.040004 14279 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:18.311002 10098 rpcz_store.cc:276] 0709 12:25:16.986660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.259822 (+273162us) service_pool.cc:224] Handling call
0709 12:25:17.285268 (+ 25446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:17.285628 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:17.695876 (+410248us) log.cc:844] Serialized 14 byte log entry
0709 12:25:17.884808 (+188932us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:17.885432 (+   624us) log.cc:844] Serialized 26 byte log entry
0709 12:25:17.885607 (+   175us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:17.885662 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:17.885749 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.141355 (+255606us) raft_consensus.cc:1685] finished
0709 12:25:18.141381 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.141536 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":569342,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":76965,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":31916,"thread_start_us":352776,"threads_started":2,"wal-append.queue_time_us":219730}
I20260709 12:25:18.221324 14236 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.351153 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080036f440 after lost signal to thread 14272
W20260709 12:25:18.407327 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080012b8c0 after lost signal to thread 14273
W20260709 12:25:18.407696 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080039c840 after lost signal to thread 14275
W20260709 12:25:18.407950 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003ca140 after lost signal to thread 14279
W20260709 12:25:18.408269 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003bc980 after lost signal to thread 14283
W20260709 12:25:18.408527 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003f22c0 after lost signal to thread 14284
W20260709 12:25:18.114431 14327 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.179s	user 0.012s	sys 0.000s
I20260709 12:25:18.352583 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.353322 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.353786 14382 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:18.175603 14350 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:18.211061 14287 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:18.353829 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:18.355401  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.414912 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.416558 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.417527 10674 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:18.045583 14214 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:18.357323  9947 pending_rounds.cc:85] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 9
I20260709 12:25:18.418680  9947 pending_rounds.cc:107] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 12:25:18.418951  9947 raft_consensus.cc:2889] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:18.357435 13235 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:18.360707 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.368492 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.369160 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:18.369979 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.370927 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.406333 14361 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:18.361986  9941 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:18.305266 14273 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:18.374197 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:18.374506  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 346696us
W20260709 12:25:18.385919 10642 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.224s (SENT)
I20260709 12:25:18.407785 14396 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:18.429899  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 120079us
I20260709 12:25:18.430787  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:18.432267  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:18.387049 10669 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:18.391031 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.391673 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:18.435081 10097 raft_consensus.cc:2376] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 11. Re-sending same reply.
I20260709 12:25:18.392194 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:18.392892 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.353772 14324 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:18.395128 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:18.305404 13996 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:25:18.396682 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:18.397986 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64574.4us
I20260709 12:25:18.314821 10065 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:18.399673  9943 raft_consensus.cc:1217] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-11.9]   Dedup: 5.6->[8.7-11.9]
I20260709 12:25:18.401386  9942 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:18.402351 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 133431us
I20260709 12:25:18.356567 14272 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:18.406617  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71823.2us
I20260709 12:25:18.409129  9946 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
W20260709 12:25:18.411285 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08002b2da0 after lost signal to thread 14287
I20260709 12:25:18.396502 14309 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.374953 14246 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.406141 14284 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:18.366519  9948 raft_consensus.cc:2889] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:18.422729 10095 raft_consensus.cc:2410] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:18.424052 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.421840 10091 raft_consensus.cc:2376] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 9. Re-sending same reply.
I20260709 12:25:18.425802 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.428208 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:18.429270 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.361666 14111 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.436144  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:18.436152 10092 raft_consensus.cc:2376] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 11. Re-sending same reply.
I20260709 12:25:18.469086  9943 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: [2.5-11.9]
I20260709 12:25:18.469403 10090 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:18.425473 14361 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.472496 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:18.527364 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57476.2us
I20260709 12:25:18.400436 14193 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:18.480067 10046 debug-util.cc:398] Leaking SignalData structure 0x7b080034cf00 after lost signal to thread 14296
W20260709 12:25:18.468909 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 154174us
W20260709 12:25:18.535899 10046 debug-util.cc:398] Leaking SignalData structure 0x7b0800360880 after lost signal to thread 14309
I20260709 12:25:18.536294 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:18.397464 13756 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.478672  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:18.540980  9948 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:25:18.540818 10022 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:25:18.541196  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62672.3us
I20260709 12:25:18.484635 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.492890 10091 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:18.492759 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.542719 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.543252 10093 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:25:18.493870 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.494201 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:18.545691 10017 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:18.545989 10025 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:18.495007 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.495252 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.495570 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.547857 10020 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:18.495704 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.495954 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.548784 10016 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 9 index: 11. (index mismatch)
I20260709 12:25:18.438886 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.412372 14236 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.554018 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:18.481845 14356 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:18.533372  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 101171us
W20260709 12:25:18.536329 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003c8120 after lost signal to thread 14313
I20260709 12:25:18.473486 14407 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:18.561537 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08003c9700 after lost signal to thread 14324
W20260709 12:25:18.561920 10046 debug-util.cc:398] Leaking SignalData structure 0x7b08002f85a0 after lost signal to thread 14327
I20260709 12:25:18.565142 10091 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:18.550247 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:18.483600 14287 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:18.401487 14402 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.503736 14111 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.553612 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:18.576399 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:18.583770 10092 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.554811 10019 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:18.584221 10093 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:18.485378 14309 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:18.533511 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.586647 10095 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:18.473578 14362 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:25:18.555207 14324 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:18.558118 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 125071us
I20260709 12:25:18.502363 14295 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
I20260709 12:25:18.576568 10090 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:18.581144 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 91386.1us
I20260709 12:25:18.533248 14361 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:18.501133 14401 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.000s	sys 0.029s
I20260709 12:25:18.588238 10025 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:18.588860 10062 leader_election.cc:336] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.514s (SENT)
I20260709 12:25:18.482556 14385 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 12:25:18.566434 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:18.485134 14380 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 12:25:18.423045 14382 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.540726 14284 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:18.545243 14396 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:18.553486 10099 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:18.556077 14279 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:18.581377 13996 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.577186 14287 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.607213 14410 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:18.602162 14310 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:25:18.662341 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 276466us
W20260709 12:25:18.651033 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.599658 14324 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:18.585810 14277 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:25:18.581987 14384 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:25:18.607369 14327 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.069s
I20260709 12:25:18.611021 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.582875 14356 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.620522 14407 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:18.731150 14362 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:18.730892 14407 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.731688 14362 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.734113 14407 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.599875 14275 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:25:18.671696 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.745693 10091 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:18.681826 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.676263 14310 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.620225 14415 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:18.646775 14279 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:18.671190 14408 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.058s	user 0.002s	sys 0.000s
I20260709 12:25:18.687000 14324 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:18.694783 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.575932 14395 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:25:18.721586 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.669099 14287 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:18.666332 14396 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.723970 10095 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:18.672698 14403 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:18.554339 14110 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.726275 14277 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:18.728456 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 888) took 2009 ms. Trace:
I20260709 12:25:18.728986 14272 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:18.729915 14246 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.735033 14380 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:18.669745 13996 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.733839 14362 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.736362 14356 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.737519 13756 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.669605 14410 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:18.666548 14426 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:18.749020 10099 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:18.683427  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 90452.6us
W20260709 12:25:18.684037 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 88574.3us
I20260709 12:25:18.672537 14416 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:18.684125 14296 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.688201 10090 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:18.756371 10093 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
W20260709 12:25:18.757683 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 151133us
I20260709 12:25:18.852425 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:18.853349 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:18.853698 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:18.854004 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.613165 14273 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.856662 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.857842 10675 raft_consensus.cc:2410] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:25:18.680045 14035 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 12:25:18.760288 14295 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:18.799718 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 190310us
I20260709 12:25:18.748502 14130 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:18.858826 14295 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:18.859548 14362 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:18.859833 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.753093 14310 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:18.860009 14295 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.861183 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:18.861359 14295 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:18.792814  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 212759us
I20260709 12:25:18.862695 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.862776  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:18.862967 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:18.863502 10093 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 8: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:18.863632 10099 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:18.864353 14356 raft_consensus.cc:2764] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:18.865046 14356 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:25:18.802851  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 366798us
I20260709 12:25:18.794617 10670 rpcz_store.cc:276] 0709 12:25:16.719294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:16.719443 (+   149us) service_pool.cc:224] Handling call
0709 12:25:16.719917 (+   474us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:25:16.998058 (+278141us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:17.431680 (+433622us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800478ea0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:25:17.836715 (+405035us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.162731 (+326016us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b1800478ea0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:18.402123 (+239392us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.402301 (+   178us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:18.403164 (+   863us) log.cc:844] Serialized 52 byte log entry
0709 12:25:18.403385 (+   221us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:18.403434 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.403531 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.728251 (+324720us) raft_consensus.cc:1685] finished
0709 12:25:18.728270 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.728396 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1998,"prepare.queue_time_us":826173,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":3129,"raft.queue_time_us":6943,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":11054,"spinlock_wait_cycles":68218368,"thread_start_us":858382,"threads_started":2,"wal-append.queue_time_us":713522}
I20260709 12:25:18.867712 14246 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:18.869364 14035 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.870040 14035 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:18.846046 10065 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:18.860267 14362 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:18.871800 14362 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:18.872121 14035 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:18.755806 14279 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:18.872350 14362 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:18.876623 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:18.866581  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:18.877619 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:18.879065 10096 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:18.881335 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:18.864748 10675 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:18.889586 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:18.889616 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.891192 10643 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:18.891383 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.914541 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.915978 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:18.794479 14277 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:18.918702 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 840) took 1165 ms. Trace:
W20260709 12:25:18.919256 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:18.921729 10097 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:18.924849 10672 rpcz_store.cc:276] 0709 12:25:17.753488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.779445 (+ 25957us) service_pool.cc:224] Handling call
0709 12:25:17.827568 (+ 48123us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:17.833813 (+  6245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:17.833842 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:17.868936 (+ 35094us) log.cc:844] Serialized 52 byte log entry
0709 12:25:18.037874 (+168938us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:18.589680 (+551806us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.917934 (+328254us) log.cc:844] Serialized 14 byte log entry
0709 12:25:18.918264 (+   330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.918360 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:18.918420 (+    60us) raft_consensus.cc:1685] finished
0709 12:25:18.918439 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:18.918621 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8966,"prepare.queue_time_us":133105,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":146797,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":22830,"thread_start_us":859550,"threads_started":3,"wal-append.queue_time_us":190461}
I20260709 12:25:18.874500 13756 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:18.927387  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60858.3us
I20260709 12:25:18.750227 14284 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:18.936780 10093 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:18.936123  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:18.876077 14429 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:18.937711 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:18.933578 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:18.953035 10019 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:18.927179 14035 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:18.885353 14416 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:18.962218 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:18.962949 10018 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:18.970065  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 121023us
I20260709 12:25:18.976786  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:18.935781 14260 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:18.909039 14295 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:25:18.985939  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:18.916222 14296 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.064s
W20260709 12:25:18.998556  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62512.5us
I20260709 12:25:18.923614 14395 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:25:19.004947  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:18.950320 14429 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:19.018432 10018 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:25:19.000578 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108086us
W20260709 12:25:19.019717 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 128574us
I20260709 12:25:18.935981 14426 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
W20260709 12:25:19.107170 10092 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.523s	user 0.013s	sys 0.001s
I20260709 12:25:18.984860 14423 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:18.997061 14273 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:19.111222 10092 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:18.993651 14236 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:18.998730 14254 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:19.108599 13756 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:19.135939 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 110619us
W20260709 12:25:19.136420 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 109578us
I20260709 12:25:18.994197 14385 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.136s
I20260709 12:25:19.110769 14384 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:25:19.007889 14260 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.153522 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:19.135116 14403 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:19.159922 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.161270 10675 raft_consensus.cc:2410] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 12 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:19.141244 14119 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.165086 14440 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
W20260709 12:25:19.171087 14425 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:25:19.132687 14391 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:25:19.174218 10672 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:19.172595 14396 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:19.004115 14277 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.103379 14431 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.158s
I20260709 12:25:19.177083 14387 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:19.177583 14387 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.177964 14387 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.152631 14236 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:19.178890 14236 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:19.179517 14236 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.149945 14273 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:19.167817 14414 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.403s	user 0.000s	sys 0.001s
I20260709 12:25:19.178833 14426 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:18.985200 14410 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.153872 14254 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:19.201278 14428 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:25:19.201776 14254 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.202782 14455 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:25:19.203694 14452 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:19.204166 14452 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.204525 14452 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.153997 14423 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:25:19.208508 14455 consensus_queue.cc:1243] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:19.146845 14416 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.166544 14429 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:19.221516 14422 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
I20260709 12:25:19.222105 14432 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:25:19.222344 14429 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:19.223963 14441 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:25:19.108404 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:19.288062 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:19.289784 10018 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:19.281064 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 127476us
W20260709 12:25:19.293704 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 355973us
I20260709 12:25:19.296975 10024 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:19.172618 13756 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.137s
I20260709 12:25:19.309451 13996 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.183137 14119 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.202790 14260 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.283558 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:19.197964 14310 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.323812  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:19.289819 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 136220us
I20260709 12:25:19.349546 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:19.352069 14464 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
W20260709 12:25:19.352723 14467 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:25:19.353457 14310 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.355199 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:19.355609 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:19.356208 14469 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:25:19.356401 10023 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.357250 10676 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 10 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 12:25:19.359781 14438 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 12:25:19.360047 14462 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:25:19.210502 14382 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:19.361569 14468 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
W20260709 12:25:19.362707 14421 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:25:19.367218 14442 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:19.368610 14435 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.371325 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:19.372496 10018 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:19.374241 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:19.162340 14403 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:19.366441 14355 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.973s	user 0.012s	sys 0.000s
I20260709 12:25:19.388356  9941 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:19.337491 14432 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:19.394009 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:19.400787 14453 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:19.401759 14463 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:19.402853 14384 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:19.403422 14434 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:19.404896 14440 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.369927 14435 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:19.422204 10064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.006s	sys 0.007s
W20260709 12:25:19.423610 10064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.753s	user 0.007s	sys 0.007s
I20260709 12:25:19.423851 14440 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.424500 14448 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.425462 14448 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.428501 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:19.429400 10093 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:19.430812  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.432802 10021 raft_consensus.cc:1217] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 5.6->[7.7-9.8]   Dedup: 9.8->[]
I20260709 12:25:19.433614 10021 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: [7.7-9.8]
I20260709 12:25:19.407030 14447 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.441368 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:19.444445 14450 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:25:19.444809 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:19.446467 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:19.462670 10019 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
W20260709 12:25:19.452553 14111 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.804s	user 0.009s	sys 0.009s
I20260709 12:25:19.470242 10018 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:19.481684  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 476645us
W20260709 12:25:19.483127 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 133876us
I20260709 12:25:19.469486  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.482178  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 818) took 1180 ms. Trace:
I20260709 12:25:19.420128 14355 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:19.478259 14435 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.497552 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.500690 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:19.501765 14401 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:19.502507 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:19.495589  9945 rpcz_store.cc:276] 0709 12:25:18.301797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.398290 (+ 96493us) service_pool.cc:224] Handling call
0709 12:25:18.398645 (+   355us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.410725 (+ 12080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.410768 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:18.418876 (+  8108us) log.cc:844] Serialized 26 byte log entry
0709 12:25:18.603713 (+184837us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:18.603748 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.481486 (+877738us) spinlock_profiling.cc:243] Waited 11.7 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:19.481552 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.481645 (+    93us) raft_consensus.cc:1685] finished
0709 12:25:19.481657 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.481972 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":26981760,"thread_start_us":166320,"threads_started":1,"wal-append.queue_time_us":240895}
I20260709 12:25:19.520324 10022 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
W20260709 12:25:19.498152 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 568325us
I20260709 12:25:19.530843 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3049) took 1163 ms. Trace:
I20260709 12:25:19.531345 10094 rpcz_store.cc:276] 0709 12:25:18.367577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.370722 (+  3145us) service_pool.cc:224] Handling call
0709 12:25:19.509227 (+1138505us) spinlock_profiling.cc:243] Waited 547 ms on lock 0x7b5400164254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:19.530650 (+ 21423us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1257418112}
I20260709 12:25:19.520835 14401 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:19.543506  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.543622 14401 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.545318 14401 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.545801  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.548229 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:19.550721 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:19.530328 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 128619us
I20260709 12:25:19.559481 10093 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:19.560706  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 480297us
I20260709 12:25:19.561934 10016 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:19.562309  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:19.563045 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:19.563717 14382 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:19.564185 14382 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:19.564767 10016 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:19.564733 14382 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:19.578765  9945 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
W20260709 12:25:19.579397 14279 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.687s	user 0.008s	sys 0.018s
I20260709 12:25:19.579802 14387 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.583405 14119 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:19.584465 14361 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.959s	user 0.008s	sys 0.013s
I20260709 12:25:19.585147 14361 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.587378 10673 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:19.579923  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.556068 10090 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:19.599968 14456 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:19.579879 14279 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:19.602381 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.510001 14111 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.582458 14387 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.564129  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:19.614310  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.614488 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:19.554440 14236 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:19.622906 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:19.625180 14111 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.627012 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.627939 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.628248 10675 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:19.629560 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.631057 14454 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:19.627095  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:19.635810  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.637125  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.637444 14452 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.638471  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.638912 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.640267  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:19.647166 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:19.648054 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.649269 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:19.649991 14236 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:19.652408 14460 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:19.653051 14460 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:19.655562  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.656939  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.657225 14393 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:19.657776  9940 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:25:19.657738 14393 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:19.558915 14447 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:19.644106  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.646135 14472 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:19.647367  9945 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.643419 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.658514 10018 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
W20260709 12:25:19.661098 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.685880 10023 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:19.687412 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:19.688875 14309 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.091s	user 0.008s	sys 0.009s
I20260709 12:25:19.689766  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.689479 14309 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.692713  9943 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
W20260709 12:25:19.694267 14422 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:25:19.696111 14353 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:19.696806 14353 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:19.697139 14485 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:25:19.694453 14277 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:19.636159 13965 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:19.669631 14362 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.796s	user 0.007s	sys 0.011s
I20260709 12:25:19.700949 14442 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:19.729367 14442 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.729715  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.731016  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.732102  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:19.736441  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 817) took 1434 ms. Trace:
I20260709 12:25:19.736920  9946 rpcz_store.cc:276] 0709 12:25:18.301587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.312901 (+ 11314us) service_pool.cc:224] Handling call
0709 12:25:18.313203 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.479402 (+166199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:19.735897 (+1256495us) spinlock_profiling.cc:243] Waited 92.8 ms on lock 0x7b54000b7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:19.736252 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":213427456}
W20260709 12:25:19.738695 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.738762 14362 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.739809 14452 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:19.740577 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48698.2us
I20260709 12:25:19.740849 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:19.741117 14362 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:19.741734 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:19.743209 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:19.743767  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.708428 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.745549 10674 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:19.746410 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:19.747525 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.748886 10674 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:25:19.708778 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.710505 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.698536 14353 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.713929  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:19.660496 14193 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.713873 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.714830 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:19.755347 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.715355  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.758008  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.759294 14273 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.760124 14273 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.761180 14442 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.715626 14279 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:19.654078 14460 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:19.763577 14442 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.764565 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:19.718161 14440 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:19.666030 14419 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.043s	user 0.003s	sys 0.000s
I20260709 12:25:19.767736 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:19.768527 10092 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:19.719282 10677 raft_consensus.cc:2368] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8.
I20260709 12:25:19.726229 10022 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
W20260709 12:25:19.714102 14287 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.912s	user 0.000s	sys 0.015s
I20260709 12:25:19.738327  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.766320 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:19.752405  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.771121 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:19.772256  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 814) took 1471 ms. Trace:
I20260709 12:25:19.774907 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:19.777683  9949 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:19.724027 13965 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:19.786152 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 156304us
I20260709 12:25:19.751187 14353 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.776419 14287 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.783409 14279 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:19.796713  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.807053  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.807425  9946 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:19.808197  9942 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:19.809337 14254 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:19.809836 14254 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:19.809195 13965 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.810184 14254 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.812929 14382 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.813894 14359 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:19.815913 14447 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:19.659128 14393 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:19.816345 14359 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.817634 14393 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:19.819743 14359 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.815269 13965 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.822405 14382 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.835286 14273 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.837937 14287 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.841846  9941 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:19.845156  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.848271  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.850664 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:19.851421 10090 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
W20260709 12:25:19.851989  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 138330us
I20260709 12:25:19.778321  9944 rpcz_store.cc:276] 0709 12:25:18.300915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.305366 (+  4451us) service_pool.cc:224] Handling call
0709 12:25:18.305722 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:18.306096 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:18.306147 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:18.306642 (+   495us) log.cc:844] Serialized 26 byte log entry
0709 12:25:18.358158 (+ 51516us) spinlock_profiling.cc:243] Waited 4.99 ms on lock 0x7b1800412060. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:25:18.430494 (+ 72336us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:18.430536 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:18.430628 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:19.665118 (+1234490us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:19.771756 (+106638us) raft_consensus.cc:1685] finished
0709 12:25:19.771787 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:19.772028 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11049,"raft.queue_time_us":8176,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":64264832,"thread_start_us":70759,"threads_started":1,"wal-append.queue_time_us":237714}
W20260709 12:25:19.854364  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 290442us
I20260709 12:25:19.856362  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.856596  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:19.858704 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 115602us
I20260709 12:25:19.858919 14350 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:19.861289 14350 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:19.861649 14350 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:19.862035 14350 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.868889 14409 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:19.869354 14324 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.079s	user 0.010s	sys 0.010s
I20260709 12:25:19.869676 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:19.869858 14324 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:19.870852 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 65589us
I20260709 12:25:19.858460  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:19.831815 14254 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:19.870901 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:19.883502 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:19.886554 14309 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.887372 14309 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:19.887650  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:19.888068 14309 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:19.888481 14309 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:19.889007 14309 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.889132  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:19.891021 14466 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:19.891451 14466 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:19.891825 14466 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:19.893533 10670 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:19.894821 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:19.895598 14327 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:19.896021 14327 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:19.871948 14472 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.897152 14327 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:19.898255 14327 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.899154 14279 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:19.878415 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:19.878850  9947 raft_consensus.cc:2368] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8.
I20260709 12:25:19.899590 14382 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:19.900022 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:19.880868 10643 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:19.902217 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:19.898916 14472 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:19.907459  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:19.911911 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.920308  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.920840  9945 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:19.922813 10092 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:19.922278 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:19.912526 14277 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:19.923846 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:19.928206  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:19.926487 10025 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:19.930727 14442 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:19.932298 14440 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:25:19.935096  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46047.9us
I20260709 12:25:19.936043  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:19.937460  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:19.938290 14279 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:19.939049 14279 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:19.939527 14279 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:19.940006  9941 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:19.940366 10091 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:19.940755 14279 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:19.942051 10669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 983) took 3027 ms (client timeout 2427 ms). Trace:
I20260709 12:25:19.943100 10096 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:19.943301 10671 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:19.943621  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:19.942477 10669 rpcz_store.cc:269] 0709 12:25:16.914098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.274420 (+360322us) service_pool.cc:224] Handling call
0709 12:25:19.941676 (+2667256us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54005b95d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:19.941843 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2539066752}
I20260709 12:25:19.945736 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:19.947566  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:19.948213 14295 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:19.935070  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.949321  9947 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:19.953812  9941 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.949301  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:19.958694  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:19.959751  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 101402us
I20260709 12:25:19.963943  9945 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:19.972012 10092 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:19.950525 14295 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:19.974521 10096 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:19.958362 14472 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:19.975662 14119 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:19.978434 10090 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:19.963846 13965 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:19.983702  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.988428 10065 leader_election.cc:400] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8.
I20260709 12:25:19.989902 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:19.991369 14295 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.978523 14432 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:25:19.994045 14432 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:19.994460 14432 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:19.995571 14432 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:19.996956 14432 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:19.997833  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:19.999471 14324 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:20.001065 14119 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:20.000645  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:20.002628 14119 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:20.005548  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61996.9us
I20260709 12:25:20.005848 14254 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:20.010550  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:20.011298 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:19.990473 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:19.977869 14455 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:25:20.043401  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:19.985852 13965 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.050866 14387 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.056437 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:20.057538  9940 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:25:20.059186  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:20.059835  9947 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:20.062026  9945 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
W20260709 12:25:20.072865  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 152950us
I20260709 12:25:20.012177 14350 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.075695  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:20.077076 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:20.078168 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 88360.5us
I20260709 12:25:20.044391 14111 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
W20260709 12:25:20.079384 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 88982.5us
I20260709 12:25:20.079648 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:20.053704 13965 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.063441 14277 raft_consensus.cc:2764] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:20.039570 14393 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:25:20.087627  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:20.068333 14254 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:20.051316 14382 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:25:20.092479 14254 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:20.102415 14273 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:20.105587 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:20.092839 10674 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:20.115520  9941 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:20.088207 14324 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:20.120208 10675 raft_consensus.cc:2368] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13.
I20260709 12:25:20.122040  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:20.123740 10094 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
W20260709 12:25:20.123952  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50296.7us
I20260709 12:25:20.125336  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:20.125797  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:20.129799 14387 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:20.130100  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:20.130331 14214 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:20.130791 14214 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:20.131446 10025 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:20.131806 14214 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.133314 14214 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:20.135075  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:20.135077 14470 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:20.136158 14470 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:20.137217 14470 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.138814 14454 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:20.139261 14454 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.139612 14454 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.139853 14350 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:20.140240 10018 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:20.141387 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:20.106956 14254 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.142097 10018 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:20.144295 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:20.147212 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:20.105499 14452 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.083290 14353 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.035s
W20260709 12:25:20.152133 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 69796.2us
I20260709 12:25:20.152882 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:20.157232  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:20.152985 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:20.168877  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:20.170920 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:20.121320 14324 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:20.177691 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:20.171844 10675 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:20.121908 14273 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:20.179054 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:20.185381 10674 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
W20260709 12:25:20.188664 14492 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:25:20.189242  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59223.9us
I20260709 12:25:20.190454  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:20.187984 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:20.196589 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:20.201668  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:20.179045 14324 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:20.207163  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 70724.7us
W20260709 12:25:20.209707  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.211865  9941 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:20.214035 10022 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:20.211081 14353 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:20.216045 14353 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:20.218976 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66074.6us
I20260709 12:25:20.219058 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:20.222808 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:20.198415 14460 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:20.205933 14401 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:20.175032 14466 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.234946 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:20.227100 14472 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:20.238986 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 70745.6us
W20260709 12:25:20.245204  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.245635  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55320.8us
I20260709 12:25:20.245674 10668 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:20.247154 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:20.248200 10022 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:20.250308 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.233711 14393 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.235520 14452 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.247855 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:20.255771 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:20.265889 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.268870 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.277485 10016 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:20.254968 14460 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.284693 14460 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:20.256692 14353 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:20.277541 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:20.289198  9912 leader_election.cc:400] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13.
I20260709 12:25:20.291023  9912 leader_election.cc:403] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:20.263482 14349 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.287855 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:20.307689 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:20.272908 14035 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:20.312995 10025 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:20.314359  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43644.7us
W20260709 12:25:20.316277 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 78726.7us
I20260709 12:25:20.318957  9941 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:20.290446 14452 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:20.321815 14494 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.208s	user 0.002s	sys 0.000s
I20260709 12:25:20.271255 14273 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.331357 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:20.303788 14393 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.332943  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:20.338891 14513 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:20.326324 14353 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:25:20.354677  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:20.366322  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:20.352370 14513 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.376021 14448 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:20.377125 10670 raft_consensus.cc:2410] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:25:20.380770 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1882) took 1407 ms. Trace:
I20260709 12:25:20.358525 14510 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:20.381197 10019 rpcz_store.cc:276] 0709 12:25:18.964339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.393413 (+429074us) service_pool.cc:224] Handling call
0709 12:25:20.370976 (+977563us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b54001c7354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:20.371145 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112426880}
I20260709 12:25:20.387720 14309 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:20.388031 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 70307us
I20260709 12:25:20.390183 14309 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:20.370252 14214 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:20.391175 14214 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:20.391685 14214 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.395661 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:20.380160 14497 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 12:25:20.381297 14513 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.406967 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:20.394603 14466 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:20.415315 10098 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:20.396782 14448 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:20.417105 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:20.377779 14119 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 12:25:20.439034 10025 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:20.440344 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:20.425140 14507 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:25:20.455013  9947 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:20.459246 14297 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13.
I20260709 12:25:20.460266 14466 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:20.460958 14466 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:20.462090 14514 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
W20260709 12:25:20.463446 14521 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:25:20.464195 14466 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:20.464382 14525 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:20.465411  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 51413.9us
I20260709 12:25:20.426182 14327 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:20.465407 14509 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:25:20.469357  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:20.470808 14279 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.472680 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:20.474644  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:20.450086 14511 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:25:20.475682 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:20.478387 14279 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:20.479254 14393 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:20.447338 14519 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:25:20.465188 14525 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:20.483270 14512 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:25:20.485594 14525 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:20.462021 14254 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.490866 10019 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:20.495281 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 686) took 1620 ms. Trace:
I20260709 12:25:20.495659 10099 rpcz_store.cc:276] 0709 12:25:18.874956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.876359 (+  1403us) service_pool.cc:224] Handling call
0709 12:25:20.494979 (+1618620us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54002fc454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:20.495111 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1423331712}
I20260709 12:25:20.494331 14254 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:20.496078 10019 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 12:25:20.501569 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:20.502943 14452 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:20.504922  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:20.502475 10019 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:25:20.474545 14510 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:25:20.508037 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:20.508711 10019 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:20.509316  9942 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:25:20.509518 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:20.510284 10093 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:20.510632 10025 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:20.512056 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3061) took 2139 ms. Trace:
I20260709 12:25:20.513660  9949 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:20.514410  9942 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:20.477486 10098 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:20.512674 10097 rpcz_store.cc:276] 0709 12:25:18.372353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.564205 (+191852us) service_pool.cc:224] Handling call
0709 12:25:18.591583 (+ 27378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.924500 (+332917us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.511528 (+1587028us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b5400161f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:20.511836 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758764928}
W20260709 12:25:20.519618 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 116093us
I20260709 12:25:20.520547 10062 leader_election.cc:400] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 7 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 7. Current term is 8.
W20260709 12:25:20.503050 14503 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.281s	user 0.000s	sys 0.004s
I20260709 12:25:20.525071 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:20.524971 14497 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:25:20.532101 14287 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.544211 14506 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:20.547174 14382 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:20.548683 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1872) took 1590 ms. Trace:
I20260709 12:25:20.550318  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 829) took 1049 ms. Trace:
I20260709 12:25:20.561209  9943 rpcz_store.cc:276] 0709 12:25:19.500868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.501299 (+   431us) service_pool.cc:224] Handling call
0709 12:25:19.501874 (+   575us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.699267 (+197393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.550033 (+850766us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b5400208354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:20.550252 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":436595584}
I20260709 12:25:20.562080 14429 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:20.563180 14429 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:20.563521 14429 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:20.533303 10016 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:20.553125  9942 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:20.554409 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:20.560487 10021 rpcz_store.cc:276] 0709 12:25:18.957891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.958089 (+   198us) service_pool.cc:224] Handling call
0709 12:25:19.432337 (+474248us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:19.434495 (+  2158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.548415 (+1113920us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54001f5c50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:20.548590 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1062038400}
I20260709 12:25:20.568022 14506 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.584308  9943 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:20.588085  9942 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
W20260709 12:25:20.593436 14502 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.374s	user 0.001s	sys 0.000s
I20260709 12:25:20.595201 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:20.560293 14382 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:20.613384 10016 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:20.628764 10099 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:20.568637 13965 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.582048 14429 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:20.637279 14416 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.238s	user 0.014s	sys 0.011s
W20260709 12:25:20.637715 14416 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.240s	user 0.014s	sys 0.011s
I20260709 12:25:20.638190 14429 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.638355 10023 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:20.638494 14470 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:20.639514 14111 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:20.639883 14111 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:20.640722 14111 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.641906 14111 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.632314 14287 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.649542 10099 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:20.651755 14353 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:25:20.652431 14454 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.003s
I20260709 12:25:20.652863 14454 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.655005 14382 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:20.655561  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.659559  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.634583 14506 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:20.673333 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 108811us
I20260709 12:25:20.676457 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:20.677042 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.667382 14384 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:20.674674 14382 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.679216 14214 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.688391 14214 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:20.691322 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:20.692451 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:20.692723 10094 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.688144  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:20.693007 10092 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.684496 14409 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:20.694746 14409 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.696040 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 908) took 1556 ms. Trace:
I20260709 12:25:20.696404 10672 rpcz_store.cc:276] 0709 12:25:19.140283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.140552 (+   269us) service_pool.cc:224] Handling call
0709 12:25:19.155576 (+ 15024us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.175489 (+ 19913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.695577 (+1520088us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b5400501ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:20.695833 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42427136}
I20260709 12:25:20.693091 14384 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.701857 14384 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:20.702198 10094 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:20.709102 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:20.691679 14454 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:20.706714 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:20.723757 14273 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:20.692984 14324 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.742120 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 907) took 2141 ms. Trace:
I20260709 12:25:20.742775 10673 rpcz_store.cc:276] 0709 12:25:18.601228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.601795 (+   567us) service_pool.cc:224] Handling call
0709 12:25:18.602236 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:19.587920 (+985684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.741662 (+1153742us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b54006ff754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:20.742017 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2264491904}
I20260709 12:25:20.750998 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:20.755084 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:20.755910 10670 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:25:20.757536 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:20.759760 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.722054 14513 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.763129 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:20.765869 10673 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:20.762105 10674 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
W20260709 12:25:20.791904 14500 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.732s	user 0.003s	sys 0.001s
W20260709 12:25:20.806123 14532 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:25:20.817622 10670 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:25:20.830667 10670 pending_rounds.cc:85] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 8
I20260709 12:25:20.818095 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 838) took 3149 ms. Trace:
I20260709 12:25:20.777956 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:20.817610  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 1996) took 1213 ms. Trace:
I20260709 12:25:20.820442 13965 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:20.777019 13996 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:25:20.822885 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3068) took 1397 ms. Trace:
I20260709 12:25:20.823602 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1871) took 1866 ms. Trace:
I20260709 12:25:20.832497  9940 rpcz_store.cc:276] 0709 12:25:19.606229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.658569 (+ 52340us) service_pool.cc:224] Handling call
0709 12:25:20.776319 (+1117750us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54001b9254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:20.817443 (+ 41124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719408000}
I20260709 12:25:20.831315 10671 rpcz_store.cc:276] 0709 12:25:17.669073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.669275 (+   202us) service_pool.cc:224] Handling call
0709 12:25:19.942892 (+2273617us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.943687 (+   795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.782518 (+838831us) spinlock_profiling.cc:243] Waited 2.27 s on lock 0x7b54005b95d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:20.818035 (+ 35517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5227868032}
I20260709 12:25:20.835683 10025 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:20.837757 10673 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:20.835487 10091 rpcz_store.cc:276] 0709 12:25:19.425965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.426231 (+   266us) service_pool.cc:224] Handling call
0709 12:25:19.426740 (+   509us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.940745 (+514005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.784661 (+843916us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b5400093154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:20.822784 (+ 38123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124359424}
I20260709 12:25:20.830977 10670 pending_rounds.cc:107] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:25:20.843439 10670 raft_consensus.cc:2889] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:20.835832 10024 rpcz_store.cc:276] 0709 12:25:18.957581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:18.957780 (+   199us) service_pool.cc:224] Handling call
0709 12:25:19.296408 (+338628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:19.337902 (+ 41494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.787451 (+1449549us) spinlock_profiling.cc:243] Waited 338 ms on lock 0x7b54001388d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:20.823529 (+ 36078us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":777718016}
I20260709 12:25:20.849216 14403 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:20.849517 14407 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:20.849862 14403 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:20.850417 14407 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.850950 14403 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.851663 14409 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.853406 14407 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:20.859022 13996 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:20.859514 13996 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:20.860725 13996 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.862660 13996 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:20.864305 14456 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:20.865063 14456 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:20.873549 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 2974) took 3507 ms. Trace:
I20260709 12:25:20.876178  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:20.877493  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1751) took 1085 ms. Trace:
I20260709 12:25:20.879913 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 918) took 1029 ms. Trace:
I20260709 12:25:20.880470 10668 rpcz_store.cc:276] 0709 12:25:19.850616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.871220 (+ 20604us) service_pool.cc:224] Handling call
0709 12:25:19.882882 (+ 11662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.249991 (+367109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.879535 (+629544us) spinlock_profiling.cc:243] Waited 360 ms on lock 0x7b5400611a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:20.879809 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":827751168}
I20260709 12:25:20.874186 10017 rpcz_store.cc:276] 0709 12:25:17.365831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:17.521064 (+155233us) service_pool.cc:224] Handling call
0709 12:25:17.538212 (+ 17148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:18.546145 (+1007933us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.872904 (+2326759us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540038d6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:20.873308 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2316504832}
I20260709 12:25:20.878019  9946 rpcz_store.cc:276] 0709 12:25:19.793838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.808270 (+ 14432us) service_pool.cc:224] Handling call
0709 12:25:20.875907 (+1067637us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:20.877402 (+  1495us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75581184}
I20260709 12:25:20.883334 14356 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:20.884680 14356 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:20.885133 14356 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:20.887391 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 122900us
I20260709 12:25:20.888259 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:20.889566 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:20.848379 14324 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:20.890832 14273 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:20.891342 14324 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:20.891551 14273 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:20.892846 14273 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:20.888070 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1885) took 1922 ms. Trace:
I20260709 12:25:20.907716 10023 rpcz_store.cc:276] 0709 12:25:18.965205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.688396 (+723191us) service_pool.cc:224] Handling call
0709 12:25:19.688914 (+   518us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.638901 (+949987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.887730 (+248829us) spinlock_profiling.cc:243] Waited 949 ms on lock 0x7b54001392d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:20.887981 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2183103744}
I20260709 12:25:20.906096 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:20.910780 14309 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:20.913446 14409 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:20.915328 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:20.919189  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:20.942576 10092 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:20.944942 14401 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:20.949965 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44164.6us
I20260709 12:25:20.953744 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:20.956228 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:20.886341 14356 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:20.963884 14356 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:20.965400 14431 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.527s	user 0.019s	sys 0.027s
W20260709 12:25:20.965842 14431 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.561s	user 0.019s	sys 0.027s
I20260709 12:25:20.936985 14309 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:25:20.961014 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:20.977687 14236 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:20.978201 14236 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:20.979528 14530 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:25:20.981084 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:20.973898 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:20.989382  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.993335 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:20.994300  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:20.991089 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:20.950174 14362 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:25:21.002153 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:21.003975 10091 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:21.008635 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47730.3us
I20260709 12:25:21.010952 10099 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
W20260709 12:25:21.002547 14534 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:25:20.994757 14401 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.016940  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:20.973606 14409 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:25:20.972733 14324 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 4, Time since last communication: 0.040s
W20260709 12:25:21.022671 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48872.8us
I20260709 12:25:21.006824 14035 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:21.007157 14254 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:21.026901 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:20.986435 14287 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:25:21.027896 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:21.028187 14254 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:21.030903 14513 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:21.032279 14324 consensus_queue.cc:1243] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:25:21.038385 14409 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:21.046815  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.039458 14272 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:21.054329 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.051506 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:21.052727 14409 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:21.069550 14402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.838s	user 0.014s	sys 0.009s
I20260709 12:25:21.056748 14254 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:21.127022 14402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.952s	user 0.014s	sys 0.010s
I20260709 12:25:21.127607 14254 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:21.120097 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 65908.1us
I20260709 12:25:21.128520 14409 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.132032 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.136953 14324 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:21.137924 14324 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:21.138371 14324 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:21.142616 14523 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.659s	user 0.006s	sys 0.000s
I20260709 12:25:21.119870 14456 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.056300 14471 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:21.055459 14273 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:21.121079 14295 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.120893 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:21.122141 14546 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.069s	user 0.003s	sys 0.000s
I20260709 12:25:21.123294 14410 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:25:21.124202 14543 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.083s	user 0.004s	sys 0.000s
W20260709 12:25:21.124917 14544 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.005s
I20260709 12:25:21.046895 14262 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.049832 14236 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.203439 14236 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.144198 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 925) took 1124 ms. Trace:
I20260709 12:25:21.206820 10677 rpcz_store.cc:276] 0709 12:25:20.020101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.020268 (+   167us) service_pool.cc:224] Handling call
0709 12:25:20.052234 (+ 31966us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:20.052472 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:20.052497 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:20.082525 (+ 30028us) log.cc:844] Serialized 26 byte log entry
0709 12:25:20.364500 (+281975us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:20.364562 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:20.364682 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.093785 (+729103us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:21.143885 (+ 50100us) raft_consensus.cc:1685] finished
0709 12:25:21.143909 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.144122 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77161728,"thread_start_us":210601,"threads_started":1,"wal-append.queue_time_us":338324}
I20260709 12:25:21.207937 14361 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:21.144798 14471 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:21.193053 10020 raft_consensus.cc:2410] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:25:21.212196 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 1884) took 2247 ms (client timeout 1782 ms). Trace:
W20260709 12:25:21.199694 14542 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.234s	user 0.005s	sys 0.000s
I20260709 12:25:21.201133  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1759) took 1332 ms. Trace:
I20260709 12:25:21.182919 14273 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:21.213346 14273 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:21.213150 10020 rpcz_store.cc:269] 0709 12:25:18.964916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:19.695218 (+730302us) service_pool.cc:224] Handling call
0709 12:25:21.047271 (+1352053us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:21.212059 (+164788us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15549184}
W20260709 12:25:21.202005 14549 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.076s	user 0.002s	sys 0.002s
W20260709 12:25:21.194754 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.212384 14471 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.184741 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:21.216327  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 366989us
W20260709 12:25:21.217456  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 97272.3us
I20260709 12:25:21.228809 10091 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
W20260709 12:25:21.230219 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 106973us
I20260709 12:25:21.213018  9947 rpcz_store.cc:276] 0709 12:25:19.868860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.469035 (+600175us) service_pool.cc:224] Handling call
0709 12:25:20.531704 (+ 62669us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:20.564635 (+ 32931us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:20.611838 (+ 47203us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.832675 (+220837us) log.cc:844] Serialized 14 byte log entry
0709 12:25:20.977226 (+144551us) spinlock_profiling.cc:243] Waited 56.4 ms on lock 0x7b1800355720. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:21.146269 (+169043us) spinlock_profiling.cc:243] Waited 68.7 ms on lock 0x7b1800355720. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:21.146465 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:21.183397 (+ 36932us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:21.186699 (+  3302us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:21.200857 (+ 14158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.200948 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:21.201074 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":382913,"prepare.queue_time_us":35830,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":2085,"raft.queue_time_us":70832,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":155,"spinlock_wait_cycles":344391168,"thread_start_us":114838,"threads_started":1,"wal-append.queue_time_us":213597}
W20260709 12:25:21.232553  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 313472us
I20260709 12:25:21.235782 10099 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:21.236397  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.233099 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:21.239498  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45973.6us
I20260709 12:25:21.234910 14361 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:21.243772 10670 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:21.232589 14513 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.247480  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.247999  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.237715 14470 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:21.246831  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.254155  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:21.255905  9945 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:21.251946 14541 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.255s	user 0.002s	sys 0.000s
I20260709 12:25:21.257508 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:21.257910  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:21.224663 14429 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:25:21.260095 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:21.261003 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:21.261518 10095 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:21.262729 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62632.6us
I20260709 12:25:21.264635 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:21.263490  9942 raft_consensus.cc:1217] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-12.9]   Dedup: 5.6->[8.7-12.9]
I20260709 12:25:21.265486  9942 pending_rounds.cc:85] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 6
I20260709 12:25:21.265738  9942 pending_rounds.cc:107] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 12:25:21.265981  9942 raft_consensus.cc:2889] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.245131 14447 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:21.267412 10094 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:21.243451 14471 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.270869 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:21.263012 14356 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:25:21.272215  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:21.280458  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:21.252012 13756 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:21.273536  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:21.268792 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:21.271752 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.279063  9947 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:21.279201  9940 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:21.251686 14361 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:21.283502  9946 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:21.285168 10092 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
W20260709 12:25:21.285465  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49157us
I20260709 12:25:21.292253  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:21.285730  9943 raft_consensus.cc:2368] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12.
I20260709 12:25:21.252343 14111 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:21.294078  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 75100.3us
I20260709 12:25:21.295723 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:21.301191 10018 raft_consensus.cc:2368] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12.
I20260709 12:25:21.303659 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:21.304383 10023 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:21.309248 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:21.310175 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:21.282752 14355 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:21.317533 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:21.318671  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:21.319825  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:21.320343  9948 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.263304 14466 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:21.322736 10017 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:25:21.326696  9945 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:21.293511 14353 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:21.331678 10065 leader_election.cc:400] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12.
I20260709 12:25:21.337870 10065 leader_election.cc:403] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:25:21.273202 14463 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:25:21.303786 14279 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:21.272812 14362 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:25:21.274202 14513 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.340052 14482 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.290859 14447 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:21.284442 13756 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:21.335623 10670 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:25:21.344100 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:21.347654 10670 pending_rounds.cc:85] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 9
I20260709 12:25:21.347936 10670 pending_rounds.cc:107] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:21.348263 10670 raft_consensus.cc:2889] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.344100  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:21.352449  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60306.2us
I20260709 12:25:21.353259  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:21.354677  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.355238 14447 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.356164 14361 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.358769 14382 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:21.359287  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:21.359000 13756 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.359892 14470 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:21.311551 14432 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.354444  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:21.359506 14382 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.364006  9948 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:21.322265 14272 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:21.322391 14355 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:21.360384 14384 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:21.381090 14539 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.416s	user 0.003s	sys 0.001s
I20260709 12:25:21.381022 14384 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.382979 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:21.383574 10668 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:21.371770 14309 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:25:21.383816 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:21.344818 14349 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.386168 14349 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.387895 14362 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:21.388329 14362 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:21.389242 14362 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.390596 14513 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:21.391141 14466 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:21.374684 14432 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.391575 14466 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:21.392627 14466 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.393710 14432 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:21.393946 14466 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.372534 14403 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12.
I20260709 12:25:21.395608 14470 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.396406  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:21.397022  9946 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:21.397696 14470 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:21.375038  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45404.6us
I20260709 12:25:21.399627 10677 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:21.390590 14362 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:21.402770 14402 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:21.403323 14402 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.407711 14510 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:21.408493 14510 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.409077 14456 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:21.408957 14510 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.383482 14384 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:21.418980  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 179342us
I20260709 12:25:21.421166  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.421368 14429 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:21.423753 14407 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:21.424224 14407 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:21.424602 14407 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.364117 14279 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:21.427479 14279 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.428678 13756 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:21.432264 10020 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:21.388883 14349 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:21.441581 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:21.450992  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.422519  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:21.455037  9949 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:21.457630 10099 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:21.458786  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:21.451476 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:21.464547  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:21.465798 14035 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:21.466238 14035 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:21.467072 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.466279  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:21.476179 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:21.484562 10668 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:21.487115 10668 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:25:21.342331 14287 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.109s
I20260709 12:25:21.434392 14355 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:21.366892 14111 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.421155 14236 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:25:21.515975 10023 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:21.482875 14356 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.516708 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:21.371333 14482 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.521637 10670 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:21.517926  9949 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:25:21.522374  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 167989us
I20260709 12:25:21.498719 13996 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:21.502103 14474 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:21.536798 14480 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:21.537214 14480 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.537559 14480 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.538312 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:21.538514 14273 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.540987 14273 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:21.531024 14035 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.542354 14035 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:21.502014 14470 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 12:25:21.545553 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61430us
I20260709 12:25:21.475509 14324 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.550776 14324 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:21.553213  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.560128 13996 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:21.561993 13996 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:21.562669 14035 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:21.563110 14035 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:21.483150 14431 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:25:21.534111 14482 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:21.534723 10675 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:21.534811 14557 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.535408 14510 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:21.535657 14236 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:25:21.572773  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44807.8us
I20260709 12:25:21.503188 14463 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:25:21.515376 14384 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.523718 14402 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:21.527449 14353 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:21.612215  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:21.612489 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 343734us
I20260709 12:25:21.572597 13996 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:21.618198  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:21.621697 10090 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:21.626457 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:21.626945  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:21.564000 14035 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.585494 14236 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:21.633086 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:21.636747 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:21.639132 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:21.641696 10095 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:21.630605  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:21.630244 14356 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.647751 14035 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.650274 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:21.652763 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:21.653534 10099 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:21.658242 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:21.659054 10062 leader_election.cc:400] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12.
I20260709 12:25:21.659794 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:21.628572 14402 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:21.585821 14557 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:21.669682  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 168278us
I20260709 12:25:21.672566 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:21.675393  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44891.8us
I20260709 12:25:21.646529 14510 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:21.617262 14384 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.688122  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:21.656200 14327 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.648824 14463 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.681813  9949 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:21.651908 14447 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:21.692543  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:21.693130  9940 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:21.660921 14324 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.656083 14556 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:21.677948 14456 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.679349 14349 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:21.704413 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 372900us
I20260709 12:25:21.721568 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:21.720947 10676 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:25:21.725558 10676 pending_rounds.cc:85] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 8
I20260709 12:25:21.725823 10676 pending_rounds.cc:107] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:25:21.725646 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:21.726251 10676 raft_consensus.cc:2889] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:21.725790  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:21.689719 14295 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.729418 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:21.745664  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:21.669101 14273 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:25:21.701874 14447 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:21.728394 14410 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 12:25:21.752475  9943 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:21.720943 14409 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:21.748301 14471 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.753509  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:21.768963 10019 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:21.750276 14401 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:21.708050 14327 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.756083 14447 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:21.773900 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66414.8us
I20260709 12:25:21.775198  9949 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:21.756851 14463 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:21.720606 14272 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:25:21.705240 14402 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.702729 14353 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.790501 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:21.707733 14236 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.749284 14295 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.780583 14436 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:21.867069  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 178523us
W20260709 12:25:21.858479  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 165996us
I20260709 12:25:21.850260  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:21.782864 14111 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:21.777899 14517 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:21.873710  9943 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:21.749720 13996 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.889005  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:21.889976  9949 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:21.841703 14401 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:21.843829 14410 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:21.911530  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 196748us
I20260709 12:25:21.913022 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:21.916124  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.914959  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:21.870658 14466 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:21.921310  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 195574us
I20260709 12:25:21.863379 14296 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.251s
I20260709 12:25:21.919305  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:21.893774 14556 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:21.935542  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:21.937587 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 154627us
I20260709 12:25:21.940892 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:21.946472 10094 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:21.947937  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.950376 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:21.934476 10675 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:25:21.924645  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.922400 14560 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:21.954036  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:21.922489  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:21.958158  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:21.959393  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.959498  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:21.906210 14517 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.947225 14432 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:21.962534  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.963565 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:21.963872 14556 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:21.965694 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3009) took 1469 ms. Trace:
I20260709 12:25:21.966068 10019 rpcz_store.cc:276] 0709 12:25:20.496059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:20.509302 (+ 13243us) service_pool.cc:224] Handling call
0709 12:25:21.965540 (+1456238us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002d5d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:21.965637 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2894691072}
I20260709 12:25:21.967639 14297 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:21.968066 14297 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.968395 14297 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:21.971973 14576 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.972133 14510 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:21.972669 14510 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.973296 14480 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.975167 14557 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:21.975747 14402 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:21.976018 14432 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:21.976444 14432 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:21.977147  9948 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:21.979592 14295 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:21.980921 14436 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:21.980719 14296 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:21.979730  9943 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:21.983431 14327 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:21.985064 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:21.988886 14356 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:21.975749 14557 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:21.980723  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.991039 14557 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:21.992015  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:21.992830 14436 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:21.993479 14436 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:25:21.997936 14407 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.000s
W20260709 12:25:21.999027 14562 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.231s	user 0.001s	sys 0.000s
I20260709 12:25:21.999313 14385 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:21.999859 14385 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:21.998296  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:22.000854  9947 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:21.959913  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:21.950984 14579 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:21.947417 14463 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:25:22.016867 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:21.999353 14557 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.029196 14272 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:25:22.030722 14563 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:25:22.026769 14254 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:22.031595 14480 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:22.032717 14432 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.035148 14432 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.036139 14327 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:22.036617 14327 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:22.036990 14327 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:22.040313 10019 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:22.016623 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:22.042472 14254 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:22.043586 14254 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.045596 14384 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.015482  9948 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:22.030757 14297 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:21.955541 14111 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 12:25:22.053375 14385 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.054064 14297 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:21.998404 14407 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.056320 14407 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.005051 14560 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:22.016688 14558 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
I20260709 12:25:22.000510 14574 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:22.060295 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:22.064688 10095 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:22.012706 14454 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:22.072717 10023 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:22.073863 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:22.074548 10017 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:22.074851 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:22.074958 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:22.075448 10016 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:22.075556 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:22.076287 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:22.076709  9949 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:22.076843 10022 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:22.078016  9949 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
W20260709 12:25:22.078665 14559 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.315s	user 0.003s	sys 0.000s
I20260709 12:25:22.078953 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:22.075800 10024 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 12:25:22.084053 10016 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:22.084831 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:22.087256 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:22.074224 14409 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:22.098224 10025 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:22.049077 14356 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:22.012670  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52854.6us
I20260709 12:25:22.107055 10095 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:22.110343  9941 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 13 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:22.017362 14482 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 12:25:22.125934 10022 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:22.125914  9949 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:21.990352 14355 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.012861 14401 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.117712 14356 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:22.124536 14361 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.011s	sys 0.006s
W20260709 12:25:22.145763  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 182693us
W20260709 12:25:22.141546 14382 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.769s	user 0.013s	sys 0.000s
I20260709 12:25:22.205338 10025 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:22.198231 10021 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:22.027948 14579 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:22.166169 14361 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.012s	sys 0.007s
I20260709 12:25:22.151482 14356 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.197757 14409 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:22.212972 14558 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:22.149055 14510 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.140172 14575 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:25:22.236963 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:22.240837 10016 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:22.214313 14382 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:22.241322 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:22.212373 14407 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:22.242187 10668 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:22.242622 14407 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.245518 14407 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.246384  9943 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:22.249265 14575 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:22.249908 14575 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:22.251106 14575 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.252439 14118 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.120s
I20260709 12:25:22.235666  9946 pending_rounds.cc:85] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 7
I20260709 12:25:22.253536  9946 pending_rounds.cc:107] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:25:22.253768 13996 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:22.253919  9946 raft_consensus.cc:2889] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.254333 13996 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:22.256186  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.256650 10025 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:22.228582 14557 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.268594 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:22.269644 10016 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:22.276643 10668 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:22.276754 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:22.277982  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:22.228334 14572 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:22.216758 14447 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.255388 13996 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.228528 14454 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:25:22.299081 14574 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.270553 14409 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.310207 14573 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:22.274571 14557 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.317516 14575 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.299386 14382 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:22.427244  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 135628us
I20260709 12:25:22.222890 14560 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:22.436012 14592 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 12:25:22.433899 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.441915 14432 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:22.443042 10643 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:22.434818 14482 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:22.446022 14482 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.447633 14355 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.448598 14482 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:22.435369 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.449829 14582 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:22.450665 14582 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.452103  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1778) took 1200 ms. Trace:
I20260709 12:25:22.452986  9945 raft_consensus.cc:1217] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-11.9]   Dedup: 10.8->[11.9-11.9]
I20260709 12:25:22.453559  9945 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: [2.5-11.9]
W20260709 12:25:22.435765 14584 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:25:22.446285 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:22.456504 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:22.469779 10016 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:22.413674 14353 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:22.497860 10021 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:25:22.503584  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.505302  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.452479  9942 rpcz_store.cc:276] 0709 12:25:21.251490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.251603 (+   113us) service_pool.cc:224] Handling call
0709 12:25:21.263247 (+ 11644us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 12:25:21.344467 (+ 81220us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:21.407009 (+ 62542us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.411282 (+  4273us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b48002a2f58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:21.425685 (+ 14403us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b18003551e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:21.505736 (+ 80051us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.505892 (+   156us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:21.511505 (+  5613us) log.cc:844] Serialized 78 byte log entry
0709 12:25:21.520990 (+  9485us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:21.537211 (+ 16221us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.537584 (+   373us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.537742 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.537835 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:21.967370 (+429535us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b18003551e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:22.451869 (+484499us) raft_consensus.cc:1685] finished
0709 12:25:22.451891 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.452047 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":96713,"prepare.queue_time_us":90114,"prepare.run_cpu_time_us":156,"prepare.run_wall_time_us":4536,"raft.queue_time_us":128530,"raft.run_cpu_time_us":2173,"raft.run_wall_time_us":56763,"spinlock_wait_cycles":83501696,"thread_start_us":66784,"threads_started":1,"wal-append.queue_time_us":222144}
I20260709 12:25:22.512965 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:22.486119 14384 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:22.538003  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:22.538930  9942 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:22.315560 14572 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.442452 14432 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.494843 14353 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:22.533018 14573 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:22.496460 14355 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.496817 14470 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.559120 10016 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:25:22.530979 14591 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.243s	user 0.000s	sys 0.001s
I20260709 12:25:22.560109 14578 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.557575 14384 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:22.613435 14600 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:25:22.473309 14382 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.502671 14361 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:22.503921 14287 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:22.502357 13996 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:22.534852 14588 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.542409 14429 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:22.605275  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.620440  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:22.563334 14254 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:22.568768 14596 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
I20260709 12:25:22.422317 14574 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.603560 14355 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.632409 14361 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:22.620375 14470 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:22.646792 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 113155us
I20260709 12:25:22.671442 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:22.672106 10672 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:22.679154 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:22.647099 14429 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.646404 14254 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.710990 14111 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.648644 14447 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.633610 14353 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:22.676513 14583 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 12:25:22.722222 10673 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:22.639045 14588 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.711691 14111 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.733523 14595 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.734117 14595 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.735404 14111 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.638636 14578 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.736315 14595 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.667007 14573 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.743813 14254 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.744630 14254 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.745306 14254 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.730237  9942 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:22.731487 14432 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.732833 14588 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.717267 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:22.759364 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:22.738366 14578 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:22.721957 14327 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.685s	user 0.013s	sys 0.000s
I20260709 12:25:22.755458 10672 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:22.744397 14429 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.711226 14384 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.780918 10668 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:25:22.758728 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 270252us
I20260709 12:25:22.788826 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:22.759542 14604 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:25:22.751490 14582 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.756860 14353 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:22.730190 14557 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.797813 14574 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.799350 14382 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.804392 14517 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:22.808763 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.811121 14596 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:22.824600 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:22.825651 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:22.785674 14327 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.820721 14565 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.830546 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:22.837293 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:22.844849 14214 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:22.845443 14214 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:22.845657 14254 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:22.828212 14615 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:22.846729 14615 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:22.847128 14615 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:22.849932 14356 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.611s	user 0.009s	sys 0.000s
I20260709 12:25:22.841444 14613 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:22.850869 14355 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:22.839593 14605 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:25:22.832293 14482 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:22.854393  9942 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:22.839330 14575 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.857616 14612 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:22.858552 14355 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.862824 14463 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:22.871889 14612 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:25:22.863651 14608 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:25:22.872318 14612 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:22.855490 14214 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:22.865037 14601 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:25:22.866674  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:22.877957  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.866887  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:22.851728 14447 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:22.856271 14607 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:25:22.879904  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:22.853737 14254 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.815507 14361 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.850415 14356 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.886951 14254 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.887429  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:22.874432 14612 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:22.900489 14610 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:25:22.854936 14613 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:25:22.902057  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1781) took 1170 ms. Trace:
W20260709 12:25:22.902956 14590 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.677s	user 0.001s	sys 0.002s
I20260709 12:25:22.896396 14572 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.888500 14617 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:22.882356 14557 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.875911 14355 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.895442 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:22.902463  9947 rpcz_store.cc:276] 0709 12:25:21.731678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.002757 (+271079us) service_pool.cc:224] Handling call
0709 12:25:22.003047 (+   290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:22.059405 (+ 56358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.059437 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:22.059889 (+   452us) log.cc:844] Serialized 26 byte log entry
0709 12:25:22.413697 (+353808us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:22.413757 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.413867 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.901794 (+487927us) raft_consensus.cc:1685] finished
0709 12:25:22.901827 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:22.901995 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11984,"raft.queue_time_us":53876,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":276114,"threads_started":1,"wal-append.queue_time_us":566500}
I20260709 12:25:22.931842  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:22.932508  9947 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:22.908031 14615 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.934883 14613 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:22.935238 14551 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.452s	user 0.005s	sys 0.002s
I20260709 12:25:22.844034 14565 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.935353 14613 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:22.936000  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:22.937283 14621 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:22.939350 14327 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:22.942941 14254 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.943342 14612 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:22.895174 14214 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:22.908663 14356 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.895187 14447 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:22.887897 14432 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:25:22.888345 14582 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:22.895550 14470 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.875754 14575 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.953261 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:22.907903 14510 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.907768 14557 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:22.955382  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:22.931430 14380 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:22.957173 14621 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:22.958737 14297 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:22.990795 14297 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:22.990975 14384 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:22.991293 14621 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.991925 10093 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:22.966073 10022 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:25:22.992403 10022 pending_rounds.cc:85] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 8
I20260709 12:25:22.991024 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:22.992640 10022 pending_rounds.cc:107] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:25:22.992925 10022 raft_consensus.cc:2889] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:22.993018 10099 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:22.953022 14355 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.956362 14463 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:22.994540 10097 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:22.994619 14463 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.995734 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:22.996338 10096 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:22.996906 14463 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:22.956177  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:22.973035 14615 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:22.973713 14557 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:22.974462  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:22.975109 14565 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:22.976780  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:22.981884 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:23.000100 10016 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:22.959950 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 883) took 1579 ms. Trace:
I20260709 12:25:22.983337 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:22.984445 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:22.984802 14613 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.002817 10091 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:22.985636 14575 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:22.970723 14582 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.991642 14384 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:22.997599  9948 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:22.999439 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:22.999365 14615 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:25:22.971742 14380 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:23.000773 10676 rpcz_store.cc:276] 0709 12:25:21.380289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.382265 (+  1976us) service_pool.cc:224] Handling call
0709 12:25:21.463989 (+ 81724us) spinlock_profiling.cc:243] Waited 987 us on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:25:21.464207 (+   218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:21.726806 (+262599us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.959677 (+1232871us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b540073afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:22.959885 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591525120}
I20260709 12:25:22.997385 14447 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.004375  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:22.991796 14297 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.003299 14557 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.049103 14480 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.063979 14480 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.065650 14579 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.066318 14480 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:23.008306  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2014) took 1334 ms. Trace:
I20260709 12:25:23.067402 14236 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.067902 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.068509 10025 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
W20260709 12:25:23.052240 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:23.069550 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.051398 13756 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.071094 13756 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.046509  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 845) took 1760 ms. Trace:
I20260709 12:25:23.073004 14470 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:23.047456 14614 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.073773 14470 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.074779 14510 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:23.075325 14510 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.076238 14470 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.076660  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.072856 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:23.077267 14510 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:23.077943  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.078667 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.074497 14614 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.079258 10019 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:23.031284  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:23.079816 14431 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:23.080072  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48915.1us
I20260709 12:25:23.080201 14431 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:23.080690 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.081059 14431 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:23.081676  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.082093 14431 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.082518 14349 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:23.007164 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 885) took 1626 ms. Trace:
I20260709 12:25:23.083071 14349 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.029102 14327 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.047765 14401 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.009450 14613 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.083972 14327 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.084147 14401 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.085263 14385 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.086202 14327 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.083688 14349 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:23.028072  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.047197  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2015) took 1373 ms. Trace:
I20260709 12:25:23.088277 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:23.088668  9943 rpcz_store.cc:276] 0709 12:25:21.673754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.982129 (+308375us) service_pool.cc:224] Handling call
0709 12:25:21.982449 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.246975 (+264526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.046957 (+799982us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54001b9754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:23.047144 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606430080}
I20260709 12:25:23.073540  9945 rpcz_store.cc:276] 0709 12:25:21.286344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.459755 (+173411us) service_pool.cc:224] Handling call
0709 12:25:22.452792 (+993037us) raft_consensus.cc:1471] Updating replica for 5 ops
0709 12:25:22.453911 (+  1119us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.046286 (+592375us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b540018a1d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:23.046442 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185570176}
I20260709 12:25:23.076776 14470 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.091413 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:23.101467 10095 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.092695 10024 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:23.022256  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.007790  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1783) took 1265 ms. Trace:
I20260709 12:25:23.007206 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 936) took 1515 ms. Trace:
I20260709 12:25:23.063876 14557 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.083619 14432 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:23.096197 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:23.096518 10023 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:23.110103 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.096988 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:23.097530 14602 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.429s	user 0.001s	sys 0.001s
I20260709 12:25:23.127689 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.099202 10019 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:25:23.100888 10016 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:23.128450 10022 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:23.083386 10675 rpcz_store.cc:276] 0709 12:25:21.380909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.382286 (+  1377us) service_pool.cc:224] Handling call
0709 12:25:21.382629 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:21.952190 (+569561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.006870 (+1054680us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54005b95d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:23.007092 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349371776}
I20260709 12:25:23.056437 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:23.104266 10677 rpcz_store.cc:276] 0709 12:25:21.491929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.492103 (+   174us) service_pool.cc:224] Handling call
0709 12:25:21.500841 (+  8738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:21.501065 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:21.511950 (+ 10885us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.512333 (+   383us) log.cc:844] Serialized 14 byte log entry
0709 12:25:21.995507 (+483174us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b180044fda0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1
0709 12:25:21.996876 (+  1369us) spinlock_profiling.cc:243] Waited 18 ms on lock 0x7b180044fda0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:23.006634 (+1009758us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b180044fda0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0910cd70 00007f4e09100293 00007f4e090f5cea 00007f4e090f623a 00007f4e09560a2a 00007f4e095700bf 00007f4e09570038
0709 12:25:23.006858 (+   224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:23.006911 (+    53us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:23.006967 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.007038 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.007152 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11058,"prepare.queue_time_us":499221,"prepare.run_cpu_time_us":172,"prepare.run_wall_time_us":172,"raft.queue_time_us":6156,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":9890,"spinlock_wait_cycles":84533248}
I20260709 12:25:23.093899  9949 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:23.103636  9941 rpcz_store.cc:276] 0709 12:25:21.742210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.032297 (+290087us) service_pool.cc:224] Handling call
0709 12:25:22.032621 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:22.133589 (+100968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.007550 (+873961us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b540003fa54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:23.007723 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178262912}
I20260709 12:25:23.025489 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:23.113145 10095 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:23.008563 13965 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:23.127671  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:23.130484  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.130905  9945 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.000746 14214 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:23.131559  9940 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:23.066906  9946 rpcz_store.cc:276] 0709 12:25:21.673536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.979151 (+305615us) service_pool.cc:224] Handling call
0709 12:25:21.979491 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:22.254920 (+275429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.254967 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:22.255392 (+   425us) log.cc:844] Serialized 26 byte log entry
0709 12:25:22.317079 (+ 61687us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:22.317226 (+   147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.007851 (+690625us) spinlock_profiling.cc:243] Waited 256 ms on lock 0x7b54001b9c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:23.007913 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.007974 (+    61us) raft_consensus.cc:1685] finished
0709 12:25:23.007993 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.008238 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":588714368,"thread_start_us":61118,"threads_started":1,"wal-append.queue_time_us":259329}
W20260709 12:25:23.115024 14594 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.631s	user 0.000s	sys 0.004s
I20260709 12:25:23.047645 14287 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:23.156769  9941 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.150036  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.049968 14361 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.159536  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1791) took 1111 ms. Trace:
I20260709 12:25:23.160908 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.161046 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:23.157624 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:23.161612  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45808.4us
I20260709 12:25:23.161628 10672 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:23.080703 14582 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.159982  9941 rpcz_store.cc:276] 0709 12:25:22.048042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.156260 (+1108218us) service_pool.cc:224] Handling call
0709 12:25:23.156582 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.159273 (+  2691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.159483 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:23.100720 14327 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:23.073088 13756 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:23.066354 14384 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:23.107122 14295 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:23.165925 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 109574us
I20260709 12:25:23.165877  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.167171  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.179261  9949 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:23.163841 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:23.167757  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.168592 10672 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:23.169143 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.179596  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:23.114473 14606 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.438s	user 0.006s	sys 0.002s
I20260709 12:25:23.128414 14432 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.169312  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.173323 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.173532 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.045293 14560 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.119s
I20260709 12:25:23.192662 10024 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:23.183801  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.184000 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.193950  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:23.194458 10672 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:23.194835 10023 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:23.195348 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:23.191493  9945 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:23.192652 14350 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.196436 14350 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.198203  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.198882  9946 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:23.026836  9947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:23.199741  9947 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 13 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:23.201102  9946 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:23.201182 14614 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.201586 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.201781 14614 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:23.202199 10673 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:23.202987 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.206024  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.206902  9941 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:23.205103  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.205677 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.208606  9941 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:23.198902 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:23.210906 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:23.202265  9947 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:25:23.206720 10019 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:25:23.214260 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 188855us
W20260709 12:25:23.211905  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:23.215723  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:23.217015 10097 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.170411 14620 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:23.218436 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.202545  9946 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:23.223289  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:23.177407 14295 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.223515 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.230690 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:23.207986  9940 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:23.225051  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:23.107789 14470 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.192339 14432 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.183597 14513 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:23.160010 13965 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.237323 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.246634 10675 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:23.235646 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.235780  9947 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.237377 10023 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:25:23.198493 14350 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.177956 14582 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:25:23.212096 14565 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:23.224285 14617 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:23.170702 14214 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.245726 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:23.174273 14287 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.250903  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1784) took 1505 ms. Trace:
I20260709 12:25:23.217187 14384 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:23.174970 14327 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.252507  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.247752 10097 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
W20260709 12:25:23.254123  9914 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.525s (SENT) [suppressed 20 similar messages]
I20260709 12:25:23.261451  9947 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:23.223104 13756 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:23.262547 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.226397 14254 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:23.244729 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.292834 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.264128 10675 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:23.265255 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.291155  9949 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:25:23.237783 14295 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.239512 14405 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:23.240566 14517 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:25:23.268630 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.270984 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.300892 10023 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:23.301250 10094 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:23.274140 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:23.276511  9944 rpcz_store.cc:276] 0709 12:25:21.745970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.066929 (+320959us) service_pool.cc:224] Handling call
0709 12:25:22.116881 (+ 49952us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:22.135606 (+ 18725us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.135650 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:22.136168 (+   518us) log.cc:844] Serialized 26 byte log entry
0709 12:25:22.228007 (+ 91839us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:22.228046 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.228118 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.055383 (+827265us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:23.243216 (+187833us) raft_consensus.cc:1685] finished
0709 12:25:23.243244 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.250838 (+  7594us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8276,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":6883,"spinlock_wait_cycles":30165760,"thread_start_us":91431,"threads_started":1,"wal-append.queue_time_us":177538}
W20260709 12:25:23.278096 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 176654us
I20260709 12:25:23.278800  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:23.283404 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 692) took 1287 ms. Trace:
I20260709 12:25:23.244674 14482 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:23.294160  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.268471 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:23.302677  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.300242 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1020) took 1077 ms. Trace:
I20260709 12:25:23.280180 14401 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:23.280480 14384 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.304951  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:23.302309 10020 rpcz_store.cc:276] 0709 12:25:21.995624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.998726 (+  3102us) service_pool.cc:224] Handling call
0709 12:25:21.999100 (+   374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:22.120357 (+121257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.120387 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:22.142572 (+ 22185us) log.cc:844] Serialized 52 byte log entry
0709 12:25:22.411337 (+268765us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:22.508868 (+ 97531us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.513292 (+  4424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.272767 (+759475us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b54001c7354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:23.272837 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.272906 (+    69us) raft_consensus.cc:1685] finished
0709 12:25:23.283075 (+ 10169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.283326 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19153,"prepare.queue_time_us":32120,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":24,"raft.queue_time_us":23751,"raft.run_cpu_time_us":789,"raft.run_wall_time_us":22135,"spinlock_wait_cycles":270697856,"thread_start_us":266741,"threads_started":2,"wal-append.queue_time_us":240436}
I20260709 12:25:23.251544 14513 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.313036 14575 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:23.321460 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.295950 14447 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:23.322221 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:23.327795 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.227442 14380 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.330561 10016 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:23.303115 10670 rpcz_store.cc:276] 0709 12:25:22.223011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:22.223129 (+   118us) service_pool.cc:224] Handling call
0709 12:25:22.246470 (+ 23341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:22.271775 (+ 25305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:22.271823 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:22.272441 (+   618us) log.cc:844] Serialized 26 byte log entry
0709 12:25:22.559639 (+287198us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:22.622126 (+ 62487us) log.cc:844] Serialized 14 byte log entry
0709 12:25:22.622583 (+   457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:22.622699 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:22.886943 (+264244us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b1800473860. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469557 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:23.059115 (+172172us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:23.300065 (+240950us) raft_consensus.cc:1685] finished
0709 12:25:23.300087 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.300195 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50494,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":51,"raft.queue_time_us":15359,"raft.run_cpu_time_us":325,"raft.run_wall_time_us":16295,"spinlock_wait_cycles":22458496,"thread_start_us":243478,"threads_started":1,"wal-append.queue_time_us":342514}
I20260709 12:25:23.345511 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:23.382140 10024 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.384876 10094 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.352244 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.301082 13756 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:23.307729 14350 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:23.391104 10016 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:23.399892 10017 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: [9.8-11.9]
I20260709 12:25:23.326104 14384 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:23.321367 14565 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.328619 14575 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.411535 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.346704 14279 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:23.387956 14624 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
W20260709 12:25:23.387959 14564 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.612s	user 0.004s	sys 0.000s
I20260709 12:25:23.288651 14630 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
W20260709 12:25:23.455154 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 161463us
I20260709 12:25:23.450314 10017 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:23.451071  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 196987us
I20260709 12:25:23.451483 10016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:25:23.448837 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.453924 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.516610 10020 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
W20260709 12:25:23.517041  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.454540 10098 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:23.392920 14409 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:23.451165 14279 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.459693 14557 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.428503 14575 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:23.545717 14279 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:23.507966 14625 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:25:23.509990 14623 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 12:25:23.511075 10023 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:23.490895 13756 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.514109 10024 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:23.347069 14401 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:23.420437 14350 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.551354 14401 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.553218 14350 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.554190 14401 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.554404 14349 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.555622 10025 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:23.519520 10676 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:23.385994 14632 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.148s
I20260709 12:25:23.426301 14565 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.524075 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.520581 14277 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:23.579007 14277 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.527946 14287 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.530927 14639 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:23.530293 14214 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.525980 14295 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.538825 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.541592 14296 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.542649 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.543490 14409 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:23.598855 10020 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:23.596717  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:23.559481 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.562260 14593 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:23.563412 14629 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:25:23.606098 14513 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:23.606343 14277 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.606904 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.607154 14296 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.567842 10098 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.608053 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.608655 10097 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:23.609445 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:23.609709 14214 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.610121 14214 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.601167  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:23.580822 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.534166 14432 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.581404 14565 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.609539 14295 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:23.609529 10099 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:25:23.612180 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.613014  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.611806 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.653343 10095 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:23.629294 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.610476 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.622560 10670 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:23.610917 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.610450 14214 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.656226 10677 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
I20260709 12:25:23.656301 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.587831 14517 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.611119 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3094) took 2177 ms. Trace:
W20260709 12:25:23.568686 14627 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 12:25:23.647716 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:23.648017  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46926.3us
I20260709 12:25:23.672365 10097 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:23.610533 14405 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:23.612005 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.664649 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.677107 10098 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:23.654963 10094 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:23.657411 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.678776 10024 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:23.657750 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.678925 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.679500 10675 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:23.571929 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.680078 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.680562 10025 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:23.665088 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.679699 10094 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:23.664556 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.605573 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.619010  9943 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:23.669047 10092 rpcz_store.cc:276] 0709 12:25:21.433751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.434306 (+   555us) service_pool.cc:224] Handling call
0709 12:25:21.434619 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:21.434979 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:21.435021 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:21.451545 (+ 16524us) log.cc:844] Serialized 26 byte log entry
0709 12:25:21.631504 (+179959us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:21.631848 (+   344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:21.634752 (+  2904us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:23.285791 (+1651039us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:23.595546 (+309755us) raft_consensus.cc:1685] finished
0709 12:25:23.595580 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:23.611042 (+ 15462us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4081,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":16474,"spinlock_wait_cycles":7090944,"thread_start_us":120683,"threads_started":1,"wal-append.queue_time_us":233864}
I20260709 12:25:23.656513 14277 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.688653 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.612730 14470 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:23.676149 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.672828 10097 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:23.683377 10091 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:23.688465 10023 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
W20260709 12:25:23.689759 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 387601us
I20260709 12:25:23.691782 10023 pending_rounds.cc:85] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 6
I20260709 12:25:23.692130 10023 pending_rounds.cc:107] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:25:23.692430 10023 raft_consensus.cc:2889] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:23.692847 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.658278 14463 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:23.694257 10098 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:23.610817 14593 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.695282 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.696556 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.673228 14279 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:23.691704 10097 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:23.697067 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 451400us
I20260709 12:25:23.704278 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:23.702128 10090 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:23.698916 10094 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:23.708946 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:23.709632 10022 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:23.689111 14277 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.703527 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:23.689111 14327 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:23.676966 14517 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.747905 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.719266 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:23.750036 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:23.720860 10025 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:23.756393 10674 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:23.679093 14557 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:25:23.754652 10023 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:23.755553 14405 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.746201 14236 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:25:23.764647 14380 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:23.764942 14035 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:23.765829 14297 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.771840 14614 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:23.772639 14640 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:23.762456 14296 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.772609 14349 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.776064 14470 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:23.808136 10091 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:25:23.697140 14463 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:23.786631 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:23.787838 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68631.9us
I20260709 12:25:23.714267 14277 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.714429 14287 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 12:25:23.868108 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:23.789196 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:23.870244  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.695719 14639 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:25:23.868856 10091 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
W20260709 12:25:23.784060 14631 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.264s	user 0.002s	sys 0.000s
I20260709 12:25:23.803483  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:23.805953  9947 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:23.782683 14621 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:23.789829 14627 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:23.846580 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.797888 14640 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.794965 14035 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:23.716887 14431 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:23.711822 14279 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.872622 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:23.869443 10020 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:23.877153  9948 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.788691 14405 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.795137 14327 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:23.881835 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:23.929665 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47927us
I20260709 12:25:23.884589 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:23.877146 14613 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.793s	user 0.014s	sys 0.000s
I20260709 12:25:23.887753 14432 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:23.876312 14575 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:23.883948 14640 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:23.843173 14480 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:25:23.899000 14560 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:23.899685 14401 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 12:25:23.911194 14463 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.909806  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:23.913880 10091 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:23.917062 10019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.011s	sys 0.006s
W20260709 12:25:23.958781 10019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.012s	sys 0.006s
I20260709 12:25:23.877749  9987 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:23.959122  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 272954us
I20260709 12:25:23.931594 14327 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:23.927479 10090 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:23.887281 14254 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.006s	sys 0.007s
I20260709 12:25:23.940433  9948 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:23.938535 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:23.871976 14470 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.902491 14384 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:23.895853 14517 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:23.951156 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:23.909026 14579 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:24.016429  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:23.958457  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 284284us
I20260709 12:25:23.921113 10095 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:23.926268 10023 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 9.
W20260709 12:25:23.964668  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 87004.2us
W20260709 12:25:24.011420 14254 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.007s	sys 0.007s
I20260709 12:25:23.967464  9940 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:25:24.067715  9940 pending_rounds.cc:85] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 5
I20260709 12:25:24.068123  9940 pending_rounds.cc:107] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:25:24.068449  9940 raft_consensus.cc:2889] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:24.069092  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:23.936017 14409 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:24.065541 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:23.954038 14560 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:23.930696  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:24.011953 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:24.012853 10096 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:24.013010 10016 raft_consensus.cc:606] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:24.013940  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:24.094784 10016 raft_consensus.cc:614] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:24.094447 10672 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:24.095305 10018 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:24.014847 13756 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.015175  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55308.9us
I20260709 12:25:24.000535 14327 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:23.908707 14214 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.120374 14214 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:23.965343  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:24.122630 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:24.123323  9945 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:24.125138  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:24.012070 14614 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:23.941004 14427 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 12:25:23.900935 14035 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.074976  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:23.984290 10668 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:24.087733 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:24.134233 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:24.134508 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46869.7us
I20260709 12:25:24.134956 10023 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:24.104138 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:24.137230 10017 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:24.012311 14349 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:25:24.104974 13756 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.112568 10671 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:24.128235  9940 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:24.016953 14557 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.059319 10677 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:24.016549 14463 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.067631 14254 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:24.146798 10096 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:23.964244 14565 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:24.150508 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:24.154243 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:24.155040 10672 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:24.065833 14579 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.130658 14593 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:24.167121 14470 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:24.167671 14279 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:24.168144 14254 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.170724 14575 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:24.149945 14648 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:24.172009 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:24.172801 10023 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:24.173617 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:24.174182 10023 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:24.149319 14621 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:24.176525 14579 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:24.147047 14613 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:24.179221 10024 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:24.179494 13756 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:24.183332 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:24.152544 14349 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:24.205621 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:24.206707 10677 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:25:24.230619  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.230940 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:24.231726 10023 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:24.233731  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:24.182473 14035 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:24.167127 14405 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:24.146023 14614 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:24.386335 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.149911 14557 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:24.437306  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:24.442219 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.185235 14287 consensus_queue.cc:1243] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:25:24.485206 14614 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:24.421057 14427 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:24.230023 14575 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:24.550932 14384 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:24.550794 14575 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.553323 14575 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:24.558133 14640 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.594s	user 0.005s	sys 0.000s
W20260709 12:25:24.529589  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.560145 10023 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:24.560688 14557 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:24.495131 14432 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.572098 14277 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.693s	user 0.009s	sys 0.000s
I20260709 12:25:24.181964 14560 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:24.574241 14349 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.575830 14427 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.562449 14565 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:24.507658 10090 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.000s
W20260709 12:25:24.595000 10021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.018s	sys 0.020s
W20260709 12:25:24.658103 10021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.019s	sys 0.021s
I20260709 12:25:24.603698 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:24.612145  9948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.013s	sys 0.014s
W20260709 12:25:24.659789  9948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.014s	sys 0.014s
I20260709 12:25:24.661180  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:24.661738  9948 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:24.662801 10023 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:24.641811 14409 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:24.654433 14635 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.144s	user 0.014s	sys 0.010s
I20260709 12:25:24.696631 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3031) took 1013 ms. Trace:
I20260709 12:25:24.697032 10024 rpcz_store.cc:276] 0709 12:25:23.683129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.683514 (+   385us) service_pool.cc:224] Handling call
0709 12:25:24.696395 (+1012881us) spinlock_profiling.cc:243] Waited 480 ms on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:24.696565 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1104876928}
I20260709 12:25:24.589562 14327 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:24.701177  9941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.000s	sys 0.050s
W20260709 12:25:24.642782 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 221305us
I20260709 12:25:24.652715 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1914) took 1561 ms. Trace:
W20260709 12:25:24.701783  9941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.001s	sys 0.050s
I20260709 12:25:24.665462 10090 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:24.667618 14614 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:24.655056 14277 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.667855 14480 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:24.719147  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.021s	sys 0.018s
W20260709 12:25:24.721215  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.022s	sys 0.018s
W20260709 12:25:24.722422  9949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.002s	sys 0.025s
I20260709 12:25:24.687811 14409 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:24.632792 14640 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:24.638070 14432 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.696241  9839 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:24.699542 10024 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:24.647820 14557 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:24.719168 14579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.006s	sys 0.021s
I20260709 12:25:24.702005 10018 rpcz_store.cc:276] 0709 12:25:23.091692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.350646 (+258954us) service_pool.cc:224] Handling call
0709 12:25:23.390861 (+ 40215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.099806 (+708945us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.652419 (+552613us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b540011ba54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:24.652623 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1619582720}
I20260709 12:25:24.722859 10021 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:25:24.723810  9949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.002s	sys 0.025s
I20260709 12:25:24.725631  9948 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:24.726665 10062 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.026s	sys 0.038s
I20260709 12:25:24.684886 14427 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:24.729027  9947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.012s	sys 0.017s
W20260709 12:25:24.774683  9947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.013s	sys 0.017s
W20260709 12:25:24.738842  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 664899us
I20260709 12:25:24.708758 14447 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:24.773217  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:24.747522 14637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.004s	sys 0.021s
W20260709 12:25:24.768568 10099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.004s	sys 0.035s
W20260709 12:25:24.771531  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.771924  9942 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:25:24.772068  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:24.771643 10062 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.026s	sys 0.038s
W20260709 12:25:24.643435  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.025s	sys 0.021s
W20260709 12:25:24.800050 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 726371us
I20260709 12:25:24.800753  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:24.801258 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:24.769449 14620 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.504s	user 0.011s	sys 0.040s
I20260709 12:25:24.801874  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:24.799559  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.029s	sys 0.024s
W20260709 12:25:24.817039  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 691947us
W20260709 12:25:24.770861 14579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.006s	sys 0.024s
I20260709 12:25:24.718935 14384 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:24.775735 14447 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.776055  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:24.817792  9948 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:25:24.803414 14637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.011s	sys 0.023s
I20260709 12:25:24.819101 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 728) took 1031 ms. Trace:
I20260709 12:25:24.776098 14640 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:24.805153 14431 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.822959 10099 rpcz_store.cc:276] 0709 12:25:23.787971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.788116 (+   145us) service_pool.cc:224] Handling call
0709 12:25:23.807865 (+ 19749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.808070 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:24.239553 (+431483us) log.cc:844] Serialized 14 byte log entry
0709 12:25:24.818777 (+579224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:24.818826 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:24.818871 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.818935 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:24.819047 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71122,"prepare.queue_time_us":91360,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":13934,"thread_start_us":926823,"threads_started":2}
I20260709 12:25:24.806586 14565 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:24.917925  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:24.815284 14620 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.553s	user 0.018s	sys 0.042s
I20260709 12:25:24.826848  9949 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:24.747814 14480 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:24.857393 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:24.722118 14614 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.919817 14431 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.924621 14649 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:24.816308 14517 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:24.805660 14663 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:24.772332 14557 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:24.935480 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 113468us
I20260709 12:25:24.936936 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:24.922350 14630 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:24.828510 14654 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.948814 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:24.933034 14637 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:25:24.922254 14620 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:24.968766  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 150830us
I20260709 12:25:24.970256  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:24.971424  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:24.941642 14035 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:24.935842 14565 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:24.991128 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:24.825282 14384 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.992056 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:24.995780  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 219776us
I20260709 12:25:24.995848 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:24.998310 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:24.999853 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:25.001439  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:24.967243 14640 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.007259 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:25.010558 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.005972 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:24.990737 14637 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:24.946516 14630 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:24.973098 14614 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:24.972815 14670 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:24.945130 14517 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.061928 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.067978 10025 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:25.063808 14427 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:25:25.047120 14287 raft_consensus.cc:2764] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:25.004297 14677 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.066s
I20260709 12:25:24.969312 14654 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.070946 14447 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.004058 14620 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:25.074229 14664 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:25:25.004861 14575 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:25.074939 14557 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.075661 14565 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:25.068413  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4176233620000858s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260709 12:25:24.999589 14405 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:24.994073 14035 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.066972 14327 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:25.110229  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2281615560000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2076s for a response
W20260709 12:25:25.126977  9912 connection.cc:256] RPC call timeout handler was delayed by -1.126023362000069s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 12:25:25.111323 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.079943 14617 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:25.090710 14277 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.106825 14660 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:25.118251 14385 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:25.136266 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87397us
I20260709 12:25:25.144042 14480 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.145570 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 900) took 1066 ms. Trace:
I20260709 12:25:25.146006 10674 rpcz_store.cc:276] 0709 12:25:24.079172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.648212 (+569040us) service_pool.cc:224] Handling call
0709 12:25:24.648644 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.649148 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.649193 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.649834 (+   641us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.738568 (+ 88734us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:24.738630 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.738713 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.145350 (+406637us) raft_consensus.cc:1685] finished
0709 12:25:25.145368 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.145508 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1452,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":138,"thread_start_us":88284,"threads_started":1,"wal-append.queue_time_us":493840}
I20260709 12:25:25.147284 10097 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.147862 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:25.148494 10674 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:25:25.149471 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:25.149991 10674 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:25.149832 14660 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:25.137954 14679 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.151062 14679 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:25.151170 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:25.151530 14679 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.151985 10674 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:25.146803  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1801) took 1191 ms. Trace:
I20260709 12:25:25.155462  9941 rpcz_store.cc:276] 0709 12:25:23.954964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.965140 (+ 10176us) service_pool.cc:224] Handling call
0709 12:25:23.965528 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.965926 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.965967 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.966523 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.702167 (+735644us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:24.702225 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.702297 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.146509 (+444212us) raft_consensus.cc:1685] finished
0709 12:25:25.146538 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.146725 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4325,"raft.queue_time_us":157527,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":12664,"thread_start_us":638447,"threads_started":1,"wal-append.queue_time_us":1056977}
I20260709 12:25:25.160418  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:25.129930 14557 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.091285 14670 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.124346 14432 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:25.119160 14447 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 12:25:25.189548  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 218219us
I20260709 12:25:25.165661 14557 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.176672 14670 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.123751 14405 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.180775 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1028) took 1937 ms. Trace:
W20260709 12:25:25.181954  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 858) took 2072 ms (client timeout 1638 ms). Trace:
W20260709 12:25:25.196148  9943 rpcz_store.cc:269] 0709 12:25:23.109185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.149692 (+ 40507us) service_pool.cc:224] Handling call
0709 12:25:25.181779 (+2032087us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b540042e854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.181889 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060107136}
W20260709 12:25:25.182648 14657 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:25:25.180775 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 948) took 1331 ms. Trace:
I20260709 12:25:25.217871 10676 rpcz_store.cc:276] 0709 12:25:23.849669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.849832 (+   163us) service_pool.cc:224] Handling call
0709 12:25:23.850206 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:23.904157 (+ 53951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.904189 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:23.904235 (+    46us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:23.904276 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.180543 (+1276267us) spinlock_profiling.cc:243] Waited 53.7 ms on lock 0x7b5400528fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:25.180585 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.180710 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123526528}
I20260709 12:25:25.121763 14575 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.190099 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.164522  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1812) took 1002 ms. Trace:
I20260709 12:25:25.136432 14617 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:25.201905  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 216519us
W20260709 12:25:25.177445 14666 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.252s	user 0.001s	sys 0.002s
I20260709 12:25:25.195091 10671 rpcz_store.cc:276] 0709 12:25:23.243434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.243549 (+   115us) service_pool.cc:224] Handling call
0709 12:25:23.268590 (+ 25041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:24.145148 (+876558us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.180558 (+1035410us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b54007f4754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:25.180711 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940563712}
I20260709 12:25:25.192859 14557 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.187773 14678 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.213848 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:25.235000 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:25.221657  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.236672  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:25.237599  9946 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:25.238500  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:25.239130  9946 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.240243  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2037) took 1327 ms. Trace:
I20260709 12:25:25.240682  9945 rpcz_store.cc:276] 0709 12:25:23.913161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.913289 (+   128us) service_pool.cc:224] Handling call
0709 12:25:25.240065 (+1326776us) spinlock_profiling.cc:243] Waited 29 ms on lock 0x7b540009bad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.240182 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66702720}
I20260709 12:25:25.241808  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.222311 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.242867 14035 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.245409 10095 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:25.245862 14630 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:25.246734 14669 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 12:25:25.194655 14327 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.226540 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:25.227324 14654 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.228683 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:25.263882 10024 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:25.265113 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:25.166843 14690 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:25:25.189087 14385 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:25.229009  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:25.266994 14690 consensus_queue.cc:1243] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:25.267508 14385 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.268103  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:25.268875 14405 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.269652  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.270741 14517 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.270825 14691 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.164125 14660 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.271603 14687 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:25.272097 14687 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.272424 14687 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.251457  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1805) took 1295 ms. Trace:
I20260709 12:25:25.274878  9940 rpcz_store.cc:276] 0709 12:25:23.955958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.069560 (+113602us) service_pool.cc:224] Handling call
0709 12:25:24.069965 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.145769 (+ 75804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.251182 (+1105413us) spinlock_profiling.cc:243] Waited 54.1 ms on lock 0x7b540003f554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:25.251396 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124381952}
I20260709 12:25:25.255286 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:25.256352 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:25.262023  9946 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:25.271073 14297 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.266325 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:25.276635 10017 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
W20260709 12:25:25.280062 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57825.9us
I20260709 12:25:25.279433 10024 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:25.281241  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.190994 14277 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:25.282927 10023 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:25.242404  9945 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:25.229591 14432 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:25.294637 14287 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.223439  9944 rpcz_store.cc:276] 0709 12:25:24.160739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.804697 (+643958us) service_pool.cc:224] Handling call
0709 12:25:24.805008 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.814482 (+  9474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.814515 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.824626 (+ 10111us) log.cc:844] Serialized 26 byte log entry
0709 12:25:25.103962 (+279336us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:25.104004 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.104073 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.162536 (+ 58463us) raft_consensus.cc:1685] finished
0709 12:25:25.162567 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.162953 (+   386us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9091,"raft.queue_time_us":113584,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":21719,"thread_start_us":177887,"threads_started":1,"wal-append.queue_time_us":225068}
I20260709 12:25:25.296701 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.300951 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.295234 14686 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:25.219964 14679 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.302793 14380 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.302845 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.302047 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.321440 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:25.303545 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:25.305717 14646 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.289s	user 0.006s	sys 0.006s
W20260709 12:25:25.306701 10670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1035) took 1761 ms (client timeout 1763 ms). Trace:
I20260709 12:25:25.330837 10091 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:25.324769 14617 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.292011 13965 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.287257 14277 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:25.287545 14327 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.319432 14660 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.387550 14327 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.322443 14432 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:25.387804 14660 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.388381 14679 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.323346 10021 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:25.388717 14432 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.319630 14677 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:25:25.267565 14690 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:25.392010 14690 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:25.392092 14557 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.392987 14690 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.394188 14690 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.399206 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:25.399948 10675 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
W20260709 12:25:25.324592 14681 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:25:25.302016 14632 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.323081 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3016) took 2071 ms (client timeout 1850 ms). Trace:
W20260709 12:25:25.320762 10025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3037) took 1474 ms (client timeout 1670 ms). Trace:
I20260709 12:25:25.329120 10017 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.329528 10018 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:25.326929 10097 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:25:25.422250 10097 pending_rounds.cc:85] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 7
I20260709 12:25:25.422569 10097 pending_rounds.cc:107] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:25:25.422900 10097 raft_consensus.cc:2889] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:25.423781 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3116) took 1813 ms. Trace:
I20260709 12:25:25.424173 10097 rpcz_store.cc:276] 0709 12:25:23.609965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.671956 (+ 61991us) service_pool.cc:224] Handling call
0709 12:25:23.672249 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:25.146789 (+1474540us) spinlock_profiling.cc:243] Waited 770 ms on lock 0x7b54004a7520. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cba1d 00007f4e094b21a4 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419
0709 12:25:25.423463 (+276674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.423712 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1770796800}
W20260709 12:25:25.331292 10670 rpcz_store.cc:269] 0709 12:25:23.535051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.538399 (+  3348us) service_pool.cc:224] Handling call
0709 12:25:25.295641 (+1757242us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b5400545bd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.295789 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63817088}
I20260709 12:25:25.307451 10090 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:25.319290 14287 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.332226  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1807) took 1375 ms. Trace:
I20260709 12:25:25.334677 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1912) took 2243 ms. Trace:
I20260709 12:25:25.338914 14663 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:25:25.339819 14575 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.281358 14678 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.325366 14686 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.342432 10095 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:25.349030 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:25.462890 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 113957us
I20260709 12:25:25.463675 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:25.350106 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.464903 10091 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:25.380209  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.385377 14277 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:25.394342 14557 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.427156  9947 rpcz_store.cc:276] 0709 12:25:23.956442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.128954 (+172512us) service_pool.cc:224] Handling call
0709 12:25:24.129346 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.129718 (+   372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.129756 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.144791 (+ 15035us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.775099 (+630308us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:24.775146 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.775218 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.176835 (+401617us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:25.331940 (+155105us) raft_consensus.cc:1685] finished
0709 12:25:25.331961 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.332171 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16875,"raft.queue_time_us":23167,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6915,"spinlock_wait_cycles":24653568,"thread_start_us":599933,"threads_started":1,"wal-append.queue_time_us":770734}
W20260709 12:25:25.420200 10025 rpcz_store.cc:269] 0709 12:25:23.846180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.846325 (+   145us) service_pool.cc:224] Handling call
0709 12:25:25.320569 (+1474244us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400265cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.320700 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2727494528}
W20260709 12:25:25.419361 10022 rpcz_store.cc:269] 0709 12:25:23.252112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.273833 (+ 21721us) service_pool.cc:224] Handling call
0709 12:25:25.322850 (+2049017us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b54002d5d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.323012 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":912639744}
I20260709 12:25:25.431581  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.470781 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:25.432036  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 271753us
I20260709 12:25:25.450444 10024 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:25.427263 10019 rpcz_store.cc:276] 0709 12:25:23.091098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.224468 (+133370us) service_pool.cc:224] Handling call
0709 12:25:23.248881 (+ 24413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:23.249347 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:23.249388 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:23.263791 (+ 14403us) log.cc:844] Serialized 26 byte log entry
0709 12:25:23.959194 (+695403us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:23.959232 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:23.959294 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.333806 (+1374512us) spinlock_profiling.cc:243] Waited 5.16 ms on lock 0x7b24001e8648. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a7fb0 00007f4e0911e488 00007f4e0911ebe8 00007f4e090f6398 00007f4e090f176a 00007f4e090f0bb6
0709 12:25:25.334352 (+   546us) raft_consensus.cc:1685] finished
0709 12:25:25.334370 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.334609 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":14108,"spinlock_wait_cycles":11864448,"thread_start_us":683536,"threads_started":1,"wal-append.queue_time_us":689733}
I20260709 12:25:25.466069  9944 raft_consensus.cc:2410] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:25:25.430830 14680 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.356s	user 0.002s	sys 0.000s
W20260709 12:25:25.431025 14673 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.341s	user 0.002s	sys 0.000s
I20260709 12:25:25.403364 13965 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:25.471791  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:25.485316  9990 connection.cc:256] RPC call timeout handler was delayed by -1.2724615689999155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
W20260709 12:25:25.485750  9990 connection.cc:256] RPC call timeout handler was delayed by -1.023281673999918s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1683s for a response
I20260709 12:25:25.481045 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:25.485842  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:25.486107  9990 connection.cc:256] RPC call timeout handler was delayed by -1.0078615689999424s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
I20260709 12:25:25.487317  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:25.482089 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:25.489411 10670 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.453812 14427 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.360898 14380 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.494370  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:25.492115 13965 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.502211 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 899) took 1515 ms. Trace:
I20260709 12:25:25.503204 10019 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:25:25.507092 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:25.506835 10672 rpcz_store.cc:276] 0709 12:25:23.987195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.187123 (+199928us) service_pool.cc:224] Handling call
0709 12:25:24.234431 (+ 47308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.234869 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:24.234910 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:24.235474 (+   564us) log.cc:844] Serialized 26 byte log entry
0709 12:25:24.737220 (+501746us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:24.737281 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:24.737376 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.396822 (+659446us) spinlock_profiling.cc:243] Waited 3.41 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:25.501901 (+105079us) raft_consensus.cc:1685] finished
0709 12:25:25.501922 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.502142 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32974,"raft.queue_time_us":4562,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":40379,"spinlock_wait_cycles":7834496,"thread_start_us":250216,"threads_started":1,"wal-append.queue_time_us":514128}
I20260709 12:25:25.469727 14409 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.510456 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.516235 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.518812 14643 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:25.519418 14643 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:25.520416 14643 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.522425 14662 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:25.522951 14662 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.523299 14662 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.532295 14297 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.516489 14630 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:25:25.534526 14630 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:25.511467 14350 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:25.534966 14630 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:25.535787 14630 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.469292 14677 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.481786 14640 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.537948 14384 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.443516 14632 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.540187 14630 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.468001 14686 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:25.511286 14617 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.507179  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:25.547019  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:25.547459 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:25.503968 14693 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:25.508042 14236 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:25.549396 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:25.557500 10673 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:25.508939 14557 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.565153  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:25.515373  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:25.553037 10024 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:25:25.510741 14565 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.509688 14678 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.534360 13965 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:25.554852 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:25.564621 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:25.508332 14517 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.568549  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.572950  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:25.572263  9948 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:25.570117  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54823.5us
I20260709 12:25:25.582479 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:25.550424 14350 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:25.584196 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:25.584900  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.508101 14427 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.586577  9945 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:25:25.548615 14633 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.877s	user 0.014s	sys 0.049s
I20260709 12:25:25.578759  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:25.591367  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:25.572718 14617 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.572567 14637 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.579252 14693 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:25:25.571934 14628 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.585922 14579 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:25.581889 14409 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:25.592769 14692 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.151s	user 0.000s	sys 0.011s
I20260709 12:25:25.593355 14617 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:25.630965 14617 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.631431 10017 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:25.604979  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:25.593583 14614 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:25.616607  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:25.591928 14633 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.928s	user 0.015s	sys 0.049s
I20260709 12:25:25.617697 14636 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:25.612828 14637 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.627924 14687 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.576542 14565 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:25.657141  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52248.3us
I20260709 12:25:25.614559 14684 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:25.606390 14236 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:25.605720 14685 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.690989 14684 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:25.692420 14684 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.635183 14579 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:25.695385 14579 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.587945 14517 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.663815  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:25.801877  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 138150us
I20260709 12:25:25.802636  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.804385  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:25.670244  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:25.672330 14633 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:25.606948 14679 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.662427 14405 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.664950 14636 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:25.810245 14636 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:25.683249 14695 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:25:25.683411 14380 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.685268 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 901) took 1599 ms. Trace:
I20260709 12:25:25.815757 10677 rpcz_store.cc:276] 0709 12:25:24.086170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.920440 (+834270us) service_pool.cc:224] Handling call
0709 12:25:24.947875 (+ 27435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.355347 (+407472us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:25.355392 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:25.403928 (+ 48536us) log.cc:844] Serialized 26 byte log entry
0709 12:25:25.605193 (+201265us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:25.605228 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.605294 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.685031 (+ 79737us) raft_consensus.cc:1685] finished
0709 12:25:25.685051 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.685206 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1629,"raft.queue_time_us":27408,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":12951,"thread_start_us":505121,"threads_started":2,"wal-append.queue_time_us":207852}
W20260709 12:25:25.817708 14667 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.744s	user 0.001s	sys 0.002s
I20260709 12:25:25.685367 14384 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.818981 14384 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:25.685878 14698 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:25:25.821142 14384 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.807024 14649 raft_consensus.cc:2764] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:25.656416 14614 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.822800 14614 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.635447 14350 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:25:25.636081 14409 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.666069 14678 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:25.674194 14693 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
W20260709 12:25:25.785621 10668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 940) took 2673 ms (client timeout 1892 ms). Trace:
W20260709 12:25:25.845273 10668 rpcz_store.cc:269] 0709 12:25:23.112383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.112562 (+   179us) service_pool.cc:224] Handling call
0709 12:25:25.718704 (+2606142us) spinlock_profiling.cc:243] Waited 774 ms on lock 0x7b540050a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.785531 (+ 66827us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1779228544}
W20260709 12:25:25.790505  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.794510 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:25.808763  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:25.809463 10670 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:25.806793 14382 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:25.798512 14575 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.849366 14382 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:25.849821 14382 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:25.849519 14575 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.830367 14409 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:25.847977 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:25.851233 14620 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:25.851796  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 872) took 1145 ms. Trace:
I20260709 12:25:25.852428 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:25.853384 14620 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.852280  9949 rpcz_store.cc:276] 0709 12:25:24.706264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.015676 (+309412us) service_pool.cc:224] Handling call
0709 12:25:25.068457 (+ 52781us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:25.202970 (+134513us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:25.203018 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:25.203864 (+   846us) log.cc:844] Serialized 52 byte log entry
0709 12:25:25.503978 (+300114us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:25.504018 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.504083 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:25.851567 (+347484us) raft_consensus.cc:1685] finished
0709 12:25:25.851585 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:25.851737 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36531,"raft.queue_time_us":96633,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":16190,"thread_start_us":430759,"threads_started":2,"wal-append.queue_time_us":408994}
I20260709 12:25:25.854568 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:25.855684 14380 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.856067 14380 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:25.856380 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:25.856426 14380 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.857813 13965 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.858196 13965 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:25.858539 13965 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.865216 14575 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.841219 14277 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.868408 14277 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.871714  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.872453  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1810) took 1915 ms. Trace:
I20260709 12:25:25.872860  9942 rpcz_store.cc:276] 0709 12:25:23.957122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:24.689655 (+732533us) service_pool.cc:224] Handling call
0709 12:25:24.771712 (+ 82057us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:24.805863 (+ 34151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:25.872265 (+1066402us) spinlock_profiling.cc:243] Waited 81.6 ms on lock 0x7b54001a40d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.872396 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187721088}
W20260709 12:25:25.876554  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67860.7us
I20260709 12:25:25.877511  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.888741  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.834025 14236 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.899178 14575 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:25.900401 14685 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.890880 14277 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:25.901018 14380 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.843667 14405 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:25.901902 14620 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.941318 14620 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.943472  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.891177 14385 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.944823  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.944922 14385 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.946026  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.947209 14385 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:25.903330 14671 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.833s	user 0.003s	sys 0.000s
I20260709 12:25:25.904628 14621 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.950291  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.950611 14621 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:25.952769 14703 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.344s	user 0.001s	sys 0.000s
I20260709 12:25:25.944404 14620 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.936605 14575 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.955262 14575 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.905905  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.907196  9948 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:25.908802  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.908870 14676 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.909415 14350 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.910000  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.910698  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.911466  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:25.961089  9949 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:25.912170 14560 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:25.914274 14431 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.915102 14632 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:25.937814 14214 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:25.916318 14287 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:25.923530 14687 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:25.898523 14679 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:25.929317 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59744 (request call id 32) took 2542 ms. Trace:
W20260709 12:25:25.933118  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44484.4us
I20260709 12:25:25.957919  9940 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:25.959352  9945 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:25.894441 14678 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:25.965255 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 727) took 2172 ms (client timeout 1638 ms). Trace:
W20260709 12:25:25.976857 10092 rpcz_store.cc:269] 0709 12:25:23.786210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.786345 (+   135us) service_pool.cc:224] Handling call
0709 12:25:25.958028 (+2171683us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400163354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.958147 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2875745664}
I20260709 12:25:25.834322 14662 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.937549 14035 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:25.975014  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:25.973327 14214 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:25.973763 10016 rpcz_store.cc:276] 0709 12:25:23.387315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:23.410904 (+ 23589us) service_pool.cc:224] Handling call
0709 12:25:25.929083 (+2518179us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b540011ba54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09550853 00007f4e0e51e60a 00007f4e07aa47d4 00007f4e07aa4762 00007f4e07aa4691 00007f4e07aa460c 00007f4e07aa38c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:25.929202 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1142963968}
I20260709 12:25:25.976336  9948 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:25.985057 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:25.990739 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 385276us
I20260709 12:25:25.991520 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:25.999468 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:26.001444 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:26.003202 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.961050 14614 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.021118 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.978370 14678 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:26.017891  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:26.032567 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:25.987792 14637 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:26.023232 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:26.024487 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.972005 14350 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.025930 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:26.039916 10097 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:26.027048 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:26.041047 10099 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:25.991226 14214 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:25.967690 14676 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:25.972906 14405 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:25:26.076561  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:26.077338  9945 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:25.980229 14035 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.037182 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:26.037196 14686 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:25.991428 14287 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:26.041397  9940 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:26.043484 10670 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:25.993420 14427 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.050321 10677 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:26.053411 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:26.087668  9944 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:26.017795 14277 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 12:25:26.115865 14517 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.120517  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:26.121115 14035 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.121714  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:26.122357  9943 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:26.123204 14662 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:26.123827 14662 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:26.004436 14693 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.124516 13965 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.125928 14693 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:26.076346 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:26.025647 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:25.992208 14679 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:25:26.017515 14685 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:26.127658 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.034303 14678 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.035764 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:26.130394 10094 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:26.050412 14680 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:26.033617 14401 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:25:26.096199 14287 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.135495 14401 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:26.135911 14401 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:26.136317 14676 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.136772 14287 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.097023 14636 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.097904 14380 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.100907 14614 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:26.102725  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:26.136701 14401 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.032474 14617 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.108078 14350 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.102761 14214 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.153239 14214 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.156940 14617 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.157586  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:26.159350 14401 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.163391  9948 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:26.164162  9947 raft_consensus.cc:2368] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:26.165572  9948 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:26.169044  9948 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:26.127377 10095 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:26.187649 14380 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:26.187872 14685 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.188289 14685 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:26.073518 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:26.139014 14636 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.145025 14686 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:26.146095 14557 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.001s
I20260709 12:25:26.145560  9839 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 6 to 10, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:26.146497 14690 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.743s	user 0.012s	sys 0.000s
W20260709 12:25:26.091312 14630 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.551s	user 0.007s	sys 0.005s
I20260709 12:25:26.149291 10675 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:26.169862  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:26.173523 14708 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.089s	user 0.000s	sys 0.002s
W20260709 12:25:26.202692 14639 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.036s	sys 0.067s
I20260709 12:25:26.174935  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:26.176433  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:26.179308 14621 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.078346 14565 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:26.065114  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 189925us
W20260709 12:25:26.194691 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.042s	sys 0.051s
W20260709 12:25:26.247512 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.042s	sys 0.052s
W20260709 12:25:26.247807 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 640761us
W20260709 12:25:26.199096  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:26.121341 14382 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.206704  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:26.215437  9839 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 7 to 10, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.219621 10090 raft_consensus.cc:2410] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:26.174194 14693 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.220816  9940 raft_consensus.cc:2368] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:26.262879  9839 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 7 to 11, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 11 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.232916 10095 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:25:26.229290  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 559065us
I20260709 12:25:26.245179 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:26.223575  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:26.265460  9948 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:26.246503 10094 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:26.257786 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:26.248615  9830 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49580.3us
I20260709 12:25:26.260460 10669 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:26.260450  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 84102.2us
I20260709 12:25:26.263652  9940 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:26.266251  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:26.271328 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:26.216655 14557 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:26.234889 14639 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.037s	sys 0.067s
I20260709 12:25:26.305583 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:26.306044 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:26.306205  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:26.214097 14432 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.274449 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:26.276075 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:26.276463 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:26.277648 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:26.281034 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:26.308734 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:26.309406 10676 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
W20260709 12:25:26.306818 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68941.8us
I20260709 12:25:26.310258  9945 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:26.221628 13965 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.218923 14384 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:26.221263 14620 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:25:26.281445  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:26.303855  9944 raft_consensus.cc:606] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:26.305706 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:26.188310 14380 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.308822 10674 raft_consensus.cc:2368] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12.
I20260709 12:25:26.309525 10023 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:26.311280 10022 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:26.311474 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:26.341357 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:26.346536  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:26.346627  9948 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:26.354655 10677 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:26.355046  9947 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:26.264667  9839 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 3 to 5, term changed from 7 to 10, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER cd983415389444e292b2330ef630850a (127.8.137.193), VOTER 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194), VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195), VOTER 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.300405 14718 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:26.357159 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:26.306613 14708 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:25:26.341239 14432 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:26.374444 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64062.1us
I20260709 12:25:26.365957 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:26.361552  9944 raft_consensus.cc:614] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:26.363478 10096 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:26.364279  9842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 329) took 33493 ms (client timeout 15000 ms). Trace:
I20260709 12:25:26.363794 10021 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.371430  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:26.377954 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 129576us
I20260709 12:25:26.378871  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:26.379743 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:26.379829  9948 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:26.382367  9948 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:26.376637  9842 rpcz_store.cc:269] 0709 12:24:52.871028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:24:52.871315 (+   287us) service_pool.cc:224] Handling call
0709 12:24:53.554262 (+682947us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094c0090 00007f4e0955ef3a 00007f4e0f314e9c 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc 00007f4e0d8d3392 00007f4e0789148c 00007f4e078908c6
0709 12:25:26.364049 (+32809787us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:18.556181 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:18.756772 (+200591us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:18.760076 (+  3304us) write_op.cc:435] Acquired schema lock
0709 12:25:18.765405 (+  5329us) tablet.cc:662] Decoding operations
0709 12:25:19.132210 (+366805us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:19.140592 (+  8382us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:19.145484 (+  4892us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:25:19.215601 (+ 70117us) tablet.cc:701] Row locks acquired
0709 12:25:19.216633 (+  1032us) write_op.cc:260] PREPARE: finished
0709 12:25:19.220306 (+  3673us) write_op.cc:270] Start()
0709 12:25:19.379331 (+159025us) write_op.cc:276] Timestamp: P: 1783599919219487 usec, L: 0
0709 12:25:19.388404 (+  9073us) op_driver.cc:348] REPLICATION: starting
0709 12:25:19.434364 (+ 45960us) log.cc:844] Serialized 2388 byte log entry
0709 12:25:19.796925 (+362561us) op_driver.cc:464] REPLICATION: finished
0709 12:25:19.966988 (+170063us) write_op.cc:301] APPLY: starting
0709 12:25:20.019233 (+ 52245us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:20.131825 (+112592us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:20.131995 (+   170us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:20.188990 (+ 56995us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:20.189443 (+   453us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 12:25:20.189922 (+   479us) write_op.cc:312] APPLY: finished
0709 12:25:20.190360 (+   438us) log.cc:844] Serialized 57 byte log entry
0709 12:25:20.203943 (+ 13583us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:20.236615 (+ 32672us) write_op.cc:454] Released schema lock
0709 12:25:20.246561 (+  9946us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:20.316115 (+ 69554us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:20.318565 (+  2450us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17998,"reported_tablets":47,"spinlock_wait_cycles":254684032,"child_traces":[["op",{"apply.queue_time_us":128508,"apply.run_cpu_time_us":13336,"apply.run_wall_time_us":354950,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"num_ops":5,"prepare.queue_time_us":454338,"prepare.run_cpu_time_us":40916,"prepare.run_wall_time_us":1095622,"raft.queue_time_us":105856,"raft.run_cpu_time_us":13114,"raft.run_wall_time_us":83979,"replication_time_us":408284,"spinlock_wait_cycles":43776,"thread_start_us":677136,"threads_started":4,"wal-append.queue_time_us":235494,"wal-append.run_cpu_time_us":22981,"wal-append.run_wall_time_us":1572965}]]}
I20260709 12:25:26.389070 14254 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:25:26.389945 14711 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:25:26.383425 10022 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:26.394191 10096 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:26.390094 14254 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:25:26.419880  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 215055us
I20260709 12:25:26.452339 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:26.457254 10675 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:26.452538 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:26.458552 10096 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:26.404886 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:26.473237  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:26.456324  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:26.403723  9839 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 9 to 10, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.480302 10669 raft_consensus.cc:2376] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11. Re-sending same reply.
I20260709 12:25:26.369350 14718 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:26.392429 14579 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.021s	sys 0.021s
I20260709 12:25:26.487931  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:26.488884 10677 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:26.391300 14297 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.508505 14382 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.353338 14575 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.495167  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:26.341455 14716 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 12:25:26.481419 14647 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:25:26.395545 14677 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.108s
I20260709 12:25:26.369585 14678 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.503472 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:26.504753 14380 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.493573  9839 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.506803 14432 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.514792  9945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:26.531288  9945 raft_consensus.cc:606] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:26.531824  9945 raft_consensus.cc:614] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:26.515842 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 136149us
I20260709 12:25:26.516558 14662 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:26.517198 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 113178us
I20260709 12:25:26.503186  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:26.537137  9947 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:26.459589 14287 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:26.360683 14713 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 12:25:26.404469 14687 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:26.500115 14579 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.021s	sys 0.021s
I20260709 12:25:26.365118 14384 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:26.509603 14707 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
W20260709 12:25:26.514973 14560 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.543s	user 0.003s	sys 0.007s
W20260709 12:25:26.516973 14712 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:25:26.500154 14690 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.544639 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:26.499759 14718 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.537194 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:26.552290  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 144683us
W20260709 12:25:26.555112  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 248997us
I20260709 12:25:26.403257 14709 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.120s
I20260709 12:25:26.568583  9940 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:26.569500 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:26.548414  9839 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.592312  9943 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:26.596813 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:26.601897  9947 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
W20260709 12:25:26.603652 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 66560us
W20260709 12:25:26.610829  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 329422us
I20260709 12:25:26.550040 14287 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:26.572865 14405 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.572865 14254 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:26.620306  9839 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 7 to 10. New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.621848  9991 leader_election.cc:400] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:26.624481  9991 leader_election.cc:403] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:26.593086 14361 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:26.628959 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59550.5us
I20260709 12:25:26.591019 14690 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:26.606865 14663 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.633754 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:26.631415  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:26.620020 14582 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:26.632479  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:26.635471 14621 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.633607 14690 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.622982 14254 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:26.638056 14678 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.634251 14662 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:25:26.638921 14677 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:26.572723 14687 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:26.639734 14721 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:25:26.639562 14614 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:25:26.640509 14722 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 12:25:26.640955 14633 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:26.642234 14715 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:25:26.644740 14647 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:26.650731 14643 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:26.620288 14565 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:25:26.632313  9991 leader_election.cc:400] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:26.620571 14405 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.666832  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:26.666824  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:26.669255  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:26.671094  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:26.672204  9947 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:26.609783 14431 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:25:26.672883  9945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:26.609232 13965 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:25:26.656097  9839 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 6 to 8, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 8 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.622638 14035 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:26.675380 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44166.7us
I20260709 12:25:26.675801  9948 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:26.633353 14361 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.683881 10021 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:26.629261 14427 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:25:26.764909  9945 raft_consensus.cc:606] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:26.765369  9945 raft_consensus.cc:614] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:26.765319  9941 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.681489 14684 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:26.766861 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:26.629204 14719 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.764599  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:26.772862 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:26.770059  9991 leader_election.cc:403] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:26.773810 10675 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 11: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:26.677521 14686 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:26.792229 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:26.788496  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 119344us
I20260709 12:25:26.704552 14733 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:25:26.790078  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 157894us
I20260709 12:25:26.797538 10669 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:25:26.770921 14405 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.816789  9839 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 5 to 7, leader changed from <none> to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 7 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:26.764427 14643 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:26.680441 14582 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:26.770712 14737 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:26.794479 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:26.793213 14693 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.003s
I20260709 12:25:26.823333  9941 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:26.837031  9839 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 7, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 7 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:26.794901 10668 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:25:26.750916 14687 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.710520 14677 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:26.771219 14361 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.770376 14617 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:26.716274 14384 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:26.773059  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 102041us
I20260709 12:25:26.824549 14734 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:26.773941 14565 consensus_queue.cc:1243] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:26.817569 14742 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.834825 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:26.773870 14035 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.681973 14663 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.773788 14647 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.790812 14719 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.791167 14714 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:26.796190 14409 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:26.824913 14722 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:25:26.745505 14724 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:26.721474 14730 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 12:25:26.750794 14517 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:26.847932  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:26.842383  9839 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 6 to 9, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 9 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:26.825299 14686 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:26.771685 14633 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:26.751932 14254 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.840622 14614 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:26.839180 14738 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:26.853319 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:26.869828 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:26.858981 14685 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.004s
I20260709 12:25:26.880403 10022 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.858680 14643 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.882637 14633 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.883407  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1822) took 1465 ms. Trace:
I20260709 12:25:26.884874 14633 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.883776  9946 rpcz_store.cc:276] 0709 12:25:25.418250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:25.418431 (+   181us) service_pool.cc:224] Handling call
0709 12:25:25.418883 (+   452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:25.419286 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:25:25.460690 (+ 41404us) log.cc:844] Serialized 14 byte log entry
0709 12:25:25.948553 (+487863us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b18000f6dc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:25:26.281390 (+332837us) log.cc:844] Serialized 14 byte log entry
0709 12:25:26.303047 (+ 21657us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:26.303500 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 12:25:26.303642 (+   142us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:26.303673 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:26.303748 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:26.385208 (+ 81460us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b18000f6dc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:26.883190 (+497982us) raft_consensus.cc:1685] finished
0709 12:25:26.883212 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:26.883349 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19530,"prepare.queue_time_us":48099,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":4391,"raft.queue_time_us":44958,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":29427,"spinlock_wait_cycles":24727552,"thread_start_us":332296,"threads_started":1,"wal-append.queue_time_us":947542}
I20260709 12:25:26.854312 14742 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:26.886103 14742 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:26.865653 14409 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:25:26.882675 14643 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:26.888125 14409 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:26.859014 14719 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:26.855815 14717 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:25:26.875034 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:26.891741 10025 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:26.857194 14709 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:26.841636 14693 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.877627  9839 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 4 to 8, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 8 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.897058  9839 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 9 to 11, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 11 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:26.851284 14737 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.895558 10021 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:26.863188 14748 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.902458 10021 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:26.875445 14731 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:26.903882 10022 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:25:26.906271 14747 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:25:26.905862 14677 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.11, Last appended by leader: 11, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:26.907311 14743 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:26.908030 14647 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.908619 14686 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:26.907352 14725 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:25:26.934773  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 446909us
I20260709 12:25:26.935367 14035 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:26.926796 14713 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.936570 14621 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:26.937026 14621 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:26.899741 14662 consensus_queue.cc:1175] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aa345bcd43b44088114b7b6f2830b17" responder_term: 13 status { last_received { term: 13 index: 7 } last_committed_idx: 6 last_received_current_leader { term: 13 index: 7 } }
I20260709 12:25:26.924417 14731 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:26.938545 14731 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.936411 14713 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.939175 14382 raft_consensus.cc:2764] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:26.939289 14633 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.939925 14382 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:26.940145 14287 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.940701 14716 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:26.940858 14382 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:26.941167 14716 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:26.941414 14382 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:26.897617 14254 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:26.913033 14401 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.753s	user 0.011s	sys 0.000s
I20260709 12:25:26.873284 14714 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.870487 14724 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.916363 14327 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:26.918655 14720 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:25:26.901925 14582 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.919504 14663 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:26.919381 14748 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:26.871042 14565 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:25:26.922917 10023 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:26.922848 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:26.876747 14614 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:26.906351 14739 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:25:26.924817  9944 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:26.923527  9839 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 9 to 10, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:26.926445 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73235us
I20260709 12:25:26.927944 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:26.936415  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:26.897783 14685 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:26.943161  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 448070us
W20260709 12:25:26.987608  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 161714us
I20260709 12:25:26.963567 14676 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.937930 14621 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.949723  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:26.951742  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.038422  9940 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:26.971964 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:27.039338 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67444us
I20260709 12:25:26.972194 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:26.973726 14384 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 12:25:26.964470  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:26.980054 14713 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.980926 14716 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.942557 14382 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:26.986410 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:26.943773 14644 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:27.033836 14678 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:26.945075 14639 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:26.945336 14693 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:26.961956 14687 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:27.032832 10022 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:27.032845 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:27.033211  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 96866.9us
I20260709 12:25:26.986464 14327 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:27.060442  9943 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:27.061233 14327 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.062382 14327 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.049307 14621 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.024456 14409 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:26.963948 14297 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:27.041015 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.041661 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 105895us
I20260709 12:25:26.985863 14427 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:25:27.041503  9839 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 5 to 8, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 8 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:27.030347 14614 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.073856 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:26.939949 14633 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.078059  9839 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:27.079397 14633 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.047817 10096 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:27.048172  9946 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:27.028605 14750 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:27.045430  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:26.943395 14579 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:26.985503 14737 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.057890 14644 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:27.087126 14737 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.060096  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.046693 14380 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:27.074407 14716 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.063866 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:27.074252 14297 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.034857  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:27.066449  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.035585 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.074802  9940 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:26.984845 10025 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:27.054044 14678 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:27.079913  9839 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:27.055544 14639 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:27.040016 14676 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.100842 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.068873 14713 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:27.101509 10019 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:27.098843  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:27.097500  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 62726us
I20260709 12:25:27.105463  9946 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:27.108362 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.110802 10090 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:27.113616 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:27.071390 14361 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.115795  9839 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 6 to 11. New cstate: current_term: 11 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:27.114239 10677 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
W20260709 12:25:27.124087 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 60310us
I20260709 12:25:27.125202 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:27.127050 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.127646 10090 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:27.127530 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.128530 10099 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:27.122854  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.137383 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.141410  9839 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 6 to 10, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 10 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:27.142408 10668 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:27.143251  9839 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 10 to 11, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 11 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:27.145017 10019 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:27.069306 14687 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.154460 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.155313 10097 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:27.141467 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.152311 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.152861 14748 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:27.121593 14380 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:27.087302 14644 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.075918 14409 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.171815 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.101406 14678 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.099826 14737 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.159721 14361 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:27.153360 14753 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:27.163064 14636 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:27.163424  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:27.218227  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:27.164625 14401 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:27.168915 14719 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.170078 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.230834 10092 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
W20260709 12:25:27.172243 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.172441 10097 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:27.173766 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.173905 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.174496 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.234364 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.174576 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.235285 10025 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:27.235769 10023 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:27.175534 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.109010 14733 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:27.180984 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:27.181310 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:27.183122 10018 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:27.180449 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:27.187242  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 128691us
I20260709 12:25:27.186023  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:27.148690 14647 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:27.249806 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.141680 14463 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:27.096841 14718 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:27.259262 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:27.246891 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.247120  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61187.4us
I20260709 12:25:27.261562  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:27.236865 10095 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:27.263370 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.249025 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.087869 14750 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.253821 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.273058 10098 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.254911 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.276868 10019 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:27.256596 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.252171 10018 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:27.267122  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:27.270422 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:27.299029 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.299930 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:27.301443 10017 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:27.299894 10024 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 13 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:25:27.260087 10092 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:27.230347 14427 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:27.289491 10016 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:27.254484 14565 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:27.240615 14753 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:27.253005 14575 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:27.252255 14463 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:27.305179 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.307260 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:27.336645 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:27.319799 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.315258 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.320909 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:27.324030 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.275204 14735 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.271929 14724 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:27.171335 14380 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.319010 14361 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.346134 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.253212 14405 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.313261 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:27.353559 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:27.344017 10017 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:27.356154 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:27.343123  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 116903us
I20260709 12:25:27.319581 14738 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.318845 14431 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.366144 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:27.385661 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72499.5us
I20260709 12:25:27.385877 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.387151 10021 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:27.260637 14639 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:27.347724 14620 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.393205 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:27.396363 14755 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:27.359328 14686 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:27.400578 14254 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:27.351148 14762 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.436971 10024 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:27.439080 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45959.2us
I20260709 12:25:27.440670 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:27.441062 14757 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:25:27.432171 14724 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:27.442106 14724 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.442788 14633 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:27.444680 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:27.409380 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55930.9us
W20260709 12:25:27.413333  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 174575us
I20260709 12:25:27.319404 14427 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.445887  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:27.385010 14738 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.370484 14361 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.392176 14431 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:27.446076 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:27.448398 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:27.460047 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:27.466626 14686 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:27.467149 14686 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.467525 14686 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:27.437826 14770 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:25:27.410231 14620 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:27.473660 14753 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:27.474745 14714 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.475000 14750 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:27.477597 14409 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.465412 10090 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Committing config change with OpId 5.5: config changed from index 3 to 5, 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:27.480346 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.433528 14642 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.493165  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:27.518831  9940 raft_consensus.cc:2368] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:27.497838 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:27.479590 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3143) took 1105 ms. Trace:
I20260709 12:25:27.520948 10094 rpcz_store.cc:276] 0709 12:25:26.374328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.374455 (+   127us) service_pool.cc:224] Handling call
0709 12:25:26.374771 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:26.869299 (+494528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:26.869342 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:26.869405 (+    63us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:25:26.869465 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.478827 (+609362us) spinlock_profiling.cc:243] Waited 2.35 ms on lock 0x7b180037af20. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419
0709 12:25:27.479344 (+   517us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b5400162454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:27.479396 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:27.479528 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18673,"raft.queue_time_us":9928,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":7983,"spinlock_wait_cycles":1119226624}
W20260709 12:25:27.446661 14731 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.003s
I20260709 12:25:27.457451 14738 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.451527 14630 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:27.489990 14759 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:25:27.522944  9991 leader_election.cc:400] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:27.550071  9991 leader_election.cc:403] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:27.471141 14431 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.472254 14361 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.491520 14687 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.478320 14557 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:27.499873 14639 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.471638 14751 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:27.550730 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.479631 14565 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:27.511019 14760 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:27.492322 14718 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:27.511859 14620 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.507985 14755 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:27.493204 14579 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:25:27.516978 14676 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.456665 14756 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.111s
I20260709 12:25:27.528795 14684 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:27.572631  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 126818us
I20260709 12:25:27.504110 14670 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:25:27.444399 14463 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:27.557402 14690 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.877s	user 0.012s	sys 0.000s
I20260709 12:25:27.602340 14756 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:27.567946 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:27.602886 14756 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:27.604071 14677 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:27.604530 14677 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:27.554800 14735 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.605559 14677 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.606210 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3145) took 1111 ms. Trace:
I20260709 12:25:27.606642 10096 rpcz_store.cc:276] 0709 12:25:26.494487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.494664 (+   177us) service_pool.cc:224] Handling call
0709 12:25:27.606018 (+1111354us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54002fbf54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:27.606157 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211891584}
I20260709 12:25:27.608136  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 891) took 1105 ms. Trace:
I20260709 12:25:27.608788 10021 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:27.609915 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3043) took 1336 ms. Trace:
I20260709 12:25:27.610348 10016 rpcz_store.cc:276] 0709 12:25:26.273673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.276159 (+  2486us) service_pool.cc:224] Handling call
0709 12:25:27.609755 (+1333596us) spinlock_profiling.cc:243] Waited 978 ms on lock 0x7b54001f8e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:27.609855 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2249822976}
I20260709 12:25:27.570892 13965 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:27.569180 14751 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:27.590641  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 200838us
I20260709 12:25:27.608547  9944 rpcz_store.cc:276] 0709 12:25:26.502698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:26.510489 (+  7791us) service_pool.cc:224] Handling call
0709 12:25:26.510821 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.031760 (+520939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.607906 (+576146us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b540003eb54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:27.608073 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":951731584}
I20260709 12:25:27.498159 14765 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 12:25:27.615824 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.616377  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:27.616461 10095 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:27.617770 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.617995 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:27.615151 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.551424 14731 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.562825 14642 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:27.618647 10099 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.615275 10094 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:27.620639 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:27.622649  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.625520  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.579998 14724 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:27.632005  9944 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:27.630184 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.586859 14684 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:27.637343 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:27.567512 14557 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:27.634812 14614 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.554s	user 0.012s	sys 0.000s
I20260709 12:25:27.640223 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:27.627097  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.574622 14401 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:25:27.584301 14755 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:27.643402  9943 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:27.708276 10096 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:27.712634  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.712793 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:27.708109  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:27.713637 10020 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:27.713876  9949 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:27.710503 10098 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:27.612506 14709 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:27.603812 14756 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:27.727499  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:27.647047 14742 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.760s	user 0.005s	sys 0.006s
I20260709 12:25:27.733376 10016 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:27.733521  9947 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:25:27.741741  9944 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.637619 14432 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:27.711464 14463 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.602353 14690 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:27.722958 14327 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.000s
W20260709 12:25:27.819466 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 75072.3us
I20260709 12:25:27.821995 10095 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
W20260709 12:25:27.826391 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.767998 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:27.768738 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:27.771705  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:27.827294 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.827776  9947 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:27.638512 14380 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.127s
I20260709 12:25:27.778652  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:27.779026 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:27.829869 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 50927.5us
I20260709 12:25:27.830880 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:27.779646  9948 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:27.833134 14254 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:27.833811 14254 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:27.634112 13965 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.835127 14409 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:27.835145 14777 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:27.835942 13965 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:27.835832 14409 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:27.837136 14732 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.092s	user 0.000s	sys 0.013s
W20260709 12:25:27.838166 14776 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:25:27.838060 14409 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:27.748832 14405 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.833916 14735 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:27.768594 14432 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:27.711738 14614 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.836088 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:27.850232 10016 raft_consensus.cc:2376] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10. Re-sending same reply.
I20260709 12:25:27.841389 10671 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 9: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:27.844185  9944 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:27.852583  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:27.860827  9949 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:27.868996 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:27.872393 10672 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:27.639952 14642 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:27.852186 14621 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.778s	user 0.008s	sys 0.004s
W20260709 12:25:27.849824 14767 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:27.883042 10016 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:27.853581 14432 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:27.768039 14742 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.891562 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:27.891812 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.892190 10016 raft_consensus.cc:2376] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12. Re-sending same reply.
I20260709 12:25:27.892373 10021 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:27.893026 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.893170 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.894114 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:27.894178 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.897109 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:27.899164 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.900702 10021 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:27.900352 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:27.876401 14405 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:27.913270 10023 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
W20260709 12:25:27.918560 14775 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 12:25:27.920367 14778 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
W20260709 12:25:27.922091 14678 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.703s	user 0.006s	sys 0.006s
W20260709 12:25:27.923218 14774 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:25:27.839076 14755 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.785369 14751 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:27.928568 14751 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:27.932144 14771 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 12:25:27.936362 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.937913 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.945432  9949 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:27.945773  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:27.949788 14382 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:27.950083 14751 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:27.951774 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.953254 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:27.953855 10092 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:27.954039 14713 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.829s	user 0.008s	sys 0.002s
I20260709 12:25:27.957758 14579 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:27.958271 14579 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:27.959167 14579 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:27.961683 14731 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.962205 14756 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:27.963021  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 368343us
W20260709 12:25:27.966847  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 350526us
I20260709 12:25:27.968451  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:27.968436  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.969295  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:27.969954 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 190996us
I20260709 12:25:27.971007 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.971781 10016 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:27.968132  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:27.976779 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:27.972509 14640 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:27.976800  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:27.980096 10099 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:27.977835 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:27.974709 14579 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:27.983937 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.981387 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:27.915128 14405 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:27.989635  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.990525  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:27.972543 14676 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:25:27.998777 14750 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:25:27.999266 14614 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.999616 14724 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:28.088507 14724 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:27.999951 14746 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:28.005330  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:28.006569  9944 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:28.007302 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:27.931262 14557 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:27.975401 14716 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.874s	user 0.009s	sys 0.002s
I20260709 12:25:27.999637 14401 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:27.973906 14678 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.098382 10017 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 11. (term mismatch)
I20260709 12:25:28.098891 10017 pending_rounds.cc:85] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 10
I20260709 12:25:28.099162 10017 pending_rounds.cc:107] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:25:28.099457 10017 raft_consensus.cc:2889] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:27.971653 14713 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:28.104344  9945 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.027963 14431 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.001065 14640 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:28.033026 14686 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.001s
I20260709 12:25:28.036407 14677 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.038023  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:28.001129 14035 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:25:28.039250 14773 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 12:25:28.121093 10097 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:28.040467 14780 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:28.041157 14405 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.123411 14780 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:28.124498 14431 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.043666 14742 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.044292 14685 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:28.048981 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:28.125969  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 112295us
I20260709 12:25:28.126436 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:28.127014 14035 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:28.127240 10090 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:28.128125 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:28.129117 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:28.130041 10090 raft_consensus.cc:2368] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:28.048695 14361 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:28.052047  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.054519 14297 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.939s	user 0.007s	sys 0.002s
I20260709 12:25:28.132009 14297 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.132463 14361 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.133260 14686 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.134183 14297 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.134567 10670 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:28.135471 14686 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.052095 14432 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.136762 10673 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:28.137866 14432 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.138185 14677 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:28.057353 14738 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.502s	user 0.005s	sys 0.002s
I20260709 12:25:28.061394 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:28.008121 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.065279 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:28.140363 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 132350us
W20260709 12:25:28.065832  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 96610.8us
I20260709 12:25:28.141700 10090 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:28.067355 14693 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:28.068125  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100069us
W20260709 12:25:28.068288 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61216.8us
I20260709 12:25:28.068137 14579 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:28.150354  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.151408 10095 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:25:28.074568 14690 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:27.975368 14777 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:25:28.026629 14781 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:28.161391 14781 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:28.162339 14781 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.165444 14427 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.165165 14693 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.167873 14614 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:28.169437 14693 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.169777 14382 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.154387 14738 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.173301 14738 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.086511 10092 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:28.094296 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:28.176527 10099 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:28.097657 14287 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.097980 14678 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.180960 10099 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:28.122941  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1845) took 1070 ms. Trace:
I20260709 12:25:28.124306 14780 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.139300  9943 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.142184  9912 leader_election.cc:400] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12.
I20260709 12:25:28.185453  9912 leader_election.cc:403] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:25:28.066902 14761 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.142565 10091 raft_consensus.cc:2368] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:28.144117  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.152541 10095 pending_rounds.cc:85] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 7
I20260709 12:25:28.182765  9941 rpcz_store.cc:276] 0709 12:25:27.053048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.053242 (+   194us) service_pool.cc:224] Handling call
0709 12:25:27.053602 (+   360us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:27.282539 (+228937us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:27.378236 (+ 95697us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.507829 (+129593us) log.cc:844] Serialized 14 byte log entry
0709 12:25:27.956617 (+448788us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b1800466ea0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:27.956816 (+   199us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:27.957481 (+   665us) log.cc:844] Serialized 52 byte log entry
0709 12:25:27.957647 (+   166us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:27.964153 (+  6506us) log.cc:844] Serialized 14 byte log entry
0709 12:25:28.037419 (+ 73266us) spinlock_profiling.cc:243] Waited 5.59 ms on lock 0x7b1800466ea0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:28.038540 (+  1121us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:28.065060 (+ 26520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.065151 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.122681 (+ 57530us) raft_consensus.cc:1685] finished
0709 12:25:28.122707 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.122875 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103100,"prepare.queue_time_us":105943,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":113,"raft.queue_time_us":588487,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":356,"spinlock_wait_cycles":95174272,"thread_start_us":448711,"threads_started":3,"wal-append.queue_time_us":208008}
W20260709 12:25:28.072808 14772 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.351s	user 0.003s	sys 0.000s
I20260709 12:25:28.153867  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:28.159713 14640 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:28.176934  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:28.178375 10092 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:28.178654  9946 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:28.188599 10095 pending_rounds.cc:107] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:25:28.191218  9949 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:28.192374  9942 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:28.199496  9944 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:28.202869  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:28.203083  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59056.5us
I20260709 12:25:28.203871  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.205021 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:28.170292 14382 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.206688 10095 raft_consensus.cc:2889] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:28.211695 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:28.207273  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:28.154719 14685 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:28.190202 14633 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.005s	sys 0.031s
I20260709 12:25:28.210678 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:28.217684 10676 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:28.122980 14684 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.175596 14432 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 12:25:28.220129  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 78140.7us
I20260709 12:25:28.220430 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:28.235615 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:28.236563 10023 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:28.237691 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:28.240154 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:28.196528 14401 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:28.244606 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:28.227394 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:28.248351 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:28.217100 14633 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.005s	sys 0.032s
I20260709 12:25:28.258440 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:28.259493 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:28.260603 10025 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:28.260990 10019 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:28.261941 10023 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:28.262550 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:28.263260 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:28.264408  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.214952 14579 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:28.214960 14685 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.268235 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:28.268952  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:28.216133 14687 raft_consensus.cc:2764] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:28.266469 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:28.245842 14750 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.379853 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:28.380496 10018 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.227859 14765 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12.
I20260709 12:25:28.375855 10090 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:28.383186 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:28.385599 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1944) took 1203 ms. Trace:
I20260709 12:25:28.178134 14724 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.387732 10022 rpcz_store.cc:276] 0709 12:25:27.182248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.234273 (+ 52025us) service_pool.cc:224] Handling call
0709 12:25:27.234779 (+   506us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.258268 (+ 23489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.258314 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.272619 (+ 14305us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.445933 (+173314us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:27.445978 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.446055 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:28.071903 (+625848us) spinlock_profiling.cc:243] Waited 11.2 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:28.373745 (+301842us) raft_consensus.cc:1685] finished
0709 12:25:28.373769 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:28.385513 (+ 11744us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14443,"raft.queue_time_us":23621,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":10842,"spinlock_wait_cycles":25732480,"thread_start_us":111191,"threads_started":1,"wal-append.queue_time_us":283696}
I20260709 12:25:28.394932 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:28.256246 14760 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:28.227250 14654 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.150s
I20260709 12:25:28.244185 14560 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.141s
I20260709 12:25:28.406534 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:28.412861 10672 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:28.408696 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:28.373605 14685 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.413859 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:28.418082 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:28.384660 14579 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:28.261358 14401 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:28.407832 14639 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.840s	user 0.007s	sys 0.004s
W20260709 12:25:28.431806  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 163076us
I20260709 12:25:28.433169  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:28.444165 10018 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:28.410971 14751 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.443776 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:28.449273 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:28.394294 14750 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.461653  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1847) took 1404 ms. Trace:
I20260709 12:25:28.462347 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:28.462093  9945 rpcz_store.cc:276] 0709 12:25:27.057175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.057313 (+   138us) service_pool.cc:224] Handling call
0709 12:25:27.057653 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.111295 (+1053642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.461391 (+350096us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540018a6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:28.461590 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2397611392}
I20260709 12:25:28.446792 14617 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.457623 14693 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:28.499570 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:28.502645  9945 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.503028 10021 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:28.492903 14620 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.908s	user 0.011s	sys 0.000s
I20260709 12:25:28.504025 10093 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:28.490707 14678 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:28.511521 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:28.512027 10021 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:28.466377 14751 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.505752 14678 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:28.516057 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 114218us
I20260709 12:25:28.411289 14760 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.507786 14620 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.502668 14617 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.530048 10018 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:28.539750 10021 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:25:28.515374 14382 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.541863 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:28.508054 14431 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 12:25:28.527277 14621 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 12:25:28.549046 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:28.553743 10674 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:28.556450  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:28.526340 14678 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.555430 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:28.511034 14579 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:28.555231  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61428.6us
W20260709 12:25:28.564069 14805 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:25:28.566025 14801 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:25:28.569056 14804 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:25:28.571185 14803 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:25:28.573400 14802 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 12:25:28.562790 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:28.576290 14794 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.549s	user 0.003s	sys 0.000s
I20260709 12:25:28.579435 14750 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.580039 14750 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:28.580382 14750 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:28.580736 14750 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.583772 14709 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.583850  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:28.584420 14760 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.555930 14686 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:28.585458 14686 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:28.583276 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:28.586460 14633 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:28.586951 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:28.587232 14654 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:28.587642 14654 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:28.588027 14746 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:28.588301 14431 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.588382 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:28.588771 14409 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:28.589167 14431 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:28.589236 14690 raft_consensus.cc:2764] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:28.592118 14751 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.550386 14724 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:25:28.593123 14724 consensus_queue.cc:1243] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:25:28.593680  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:28.559942 14799 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:25:28.595052  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:28.554531 14797 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:28.596132  9940 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:28.596499 14797 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:28.597062 14797 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.597522 14677 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.597841 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.598039 14677 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.584471 14709 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.586114 14684 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.586741 14636 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.592183 14401 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.603928  9941 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.573079 14380 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:28.618904 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:28.590727  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.604790  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:28.623987 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:28.630393 10090 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:28.631959 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:28.632766 10095 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.625089 10091 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:28.542613 14432 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:25:28.630550  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:28.635205 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:28.648192  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:28.648895  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44223.9us
I20260709 12:25:28.649901  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:28.651294  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:28.629056 14686 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:28.655474 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67168.8us
I20260709 12:25:28.655917 14686 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.656327 14686 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.656250 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:28.657083 14431 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:28.657828 14431 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:28.658152 14431 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.658468 14431 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.588529 14654 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.661972 14297 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:28.554353 14790 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:25:28.662616 14297 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:28.663705 14287 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:28.664156 14297 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.669806  9990 leader_election.cc:400] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:28.670110 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:28.671743 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:28.672055 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:28.630614 14810 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.091s	user 0.003s	sys 0.000s
I20260709 12:25:28.672891 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:28.676023 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:28.588426 14746 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:28.589195 14409 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:28.678354 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:28.635774 14809 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.051s	user 0.001s	sys 0.001s
I20260709 12:25:28.681409  9990 leader_election.cc:400] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13.
I20260709 12:25:28.681252 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:28.681957 10672 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:28.690524 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:28.691715 10095 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:28.693770 10095 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: [12.6-12.6]
W20260709 12:25:28.767788  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79074.6us
I20260709 12:25:28.771549 10090 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:28.667780 14751 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:28.666994 14684 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:28.586495 14640 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 12:25:28.676959 14753 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:28.684828 14621 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:28.630461 14636 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.694941 14401 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:28.791867 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 739) took 1740 ms. Trace:
I20260709 12:25:28.802793 10020 rpcz_store.cc:276] 0709 12:25:27.051747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.051888 (+   141us) service_pool.cc:224] Handling call
0709 12:25:27.052244 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:27.714022 (+661778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.791631 (+1077609us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400137ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:28.791811 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1520684928}
I20260709 12:25:28.803292  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:28.803980  9943 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:28.805459 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.693298 14731 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 12:25:28.792990 14760 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:28.812819 10091 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:28.783694 14643 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:28.788359 14582 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:28.809828 14560 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:28.808408 14620 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.810712 14709 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:28.807291 14751 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:25:28.815346 14678 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.810283 14690 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.021s
I20260709 12:25:28.806569 14401 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:28.845582 14820 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:25:28.847784 14636 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:28.831722 14560 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.810057 14405 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 12:25:28.851254  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:28.852926  9948 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:28.853406  9945 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:28.830713 14582 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.854296 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.810127 14409 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:28.849876 14756 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.887s	user 0.012s	sys 0.000s
W20260709 12:25:28.867606 14822 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
W20260709 12:25:28.868187 14808 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.338s	user 0.003s	sys 0.000s
I20260709 12:25:28.858618 14719 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:28.869602 14636 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:28.870182 14636 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:28.816730 14746 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.870514 14636 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:28.871141 14746 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.867547 14409 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:28.875666 14760 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:28.875922 14401 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.876704 14760 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:28.851897 14560 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:28.831354 14643 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.851222 14762 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:28.894699  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 115295us
I20260709 12:25:28.901403 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 921) took 1177 ms. Trace:
I20260709 12:25:28.901901 10673 rpcz_store.cc:276] 0709 12:25:27.723839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.724527 (+   688us) service_pool.cc:224] Handling call
0709 12:25:28.901206 (+1176679us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b54005b95d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:28.901331 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":712265600}
I20260709 12:25:28.904343 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 747) took 1472 ms. Trace:
I20260709 12:25:28.904817 10017 rpcz_store.cc:276] 0709 12:25:27.432384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.435744 (+  3360us) service_pool.cc:224] Handling call
0709 12:25:27.436028 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.099967 (+663939us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:28.904048 (+804081us) spinlock_profiling.cc:243] Waited 662 ms on lock 0x7b5400021054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:28.904253 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1522367872}
I20260709 12:25:28.905915 14678 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:28.908411 14686 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.896765 14762 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:28.911509  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:28.916050 14643 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.917045 14685 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.916038  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.923511  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:28.926119  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:28.931861  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:28.935644 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3146) took 1830 ms (client timeout 2000 ms). Trace:
I20260709 12:25:28.943205 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:28.936141 10093 rpcz_store.cc:269] 0709 12:25:27.105411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.105987 (+   576us) service_pool.cc:224] Handling call
0709 12:25:28.935475 (+1829488us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b540028fd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:28.935583 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2964682624}
I20260709 12:25:28.933511  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.955327  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:28.960240  9942 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:28.928826 14678 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:28.929095 14716 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.928689 14731 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:28.930667 14557 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:28.967152  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:28.968665 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:28.934000 14579 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:28.974310 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:28.974978 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:28.951673 14751 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:28.979130  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 274251us
I20260709 12:25:28.895282 14750 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:28.985785  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:28.986523  9948 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:28.958945 14684 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:25:28.968369 14753 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:28.970295 14831 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:29.010715 14750 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:28.932071 14762 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 3, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:28.983103 14557 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:29.014724 14633 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:29.014899 14431 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.003959 14753 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.037273 14753 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.037837 14753 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:29.041080 14797 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.021576 14297 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:28.982383 14678 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:29.045584 14678 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:29.045995 14678 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:28.968225 14620 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:25:28.976483 14749 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:25:28.977922 14731 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.034060 14382 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 12:25:29.034674 14685 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:28.976493 14716 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.085368 14557 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.045094 14621 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:29.071005 14760 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.092089 14427 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:29.092526 14427 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:29.092888 14760 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.093338 14427 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:29.095969 10097 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 755) took 1925 ms (client timeout 1899 ms). Trace:
W20260709 12:25:29.096418 10097 rpcz_store.cc:269] 0709 12:25:27.170253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.248725 (+ 78472us) service_pool.cc:224] Handling call
0709 12:25:29.095772 (+1847047us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b5400163854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:29.095896 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1925088128}
I20260709 12:25:29.100104 10096 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:29.100765 14617 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.072157 14382 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:29.106734 14642 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.112118 14716 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.114197 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 768) took 1282 ms. Trace:
I20260709 12:25:29.114634 10096 rpcz_store.cc:276] 0709 12:25:27.832209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.837799 (+  5590us) service_pool.cc:224] Handling call
0709 12:25:29.099778 (+1261979us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.100505 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.102458 (+  1953us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400163850. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:29.114125 (+ 11667us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2901554560}
I20260709 12:25:29.023903 14831 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:29.001170 14751 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:25:29.084457 14431 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.114938 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:29.052567  9948 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:25:29.122800 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:29.185957  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 222001us
I20260709 12:25:29.187476 14829 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.188076 14835 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:29.031608 14633 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:29.083895 14731 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.149784 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:29.150112 14719 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:29.083935 14797 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:29.188002 14829 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.191188 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:29.094528 14427 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:29.193732 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.194422 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:29.208137 10017 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:29.196983 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:29.209179 10021 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:29.153312 14831 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:29.197103 10640 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:29.139068 10672 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:29.210896 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:29.212090  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:29.203580 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53919.9us
I20260709 12:25:29.219269 10098 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:29.085476 14636 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.192220 14685 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:29.192242 14753 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:29.219551 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:29.244457 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 271207us
I20260709 12:25:29.184247 14678 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.206022 14431 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:29.185593 14560 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.205415 14731 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:29.250547  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:29.220957 14812 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:29.267840 10021 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:29.269519 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:29.270640  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:29.223639 14582 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:29.220417  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 236571us
I20260709 12:25:29.255421  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:29.272425  9949 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:29.272531  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:29.212672 14751 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.223888 14686 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.191918 14835 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.277429 10017 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
W20260709 12:25:29.278787  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79575.8us
I20260709 12:25:29.252948 14797 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:29.211860 14719 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.309693 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:29.255941 14431 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:29.313557  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 346498us
I20260709 12:25:29.314632  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:29.312275  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:29.289692 14807 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.766s	user 0.004s	sys 0.000s
I20260709 12:25:29.319542 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:29.319895 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:29.321117  9942 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:29.249289 14685 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.249852 14753 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.324854 10677 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:29.335477 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:29.336220 10677 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:29.328035 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:29.328620  9949 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:29.332221  9946 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:29.346951 10017 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:29.359575 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:29.269809 14731 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.362746  9949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:29.392095  9949 raft_consensus.cc:606] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:29.392614  9949 raft_consensus.cc:614] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:29.320815 14431 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:29.321879 14620 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:29.302414 14751 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.256318 14633 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.409368 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:29.420564 10021 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:29.354063 14752 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:29.429795  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 217806us
I20260709 12:25:29.431686  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:29.432440 10017 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:29.363226 14749 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.114s
I20260709 12:25:29.480132 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:29.539762 10671 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:29.423785 14642 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:25:29.512058 10673 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:29.433835 14837 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:25:29.480415 14633 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:29.438261 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.521541 14719 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.522982 14685 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.523743 14829 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.524559 14636 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:29.524994 14839 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
I20260709 12:25:29.569634 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3062) took 1331 ms. Trace:
I20260709 12:25:29.527659 14751 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.507424 14557 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:29.528695 14845 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:29.529587 14731 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.530030 14835 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.531690 14560 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:29.535197 14678 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.535425  9949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:29.547713 14850 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.598407  9949 raft_consensus.cc:606] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:29.599000  9949 raft_consensus.cc:614] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:29.538590 10677 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:29.362485 14670 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:29.554749 10672 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:29.599782  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 277290us
W20260709 12:25:29.562527 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.567938 14685 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:29.570677 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:29.601874 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.569819 10019 pending_rounds.cc:85] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 6
I20260709 12:25:29.442296 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:29.602813 10019 pending_rounds.cc:107] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:25:29.570086 10018 rpcz_store.cc:276] 0709 12:25:28.238129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.448978 (+210849us) service_pool.cc:224] Handling call
0709 12:25:29.569457 (+1120479us) spinlock_profiling.cc:243] Waited 78.3 ms on lock 0x7b54001f7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:29.569577 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180000000}
I20260709 12:25:29.603207 10019 raft_consensus.cc:2889] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:29.569773 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:29.573249 14409 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.578250 14842 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:29.578313 14833 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:29.586020 14832 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.618s	user 0.002s	sys 0.000s
I20260709 12:25:29.555459 14819 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:29.603173 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 160976us
I20260709 12:25:29.578863 14713 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.668187 14713 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.668612 14654 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:29.617728 14409 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.672746 14409 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:29.674881 14361 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:29.675431 14361 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:29.677008 14361 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.678697 14833 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.680137 14753 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.616458 14427 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:29.681939 14427 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:29.685606 14797 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:29.605569 14670 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:29.659655 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:29.690737 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.667568 14463 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:29.684373 14427 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.692083 14361 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:29.719314 10640 leader_election.cc:336] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.871s (SENT)
I20260709 12:25:29.619783 14842 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.614833 14560 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:29.612064 14845 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:29.681763 14833 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.727249 14463 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.710294 14670 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:29.728971 14831 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.670460 14713 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:29.739707 14560 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:29.742975 14792 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.664s	user 0.004s	sys 0.001s
I20260709 12:25:29.610993 14850 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:29.611245 14685 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:29.746860 14795 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.705s	user 0.000s	sys 0.004s
I20260709 12:25:29.762938 14842 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.789228 14831 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.790346 14850 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:29.746646 14845 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:29.820544  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 219899us
I20260709 12:25:29.822914  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:29.824785  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:29.830068  9949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:29.831449  9949 raft_consensus.cc:606] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:29.832229  9949 raft_consensus.cc:614] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:29.851996  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:29.854681  9949 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:29.856981  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:29.859419  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 427658us
I20260709 12:25:29.859639 14878 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:29.866035 14878 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:29.879920  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:29.887202  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:29.889333  9949 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:29.890005  9948 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:29.894270  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:29.936197 10022 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:29.938076 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:29.876871 14878 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:29.967671 14751 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:25:29.968004 10020 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.004s	sys 0.037s
W20260709 12:25:29.968493 10020 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.037s
W20260709 12:25:29.969935 10017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.000s	sys 0.019s
W20260709 12:25:29.970466 10017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.001s	sys 0.019s
I20260709 12:25:29.982380  9946 raft_consensus.cc:2368] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:29.980026  9943 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:25:29.995370  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.001s	sys 0.017s
W20260709 12:25:29.995642 14836 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.029s	sys 0.058s
W20260709 12:25:29.995982  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.001s	sys 0.017s
W20260709 12:25:29.996279 14836 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.030s	sys 0.058s
I20260709 12:25:29.986788 14751 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:29.755935 14677 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:29.998289  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 725826us
I20260709 12:25:30.018739 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:29.734122 14431 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.028654  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:30.025097 14713 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:30.027158  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.031628 14713 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.033686  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.034765  9949 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:30.035687  9940 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.036211 14560 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:30.038115 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.039654  9946 raft_consensus.cc:2410] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:25:30.021348 14620 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.007s	sys 0.046s
I20260709 12:25:30.043771 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:30.045233 14755 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:30.049593 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:30.050565 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 919) took 2334 ms. Trace:
I20260709 12:25:30.050771 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:30.052053 10018 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:30.051316 10668 rpcz_store.cc:276] 0709 12:25:27.716541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.716908 (+   367us) service_pool.cc:224] Handling call
0709 12:25:27.717413 (+   505us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:27.717893 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:27.717938 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:27.718558 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 12:25:27.887374 (+168816us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:27.887407 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:27.887463 (+    56us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:29.744546 (+1857083us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b2400329828. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a809d 00007f4e090da10e 00007f4e090f188c 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221
0709 12:25:30.049803 (+305257us) raft_consensus.cc:1685] finished
0709 12:25:30.049845 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.050238 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":950,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":89530752,"thread_start_us":168095,"threads_started":1,"wal-append.queue_time_us":292833}
I20260709 12:25:30.054075 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:30.031972 14819 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:30.060806  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.050642 10024 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:30.055790 14685 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:30.055145 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.073714  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.050s	sys 0.064s
W20260709 12:25:30.090054 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.090709 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.091584 14677 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:30.094195 14677 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.092125 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:30.091298 10022 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:30.107264  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.051s	sys 0.064s
I20260709 12:25:30.065244 14620 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:30.064168 14751 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.112671  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.115212 14677 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:30.115556  9946 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:30.119295 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:30.120347 10022 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.120067 14620 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.123945 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:30.125403 10065 leader_election.cc:400] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:30.125833 10065 leader_election.cc:403] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:30.126082 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 772) took 1009 ms. Trace:
W20260709 12:25:30.127368  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 261161us
I20260709 12:25:30.128598  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:30.128782 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:30.129902 10021 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
W20260709 12:25:30.080219 14709 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.013s	user 0.007s	sys 0.047s
W20260709 12:25:30.130884 14709 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.147s	user 0.008s	sys 0.048s
I20260709 12:25:30.131940 14709 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:30.132428 14746 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.132634  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.133210 14709 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:30.133606  9948 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:25:30.133668 14709 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:30.129902 14871 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:30.125461 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:30.136140 10016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 761) took 2243 ms (client timeout 1871 ms). Trace:
I20260709 12:25:30.143764  9940 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:30.046343 14755 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.135939 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:30.141863  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 819442us
W20260709 12:25:30.142859 10016 rpcz_store.cc:269] 0709 12:25:27.892894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.895755 (+  2861us) service_pool.cc:224] Handling call
0709 12:25:30.135685 (+2239930us) spinlock_profiling.cc:243] Waited 69.3 ms on lock 0x7b54001f6654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:30.135900 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":159386496}
I20260709 12:25:30.151015 10023 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:30.146783  9943 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:30.092355 14738 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:30.156800 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 781) took 1031 ms. Trace:
I20260709 12:25:30.162842 10023 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:30.164954 10023 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:30.159513 14819 raft_consensus.cc:2764] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:30.166889  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.167016 14709 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.167624 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:30.167959  9949 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:30.168825 14755 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:30.169134  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.171298 13965 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.172204 13965 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.172622 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.174808 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 770) took 1063 ms. Trace:
I20260709 12:25:30.175204 13965 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:30.175277 10020 rpcz_store.cc:276] 0709 12:25:29.111826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.111958 (+   132us) service_pool.cc:224] Handling call
0709 12:25:29.112429 (+   471us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:29.112835 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.112864 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:29.113593 (+   729us) log.cc:844] Serialized 78 byte log entry
0709 12:25:29.311985 (+198392us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:29.545846 (+233861us) spinlock_profiling.cc:243] Waited 9.92 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:29.969175 (+423329us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.020344 (+ 51169us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.068324 (+ 47980us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.113219 (+ 44895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.156612 (+ 43393us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.159573 (+  2961us) raft_consensus.cc:1685] finished
0709 12:25:30.159587 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.174719 (+ 15132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51196,"prepare.queue_time_us":145746,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":3648,"raft.queue_time_us":4864,"raft.run_cpu_time_us":378,"raft.run_wall_time_us":2445,"spinlock_wait_cycles":31316608,"thread_start_us":763654,"threads_started":2,"wal-append.queue_time_us":412043}
I20260709 12:25:30.177909 14677 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:30.178614 14677 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:30.180166 14677 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.182116 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:30.171343 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52662.6us
I20260709 12:25:30.126523 10025 rpcz_store.cc:276] 0709 12:25:29.116822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.120493 (+  3671us) service_pool.cc:224] Handling call
0709 12:25:29.192973 (+ 72480us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:29.204712 (+ 11739us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:29.204759 (+    47us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:29.242880 (+ 38121us) log.cc:844] Serialized 78 byte log entry
0709 12:25:29.619676 (+376796us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:30.016046 (+396370us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.109216 (+ 93170us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.125230 (+ 16014us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.125512 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.125625 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.125705 (+    80us) raft_consensus.cc:1685] finished
0709 12:25:30.125717 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.125894 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2708,"prepare.queue_time_us":386991,"prepare.run_cpu_time_us":180,"prepare.run_wall_time_us":14435,"raft.queue_time_us":48605,"raft.run_cpu_time_us":494,"raft.run_wall_time_us":26091,"spinlock_wait_cycles":108872448,"thread_start_us":318399,"threads_started":1,"wal-append.queue_time_us":492703}
I20260709 12:25:30.166728  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:30.175441  9947 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:30.187909 14731 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.188647 14731 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.178897  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:30.203917 10022 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:30.190970 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:30.206125  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.168846 14709 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.210031  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:30.190579 14654 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.521s	user 0.007s	sys 0.012s
I20260709 12:25:30.191803 14731 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:30.227351  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80594.1us
I20260709 12:25:30.227542 10022 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:30.228195 10016 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:30.228360  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.228667 14870 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:30.229883 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.231468 14755 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.232126 14755 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.232607 14755 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.233263 14642 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:30.233989 14642 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.235003 14831 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:30.236301  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.237097  9941 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:30.237597 14361 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.009s
I20260709 12:25:30.237598 14642 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:30.238785 10676 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:30.239975  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.241400  9941 raft_consensus.cc:2368] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15.
W20260709 12:25:30.242563  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.242393 14617 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:30.244920 14686 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:30.174811 14409 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.256206 14409 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.256727 14409 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.185370 13965 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 12:25:30.258852 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.238804  9943 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:30.260567 14762 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.261179  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.262051  9943 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:30.160004 10017 rpcz_store.cc:276] 0709 12:25:29.125201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.441047 (+315846us) service_pool.cc:224] Handling call
0709 12:25:29.441316 (+   269us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:29.441616 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:29.441872 (+   256us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.971188 (+529316us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.972089 (+   901us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.972303 (+   214us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:29.972347 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.972455 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.136597 (+164142us) raft_consensus.cc:1685] finished
0709 12:25:30.136616 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.156739 (+ 20123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34612,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":4486,"raft.queue_time_us":75804,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":197,"thread_start_us":528791,"threads_started":1,"wal-append.queue_time_us":493215}
I20260709 12:25:30.219777  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.222082  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.238312 14361 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:30.254662 14829 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.255540 14873 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:30.272341 10024 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:30.186686 14670 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.275099  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.275213 10025 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:30.262972 14749 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:30.252583 14686 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:30.276568  9946 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:30.251022 14617 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:30.285065 14670 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:30.285894 14686 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.287379 14686 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.288072 14831 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:30.288735 14831 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.289222 14831 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.267395 14654 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:30.295001 14654 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:30.295073 14841 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:30.295938 14841 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.296324 14654 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.298835 14734 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:30.299439 10021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:25:30.299511 14734 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.300362 10021 raft_consensus.cc:606] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:30.302384 14829 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.305130 14829 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:30.305757 14873 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.308583 14873 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:30.309397 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:30.311228  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.316922  9940 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:30.319121 14382 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.319902 14382 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.322928 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:30.324683 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66150.6us
I20260709 12:25:30.325850 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:30.331526 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:30.332538 10677 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:30.338126 14746 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:30.278695 14749 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:30.340378 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.340668 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:30.343343 10668 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:30.377103 14749 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.379024 14829 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:30.379199  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.379503 14829 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:30.380046  9948 raft_consensus.cc:2410] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 12:25:30.380558 14874 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.106s	user 0.002s	sys 0.000s
I20260709 12:25:30.381140  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.381371  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.382520  9946 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:30.382802  9991 leader_election.cc:400] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15.
I20260709 12:25:30.383112  9991 leader_election.cc:403] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:30.384770 14621 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:30.385396 14621 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:30.386396 14621 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.387534 14617 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.389443 14617 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.393033 10671 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:30.394284 14361 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.397068 14361 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:30.298282 14841 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:30.399185 14841 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 15
W20260709 12:25:30.400707 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60421.4us
I20260709 12:25:30.397320 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:30.399292 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 73669.9us
I20260709 12:25:30.379899 14829 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:30.398957  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 86482.6us
I20260709 12:25:30.382515  9948 raft_consensus.cc:2410] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:30.419960 10671 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:30.302428 14734 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:30.475208  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.448529 14620 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:30.491346 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.379801 14670 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.091s
W20260709 12:25:30.486680 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.487701 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:30.488668 14872 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.476418 14746 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.500903  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.477144 14642 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:30.498441  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.493467  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.484392 14749 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.517355  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:30.508136 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.510197 10670 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 918) took 2837 ms (client timeout 1991 ms). Trace:
I20260709 12:25:30.523146 14872 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:30.523573 10670 rpcz_store.cc:269] 0709 12:25:27.672976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.710785 (+ 37809us) service_pool.cc:224] Handling call
0709 12:25:30.509587 (+2798802us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540050a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:30.509918 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":967870848}
I20260709 12:25:30.510399 10675 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:30.524718  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1867) took 1934 ms. Trace:
I20260709 12:25:30.525148  9945 rpcz_store.cc:276] 0709 12:25:28.590242 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.591349 (+  1107us) service_pool.cc:224] Handling call
0709 12:25:28.591663 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:28.853800 (+262137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.524217 (+1670417us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400207454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:30.524536 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":601531648}
I20260709 12:25:30.525769 14872 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:30.526558 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:30.520951 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.522626 14633 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.529065 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:30.529511 14633 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:30.530571 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.532124 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 769) took 1420 ms. Trace:
I20260709 12:25:30.532392 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 1986) took 1592 ms. Trace:
I20260709 12:25:30.532477 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.534185 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.535351 14361 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:30.506469  9946 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:30.519222 14734 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.538297 14734 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.491336 14620 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.532802 10099 rpcz_store.cc:276] 0709 12:25:28.940064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.940267 (+   203us) service_pool.cc:224] Handling call
0709 12:25:28.940681 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.941163 (+   482us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:29.069237 (+128074us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.291123 (+221886us) log.cc:844] Serialized 14 byte log entry
0709 12:25:29.291284 (+   161us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.292445 (+  1161us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.292579 (+   134us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:30.530951 (+1238372us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b18003949c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:25:30.531404 (+   453us) log.cc:844] Serialized 14 byte log entry
0709 12:25:30.531766 (+   362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.531946 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.532026 (+    80us) raft_consensus.cc:1685] finished
0709 12:25:30.532043 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.532221 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30057,"prepare.queue_time_us":64366,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":6265,"raft.queue_time_us":24614,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":18368,"spinlock_wait_cycles":25947648,"thread_start_us":312245,"threads_started":2,"wal-append.queue_time_us":349380}
I20260709 12:25:30.526547 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 982) took 1614 ms. Trace:
I20260709 12:25:30.526760  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:30.544152 10675 rpcz_store.cc:276] 0709 12:25:28.912354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.912486 (+   132us) service_pool.cc:224] Handling call
0709 12:25:30.509881 (+1597395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.525149 (+ 15268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.526339 (+  1190us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540050a5d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:30.526470 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3672715136}
I20260709 12:25:30.536089 14361 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:30.555263  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 109437us
I20260709 12:25:30.502068 14709 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:30.540433 10019 rpcz_store.cc:276] 0709 12:25:29.111509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.115466 (+  3957us) service_pool.cc:224] Handling call
0709 12:25:29.569607 (+454141us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:30.005237 (+435630us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.007548 (+  2311us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:30.058266 (+ 50718us) log.cc:844] Serialized 52 byte log entry
0709 12:25:30.089571 (+ 31305us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:25:30.089616 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.089741 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.293904 (+204163us) raft_consensus.cc:1685] finished
0709 12:25:30.293939 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.531730 (+237791us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54001f7050. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:30.531947 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":312907,"raft.queue_time_us":366103,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":304,"spinlock_wait_cycles":873018752,"thread_start_us":30796,"threads_started":1,"wal-append.queue_time_us":163068}
I20260709 12:25:30.562142  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.545843 14620 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.565845  9948 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:30.573264  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.576653 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.579403  9945 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:30.580565  9945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:30.581220  9945 raft_consensus.cc:606] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 9 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:30.597469 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.598299 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.599534 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.600581 14829 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.602035 10098 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:30.603161 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:30.603883 10091 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:30.609876 14780 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:30.615108 14782 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:30.615971 14782 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:30.638039  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.640601 14684 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:30.641603 14684 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.647684 14742 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:30.648397 14742 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:30.648985 14742 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.623898 14709 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.658257 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:30.659762 14887 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:25:30.661005 10099 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:30.662699 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:30.664510 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.575042 14746 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.644903 14684 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:30.674302 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.677386 14761 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.681749 14463 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:30.686276 14849 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.168s	user 0.008s	sys 0.002s
I20260709 12:25:30.670078 10095 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:30.697758 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:30.711124 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:30.711869  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 151092us
I20260709 12:25:30.694204 14709 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:30.693439 14782 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:30.775079 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 287546us
W20260709 12:25:30.776265  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 65142.6us
W20260709 12:25:30.780274 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 142998us
I20260709 12:25:30.782630  9947 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:30.787866  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.702996 14831 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.774577 10677 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:30.794342 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.700155 14677 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:30.775751 14755 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.543s	user 0.012s	sys 0.002s
I20260709 12:25:30.796543 10093 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:30.723299 14734 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:30.793466 10094 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:30.721743 14633 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:30.801788 14684 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:30.801507 14677 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.802641 14684 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.804589 14831 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:30.805078 14684 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:30.805757 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.806572 10095 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:30.807180 14621 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:25:30.807858 14409 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.551s	user 0.011s	sys 0.001s
I20260709 12:25:30.808290 14409 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.812253  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.812871 14761 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.815408 14761 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:30.816443 14903 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:30.817054 14903 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.817497 14903 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:30.820478 14746 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:30.826787 14297 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:25:30.827368 14762 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.566s	user 0.009s	sys 0.003s
W20260709 12:25:30.835462  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47789us
I20260709 12:25:30.839841 10099 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:25:30.840595 10090 raft_consensus.cc:2368] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12.
W20260709 12:25:30.847254  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 204694us
I20260709 12:25:30.848498 10091 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:30.832376 14409 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:30.848153 10098 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:30.852672  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.857214 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.847920 14831 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:30.855796 14782 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:30.842535 14677 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:30.866457 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:30.861207  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:30.867658 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 73585.1us
I20260709 12:25:30.869429  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.831693 14755 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.867148 10094 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:30.874799 14760 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:30.875260 14760 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:30.875962  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:30.876308 14760 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:30.876897  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:30.877416 14760 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.879359 14617 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:30.879758 14617 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.880122 14617 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.881649 14755 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:30.882515 10097 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:30.882704 14873 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:30.883399 10095 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:30.884995 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 1962) took 1049 ms. Trace:
W20260709 12:25:30.886010 14881 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.662s	user 0.000s	sys 0.009s
I20260709 12:25:30.868538 10098 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:30.886530 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 772) took 2782 ms. Trace:
I20260709 12:25:30.886977 10092 rpcz_store.cc:276] 0709 12:25:28.067725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.204929 (+137204us) service_pool.cc:224] Handling call
0709 12:25:28.205324 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:28.206211 (+   887us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:28.227400 (+ 21189us) log.cc:844] Serialized 14 byte log entry
0709 12:25:28.572169 (+344769us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b1800425860. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0d42a58f 00007f4e06469571 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd
0709 12:25:28.918245 (+346076us) spinlock_profiling.cc:243] Waited 7.45 ms on lock 0x7b1800425860. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:25:29.434854 (+516609us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:29.475837 (+ 40983us) log.cc:844] Serialized 26 byte log entry
0709 12:25:29.516431 (+ 40594us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:29.516484 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:29.535576 (+ 19092us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.685171 (+1149595us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:30.830919 (+145748us) raft_consensus.cc:1685] finished
0709 12:25:30.830940 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.849718 (+ 18778us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2731,"prepare.queue_time_us":38042,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":44,"raft.queue_time_us":17945,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":27497,"spinlock_wait_cycles":132851200,"thread_start_us":431839,"threads_started":1,"wal-append.queue_time_us":1258079}
I20260709 12:25:30.883337 14873 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:30.886117 10094 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:30.900789  9947 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:30.868602 10642 leader_election.cc:336] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.760s (SENT)
I20260709 12:25:30.902796 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:30.904743  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 67691.6us
I20260709 12:25:30.906987 14831 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:30.909999 14831 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.885485 10023 rpcz_store.cc:276] 0709 12:25:29.835401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.165945 (+330544us) service_pool.cc:224] Handling call
0709 12:25:30.166340 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:30.169459 (+  3119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.169489 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:30.170034 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:25:30.544890 (+374856us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:30.544927 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.545015 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:30.884733 (+339718us) raft_consensus.cc:1685] finished
0709 12:25:30.884755 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:30.884923 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":115,"thread_start_us":374330,"threads_started":1,"wal-append.queue_time_us":627511}
I20260709 12:25:30.911732  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:30.918222 14762 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:30.920213 14762 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:30.920703 14762 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:30.931494  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 378424us
I20260709 12:25:30.956497  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:30.962971  9947 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:30.932292 14871 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:30.970834 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 119226us
I20260709 12:25:30.935117 14361 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:30.931466 14297 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:25:30.872272 14782 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:30.977455  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:30.980728  9947 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:30.973927 14871 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:30.981740 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 113213us
I20260709 12:25:30.972755 14621 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:30.962965 14761 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:25:30.993091 10093 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:30.995991 14782 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:31.002892 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 134350us
I20260709 12:25:30.999837  9945 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:31.003345  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:31.004714  9945 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:31.016045 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:31.018451  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:30.984817 14829 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:31.026384 14904 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:30.988947 14871 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:31.076406  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48463.5us
W20260709 12:25:31.079102 14686 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.791s	user 0.015s	sys 0.005s
I20260709 12:25:31.079514 14904 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.004331 14742 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:31.082051 14742 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:31.082109 14904 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:31.045284  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.049033 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.022008 14761 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:25:31.083053 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:31.097039  9947 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:25:31.083609 14901 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:31.083982 14906 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:31.084036 14621 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 9.11, Last appended by leader: 11, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.086458 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 922) took 1767 ms. Trace:
I20260709 12:25:31.086901  9945 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:31.075318 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.772s (SENT) [suppressed 17 similar messages]
I20260709 12:25:31.026160 14617 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.103353 10093 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:31.102977 14812 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:31.100247 14762 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:31.127519 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 111624us
I20260709 12:25:31.128165 14686 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:31.128566 14686 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:31.128798 14762 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:31.128952 14686 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:31.132088  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:25:31.131994 14910 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:25:31.127668 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 112434us
I20260709 12:25:31.134974  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:31.110520 10673 rpcz_store.cc:276] 0709 12:25:29.318867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:29.319432 (+   565us) service_pool.cc:224] Handling call
0709 12:25:29.352438 (+ 33006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:29.541641 (+189203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.085980 (+1544339us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54007b0cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:31.086292 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355574528}
I20260709 12:25:31.141512  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:31.144264 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:31.140959  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:31.147135 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:31.147795 10098 raft_consensus.cc:2376] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12. Re-sending same reply.
I20260709 12:25:31.148669 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:31.149955 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:31.151635 14845 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.162s	user 0.004s	sys 0.012s
W20260709 12:25:31.152061 14845 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.165s	user 0.004s	sys 0.012s
W20260709 12:25:31.153116 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.153386 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:31.154253 10098 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.163350 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:31.107415 14297 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:31.172698  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:31.174273 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.170441 10099 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:31.170034 14762 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:31.165053 14904 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:31.187855 14904 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:31.189245 14904 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:31.171627 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:31.159745 14742 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:31.204262 14431 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:31.206142  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 107641us
I20260709 12:25:31.197633 10097 raft_consensus.cc:2376] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14. Re-sending same reply.
I20260709 12:25:31.175015 14297 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:31.211383 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:31.211782  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.212321 14742 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.219616 14742 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.097460 14873 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:31.213186 10099 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:31.221599 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:31.223533 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59163.9us
I20260709 12:25:31.224833 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:31.231259 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:31.236258 10098 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:31.237609 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:31.239395 10093 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:25:31.239804 10093 pending_rounds.cc:85] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 9
I20260709 12:25:31.240051 10093 pending_rounds.cc:107] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:31.240304 10093 raft_consensus.cc:2889] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:31.241868 14876 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.145s	user 0.002s	sys 0.001s
I20260709 12:25:31.247390 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:31.210065 14871 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:31.244452 14841 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.844s	user 0.012s	sys 0.000s
I20260709 12:25:31.254468 14617 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:31.256489 14863 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.226s	user 0.010s	sys 0.005s
I20260709 12:25:31.258013 14841 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15.
I20260709 12:25:31.258816 10097 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:31.250804 14620 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.621s	user 0.003s	sys 0.008s
I20260709 12:25:31.264542 14620 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.267490  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:31.268786  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:31.270092 14780 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.643s	user 0.007s	sys 0.000s
I20260709 12:25:31.265471 14297 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.273465 14297 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:31.270246  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:31.280234  9940 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:31.283838  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:31.288079  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:31.289270  9948 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.292048  9947 raft_consensus.cc:2368] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
W20260709 12:25:31.294870 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 219579us
I20260709 12:25:31.293517 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:31.297147  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.297430 14617 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.297606  9944 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:31.303001 10091 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:31.297482 10090 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:31.279544 14919 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:31.291767 14751 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.312505  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:31.288071 14620 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:31.313344  9949 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:31.321038 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.291582 14760 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:25:31.338101 10097 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:31.343890 14871 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:31.349326 14903 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.531s	user 0.005s	sys 0.006s
I20260709 12:25:31.335137 14751 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.298060 14617 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:31.352530 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 104119us
I20260709 12:25:31.352674 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:31.356228 14693 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.356756 14693 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.349936 14903 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.368055 14617 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.319886 14686 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.375511 14780 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:31.377249 10097 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:25:31.378265  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.382355 10090 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:31.394032 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:31.400112  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.403683 14781 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:31.404357 14781 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.413944 14781 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:31.415326  9944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1869) took 2824 ms (client timeout 2184 ms). Trace:
W20260709 12:25:31.415827  9944 rpcz_store.cc:269] 0709 12:25:28.590872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:28.593431 (+  2559us) service_pool.cc:224] Handling call
0709 12:25:31.415048 (+2821617us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b540042de54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:31.415228 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6177097984}
I20260709 12:25:31.417073  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:31.418376  9944 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:25:31.487030 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 98398.9us
W20260709 12:25:31.495168 10642 leader_election.cc:336] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.899s (SENT)
W20260709 12:25:31.496073 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.504920 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:31.507433  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:31.515823 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:31.494011 14919 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.418680 14760 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.114s
W20260709 12:25:31.530133 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 140715us
I20260709 12:25:31.523914 14677 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:31.532330 14677 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.532709 14677 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.522513 14870 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.535502 14870 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.548628 14910 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.549240 14910 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.560302 14910 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:31.536178 10093 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:31.564499  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:31.574165  9949 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:31.575837 10677 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:31.578500  9948 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:31.579761 10673 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:31.503758 14755 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:25:31.561580 14870 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:31.584069  9949 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:31.561664 14686 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:31.568886 14751 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.598271 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.599412 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:31.600013 10677 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:31.610384 10677 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:31.596225 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.616391 10669 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:31.616864 14871 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:31.617518 14871 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 9 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:31.633035 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 129584us
I20260709 12:25:31.631158 14751 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.649020 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.648842 14677 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.649276 10669 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:31.649860 10677 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:31.652976 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:31.634575 14856 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 2.116s	user 0.009s	sys 0.009s
I20260709 12:25:31.661043 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:31.661022 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:31.670791  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 149260us
I20260709 12:25:31.651746 14677 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.593370 14919 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.674639  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:31.677321  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:31.677585 10023 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:25:31.680768 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.681734 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.688040 10669 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:31.696609 10671 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:31.691039 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.677052 14780 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:31.699554 10670 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:31.685539 14297 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:31.701673 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.709116 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.704633 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.705124 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.711199 10024 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:31.712071 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.711563 10675 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:31.720336 10676 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:31.720608 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:31.721217 10677 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:31.672302 14751 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.726089 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.726248 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.699976 14903 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.710629 14297 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.730561  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:31.705683 14670 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:31.713205 14780 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:31.731596 14760 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:31.726799 10023 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:25:31.735158 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.731671 14297 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:31.736272 10674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 916) took 4058 ms (client timeout 2111 ms). Trace:
I20260709 12:25:31.724786 14560 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.743928 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.745160 10670 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:31.741480 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:31.739915 10674 rpcz_store.cc:269] 0709 12:25:27.631413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:27.631544 (+   131us) service_pool.cc:224] Handling call
0709 12:25:31.652694 (+4021150us) spinlock_profiling.cc:243] Waited 915 ms on lock 0x7b54007f2e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:31.689248 (+ 36554us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2105412864}
I20260709 12:25:31.736756 14760 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:31.737771 14405 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:31.731695  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:31.767637 14035 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:31.768388 14035 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:31.709817 14871 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.783896 14871 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:31.783795 14297 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.786172 14382 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.742224 14670 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:31.788755 14932 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:25:31.789021 14670 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:31.759465 14560 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.796594 14919 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:31.798730 14405 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.801517 14405 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.804030 14713 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.806295  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:31.813100  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2083) took 1579 ms (client timeout 1980 ms). Trace:
W20260709 12:25:31.813606  9943 rpcz_store.cc:269] 0709 12:25:30.233680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.339486 (+105806us) service_pool.cc:224] Handling call
0709 12:25:31.806107 (+1466621us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:31.812991 (+  6884us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4664576}
I20260709 12:25:31.821882 14751 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.822801 14819 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.823514 14819 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.823740 14780 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.826190 14819 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.779410 14035 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:31.781832  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:31.830859 14361 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.763649 14760 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:31.832104 14930 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:25:31.835556 14896 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.818s	user 0.002s	sys 0.000s
I20260709 12:25:31.835964 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.837175 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.838402 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.773145 14380 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:25:31.840577 14847 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.903394 10670 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:31.842391  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:31.844447 14684 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.904901 14684 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.844832 14405 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.846077 14716 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:31.846616 14864 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.847324 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:31.924666 10024 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:31.849990  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:31.850919  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.863613 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.927739 10022 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:31.595909  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:31.871812  9948 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:31.866124  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:31.794221 14560 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.873672 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.897753 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.931370 10676 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 7.
I20260709 12:25:31.841322  9945 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:31.896939 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.894822  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:31.920661 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:31.920812  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:31.883450 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:31.926251  9946 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:31.934799 10016 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:31.946382 14790 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:31.923694 14405 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:31.947038 14790 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.952594 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.953423 10022 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:31.954445 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.955588 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.958525 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:31.952317 10018 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 12:25:31.959851 10677 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:31.962169 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:31.965649 14684 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:31.967851 14405 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:31.963411 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.974491  9945 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:25:31.894308 14361 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.992338 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.982429 10020 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:31.969796 14760 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:31.996045 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:31.996877 10018 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:31.998035 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:32.000535 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:32.004119  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2104) took 1113 ms. Trace:
I20260709 12:25:32.004405 10018 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:32.004602  9945 rpcz_store.cc:276] 0709 12:25:30.870348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.999308 (+128960us) service_pool.cc:224] Handling call
0709 12:25:30.999609 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.840514 (+840905us) spinlock_profiling.cc:243] Waited 699 ms on lock 0x7b580091e2d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:25:31.983880 (+143366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:31.984075 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1606802816}
I20260709 12:25:32.010579 10016 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:32.013566 10676 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:32.015697 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:31.924118 14847 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.016791 14380 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:32.017256 14750 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:32.024070 14361 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:32.028030 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:32.036782 10020 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:32.003222 14919 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:32.032802 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:31.957078 14790 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:32.040354 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 51663.3us
I20260709 12:25:32.040896 10016 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.041826 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:32.042930 14911 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:32.046170 13965 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:31.949286 14864 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.045534 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:32.050199 10676 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:31.951440 14716 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.047084 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:32.045002 14847 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.053298 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:32.047798 14911 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.060123 10017 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:32.062706 14361 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:25:32.064347 14848 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.507s	user 0.028s	sys 0.042s
I20260709 12:25:32.064395 10016 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:32.064908 14848 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.537s	user 0.028s	sys 0.042s
I20260709 12:25:32.065828 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:32.066304 14760 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.066540 10016 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:32.060736 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:32.073252 10676 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:32.073926 13965 raft_consensus.cc:2764] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:25:32.047606 14686 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:25:32.054857 14750 raft_consensus.cc:2764] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:32.067474 14380 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:32.068027 14911 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.090783 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:32.084775 10016 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:32.085595 10024 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:32.096349 10016 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:32.117909 10016 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:32.117909 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:32.081878 14380 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:32.129619 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.126976 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:32.140136 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:32.142325 14427 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:32.144539 10023 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 11. Preceding OpId from leader: term: 11 index: 12. (index mismatch)
I20260709 12:25:32.145346 10675 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:32.146250 10024 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:32.132658 14380 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:32.149780 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 89121.6us
I20260709 12:25:32.152387 10677 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
W20260709 12:25:32.153177  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 134861us
W20260709 12:25:32.147117 10062 connection.cc:256] RPC call timeout handler was delayed by -2.4296980939998321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21110000000000004s for a response
I20260709 12:25:32.155609 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:32.156181 10024 raft_consensus.cc:2376] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 11. Re-sending same reply.
I20260709 12:25:32.142817 14427 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.162300 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:32.149008 14910 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:32.159343 14919 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:25:32.157837 14297 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.154874 14716 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.161988 14427 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.185345 10020 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:32.188155 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:32.195874 10020 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:25:32.188576 14941 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:25:32.200536 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:32.167387 14910 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:32.201691  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.202319 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.203280 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.204011 14560 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:32.204762 14780 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:32.205330 14780 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.206502 14297 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.161810 14750 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:25:32.193377 14864 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:32.188920 14942 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:25:32.222945 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:32.206859 14713 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:32.229056  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:32.208391 14361 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:32.229846  9941 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:32.230484  9945 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:32.200574  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:32.231199 10024 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:32.231772 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:32.228646  9943 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:32.232347 10017 raft_consensus.cc:2376] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 13. Re-sending same reply.
I20260709 12:25:32.247578 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 991) took 1893 ms. Trace:
I20260709 12:25:32.252717  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.257774  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.258008 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:32.253374  9942 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:32.259286 10017 raft_consensus.cc:2376] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 14. Re-sending same reply.
I20260709 12:25:32.206570 14871 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.208173 14780 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.266279  9943 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:32.206998 14409 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:25:32.207050 14693 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:32.270143 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.207048 14297 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.248714 10672 rpcz_store.cc:276] 0709 12:25:30.255845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.283240 (+ 27395us) service_pool.cc:224] Handling call
0709 12:25:30.283701 (+   461us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.284229 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:30.563928 (+279699us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.860494 (+1296566us) spinlock_profiling.cc:243] Waited 4.78 ms on lock 0x7b18001128a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:25:32.148745 (+288251us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.148803 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:32.148861 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.149168 (+   307us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.149325 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":277868,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":45129,"raft.queue_time_us":1542,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":4540,"spinlock_wait_cycles":10989440,"thread_start_us":565393,"threads_started":2,"wal-append.queue_time_us":1612925}
W20260709 12:25:32.282056 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 77676.7us
I20260709 12:25:32.291049  9942 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:25:32.286501 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 124468us
I20260709 12:25:32.265810 14713 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:32.383600  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.386369  9943 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:32.384342  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.388535  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:32.282347 14560 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:32.271617 14871 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.405913 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3167) took 1765 ms. Trace:
I20260709 12:25:32.406533 10095 rpcz_store.cc:276] 0709 12:25:30.640429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.884523 (+244094us) service_pool.cc:224] Handling call
0709 12:25:30.993667 (+109144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:30.994059 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:30.994095 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:30.994144 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:30.994185 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:30.994241 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.405638 (+1411397us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540028fd50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:32.405821 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":62940,"raft.queue_time_us":9754,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6249,"spinlock_wait_cycles":250063744}
I20260709 12:25:32.391360 14910 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:32.439191 14560 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.442246 14361 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.442826 14361 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.443931 14297 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.453271  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.454365  9946 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
W20260709 12:25:32.461431  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.445039 14953 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.450700 14910 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:32.488168  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.489182  9946 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:32.475142 14361 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:32.506423  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.508097 14614 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.508937 14614 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.509228  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.475725 14911 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.522219 14954 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:32.522888 14954 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.527088  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.527889  9946 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:32.476681 14713 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.532855 14380 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.538582 14427 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.548902 14427 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:32.549221 14871 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.542222  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.551590 14693 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:32.551915 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:32.552035 14693 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.552608 14693 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.552759 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:32.553246 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:32.553407 10677 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:32.544301 14819 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:32.555274 14819 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:32.555847 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.557402 14819 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:32.551630 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:32.558499 10672 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:32.549593 14409 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.561192  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.552078 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:32.566018  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:32.485142 14953 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:32.569777 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.575734 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.576725 10676 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:25:32.577481 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.581080 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:32.496151 14910 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.588094 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.590917 10091 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:32.544118 14382 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.546169 14380 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:32.594857 10091 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:32.607483 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.605144 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.593863 10090 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:32.610476 10091 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:32.574083 14780 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:32.614008 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:32.614763 10093 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.545289 14957 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.620139 10677 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:32.620476 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:32.621491 10099 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:32.616626 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.623323 10094 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:32.617286 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:32.624827 10096 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:32.618243 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.618613 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.626003 10091 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:32.627079 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:32.628974 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82428.9us
W20260709 12:25:32.636993 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 118499us
I20260709 12:25:32.640756 10095 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:32.652334 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.653075 10094 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:32.654649 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.616451 14780 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.670298 14560 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:32.671098 10018 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:32.672132 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.672933 14560 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.674865  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.675597 14819 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:32.675860  9942 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:32.676173 14819 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.679461 14819 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.673431 14560 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.680584  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:32.686615  9943 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:32.683075 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 791) took 1036 ms. Trace:
W20260709 12:25:32.682781  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 98248us
W20260709 12:25:32.684221  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 841860us
I20260709 12:25:32.653105 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:32.651360 14746 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:32.640856 14327 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.605785 14409 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.619020 14382 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.699314 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 787) took 1059 ms. Trace:
W20260709 12:25:32.700984 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47906.8us
I20260709 12:25:32.710305 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2003) took 2065 ms. Trace:
I20260709 12:25:32.687237 10025 rpcz_store.cc:276] 0709 12:25:31.647620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.658567 (+ 10947us) service_pool.cc:224] Handling call
0709 12:25:31.679127 (+ 20560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.686777 (+  7650us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:31.695649 (+  8872us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.875050 (+179401us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.207697 (+332647us) spinlock_profiling.cc:243] Waited 5.68 ms on lock 0x7b180048afc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:32.682378 (+474681us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b180048afc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:32.682561 (+   183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.682665 (+   104us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:32.682735 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.682829 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.683002 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":781644,"prepare.queue_time_us":346747,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":9008,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9373,"spinlock_wait_cycles":38015616,"thread_start_us":172911,"threads_started":1,"wal-append.queue_time_us":265242}
I20260709 12:25:32.694257 14746 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:32.701658 14777 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:25:32.701951 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.709287 14614 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:32.709959 14954 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:32.710088 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.688882 14841 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:32.762029 14841 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:32.762552 14841 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:32.759272 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.711469 14910 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:32.715772 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:32.680838 14780 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:32.718525 10092 rpcz_store.cc:276] 0709 12:25:30.648123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:30.955672 (+307549us) service_pool.cc:224] Handling call
0709 12:25:31.003845 (+ 48173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.077112 (+ 73267us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:25:31.139938 (+ 62826us) log.cc:844] Serialized 14 byte log entry
0709 12:25:31.783593 (+643655us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b1800423880. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:25:32.081764 (+298171us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.081827 (+    63us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:32.081872 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.081934 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.710116 (+628182us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b5400094550. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:32.710243 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47152,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":609,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":25076,"spinlock_wait_cycles":88536192,"thread_start_us":297309,"threads_started":1,"wal-append.queue_time_us":995262}
I20260709 12:25:32.722266 14908 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:32.717144 10019 rpcz_store.cc:276] 0709 12:25:31.639487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:31.639624 (+   137us) service_pool.cc:224] Handling call
0709 12:25:31.665489 (+ 25865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:31.665782 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:31.739092 (+ 73310us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.698942 (+959850us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:32.699004 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:32.699056 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:32.699125 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:32.699254 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":761455,"prepare.queue_time_us":20000,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":50,"raft.queue_time_us":770401,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":80,"thread_start_us":247227,"threads_started":2,"wal-append.queue_time_us":277818}
I20260709 12:25:32.783156 10672 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:32.734724 14409 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:32.738276  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 172348us
I20260709 12:25:32.739094 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:32.739250 14911 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:32.722982 14327 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.740408 14953 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:32.740862  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 540311us
I20260709 12:25:32.741353 14864 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:32.741950  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50128.6us
I20260709 12:25:32.738350 14405 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:32.747561 14746 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.651454 14957 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.805884 14746 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:32.806782 10090 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:32.807287 14957 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.685348 14960 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:25:32.762032 14382 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:32.810967 10095 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.812064 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:32.812721 10016 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.752027 14959 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:32.815490 14959 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:32.816004 14959 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.753551 10093 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:32.758042 10096 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:32.811728 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:32.819836  9991 leader_election.cc:400] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:32.820192  9991 leader_election.cc:403] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:25:32.777140  9943 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:32.775357 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:32.791960 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 70743.7us
I20260709 12:25:32.798121  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:32.799592 14409 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.005s
I20260709 12:25:32.804610 14327 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.804591 14693 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.811861 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:32.867712 10018 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:32.816545  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.787405 14463 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:32.862149 10095 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:32.865365 10677 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:25:32.869371  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52915.1us
W20260709 12:25:32.869599  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.874377 10016 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
W20260709 12:25:32.879097  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79492us
W20260709 12:25:32.880366 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44814.3us
I20260709 12:25:32.879038 14864 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:32.886749 14953 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:32.886945 14716 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:32.881183 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:32.887374 14953 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.887203 14864 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.888978  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:32.889361 14911 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:32.886792 14617 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:32.889918 14911 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:32.891016 14911 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.891731 14847 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:32.894985  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 96938us
I20260709 12:25:32.887410 14716 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.896404 14716 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.900262  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:32.901180 14684 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:32.879401 14760 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:32.903067 14760 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.903492 14760 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:32.907231 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.880414 14870 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:32.908406 10094 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:32.911762  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:32.878628  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.903388 14035 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:25:32.913820 14967 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:32.914359 14967 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:32.915313 14967 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.915920 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.917132 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.917640 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.918465 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.918982 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.920105 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.921303 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.921914 10093 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:32.922089 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.922703 10099 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:32.820648 14954 consensus_queue.cc:1175] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd983415389444e292b2330ef630850a" responder_term: 11 status { last_received { term: 10 index: 6 } last_committed_idx: 6 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:25:32.925552 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.926438 10092 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:32.927038 14970 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:32.927426 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.927769 14970 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:32.929126 14970 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:32.932754  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.934527  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.936100  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.936802 14953 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.919320 10096 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:32.943135  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.829257 14904 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.121s
I20260709 12:25:32.876240 14463 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.952873 14463 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:32.875321 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 100123us
I20260709 12:25:32.868532 14960 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:32.955830 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:32.956677 14870 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:32.957077 14870 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:32.957433 14870 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:32.958565  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:32.960004 14693 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:32.963271 14684 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:32.965065 14959 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.894330  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:32.971410 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:33.012167  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 117913us
I20260709 12:25:33.014974 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:33.015528  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:33.016218  9942 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:32.977001 14297 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:32.978693 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:32.982076 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:33.017539 10677 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.010982  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:32.972005 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 160391us
I20260709 12:25:32.988173 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:33.018736  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:32.875031  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:32.991607 14969 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:25:32.994287 14870 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.999145 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:33.046854 10090 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:33.047021 10094 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:33.002945  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:33.048483  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45674.7us
I20260709 12:25:33.015591 10095 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:25:33.019232 10676 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:32.990067 14841 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:32.987998 14716 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.009285 14864 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:33.066684 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:33.067519 10097 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:33.068629 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:33.024742  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:33.029214 10018 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
W20260709 12:25:33.038954 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.050447  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:33.012428 14970 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.079295  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:33.080017  9945 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:33.081338 10022 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:33.082304 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:33.011768 14687 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 12:25:33.028065 14693 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:33.125411 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:33.133332 10095 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.050925 14910 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:33.014038 14972 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
W20260709 12:25:33.132027 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.137244 14959 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:33.088299 14847 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:33.131171 14819 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.138768 14752 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:25:33.140359 14716 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:33.017705 14848 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
I20260709 12:25:33.137326 14870 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:33.161273 10677 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:33.162976 10098 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.035s	sys 0.057s
W20260709 12:25:33.163400 10098 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.036s	sys 0.057s
I20260709 12:25:33.149588 10095 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:33.137566 14693 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:33.171682 14693 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:33.086010 14864 raft_consensus.cc:2764] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:33.176321 14864 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:33.176764 14864 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:33.180354 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.180637  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 131173us
I20260709 12:25:33.164853 14297 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:33.190543 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 125515us
I20260709 12:25:33.163612 14910 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:33.193351  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:33.167960 14687 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:33.177687 14847 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:33.178056 14405 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 12:25:33.227203  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:33.271322  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 246672us
I20260709 12:25:33.275678  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:33.278954  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:33.289562  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:33.177687 14864 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:33.306715  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 246470us
I20260709 12:25:33.195330 14297 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.261365 14035 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:33.235366 14560 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:33.287896 14847 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:33.298898 14617 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:33.312537 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.313580 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:33.318296 10018 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.314826 10016 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
W20260709 12:25:33.321002 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 101202us
I20260709 12:25:33.302948 14972 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:33.301172 14819 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:33.327956 10025 raft_consensus.cc:2368] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
W20260709 12:25:33.336246 14978 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:25:33.287098 14687 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.342303 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.344017 10016 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:33.314286 14035 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:33.346989  9942 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:33.327272 14841 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:33.352872 10642 leader_election.cc:336] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.747s (SENT)
I20260709 12:25:33.357048 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:33.338102 14819 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:33.360225 10018 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:33.363229 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.364571 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.365782 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.343096 14864 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.368139 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:33.370383 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.316884 14910 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.348196 14035 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.377296 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:33.351138 14871 raft_consensus.cc:2764] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:33.378748 10016 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
W20260709 12:25:33.370565 14966 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:25:33.365303 14819 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.366304 14986 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:33.422736 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:33.433619 10018 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 12
W20260709 12:25:33.376016 14976 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 12:25:33.445751 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:33.459362 10672 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:33.460579 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.462129 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:33.463271 14979 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:25:33.450137 14297 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:33.466269 14297 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.408146 14968 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.466818 14297 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:33.467985 14982 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:25:33.469475 14991 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.471077 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 118231us
W20260709 12:25:33.471913 10642 leader_election.cc:336] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.772s (SENT)
I20260709 12:25:33.375183 14777 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:33.479244 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 126460us
I20260709 12:25:33.453496 14971 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:25:33.457499 14780 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.678s	user 0.008s	sys 0.000s
I20260709 12:25:33.447219 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.485365 10022 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:33.462167 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:33.450230  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 437112us
I20260709 12:25:33.359797 14560 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:25:33.492311 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:33.498004 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:33.498476 10671 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:33.470127 14991 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:33.508188 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:33.516458 10018 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:33.517824 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.557111 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.557695 10018 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:33.518335 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:33.558238 10016 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:33.478821 14382 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.013s	sys 0.031s
I20260709 12:25:33.491127 14972 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:33.507472 14617 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:33.524076 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:33.525664  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:33.572765  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47198.3us
I20260709 12:25:33.527966 14297 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:33.535537 14560 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:25:33.540450  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.543607 14873 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:33.546988 14973 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.289s	user 0.000s	sys 0.003s
W20260709 12:25:33.547699 14977 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
W20260709 12:25:33.548383 14974 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.291s	user 0.002s	sys 0.000s
I20260709 12:25:33.594110 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3091) took 1545 ms. Trace:
I20260709 12:25:33.595288 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 801) took 1438 ms. Trace:
I20260709 12:25:33.595113 14617 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.595616 10020 rpcz_store.cc:276] 0709 12:25:32.157172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.196686 (+ 39514us) service_pool.cc:224] Handling call
0709 12:25:32.196961 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.715369 (+518408us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:32.722507 (+  7138us) log.cc:844] Serialized 14 byte log entry
0709 12:25:32.902759 (+180252us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b180009f520. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:33.101063 (+198304us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.150170 (+ 49107us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.164733 (+ 14563us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:33.164770 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.164850 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.547153 (+382303us) spinlock_profiling.cc:243] Waited 65 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:33.595066 (+ 47913us) raft_consensus.cc:1685] finished
0709 12:25:33.595088 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.595233 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":518020,"prepare.queue_time_us":9439,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":599519,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":24773,"spinlock_wait_cycles":153998080,"thread_start_us":317979,"threads_started":1,"wal-append.queue_time_us":396205}
I20260709 12:25:33.597673 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:33.598201 10020 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.548802 14990 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:33.551028 14463 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.002s
I20260709 12:25:33.552735 14991 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:33.553244 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81372us
W20260709 12:25:33.604820 10642 leader_election.cc:336] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.776s (SENT)
I20260709 12:25:33.605252 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:33.554170 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46963.7us
I20260709 12:25:33.558501 10674 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:33.567847 10021 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:33.606753 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.482888 14971 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:33.579017  9942 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:33.507253 14780 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:33.609815 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.583978 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.587601 14848 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:33.579998 14984 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.008s
I20260709 12:25:33.582931 14297 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:33.580127 14560 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:33.580391 14873 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:33.588495 14847 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.589973  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:33.568957 14382 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s	user 0.013s	sys 0.031s
I20260709 12:25:33.594556 10023 rpcz_store.cc:276] 0709 12:25:32.048870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.149448 (+100578us) service_pool.cc:224] Handling call
0709 12:25:32.161598 (+ 12150us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.713663 (+552065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.713701 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.714241 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.013891 (+299650us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:33.013931 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.014000 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.546351 (+532351us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:33.593802 (+ 47451us) raft_consensus.cc:1685] finished
0709 12:25:33.593827 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.594047 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":549254,"raft.queue_time_us":550768,"raft.run_cpu_time_us":52,"raft.run_wall_time_us":52,"spinlock_wait_cycles":23314816,"thread_start_us":299274,"threads_started":1,"wal-append.queue_time_us":470835}
I20260709 12:25:33.638146  9948 raft_consensus.cc:2368] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:33.602530 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 802) took 1171 ms. Trace:
I20260709 12:25:33.601236  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:33.640038  9941 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:33.602171 14990 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:33.581637 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.618806 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.621574  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:33.642011 10021 raft_consensus.cc:2376] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 14. Re-sending same reply.
I20260709 12:25:33.642166  9943 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:33.623062  9942 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:33.623848  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:33.644193 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.645009 10018 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:33.624923  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:33.638541 10017 rpcz_store.cc:276] 0709 12:25:32.431641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.434248 (+  2607us) service_pool.cc:224] Handling call
0709 12:25:32.434618 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.735531 (+300913us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.735566 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.736081 (+   515us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.082108 (+346027us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:33.082150 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.082211 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.547710 (+465499us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:33.598694 (+ 50984us) raft_consensus.cc:1685] finished
0709 12:25:33.598714 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:33.602469 (+  3755us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":300481,"raft.queue_time_us":301660,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":85,"spinlock_wait_cycles":29508480,"thread_start_us":335156,"threads_started":1,"wal-append.queue_time_us":442129}
I20260709 12:25:33.627764 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:33.647480 10642 leader_election.cc:336] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.828s (SENT)
I20260709 12:25:33.628616 14908 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.631572 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:33.633055 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.623926  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:33.655864  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 346786us
I20260709 12:25:33.657773  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:33.662366 10020 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:33.616571 14361 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:33.665618 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:33.604271 14463 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.666731  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:33.666610 10017 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:33.672735 10021 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:33.681365  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:33.636750 14297 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.686865  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:33.688843 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78340.3us
I20260709 12:25:33.686774 14560 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.12, Last appended by leader: 12, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.692399 14297 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:33.692903 14560 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:33.674669 14972 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:33.676417 14713 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:33.665215 14962 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:33.706861 14962 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:33.707871 14962 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.699337  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:33.686847 14991 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.722251 14991 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:33.723575 14873 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.724946 14991 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:33.726049 14990 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:33.669255 14908 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.729869  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:33.685341 14984 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:33.743877  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.699945 14716 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:33.684330 14986 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:33.745074  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:33.742302  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:33.685103 14687 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:33.746819 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:33.728667 14988 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:33.724975 14873 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:33.743309 14760 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.839s	user 0.012s	sys 0.001s
I20260709 12:25:33.756354  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:33.757284 14986 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:33.762228 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.684970 14971 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:33.737218 14996 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:33.731837 14713 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:33.751106 14963 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.875s	user 0.003s	sys 0.003s
I20260709 12:25:33.743501 14848 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.747762 14984 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:33.771567 14971 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:33.773573  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:33.766198  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:33.747475 14998 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:25:33.813105  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46991us
I20260709 12:25:33.748462 14560 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:25:33.814141  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:33.747601 14904 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:25:33.765353 14716 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.775780 14405 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:25:33.822803  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49327.6us
I20260709 12:25:33.801141 14995 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:33.824360 14988 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:33.775671 14760 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.816130  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:33.775400 14964 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.804605 14984 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.835049  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:33.803695 14998 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.826843 14841 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:25:33.859541 10642 leader_election.cc:336] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.939s (SENT)
I20260709 12:25:33.854740  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:33.855176  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:33.830315 14405 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:33.860874  9945 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:33.833648 14988 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:33.832577 14847 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:33.870563  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:33.856546 14716 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:33.857720  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:33.775226 14996 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:33.840348 14980 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.854533  9946 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:33.860828 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 98417.7us
I20260709 12:25:33.856074 14998 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:33.858831 15006 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:33.877924  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:33.883252 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:33.884038 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.892005 14684 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:33.892665 14684 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:33.905377 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:33.906380 10099 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:33.893796 14684 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.908917 14684 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:33.840781 14780 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:33.917335  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59694.7us
I20260709 12:25:33.918933  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:33.921444  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:33.850250 14968 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
W20260709 12:25:33.874662 15001 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:25:33.825917 15003 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
W20260709 12:25:33.937906  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60081us
I20260709 12:25:33.858882 14760 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:33.938542 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.929196  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:33.867236 14405 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:33.876271 14988 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:33.950623  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:33.934993 14693 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.763s	user 0.012s	sys 0.000s
I20260709 12:25:33.914336 14687 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:33.989457  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:33.989914 10672 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:25:33.990155  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:33.937705 14617 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.928200 14752 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:33.943805 14463 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:33.968403 14035 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.538s	user 0.003s	sys 0.006s
I20260709 12:25:33.972345 10062 leader_election.cc:400] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:34.017445 10062 leader_election.cc:403] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:33.977670 14998 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:33.979349 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:33.980242 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:33.986707 14693 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:33.958034  9949 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
W20260709 12:25:33.992496 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 136384us
I20260709 12:25:33.914170 14999 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 12:25:33.994917 14996 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:33.879742 15000 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:33.934517 14841 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:33.996539  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 1918) took 1060 ms. Trace:
I20260709 12:25:34.002928 14995 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:33.945113 14864 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.571s	user 0.011s	sys 0.000s
W20260709 12:25:33.997980  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54442.4us
I20260709 12:25:34.008581 14687 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:33.937892 14990 raft_consensus.cc:2764] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:25:34.002285 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:34.005426  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:34.016189  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.051467 10022 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:33.914530 14382 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
I20260709 12:25:34.021389 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:33.963224 14980 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:34.052799 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:34.037700 10676 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:34.022689 14993 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 12:25:34.041852 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 156200us
I20260709 12:25:34.047829  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:33.999357  9941 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:34.044142  9946 rpcz_store.cc:276] 0709 12:25:32.935728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.935839 (+   111us) service_pool.cc:224] Handling call
0709 12:25:33.996275 (+1060436us) spinlock_profiling.cc:243] Waited 918 ms on lock 0x7b540042e854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:33.996464 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2110286720}
I20260709 12:25:34.049846  9944 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:33.928561 14713 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:34.060393  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55096.8us
I20260709 12:25:34.066915  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:34.045287 14841 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:34.078713  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:34.035140 14035 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:34.051165 14687 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.047057 14908 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:34.067902 15006 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:25:34.075511 14970 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.993s	user 0.013s	sys 0.000s
I20260709 12:25:34.069803 14991 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:34.034847 14777 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:25:34.066180 15000 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:34.093299 10022 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
W20260709 12:25:34.082839 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 110680us
I20260709 12:25:34.082933 14841 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:34.081519 14990 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:34.090430 14687 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:34.088336 14760 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:34.053061 14995 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:34.099633 14361 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:34.105849 14819 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.002s
I20260709 12:25:34.122167 15019 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:34.123173 14975 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.932s	user 0.003s	sys 0.003s
I20260709 12:25:34.124373 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 803) took 1692 ms. Trace:
I20260709 12:25:34.124794 14693 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:34.124711 10024 rpcz_store.cc:276] 0709 12:25:32.431963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.434950 (+  2987us) service_pool.cc:224] Handling call
0709 12:25:32.435245 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:32.736956 (+301711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:32.736989 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:32.737444 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.005883 (+268439us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:33.005923 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.006000 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.100877 (+1094877us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:34.124044 (+ 23167us) raft_consensus.cc:1685] finished
0709 12:25:34.124072 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.124298 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":301338,"raft.queue_time_us":302512,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"spinlock_wait_cycles":5253504,"thread_start_us":268038,"threads_started":1,"wal-append.queue_time_us":415777}
I20260709 12:25:34.127000 15000 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:34.128834 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.109789 14990 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:34.129544 14990 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:34.131258 14998 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:34.135484 15007 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:25:34.137548 15016 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:25:34.137778 15012 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
W20260709 12:25:34.138834  9839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 213) took 19909 ms (client timeout 15000 ms). Trace:
I20260709 12:25:34.138887 14035 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:34.135506 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:34.122759 15019 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:34.139424  9839 rpcz_store.cc:269] 0709 12:25:14.167726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:14.173740 (+  6014us) service_pool.cc:224] Handling call
0709 12:25:34.068126 (+19894386us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:27.767565 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:27.831651 (+ 64086us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:27.831701 (+    50us) write_op.cc:435] Acquired schema lock
0709 12:25:27.831720 (+    19us) tablet.cc:662] Decoding operations
0709 12:25:27.833307 (+  1587us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:27.833403 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:27.833434 (+    31us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:25:27.999372 (+165938us) tablet.cc:701] Row locks acquired
0709 12:25:27.999433 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:25:27.999623 (+   190us) write_op.cc:270] Start()
0709 12:25:27.999771 (+   148us) write_op.cc:276] Timestamp: P: 1783599927999555 usec, L: 0
0709 12:25:27.999795 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 12:25:28.000529 (+   734us) log.cc:844] Serialized 8769 byte log entry
0709 12:25:28.831694 (+831165us) op_driver.cc:464] REPLICATION: finished
0709 12:25:29.071968 (+240274us) write_op.cc:301] APPLY: starting
0709 12:25:29.079158 (+  7190us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:29.674016 (+594858us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:29.674112 (+    96us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:30.056886 (+382774us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:30.057483 (+   597us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=44,mrs_lookups=0
0709 12:25:30.057558 (+    75us) write_op.cc:312] APPLY: finished
0709 12:25:30.059320 (+  1762us) log.cc:844] Serialized 196 byte log entry
0709 12:25:30.221558 (+162238us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:30.283264 (+ 61706us) write_op.cc:454] Released schema lock
0709 12:25:30.283353 (+    89us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:30.284082 (+   729us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:30.284121 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2164005,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":197285,"apply.run_cpu_time_us":95688,"apply.run_wall_time_us":1216043,"cfile_cache_hit":90,"cfile_cache_hit_bytes":96131,"num_ops":22,"prepare.queue_time_us":286048,"prepare.run_cpu_time_us":21008,"prepare.run_wall_time_us":540284,"raft.queue_time_us":141461,"raft.run_cpu_time_us":42584,"raft.run_wall_time_us":154555,"replication_time_us":814869,"thread_start_us":625514,"threads_started":5,"wal-append.queue_time_us":707701,"wal-append.run_cpu_time_us":13214,"wal-append.run_wall_time_us":2263673}]]}
W20260709 12:25:34.147291 15014 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:25:34.165484 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3190) took 1127 ms. Trace:
I20260709 12:25:34.110476 15023 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:34.166266 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62652.8us
I20260709 12:25:34.108819 14991 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:34.100533 14972 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:25:34.169616 10091 rpcz_store.cc:276] 0709 12:25:33.038529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.054420 (+ 15891us) service_pool.cc:224] Handling call
0709 12:25:33.054826 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.055102 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.055137 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.990781 (+935644us) spinlock_profiling.cc:243] Waited 48 ms on lock 0x7b2c00115cd8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:25:33.991207 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.081167 (+ 89960us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:34.081879 (+   712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.081959 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:34.165193 (+ 83234us) raft_consensus.cc:1685] finished
0709 12:25:34.165213 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:34.165421 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110321024,"thread_start_us":89554,"threads_started":1,"wal-append.queue_time_us":129354}
I20260709 12:25:34.165863 14405 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:34.180871 10065 leader_election.cc:400] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:34.118499 14361 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.153190 14990 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:34.146231 15020 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.200620 10065 leader_election.cc:403] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 12:25:34.181435 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45996.7us
W20260709 12:25:34.205744 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 153021us
W20260709 12:25:34.208897  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 118807us
I20260709 12:25:34.152542 14687 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:34.210774  9941 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:34.212263 14687 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:34.212433 14990 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:34.172156 14970 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:25:34.172665 15023 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:34.214350 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.214665 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:34.216926 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.120998 14819 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:34.203285 14968 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.203505 14361 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:34.218037 10024 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:34.229632 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:34.230291 10022 raft_consensus.cc:2376] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 14. Re-sending same reply.
I20260709 12:25:34.218950 14970 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.233443 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.239956 10022 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:34.235317 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.240900 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:34.241626  9947 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:34.243731 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 224926us
I20260709 12:25:34.151333 14841 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:34.247041 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.223069 14910 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:34.251152 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:34.253207 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.255188 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.256523 10025 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:34.267915 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:34.268687 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.271014 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.271683 10022 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:34.277324 10023 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: [11.11-11.11]
W20260709 12:25:34.268211 14967 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.311s	user 0.019s	sys 0.005s
I20260709 12:25:34.299714 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:34.300551 10099 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:34.302915 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:34.306358 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:34.309252 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.309854 10099 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:34.314226 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.320220  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:34.325345 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.326191 10093 raft_consensus.cc:2368] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:34.327044 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:34.327638 10093 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:34.328495 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.329010 10093 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:34.282913 10642 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.010s (SENT)
I20260709 12:25:34.287528 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:34.362365  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:34.369290  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49193.6us
I20260709 12:25:34.371414  9990 leader_election.cc:400] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:34.372606  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:34.374969  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:34.424361  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49588.7us
I20260709 12:25:34.426048  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:34.289214 14967 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.343s	user 0.020s	sys 0.005s
I20260709 12:25:34.429482  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:34.462543 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:34.449124 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.463194 10025 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:34.223181 14991 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:34.465253 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.253911 15025 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.176s
I20260709 12:25:34.466063 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:34.466574  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:34.439980 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 220257us
I20260709 12:25:34.466701 10023 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:34.448810 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.468261 10097 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:34.448324  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 75793.5us
I20260709 12:25:34.481688  9949 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:34.482429  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:34.476121  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:34.449193 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.484193 10094 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.485064 14985 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:34.485637 15049 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:34.485688 14760 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:34.486367 14760 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:34.486814 14760 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:34.488842 15027 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.205s	user 0.000s	sys 0.000s
I20260709 12:25:34.423524 15028 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:34.492035 10095 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:34.285462 15015 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.423756 14990 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.434437 10093 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:34.516165 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 233289us
W20260709 12:25:34.610116 10642 leader_election.cc:336] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.028s (SENT)
I20260709 12:25:34.610634 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:34.461848 15006 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:34.448493 14687 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:34.465015 15026 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:34.477722 14693 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:34.466454 14684 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:34.547008 15031 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:25:34.498436 15028 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.539564 14382 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:34.607921 10097 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:34.546512 15019 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:34.586535 14995 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:34.655673 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.691437 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.639212 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.288070 15023 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:34.485744 14985 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:34.679425 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:34.692302 10092 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:34.612560 15006 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.699805 10097 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:25:34.703693 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 454601us
I20260709 12:25:34.630393 14687 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:34.599325 14999 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.087s
I20260709 12:25:34.636406 14873 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:34.678691 14382 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.665257 14405 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:34.640057 14684 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:34.740540 14687 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:34.748284 10019 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:34.748559 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:34.762629 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:34.769660 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:34.765880 10017 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:34.702127 15025 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:34.774933 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 487511us
I20260709 12:25:34.682814 15028 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:34.714326 15006 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:34.730290 14841 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:34.696703 14819 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:34.785790 10017 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:34.755834 14382 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:34.741547 14919 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:34.741431 15029 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.721483 15015 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:25:34.764225 14684 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:34.775568 14777 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.020s	sys 0.014s
I20260709 12:25:34.758323 14910 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:34.811491 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:34.815663 15051 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:25:34.790324 10097 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
W20260709 12:25:34.791496 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 181405us
W20260709 12:25:34.815819 15035 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:25:34.826177 10642 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.051s (SENT)
I20260709 12:25:34.827383 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:34.836851 14910 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:34.844170 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:34.864883 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.788226 15030 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:34.869573 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:34.759574 14873 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:34.856464 15034 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.433s	user 0.000s	sys 0.002s
W20260709 12:25:34.906353 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 106989us
I20260709 12:25:34.832185 15054 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:34.914213  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:34.914423 14841 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:34.914245  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:34.926410  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:34.920933 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94792.5us
I20260709 12:25:34.929939 10097 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:25:34.943338 10642 leader_election.cc:336] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.061s (SENT)
W20260709 12:25:34.949107 10099 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.639s	user 0.010s	sys 0.001s
I20260709 12:25:34.838663 15021 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.075s
W20260709 12:25:34.938431 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46309.4us
I20260709 12:25:34.954227  9946 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:34.943994 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:34.956215 14995 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:34.957208  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 464227us
I20260709 12:25:34.891238 15037 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:34.810561 15029 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:34.963876 10099 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:34.810185 14919 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:34.956426 14617 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:25:34.989516 10641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.500s	user 0.058s	sys 0.077s
W20260709 12:25:35.019948 10023 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:25:35.020488 10023 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:34.989766 15055 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:35.015034 15044 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.394s	user 0.001s	sys 0.000s
I20260709 12:25:34.946702 15054 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:35.063932 10094 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.002s
W20260709 12:25:35.064455 10025 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.601s	user 0.007s	sys 0.006s
I20260709 12:25:35.068744 10094 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:35.069386 10025 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:35.023222 15026 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:35.074661  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 106394us
W20260709 12:25:35.078042 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 150157us
I20260709 12:25:35.023113 14985 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:35.060227 15020 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.847s	user 0.000s	sys 0.012s
W20260709 12:25:35.079380 10642 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.245s (SENT)
I20260709 12:25:35.047601 14962 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:25:35.077414  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 152280us
I20260709 12:25:35.082082 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:35.087450  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:35.090276  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:35.007493 15037 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.111375 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:35.084007 14405 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:25:35.112474 15037 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.114002 14985 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.079924 14986 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:35.085770 15061 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:35.128937 10016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.008s	sys 0.001s
W20260709 12:25:35.139299 10016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.008s	sys 0.002s
I20260709 12:25:35.124114 10092 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:35.089285 15020 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:35.066633 15055 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.141461  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:35.109733 15000 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:35.150333 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 91352.6us
I20260709 12:25:35.117237 15047 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:35.152825  9947 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.005s	sys 0.010s
I20260709 12:25:35.117094 15062 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:35.172127  9947 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.005s	sys 0.010s
W20260709 12:25:35.174844 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 95493.1us
I20260709 12:25:35.139097 15061 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:35.126984 14995 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:35.149837 15048 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:25:35.141103 14991 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:35.171391 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 92746.6us
I20260709 12:25:35.114908 15030 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:25:35.145494 14986 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:35.148378 14750 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:25:35.218258 10642 connection.cc:256] RPC call timeout handler was delayed by -1.0976056340000468s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
I20260709 12:25:35.219493 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:35.213300 15033 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 12:25:35.143267 14985 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:35.174834 14617 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:35.177776 15000 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:35.266995  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2124) took 1645 ms. Trace:
I20260709 12:25:35.218266 15041 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.276150  9943 rpcz_store.cc:276] 0709 12:25:33.622018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.643133 (+ 21115us) service_pool.cc:224] Handling call
0709 12:25:33.665311 (+ 22178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:33.669364 (+  4053us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:33.669411 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:33.669467 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:33.669502 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.266620 (+1597118us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b54001a40d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:35.266739 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.266927 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8683776}
I20260709 12:25:35.225353 15063 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:35.277206 14970 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.033s	user 0.009s	sys 0.004s
W20260709 12:25:35.295722 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80234.7us
I20260709 12:25:35.217679 15062 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:35.277652 15037 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:35.310899 15063 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.284725 15041 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.236310 15030 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 12:25:35.334494  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.313702 15070 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:35.308027 14970 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
W20260709 12:25:35.314990 15065 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
W20260709 12:25:35.316948 14361 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.085s	user 0.010s	sys 0.001s
I20260709 12:25:35.317799 14985 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:35.320909 15004 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.463s	user 0.003s	sys 0.001s
W20260709 12:25:35.321791 14760 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.834s	user 0.010s	sys 0.001s
I20260709 12:25:35.322852 14617 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:35.325352 15053 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.488s	user 0.004s	sys 0.000s
I20260709 12:25:35.326311 10025 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:35.338778 14873 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.234536 15046 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:35.347978 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 125776us
I20260709 12:25:35.224466 14991 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:35.352509  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:35.355845 10024 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:35.311228 14984 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.439s	user 0.011s	sys 0.004s
I20260709 12:25:35.358534 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3192) took 1062 ms. Trace:
I20260709 12:25:35.360630  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 983) took 1782 ms. Trace:
I20260709 12:25:35.390244  9944 rpcz_store.cc:276] 0709 12:25:33.577772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:33.577893 (+   121us) service_pool.cc:224] Handling call
0709 12:25:33.589680 (+ 11787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.066998 (+477318us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.360271 (+1293273us) spinlock_profiling.cc:243] Waited 460 ms on lock 0x7b54000168d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:35.360534 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1057785728}
W20260709 12:25:35.326543 14988 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.330s	user 0.008s	sys 0.004s
I20260709 12:25:35.369081 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:35.371984  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:35.393604  9946 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:35.378774 15074 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.337764 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:35.378333  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:35.340274 14919 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:35.357901 14841 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.388025 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:35.358845 15029 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:35.390197 10091 rpcz_store.cc:276] 0709 12:25:34.296628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.298721 (+  2093us) service_pool.cc:224] Handling call
0709 12:25:34.299164 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.388342 (+ 89178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.388392 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:34.389215 (+   823us) log.cc:844] Serialized 26 byte log entry
0709 12:25:34.402287 (+ 13072us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:34.402350 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:34.402458 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.358290 (+955832us) raft_consensus.cc:1685] finished
0709 12:25:35.358309 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.358471 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":185705,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":144,"thread_start_us":101233,"threads_started":2,"wal-append.queue_time_us":517575}
I20260709 12:25:35.416028 10019 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:35.371960  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:35.402456 14841 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.402976 15081 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:35.394389 14984 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.405834 14967 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:35.406726 15025 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.272524 15061 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:35.408658 15064 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:25:35.409595 15072 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:25:35.408090 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:35.327256 15000 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.410241 15062 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:35.411177 15059 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:25:35.412142  9943 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:25:35.422546  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:35.497936  9943 pending_rounds.cc:85] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 7
W20260709 12:25:35.414378 14687 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.630s	user 0.011s	sys 0.000s
I20260709 12:25:35.424604 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:35.370551 14361 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.393812 14910 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:35.430511  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:35.381526 14760 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.498225  9943 pending_rounds.cc:107] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:25:35.503033  9943 raft_consensus.cc:2889] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:35.422135 15041 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:35.424736 14713 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:35.506067 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:35.507422 10677 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:35.506070 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:35.509483 10018 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:35.415517 15074 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.366274 14995 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:35.511219 10673 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:35.448729 15043 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:35.424482 15029 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:25:35.398412 14988 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.391032 15046 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.544697 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:35.592609 10676 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:25:35.594445 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 169263us
I20260709 12:25:35.430653 14919 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.084s
I20260709 12:25:35.548360 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:35.595515 10677 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.496811 14990 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:35.553416 10675 raft_consensus.cc:2368] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:35.602370 10019 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:35.507882 15061 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:35.509816 15070 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:35.584738 15079 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
W20260709 12:25:35.584903 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 247211us
I20260709 12:25:35.449685 14750 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.092s
W20260709 12:25:35.671031  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:35.540923 15073 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.671255 10673 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:35.587002 15063 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.393090 14986 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.691531 10672 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:35.693296 15086 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.693907 15086 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.696466 15086 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:35.503070 15025 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.668015 14990 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.698879 15025 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:35.699039 10674 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:35.624047 14904 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.700066 14904 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.503741 14687 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.504757 14967 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.702836 14687 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.705017 14967 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.706059 15070 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.700650 14990 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.710798 14904 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.712849 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3172) took 3127 ms. Trace:
I20260709 12:25:35.716471 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:35.718020 10019 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:25:35.718914  9944 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:35.713608 10098 rpcz_store.cc:276] 0709 12:25:32.585100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.585246 (+   146us) service_pool.cc:224] Handling call
0709 12:25:32.585557 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:33.163829 (+578272us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:33.367227 (+203398us) log.cc:844] Serialized 14 byte log entry
0709 12:25:33.729232 (+362005us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:33.773169 (+ 43937us) log.cc:844] Serialized 26 byte log entry
0709 12:25:33.773325 (+   156us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:33.773357 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:33.773426 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:33.849894 (+ 76468us) spinlock_profiling.cc:243] Waited 4.87 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:35.320153 (+1470259us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:35.712482 (+392329us) raft_consensus.cc:1685] finished
0709 12:25:35.712519 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.712749 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29170,"prepare.queue_time_us":385624,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":122743,"raft.run_cpu_time_us":2858,"raft.run_wall_time_us":181462,"spinlock_wait_cycles":15740032,"thread_start_us":1004370,"threads_started":3,"wal-append.queue_time_us":368213}
I20260709 12:25:35.650622 15083 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:35.718137 15059 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:25:35.623134 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:35.653448  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:35.449486 15009 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 12:25:35.542040 15043 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:35.717794 15072 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:35.751982 14361 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:35.759339 15078 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 12:25:35.766870 10677 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:35.769541 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 818) took 1767 ms. Trace:
I20260709 12:25:35.581526  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:35.787916 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:35.846329 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 115907us
I20260709 12:25:35.846578 10019 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:35.509128 14910 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.719970 15063 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:35.846089 10021 rpcz_store.cc:276] 0709 12:25:34.002468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.002615 (+   147us) service_pool.cc:224] Handling call
0709 12:25:34.006764 (+  4149us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:34.037684 (+ 30920us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:34.037718 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:34.037760 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:34.037799 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.766471 (+1728672us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b5400021054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:35.766535 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.769337 (+  2802us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":70627584}
I20260709 12:25:35.847182 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:35.725095 14990 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:35.501875 15081 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:35.842386 15079 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:25:35.717963 15073 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.850903 14750 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:35.887044 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3120) took 1632 ms. Trace:
I20260709 12:25:35.889228 10020 rpcz_store.cc:276] 0709 12:25:34.254971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.255119 (+   148us) service_pool.cc:224] Handling call
0709 12:25:34.255452 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.272889 (+ 17437us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:34.732796 (+459907us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.109303 (+376507us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.110129 (+   826us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.110301 (+   172us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:35.110350 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.110437 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.886804 (+776367us) raft_consensus.cc:1685] finished
0709 12:25:35.886826 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.886979 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43347,"prepare.queue_time_us":212417,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":112341,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":7203,"thread_start_us":651377,"threads_started":3,"wal-append.queue_time_us":354995}
W20260709 12:25:35.896641 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49525.3us
I20260709 12:25:35.898538 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:35.900744 15008 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 2.043s	user 0.004s	sys 0.011s
I20260709 12:25:35.900888 15085 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:35.901809 15085 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.904788 15085 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.857036 15047 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:35.913527 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:35.917258 15037 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:25:35.917487 15082 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 12:25:35.922443 14760 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.923343 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3119) took 1670 ms. Trace:
I20260709 12:25:35.924070 14841 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.889137 14750 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:35.929207 10023 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:35.764717 15083 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.857405 15063 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.868707 14990 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:35.926227 15006 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.103s	user 0.013s	sys 0.001s
I20260709 12:25:35.918380 14970 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:35.915685 14991 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:35.927031 10016 rpcz_store.cc:276] 0709 12:25:34.252673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.252833 (+   160us) service_pool.cc:224] Handling call
0709 12:25:34.253180 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:34.255495 (+  2315us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:34.314509 (+ 59014us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.147211 (+832702us) log.cc:844] Serialized 14 byte log entry
0709 12:25:35.147346 (+   135us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:35.147800 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 12:25:35.147944 (+   144us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:35.147976 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:35.148047 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:35.902637 (+754590us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b4800002958. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:35.923087 (+ 20450us) raft_consensus.cc:1685] finished
0709 12:25:35.923119 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:35.923268 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30922,"prepare.queue_time_us":220648,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":95,"raft.queue_time_us":255726,"raft.run_cpu_time_us":13002,"raft.run_wall_time_us":662393,"spinlock_wait_cycles":5464448,"thread_start_us":708386,"threads_started":2,"wal-append.queue_time_us":766112}
I20260709 12:25:35.953781 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:35.910971 15043 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:35.963896 14988 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:35.950896 14991 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:35.966790 14991 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.967870 14970 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:35.968726 14970 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:35.969970 14970 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.976111 10098 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:35.977710 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:35.979147 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:35.948683 15006 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:35.980945  9945 raft_consensus.cc:2368] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:35.978477 10098 raft_consensus.cc:2376] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13. Re-sending same reply.
I20260709 12:25:35.982110 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:35.982880 10099 raft_consensus.cc:2410] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 11 because replica has last-logged OpId of term: 10 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 11.
I20260709 12:25:35.980875 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:35.983246 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:35.983793 10094 raft_consensus.cc:2368] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12.
I20260709 12:25:35.979959 10092 raft_consensus.cc:2376] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12. Re-sending same reply.
I20260709 12:25:35.984694 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:35.985570 10098 raft_consensus.cc:2376] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13. Re-sending same reply.
I20260709 12:25:35.988636 15074 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:36.002713  9946 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.002s
I20260709 12:25:36.003243  9946 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:36.005300 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.006014 10099 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:36.007687 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:36.008590 10098 raft_consensus.cc:2376] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12. Re-sending same reply.
I20260709 12:25:35.957953 14908 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:36.010486 14908 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:36.011775 14908 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.013672 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:35.971686 15028 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.181s	user 0.011s	sys 0.001s
I20260709 12:25:36.016451 10097 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:36.017020 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.023033 10092 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:25:36.020339 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.020495 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:36.024379 10095 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:36.024492 10099 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:36.029168 10097 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 12:25:36.030841 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:36.030990 15028 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.033249 15028 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.034080  9949 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 12:25:36.039017 15084 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.294s	user 0.001s	sys 0.001s
I20260709 12:25:36.039831 14990 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.001819 15043 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:36.040297 14750 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.041414 14750 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:36.044104 14910 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.045871  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:36.049888 10095 raft_consensus.cc:2410] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:36.052011 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.054428 10095 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:35.973024 15063 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.059839 10098 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 12. Preceding OpId from leader: term: 11 index: 12. (term mismatch)
I20260709 12:25:36.065991 10098 pending_rounds.cc:85] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 11
I20260709 12:25:36.066315 10098 pending_rounds.cc:107] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 10.12
I20260709 12:25:35.988682 15006 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.066565 10098 raft_consensus.cc:2889] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:35.970001 14991 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:36.030876 14382 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.221s	user 0.013s	sys 0.000s
I20260709 12:25:36.071944 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:36.059019 14990 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:36.082376  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.008s	sys 0.016s
W20260709 12:25:36.083127  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.009s	sys 0.016s
W20260709 12:25:36.083714  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 718724us
I20260709 12:25:36.087772 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:36.093807 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:36.094780 10016 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:25:36.084405 15006 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:36.114030 10095 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:36.084563 15046 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:36.074860 14382 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.115500 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.119717 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.118530 15086 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:36.091559 15074 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:36.116555  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:36.118672 10096 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:36.148208 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.132076 10673 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:36.155445 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.156100 10669 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:36.156829 10019 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:36.157392  9948 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:36.084167 14990 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.154728 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.160017 10671 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:36.114753 14361 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.159102 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.166523 10670 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:36.161672 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.162276 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:36.162719 15081 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.163300 15083 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.164199 15074 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.154793 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.167541 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:36.175765  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59307.9us
I20260709 12:25:36.177510 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.180163 10669 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 12:25:36.180608 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.150632 15073 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.130468 15085 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:25:36.129745 15006 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.185004 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.191267 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:36.178079 15070 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:36.262890 14684 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.431s	user 0.010s	sys 0.003s
I20260709 12:25:36.263384 14684 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.265601 14684 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.205844 10096 raft_consensus.cc:2368] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12.
I20260709 12:25:36.267522 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.268085 10096 raft_consensus.cc:2376] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13. Re-sending same reply.
I20260709 12:25:36.268970 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:36.269400 14684 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.269677 10096 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:36.209220 15074 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.153648 15086 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.178586 14361 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.275512  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:36.211687 15006 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.214073 15085 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:36.216553 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:36.217190 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.205760 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.231181 14750 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:36.232384 14986 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:36.235047  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.298265  9946 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:36.236601  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.299039  9942 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:36.244686 15015 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:36.252259 14990 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.256408 15055 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.270174 14684 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.272614 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.273077 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:36.292989 10668 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:25:36.296062 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.235339 10675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1020) took 1495 ms (client timeout 1820 ms). Trace:
W20260709 12:25:36.303246 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 86788us
W20260709 12:25:36.332504 10675 rpcz_store.cc:269] 0709 12:25:34.739621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.739819 (+   198us) service_pool.cc:224] Handling call
0709 12:25:36.235122 (+1495303us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b5400690d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:36.235256 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1794616192}
I20260709 12:25:36.309289 10098 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:36.291846 14777 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.278846 15081 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:36.335909 10096 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
W20260709 12:25:36.337340  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 157976us
I20260709 12:25:36.338528 10091 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:25:36.333153 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:36.303973 10095 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:36.340543  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.290815 15083 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:36.345606 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:36.308635 15036 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:36.348510 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:36.299882 15085 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:36.333642 14919 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:36.291033 15067 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:36.307107 15087 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:36.304338 15015 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:36.425199 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:36.425969 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 80477.1us
I20260709 12:25:36.427397 10020 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:36.310752 15061 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:25:36.308095 15006 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.056s
I20260709 12:25:36.333423 15040 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:25:36.430440  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.363566 14990 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:36.307988 14986 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.437182 14986 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.439455 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 821) took 1698 ms. Trace:
I20260709 12:25:36.440026 10018 rpcz_store.cc:276] 0709 12:25:34.741182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.741615 (+   433us) service_pool.cc:224] Handling call
0709 12:25:34.770794 (+ 29179us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.509885 (+739091us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.439192 (+929307us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54001c7354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:36.439387 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1698409728}
W20260709 12:25:36.443825 15089 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:25:36.448058 15054 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.426282 15086 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:36.427340 15087 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:36.452147 15087 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:36.453464 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.454864 15037 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.455417 15037 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:36.456312 15096 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.243s	user 0.002s	sys 0.000s
I20260709 12:25:36.457638 15037 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:36.458051 15021 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.458209 15037 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:36.344105 15029 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.087s
I20260709 12:25:36.458570 15021 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.459318 15029 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:36.459687 15029 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.460032 15029 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:36.460805 15021 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:36.462245 14382 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.464421  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:36.451156 15015 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.466739 15015 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:36.472869 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.475445  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2133) took 1124 ms. Trace:
I20260709 12:25:36.475881  9949 rpcz_store.cc:276] 0709 12:25:35.351428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.353148 (+  1720us) service_pool.cc:224] Handling call
0709 12:25:35.353572 (+   424us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:36.034514 (+680942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.475157 (+440643us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b54001b9754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:36.475375 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1564315520}
W20260709 12:25:36.476616 15098 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:25:36.477221  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.478044  9949 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:36.422678 14919 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:36.485275 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.488027  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.366046 15099 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:36.341512 14750 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.092s
I20260709 12:25:36.432512 15036 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:36.505511 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 168051us
I20260709 12:25:36.508029 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:36.512080 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:36.362036 15046 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.131s
W20260709 12:25:36.514412 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 170985us
I20260709 12:25:36.520386  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:36.494666 15043 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.524111  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:36.526371  9991 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.820s (SENT) [suppressed 34 similar messages]
I20260709 12:25:36.360266 15081 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.347007 14777 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:36.529947  9987 connection.cc:256] RPC call timeout handler was delayed by -1.5304198240000915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
W20260709 12:25:36.530373  9987 connection.cc:256] RPC call timeout handler was delayed by -1.5016198240000449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
I20260709 12:25:36.494443 15099 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:36.498188 15002 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.532296 10023 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:36.432250 15067 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.482892 14990 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.364075 15083 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.517560 15047 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:36.543706 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.487461 15037 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.544736 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:36.544616 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.548126 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.548230 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.555267 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:36.555398 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.556628 10677 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:36.558941 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:36.559569 10670 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:36.562142 10672 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:36.538661 15056 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:36.533311 15036 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:36.543169 14617 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.567216  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.533305 14986 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:36.533675 15099 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:36.532604 15092 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:36.579645  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.580523  9947 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:36.581725  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.583184  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:36.566115 15062 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.006s	sys 0.011s
I20260709 12:25:36.592501  9947 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:25:36.536041 15043 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.566265 14819 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:36.584476 15024 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.129s	user 0.017s	sys 0.020s
I20260709 12:25:36.605373  9947 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:36.542309 15002 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.603487 14990 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.550395 15025 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
W20260709 12:25:36.607435  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81114.4us
W20260709 12:25:36.594894 15062 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.006s	sys 0.011s
W20260709 12:25:36.604236 15024 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.157s	user 0.017s	sys 0.021s
I20260709 12:25:36.609249 10670 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:36.529440 15085 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.612294 15056 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:36.613616 14684 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.631879  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1928) took 2146 ms. Trace:
I20260709 12:25:36.587090 14617 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.599237 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.618245 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.633425 10670 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:36.633663 10668 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:36.624264  9941 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:36.614629 15067 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.636066  9941 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:36.632350  9948 rpcz_store.cc:276] 0709 12:25:34.479512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.479646 (+   134us) service_pool.cc:224] Handling call
0709 12:25:34.538017 (+ 58371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.158006 (+1619989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:36.625648 (+467642us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b5400016dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:36.625831 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3723813248}
W20260709 12:25:36.637059 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 125098us
I20260709 12:25:36.615406 14361 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:36.600569 15092 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.650629 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:36.651758 10670 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:36.653049 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:36.607266 14819 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:36.661260  9987 connection.cc:256] RPC call timeout handler was delayed by -1.3009198240001751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
I20260709 12:25:36.667590 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:36.669185 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.632736 14986 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:36.653710 14684 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.673905 14986 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.685361 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.653800 15101 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:36.683236 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:36.689733 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.690526 10670 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:36.666311 15021 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:36.694142 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.675628 15036 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:36.702906 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:36.704690 10670 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 9: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:36.707595  9948 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:36.711882 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.711226 15111 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:36.715298 15111 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:36.715646 15111 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:36.716266 15092 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.716830 15092 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:36.717289 15092 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:36.717833 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.718855 14777 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:36.719244  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.719648  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.719805  9945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2135) took 1367 ms (client timeout 1820 ms). Trace:
W20260709 12:25:36.720155  9945 rpcz_store.cc:269] 0709 12:25:35.352116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.352254 (+   138us) service_pool.cc:224] Handling call
0709 12:25:36.719615 (+1367361us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b54001b9254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:36.719751 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1364267392}
I20260709 12:25:36.725514  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:36.719251 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.735050 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:36.690106 14684 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.719821 15037 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:36.700631 15101 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.694191 15056 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:37.066035 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.349701 10670 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:36.783453 10669 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:37.351137 10669 raft_consensus.cc:606] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:37.096067  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:36.736199  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:36.878225 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.903497 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.355543 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:37.361974 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:36.722512 15117 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:37.367573 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:36.828337 15119 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:25:36.824890 15099 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.373514 14617 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.388937 14871 raft_consensus.cc:2764] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:37.390111 15099 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:37.393575 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.066948 14463 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:36.743839 14984 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.750097 15041 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:37.374475 14684 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:36.919121 15029 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:36.742522 15002 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.747812 14819 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:37.396806 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.406054 10019 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:37.394537 10020 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:37.394816 14463 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:36.734735 15081 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:36.749773 14988 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:36.784866 15019 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:36.777653 15083 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.372221 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:36.745237 15077 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.367995 15087 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.914s	user 0.018s	sys 0.000s
W20260709 12:25:37.412133 15003 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.022s	sys 0.036s
I20260709 12:25:37.418716 15077 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.420634 15029 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.421350 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:37.421758 15077 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:37.422667 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:37.414752 15111 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.699s	user 0.011s	sys 0.003s
W20260709 12:25:37.415473 14780 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.028s	sys 0.028s
I20260709 12:25:37.403913 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:37.427520  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.428196 10016 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:37.428931 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:37.429180  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:37.360131 15092 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.409693 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:37.409135  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.004s	sys 0.024s
I20260709 12:25:37.410239  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:37.395210 15041 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:37.447984  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:37.448823 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:37.396272 15117 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.424935 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2041) took 2746 ms. Trace:
W20260709 12:25:37.422607 15003 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.022s	sys 0.036s
I20260709 12:25:36.797971 15043 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:37.417865 14463 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:37.429945 14780 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.029s	sys 0.028s
I20260709 12:25:37.435626 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:37.438865  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.004s	sys 0.024s
I20260709 12:25:37.439108 10669 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:37.440286  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:37.444502  9944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2140) took 2087 ms (client timeout 2026 ms). Trace:
I20260709 12:25:37.425825  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:37.458158  9945 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:37.457980  9944 rpcz_store.cc:269] 0709 12:25:35.357305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.396794 (+ 39489us) service_pool.cc:224] Handling call
0709 12:25:37.444297 (+2047503us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54000163d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:37.444432 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507360128}
I20260709 12:25:37.450868 10090 rpcz_store.cc:276] 0709 12:25:34.678606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:34.916214 (+237608us) service_pool.cc:224] Handling call
0709 12:25:34.916575 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.415683 (+499108us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:35.415714 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:35.415756 (+    42us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:35.415788 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.424691 (+2008903us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54002fd354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:37.424745 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.424879 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1147343616}
I20260709 12:25:37.453814  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:37.460150 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.462164  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:37.450680 15057 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 2.228s	user 0.013s	sys 0.028s
I20260709 12:25:37.415339 15019 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.463892  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:37.460530 15127 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:37.456966 14463 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:37.471539 15127 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.472990  9945 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:37.473820 15127 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.465320 15117 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.453400 15081 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:37.478053 15081 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.462793  9940 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:37.447391 15083 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:37.468781  9949 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:37.421759 15029 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:25:37.415560 14988 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.422816 15087 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.480993  9944 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:37.484671  9944 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:37.428607 15111 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:37.488155 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 125317us
I20260709 12:25:37.489424 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:37.483621 15129 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:37.490957  9948 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:37.497586 10018 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:37.491927 15129 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.498469 15129 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:37.499691 15111 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:37.500265 10050 heartbeater.cc:646] Failed to heartbeat to 127.8.137.254:38721 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.8.137.254:38721 timed out after 15.000s (SENT)
I20260709 12:25:37.497040 15117 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 12:25:37.484225 15101 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:37.501701  9944 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.505069  9941 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:37.508272 15087 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:37.484266 15083 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.512602 15083 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:37.516083 14985 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.517835 14985 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.506665 15127 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.519080 15127 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.519992 14985 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.515405 15077 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:37.487459 15041 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.532621 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:37.532825 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:37.533741 10093 raft_consensus.cc:2368] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12.
I20260709 12:25:37.535029 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:37.535773 10091 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 7.
I20260709 12:25:37.534397 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:37.536904 10095 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:37.533706 10099 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:37.537511 10093 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 12:25:37.538146 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:37.547464 15041 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:37.549317 15037 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:37.549775 10093 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:37.549889 15037 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.552441 15037 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:37.526003 15111 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:37.557288 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:37.558138 10094 raft_consensus.cc:2376] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13. Re-sending same reply.
I20260709 12:25:37.559722 10092 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:37.523747 15077 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:37.555948 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:37.572790 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:37.574472 10097 raft_consensus.cc:2376] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13. Re-sending same reply.
I20260709 12:25:37.547675 15043 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.624037 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 218829us
I20260709 12:25:37.554176 14560 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.581239 15111 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.631815 15037 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:37.634199 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:37.635713 14560 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:37.637051 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 267558us
W20260709 12:25:37.652956 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 201189us
I20260709 12:25:37.653134 14463 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:25:37.653587 15131 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:25:37.654120 14463 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:37.636562 15081 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.629529 15043 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:37.638047  9987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.001s	sys 0.013s
I20260709 12:25:37.653029 15111 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:37.641096 15019 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:37.680056 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 839) took 1420 ms. Trace:
I20260709 12:25:37.680645 10023 rpcz_store.cc:276] 0709 12:25:36.259917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.260300 (+   383us) service_pool.cc:224] Handling call
0709 12:25:36.260667 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:36.532793 (+272126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.679690 (+1146897us) spinlock_profiling.cc:243] Waited 271 ms on lock 0x7b540011b054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:37.679984 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":624208128}
I20260709 12:25:37.642110 14988 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:37.641487 15037 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.665943 15023 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:37.638170  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 539336us
W20260709 12:25:37.670557  9987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.937s	user 0.002s	sys 0.013s
I20260709 12:25:37.638870 15077 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.673465 15043 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:37.674407 10090 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:25:37.694581  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 97462.7us
I20260709 12:25:37.644403 14560 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:37.702415 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.703080 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.704592 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.706784 10024 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:25:37.709985  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 144974us
I20260709 12:25:37.713765 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.716722 10019 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:37.717711 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.718532 10019 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:37.719682 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:37.720595 10019 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:37.722913 10019 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:37.723981 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:37.715481 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 226103us
I20260709 12:25:37.726712 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.726956 10022 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:25:37.727463 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.732246 10020 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:37.728138 10016 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: [12.9-14.10]
I20260709 12:25:37.728116 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.728899 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.693886 15111 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:37.729836 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 77291.5us
W20260709 12:25:37.725762 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71924.5us
I20260709 12:25:37.733795 10017 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:37.729759 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:37.731376 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 90025.5us
I20260709 12:25:37.736881 10023 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:37.776571 10022 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 12:25:37.779506 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44763.9us
I20260709 12:25:37.780498 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:37.782547 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45578.7us
W20260709 12:25:37.782681 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44894.5us
I20260709 12:25:37.783622 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:37.782037 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:37.792878 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:37.797451  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.754379 15077 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:37.813514 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.815147 15139 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:37.788209 15127 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:37.790021 15000 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:37.851703 10019 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:37.853150 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:37.765254 15133 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:25:37.859884 15133 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:37.833441 15149 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:37.861253 15149 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.861688 15149 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:37.863191 10016 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:37.877969 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.765771 15111 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.846935 10017 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:37.848632 10022 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: [11.11-11.11]
I20260709 12:25:37.880103 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:37.754684 15146 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:25:37.887849 10020 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:37.889541 15019 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:37.890278 15019 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:37.891276 15019 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.892520 15019 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:37.897218  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.899185  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:37.840283 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56834.6us
I20260709 12:25:37.904112 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:37.803470 15043 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:25:37.838366 15148 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:37.914290 15002 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:37.927136 15002 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.927119 15148 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:37.927749 15002 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:37.929601 15148 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:37.879068 15000 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.935698  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1002) took 1700 ms. Trace:
I20260709 12:25:37.936195  9947 rpcz_store.cc:276] 0709 12:25:36.235608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.610137 (+374529us) service_pool.cc:224] Handling call
0709 12:25:37.447355 (+837218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.447553 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.447578 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:37.447620 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:37.447651 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.447703 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:37.935517 (+487814us) spinlock_profiling.cc:243] Waited 818 ms on lock 0x7b54000163d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:37.935632 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1881839360}
I20260709 12:25:37.936604 15002 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:37.939960  9944 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:37.943429 10098 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:37.944573 15002 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:37.945473 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.946885 15139 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:37.912251  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:37.948377 15002 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:37.915094  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2131) took 2564 ms. Trace:
I20260709 12:25:37.949906  9943 rpcz_store.cc:276] 0709 12:25:35.350660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.350827 (+   167us) service_pool.cc:224] Handling call
0709 12:25:35.373974 (+ 23147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:35.508358 (+134384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:37.914861 (+2406503us) spinlock_profiling.cc:243] Waited 37.8 ms on lock 0x7b54003a0554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:37.915038 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86851456}
I20260709 12:25:37.915122 10671 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:37.951830 15002 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:37.953104 15002 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:37.916653 15111 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:37.917650 15037 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:37.924042 15150 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:25:37.914934 14962 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:37.957906 14908 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:37.961431 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57449us
I20260709 12:25:37.962217 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:37.967381  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:37.967548 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:37.969532  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:37.977584  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:37.983155  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:37.983263  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:37.984185  9947 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:37.987267  9949 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:25:37.991173  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.000101  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.001451  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.003098 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:38.004864 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:38.005461 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:38.006088 10025 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:38.006764 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44648.8us
I20260709 12:25:38.007807 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:38.008488 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.009132 10098 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.004725 14962 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:38.010146 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 951) took 1844 ms. Trace:
I20260709 12:25:38.010511 10671 rpcz_store.cc:276] 0709 12:25:36.165984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.177279 (+ 11295us) service_pool.cc:224] Handling call
0709 12:25:36.188912 (+ 11633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.951127 (+1762215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.009908 (+ 58781us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54005b95d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:38.010091 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3969107328}
I20260709 12:25:38.014083 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.014962 10095 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:38.006649 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:38.003229 15129 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.504s	user 0.007s	sys 0.004s
I20260709 12:25:38.017995 15111 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.024570 15111 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.028417 15148 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:38.030519 15148 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:38.033507 15148 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:38.034628 15148 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.036379 15148 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.048574 10092 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:38.050267 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.051194 10094 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:37.984082 15073 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:25:38.054723 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:38.055863 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.065477 10093 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:38.056051 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:38.067682 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:38.071527 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.072542 10092 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:38.073817 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.019249 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.076308 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.077910 10092 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:38.079385 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.080497 10092 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:38.081583 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.018438 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:38.061486 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:38.064388 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 280671us
I20260709 12:25:38.082285 10092 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:38.085351 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.086427 10025 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:38.088248 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.017218 15154 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:38.090152 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 308181us
I20260709 12:25:38.091068 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.095266 10095 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.102121 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.102919 10093 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:38.107393 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:38.107554 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.108052 10020 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.108711 10098 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:38.109817 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.062891 14560 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.113747 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.116232 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:38.117057 10017 raft_consensus.cc:2368] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15.
I20260709 12:25:38.118979 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:38.119843 10025 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:38.089092 15021 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.021662 15145 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.124017  9944 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:25:38.134938  9944 pending_rounds.cc:85] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 6
I20260709 12:25:38.135396  9944 pending_rounds.cc:107] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:25:38.135712  9944 raft_consensus.cc:2889] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:38.131137 10020 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:38.131137 10095 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:38.052953 15129 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.128592 10022 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:38.111577 15043 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.139978 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:38.139358 10095 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:38.145015 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.143527 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:38.147512  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:38.147941 10095 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:38.149642 10642 leader_election.cc:400] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12.
I20260709 12:25:38.150148 10642 leader_election.cc:403] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:38.131294 15111 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:38.138684 14910 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:38.168020 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.161727 15041 raft_consensus.cc:993] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:25:38.129146 14560 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.138484 15145 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:38.202422 15145 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.156359 15153 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:38.156711 15139 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:38.209438 15139 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:38.209908 15139 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.209707 14560 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:38.177317 15073 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.211557 15073 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.212095 15073 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.212321 14760 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:38.212946 14760 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:38.179334 15099 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:25:38.216109 14760 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:38.156149 15043 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:38.142163 14988 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:25:38.138154 15021 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.229444  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.190049 14962 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12.
I20260709 12:25:38.237826 14760 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:25:38.248754 15021 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:38.232092 15155 raft_consensus.cc:1081] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:25:38.228931 15061 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:25:38.194245 14910 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:38.236407 15163 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:25:38.266896 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 248574us
I20260709 12:25:38.262696 15023 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:38.272998  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.273058 15023 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:38.274260 15085 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.237370 15060 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:38.277092 15060 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.277468 15060 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.279635 14910 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:38.283821 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 103016us
I20260709 12:25:38.284438 10092 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:25:38.288491 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 181059us
I20260709 12:25:38.239451 15153 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:38.292789 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:38.289750 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.293339 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:38.283608 15156 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:25:38.307864 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1119) took 1214 ms. Trace:
I20260709 12:25:38.308466 10672 rpcz_store.cc:276] 0709 12:25:37.093625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:37.461093 (+367468us) service_pool.cc:224] Handling call
0709 12:25:37.482893 (+ 21800us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:37.667962 (+185069us) spinlock_profiling.cc:243] Waited 7.29 ms on lock 0x7b480002ea58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:37.714432 (+ 46470us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:37.714483 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:37.727861 (+ 13378us) log.cc:844] Serialized 52 byte log entry
0709 12:25:38.004497 (+276636us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:38.004569 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.007860 (+  3291us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:38.307445 (+299585us) raft_consensus.cc:1685] finished
0709 12:25:38.307472 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:38.307770 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":168223,"raft.run_cpu_time_us":2781,"raft.run_wall_time_us":16615,"spinlock_wait_cycles":16766208,"thread_start_us":448000,"threads_started":2,"wal-append.queue_time_us":377220}
I20260709 12:25:38.304746 15021 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:38.304839 15002 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.299170 10091 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:38.298524 10097 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:38.312790 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.313918 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.238581 15081 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.052s
W20260709 12:25:38.325317 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:38.358438 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.356580 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.332069 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.387882 10097 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:38.388053 10091 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:38.357616 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.358659  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.324183 15161 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.359423 15129 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:38.362965 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:38.363816 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.363951 15002 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:38.295909 15154 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:38.364115 14911 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:38.404619 14911 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.406821 14911 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:38.408794 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.409430 10099 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:38.410446 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.411000 10097 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:38.411791 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.412325 10097 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:38.413139 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.414156 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.366201 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:38.403350  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6c7bd3fa1bfa431c89fc9c9096452a76 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:38.367063 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.415652 10677 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:25:38.369556 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.369043 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.416381 10676 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:38.370339 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.417297 10670 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:38.368572 15139 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.418126 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:38.377079 15108 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.763s	user 0.012s	sys 0.014s
I20260709 12:25:38.419371 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:38.380584 15077 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.000s
I20260709 12:25:38.419649 15139 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:38.406364 15043 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:38.383805 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.420853 15015 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.421063 10098 raft_consensus.cc:2376] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 14. Re-sending same reply.
I20260709 12:25:38.421290 15015 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.421695 15015 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:38.384743  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.422091 10021 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:25:38.422482 15139 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:38.410316 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.300770  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:38.424113 10099 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:38.424613  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 123942us
I20260709 12:25:38.424870 14760 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.431280 14919 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.432463 14919 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.433772  9947 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:38.422139 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.435285 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.398574 10671 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:38.436419 10668 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:38.436496 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.437197 10092 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 7.
I20260709 12:25:38.437425 15000 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:38.437662 10668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1032) took 1881 ms (client timeout 1780 ms). Trace:
I20260709 12:25:38.438009 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.401515 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.438294 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.438905 15135 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:38.438123 10668 rpcz_store.cc:269] 0709 12:25:36.556367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.649040 (+ 92673us) service_pool.cc:224] Handling call
0709 12:25:38.437507 (+1788467us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:38.437601 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4062717824}
I20260709 12:25:38.439388 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.439631 10097 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:38.440048 10096 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:38.440950 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.441746 10096 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:38.443917 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.397286 15161 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:38.403231  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44658.3us
I20260709 12:25:38.451568 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.451763 10023 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:38.403337 14819 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.404937  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 1939) took 1929 ms. Trace:
I20260709 12:25:38.457283 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:38.457813 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.457983 14819 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:38.458040 10675 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:38.458809 15161 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.458859 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.460263 10676 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:38.461314 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.458496 10677 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:38.452535  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.457355 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.462483 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:38.383718 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 80766.7us
W20260709 12:25:38.369977 15149 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.508s	user 0.000s	sys 0.009s
I20260709 12:25:38.372138 15023 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:38.311677 14463 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:25:38.420241 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:38.420475 15168 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.385547  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:38.423048 15139 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.397791 10673 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:38.447531 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.447801  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:38.475649  9942 raft_consensus.cc:2368] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:38.475483 10090 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:38.476673  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.448596  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.477205 10676 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:38.441741 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.440940 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.478243 10095 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:38.478821 15023 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:38.479166 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.470283 15055 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.479342 15023 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.479794 10095 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:38.449257  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.480264 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.480748 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.481009  9943 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:38.481326 10098 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:25:38.481727 10093 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:38.481541 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.401844 15154 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.439479 15135 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.485723  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:38.478385 10097 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:38.486439  9946 raft_consensus.cc:2376] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11. Re-sending same reply.
I20260709 12:25:38.487560  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.453540 10092 raft_consensus.cc:2376] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 12. Re-sending same reply.
I20260709 12:25:38.484580  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:38.399487 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.454684  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.498240  9947 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:38.500305  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.455709 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.457055 15127 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.502110 15127 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:38.505678  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.506881 15127 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:38.467860  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:38.457595  9941 rpcz_store.cc:276] 0709 12:25:36.475039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:36.644288 (+169249us) service_pool.cc:224] Handling call
0709 12:25:36.644612 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:37.505499 (+860887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.404704 (+899205us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b540042e354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:38.404877 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978477312}
I20260709 12:25:38.508265  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.508471  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.508638 10099 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:38.471302  9945 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:38.471251 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 51081.7us
I20260709 12:25:38.509020 14760 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:38.509522 14760 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:38.509888 14760 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:38.511125 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:38.511559  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.512590 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:38.514161 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.514384 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:38.514869 14991 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:38.515136 15136 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.515309 14991 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.515571 15136 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:38.515727 14991 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.515995 15136 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.517374 15168 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.517805 15168 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:38.518138 15168 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:38.518989  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.522888 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.509079  9941 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:38.524933 15043 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:38.525547 15043 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.525624  9941 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:38.526039 15043 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:38.530249  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.531189 14693 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:38.531692 14693 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:38.532538 14693 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:38.533789  9945 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:38.534790 15061 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:25:38.535198 15061 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:38.535557 15061 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:38.536801 15136 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.545828 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.546983 14463 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:38.547576 14463 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:38.548576 14463 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.415254 15060 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.552546  9940 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:25:38.473177 14908 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:25:38.474004  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 297502us
I20260709 12:25:38.558200  9987 leader_election.cc:400] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:38.558647  9987 leader_election.cc:403] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:25:38.559556  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:38.560531 15085 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
I20260709 12:25:38.561216 15085 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.470223 15149 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.389482  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:38.458503 14819 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.565954  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:38.566848  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 177433us
I20260709 12:25:38.567241 15127 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:38.567833 15127 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:38.568328 15127 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.568410 15074 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.568919 15074 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:38.569252 15074 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.569710  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:38.571642 15085 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.572068 15085 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:38.572455 15085 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.483255  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:38.483016 15023 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:38.575563  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:38.578713 14841 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:38.579839 15074 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.582293 15074 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:38.583268 14841 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:38.584007  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:38.586756 15060 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:38.587412  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:38.606237 15074 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.606851  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:38.607857 15074 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:38.591032  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 60803.3us
I20260709 12:25:38.608947  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:38.610359  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:38.611465 15063 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:38.611871 15063 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:38.612267 15063 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.614473 10674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:38.594907 14760 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.683415  9843 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 8 to 9, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 9 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:38.684160 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.815140 10093 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:38.685001 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:38.697062 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.817180 10097 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:38.703975 15149 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:38.472373 10675 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:38.711484 15136 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:38.713986 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.827070 10090 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:38.479872 15055 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:38.681872 15073 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.483546 15040 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:38.789847 15139 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:38.791886 15135 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:38.837060  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.834304 15040 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:38.839846 15040 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.841725 15040 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:38.792059 10676 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:38.440891 15153 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:38.792948 10670 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:38.618007 10674 raft_consensus.cc:606] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:38.847486 10674 raft_consensus.cc:614] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:38.794277 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:38.848436 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:38.677003 14297 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:25:38.807412  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:38.811189  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.814002  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:38.818029  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:38.798779 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 185288us
I20260709 12:25:38.831076  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:38.795015 15145 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.005s
I20260709 12:25:38.813064 15137 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:38.799259 15074 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:38.863395  9942 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:39.032121  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:38.864073  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56808.3us
I20260709 12:25:38.865801 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:39.035820 10098 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.861552  9843 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 6 to 12, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 12 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:38.813278 15060 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:38.862658 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.868415  9946 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 10: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:38.833981  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:39.042464 10092 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:38.864027 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:38.885975 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:38.965775 10674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:39.043169 10095 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.043493 10674 raft_consensus.cc:606] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:39.043836 10674 raft_consensus.cc:614] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:25:39.044623 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 158755us
I20260709 12:25:38.835041 15127 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:38.884737 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:38.930449  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:39.051790  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.910570  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:38.835399 14819 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:38.795329 15015 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.060338  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:39.061226  9946 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:38.831485 15023 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:38.909591 15180 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:25:38.835201 15168 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.089955 10098 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:38.899292 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:39.095549 10093 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 9.
I20260709 12:25:38.869161 14904 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:38.898182 10096 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:25:38.888934 10091 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:38.869994 15145 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:38.926021  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 316777us
I20260709 12:25:38.886991  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:38.960892 15161 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.001s
I20260709 12:25:38.795560 14991 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:39.027549 15137 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:39.168208 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:39.176282 10095 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:39.178247 10098 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:25:38.835583 15043 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.040270 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:39.067447 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.072371 10092 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:39.073297 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:38.956825 15170 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.105s
W20260709 12:25:39.212245  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 381267us
I20260709 12:25:38.830786 15055 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.168488  9945 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:39.184275 10095 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:39.217978 10095 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:39.188638 10671 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:39.030789 14971 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:39.189393  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:39.189531 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:39.192755  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:39.194644 10676 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:39.233179  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:39.194782 10096 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:39.239073 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2061) took 1017 ms. Trace:
W20260709 12:25:39.246815  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.247711  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:39.169114 15061 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.633s	user 0.009s	sys 0.006s
W20260709 12:25:39.248528  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55862.5us
I20260709 12:25:39.239588 10096 rpcz_store.cc:276] 0709 12:25:38.221221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.194248 (+973027us) service_pool.cc:224] Handling call
0709 12:25:39.194575 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.238713 (+ 44138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.239001 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:39.258366 10091 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:39.262701 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2059) took 1042 ms. Trace:
I20260709 12:25:39.261159  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:39.264391  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:39.104120 14919 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.671s	user 0.011s	sys 0.004s
I20260709 12:25:39.271643 10091 rpcz_store.cc:276] 0709 12:25:38.220677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.185364 (+964687us) service_pool.cc:224] Handling call
0709 12:25:39.185600 (+   236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.262313 (+ 76713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.262620 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:39.104302 15074 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:39.248383  9843 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.331817  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:39.332283 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:39.332947  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:39.333078  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 65932us
W20260709 12:25:39.333812  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72744us
I20260709 12:25:39.333771  9941 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 15
W20260709 12:25:39.336094  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.335593  9843 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 5 to 9, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 9 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:39.339491 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.341542 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:39.342273 10091 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:39.343271 10668 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:39.343746 10094 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:39.334625  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:39.348783 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.350106 10021 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:39.354744 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.356374 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.171527 14760 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:39.358212 10093 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:39.361276 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:39.355659 10021 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:39.362392 10671 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.361231 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.363241 10025 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:39.367079 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.367743 10023 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:39.362396 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2042) took 1156 ms. Trace:
I20260709 12:25:39.363080  9843 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a reported cstate change: term changed from 7 to 10, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 10 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:39.366305 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.371627 14971 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.371886 15192 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:39.372206 14971 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.358364 14904 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:39.373276 14904 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.373661 14904 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:39.372575 15192 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:39.375463 15192 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:39.366783 15184 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.457s	user 0.000s	sys 0.001s
I20260709 12:25:39.191010 15161 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.331544 14995 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:39.368223 10093 rpcz_store.cc:276] 0709 12:25:38.205743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.170275 (+964532us) service_pool.cc:224] Handling call
0709 12:25:39.305865 (+135590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.361910 (+ 56045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.362304 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:39.342466 10096 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:57396
I20260709 12:25:39.396832 10096 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:39.397362 10096 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.331851 15189 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:39.393987  9945 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
W20260709 12:25:39.401181  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.359059 15099 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.407485 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.410074 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.365679 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.412534 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.414106 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.415392 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.415758 10091 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:39.416189 10024 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:39.374441 14971 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:39.386157 15182 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.340s	user 0.001s	sys 0.000s
W20260709 12:25:39.404629 15040 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.562s	user 0.002s	sys 0.009s
W20260709 12:25:39.386801 15085 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.814s	user 0.013s	sys 0.000s
I20260709 12:25:39.418645 10091 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: [12.10-14.11]
I20260709 12:25:39.343120 15188 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.170s
I20260709 12:25:39.457437 10671 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:39.418916 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.419081  9843 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 8 to 11, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 11 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:39.378386 15147 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:25:39.424438 15187 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.442s	user 0.000s	sys 0.001s
I20260709 12:25:39.424690 10672 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:39.424965 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.474709 10022 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:25:39.371335 15030 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:39.475335 15186 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.225s	user 0.000s	sys 0.000s
I20260709 12:25:39.475672 15030 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.476109 15030 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.461349 15085 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.427500 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:39.478547  9944 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:39.474335  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:39.479769 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:39.480878 10677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:39.481540 10677 raft_consensus.cc:606] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:39.417902 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 372461us
I20260709 12:25:39.481974  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:39.434744  9941 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:39.482842  9946 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.436378 15161 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:39.438251 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:39.439865 10097 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:25:39.440860 15183 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
I20260709 12:25:39.442787 15198 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:39.416599 15189 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:39.443835 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.444188 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:39.395355 15191 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.521679 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.524163 10670 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.524427 10024 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:39.420506 15063 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.808s	user 0.010s	sys 0.003s
I20260709 12:25:39.358165 15061 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.463268 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.466253 14991 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:39.467061 15015 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:39.467437 15043 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:39.410429 15073 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:39.418242 14995 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:39.422000 15077 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:39.421718 15099 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.468174 14819 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:39.468904 15199 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.470266 15145 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:39.461092 15074 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.471673 15127 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:39.473177  9991 leader_election.cc:400] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14.
W20260709 12:25:39.652459  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 179421us
I20260709 12:25:39.473555 15147 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:39.481971 10677 raft_consensus.cc:614] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:39.459933 10096 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.479164  9843 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 7 to 9, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 9 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:25:39.657351  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 315206us
I20260709 12:25:39.659826  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:39.625694 14693 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.085s	user 0.005s	sys 0.026s
W20260709 12:25:39.500293  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.501538 10097 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:39.501982 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63813.3us
I20260709 12:25:39.404533 15060 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:25:39.522276 10025 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:39.522063 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.520750 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:39.519285 10675 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:39.542317 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:39.523401 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.461833 15168 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:39.607612 10674 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:25:39.527577 14911 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:39.525178 15198 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:39.502169 15201 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:39.607656 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.609448 10670 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:39.609824 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:39.541445 15063 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.611218 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:39.619719 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.628927  9946 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:25:39.625576 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 453266us
W20260709 12:25:39.632912  9943 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.009s	sys 0.001s
W20260709 12:25:39.637604 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.642133 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 959) took 1274 ms. Trace:
I20260709 12:25:39.647630  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:39.655808  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:39.923879 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1121) took 1471 ms. Trace:
I20260709 12:25:39.654407  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:39.833241 10099 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.002s	sys 0.017s
W20260709 12:25:39.784636 10098 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.013s	sys 0.002s
I20260709 12:25:39.940881 10668 rpcz_store.cc:276] 0709 12:25:38.452715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.453113 (+   398us) service_pool.cc:224] Handling call
0709 12:25:38.474871 (+ 21758us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.343621 (+868750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.923568 (+579947us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b54005267d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:39.923809 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996430592}
W20260709 12:25:39.941977 10098 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.014s	sys 0.007s
I20260709 12:25:39.676710  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:39.943228 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3249) took 1139 ms. Trace:
I20260709 12:25:39.608831 15139 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:39.944839  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 290507us
W20260709 12:25:39.906284  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 246592us
W20260709 12:25:39.945135  9943 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.971s	user 0.015s	sys 0.006s
I20260709 12:25:39.959400  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:39.960074  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2154) took 1514 ms. Trace:
W20260709 12:25:39.945420  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 268779us
I20260709 12:25:39.946265 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:39.699625 10096 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:39.784941  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 450574us
I20260709 12:25:39.961597 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:39.784432 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 851) took 1374 ms. Trace:
W20260709 12:25:39.947278 10099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.002s	sys 0.020s
I20260709 12:25:39.946511 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.964627 10016 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
W20260709 12:25:39.670089  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 335525us
I20260709 12:25:39.963864  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:39.948083 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.967413 10019 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:39.950196 10672 rpcz_store.cc:276] 0709 12:25:38.368025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.368801 (+   776us) service_pool.cc:224] Handling call
0709 12:25:39.641907 (+1273106us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400615b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:39.642069 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2318270208}
I20260709 12:25:39.804092  9843 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:39.969125  9843 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 9 to 10, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 10 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:39.940443 10022 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:39.971797 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.972366 10022 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:39.973104 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.973630 10022 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
W20260709 12:25:39.627596 15000 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.008s	sys 0.001s
I20260709 12:25:39.955755 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:39.956218 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.974628 10021 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:39.975345  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.975463 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:39.960613  9943 rpcz_store.cc:276] 0709 12:25:38.445999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.490207 (+ 44208us) service_pool.cc:224] Handling call
0709 12:25:38.490579 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:38.490980 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:38.491402 (+   422us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.959590 (+1468188us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.959662 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:39.959732 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.959824 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.959979 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":150,"prepare.run_cpu_time_us":7130,"prepare.run_wall_time_us":109160,"raft.queue_time_us":164044,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":1467649,"threads_started":2,"wal-append.queue_time_us":569993}
I20260709 12:25:39.956079 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:39.978621 10025 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:25:39.943699 10097 rpcz_store.cc:276] 0709 12:25:38.803629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.499144 (+695515us) service_pool.cc:224] Handling call
0709 12:25:39.499459 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.942994 (+443535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.943175 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:39.886423 15178 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.091s	user 0.003s	sys 0.011s
I20260709 12:25:39.979480 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:39.660578 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.630537 15077 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:39.970492  9843 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a reported cstate change: term changed from 5 to 7, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 7 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:39.957274 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.976027 10021 raft_consensus.cc:2376] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 9. Re-sending same reply.
I20260709 12:25:39.989235 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.990113 10021 raft_consensus.cc:2368] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15.
I20260709 12:25:39.963835  9942 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:39.990981  9942 raft_consensus.cc:606] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:39.979626  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:39.962772 10093 rpcz_store.cc:276] 0709 12:25:38.409978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.412868 (+1002890us) service_pool.cc:224] Handling call
0709 12:25:39.432583 (+ 19715us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.432795 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:39.482375 (+ 49580us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.783891 (+301516us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:39.783971 (+    80us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:39.784043 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.784140 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:39.784353 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":50122,"prepare.run_cpu_time_us":9016,"prepare.run_wall_time_us":90283,"raft.queue_time_us":126165,"spinlock_wait_cycles":48384,"thread_start_us":300991,"threads_started":1,"wal-append.queue_time_us":208103}
I20260709 12:25:39.992832 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:39.993050  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:39.993410 10093 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:39.980505 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.994895 14991 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:39.995391 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:39.995761 15197 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:39.996009 10016 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:39.997310 10016 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:25:39.997740 10021 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:39.998785 15212 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:25:39.945475 14693 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.405s	user 0.015s	sys 0.035s
I20260709 12:25:39.982167 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:40.002261 10018 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
W20260709 12:25:39.899562 14904 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.004s
I20260709 12:25:40.003144 14904 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:40.005486 14904 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.006119 14904 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:39.983507 10097 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:39.983530 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:39.994391 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:40.006578 14904 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:40.010663 14904 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.007395 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:40.012948  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47863.4us
I20260709 12:25:40.008678  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:40.009096 10093 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.013267 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:57396 (request call id 850) took 1566 ms. Trace:
I20260709 12:25:39.875744 15041 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:39.954972 14908 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.020711 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:39.868474 15204 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.661806 15085 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.025933 15085 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:39.977365 15139 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.025785 15204 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.026512 15085 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:40.028687 15055 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.697s	user 0.003s	sys 0.007s
I20260709 12:25:40.028815 15085 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.028877 15204 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.014844 10096 rpcz_store.cc:276] 0709 12:25:38.408980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.259141 (+850161us) service_pool.cc:224] Handling call
0709 12:25:39.961602 (+702461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:39.692836 15145 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:39.960755 15127 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:39.987133 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:39.720180 15073 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.302s
I20260709 12:25:39.876348 15137 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.013455 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:40.019830  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:39.624791 15191 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:39.978034 14986 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:39.955260 14911 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:39.977989 15000 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.010s	sys 0.003s
I20260709 12:25:39.977453 14841 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:39.998445 10016 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:40.024439 10097 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.955832 15198 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:39.986716 15168 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:40.059386  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 97920.3us
I20260709 12:25:40.077927 10019 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:39.692895 15169 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:39.910360 15188 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.112s
I20260709 12:25:40.013427 15077 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:39.828573 15147 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.013633 15029 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:40.095376 15212 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:40.095950 14991 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:40.098656 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 135726us
W20260709 12:25:40.317312 15223 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:25:40.031240 15030 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.555s	user 0.009s	sys 0.003s
I20260709 12:25:40.379611 10668 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:40.017449 15155 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:39.987613 15081 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 12:25:39.987879 15193 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.181s
I20260709 12:25:39.651870 15199 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:39.837324 14995 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:39.798179 15189 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:40.435981 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.437088 14986 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:40.437724 14986 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:39.978904 15201 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.202814 10093 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:40.439520 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:40.439666 14693 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:40.440049 14693 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:40.440135 15201 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.031323 15041 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.440622 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:40.440873 14693 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.441749 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:40.442109 14693 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:40.442262 10093 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.015424 15206 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.213s
I20260709 12:25:40.442876 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 849) took 2048 ms. Trace:
I20260709 12:25:40.443285 15206 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 13.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:25:40.443434 10095 rpcz_store.cc:276] 0709 12:25:38.394364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.251459 (+857095us) service_pool.cc:224] Handling call
0709 12:25:39.251829 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.723118 (+471289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.723166 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.723814 (+   648us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.199239 (+475425us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:40.199299 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.199399 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.442674 (+243275us) raft_consensus.cc:1685] finished
0709 12:25:40.442694 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.442819 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":228227,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":51,"thread_start_us":872071,"threads_started":2,"wal-append.queue_time_us":692237}
I20260709 12:25:40.040653 14908 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.447674 14908 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.076066 15055 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.200410 15060 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:25:40.633980  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 614200us
I20260709 12:25:40.708241 15060 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:40.360875  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:40.711804  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 350861us
I20260709 12:25:40.432788 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:40.359583  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 400252us
I20260709 12:25:40.715747 15229 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:40.716032  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:40.716585 15229 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:40.716974 15074 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:40.645809 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:40.647727 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:40.648293 10025 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:40.380859 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:40.442955 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3242) took 2131 ms. Trace:
I20260709 12:25:40.141552 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3220) took 2001 ms. Trace:
I20260709 12:25:40.724857 10094 rpcz_store.cc:276] 0709 12:25:38.084422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.093451 (+  9029us) service_pool.cc:224] Handling call
0709 12:25:40.085103 (+1991652us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:40.085299 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2826917632}
I20260709 12:25:40.726671 10017 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:40.721987 10092 rpcz_store.cc:276] 0709 12:25:38.311928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.194871 (+882943us) service_pool.cc:224] Handling call
0709 12:25:39.195264 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:39.637101 (+441837us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.637136 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:39.657597 (+ 20461us) log.cc:844] Serialized 52 byte log entry
0709 12:25:39.990391 (+332794us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:39.990471 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.990657 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.435573 (+444916us) spinlock_profiling.cc:243] Waited 27.5 ms on lock 0x7b2400223a18. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a7fb0 00007f4e0911e488 00007f4e0911ebe8 00007f4e090f6398 00007f4e090f176a 00007f4e090f0bb6
0709 12:25:40.442665 (+  7092us) raft_consensus.cc:1685] finished
0709 12:25:40.442685 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.442901 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":284298,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":107,"spinlock_wait_cycles":63319424,"thread_start_us":628589,"threads_started":2,"wal-append.queue_time_us":449411}
I20260709 12:25:40.714489 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:40.729308 10020 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:40.730020 10092 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:40.374255 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:40.718833 15227 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:40.731154 10096 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.434139 10097 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:40.735109 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3239) took 2529 ms. Trace:
W20260709 12:25:40.736691 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.736972 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3252) took 1474 ms. Trace:
I20260709 12:25:40.737413 10097 rpcz_store.cc:276] 0709 12:25:39.262240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.735553 (+1473313us) service_pool.cc:224] Handling call
0709 12:25:40.736082 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.736480 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.736510 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.736573 (+    63us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:40.736601 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.736671 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.736795 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.652295 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3247) took 1849 ms. Trace:
I20260709 12:25:40.738807 10090 rpcz_store.cc:276] 0709 12:25:38.803034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.448038 (+645004us) service_pool.cc:224] Handling call
0709 12:25:39.448432 (+   394us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:39.500152 (+ 51720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.500201 (+    49us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:39.541744 (+ 41543us) log.cc:844] Serialized 78 byte log entry
0709 12:25:39.882541 (+340797us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:39.882990 (+   449us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.883339 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 12:25:39.883526 (+   187us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.883624 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.651329 (+767705us) raft_consensus.cc:1685] finished
0709 12:25:40.651371 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.651905 (+   534us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350,"prepare.queue_time_us":300,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":171859,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":24781,"thread_start_us":151867,"threads_started":1,"wal-append.queue_time_us":355275}
I20260709 12:25:40.740932 15227 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:40.201984 14911 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:40.653652 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2045) took 2447 ms. Trace:
I20260709 12:25:40.744578 10098 rpcz_store.cc:276] 0709 12:25:38.206422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.179404 (+972982us) service_pool.cc:224] Handling call
0709 12:25:39.179772 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.942610 (+762838us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.942992 (+   382us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.943818 (+   826us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.092657 (+148839us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:40.092713 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.092825 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.653015 (+560190us) raft_consensus.cc:1685] finished
0709 12:25:40.653066 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.653406 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":275639,"raft.run_cpu_time_us":58,"raft.run_wall_time_us":58,"thread_start_us":753558,"threads_started":2,"wal-append.queue_time_us":492499}
I20260709 12:25:40.747556  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:40.748854  9943 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:40.655764 10095 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:40.753367 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3251) took 1949 ms. Trace:
I20260709 12:25:40.243413 14841 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:40.755615 15015 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:40.756553 14841 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.756718 15015 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.759778 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3255) took 1493 ms. Trace:
I20260709 12:25:40.760571 15043 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.761701 15043 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:40.663086 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.019s	sys 0.035s
W20260709 12:25:40.774921 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.019s	sys 0.035s
I20260709 12:25:40.760279 10090 rpcz_store.cc:276] 0709 12:25:39.266152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.741574 (+1475422us) service_pool.cc:224] Handling call
0709 12:25:40.742149 (+   575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.759061 (+ 16912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.759106 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.759178 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:40.759220 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.759298 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.759486 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:40.775341 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 804303us
I20260709 12:25:40.776753 15043 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:40.778000 10675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1034) took 1740 ms (client timeout 1798 ms). Trace:
I20260709 12:25:40.778162 10090 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 12:25:40.778548 10675 rpcz_store.cc:269] 0709 12:25:39.037414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.039934 (+  2520us) service_pool.cc:224] Handling call
0709 12:25:40.719931 (+1679997us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54007f2e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:40.777727 (+ 57796us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603285376}
I20260709 12:25:40.780272 15149 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:40.380275 10668 raft_consensus.cc:606] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:40.669200 14986 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:40.781385  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.781136 15149 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.783354 10672 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:40.382663 14991 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.672746 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:40.784097 15149 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:40.785780 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:40.530258 15023 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.379411 15029 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:40.392336 15030 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:40.442795 15041 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.385588 15061 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.079563  9843 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a reported cstate change: term changed from 7 to 11, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 11 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:40.431869 15063 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.598431 15077 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:40.535959 15099 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.832617 15030 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:40.084465 15145 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.383085 15139 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:40.375516 15153 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:25:40.460616 15155 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:40.844044 15041 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.843796 15155 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.844745 15041 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.845125 14911 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.379815 15168 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:40.848275 14911 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:40.849561 15168 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:40.376770 15169 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:40.523772 15177 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.729s	user 0.013s	sys 0.005s
W20260709 12:25:40.853520  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:40.538231 15179 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.743s	user 0.002s	sys 0.018s
I20260709 12:25:40.384521 15191 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.282958 15194 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:40.855412 10668 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:40.855582 15194 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.858079 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:40.143299 15197 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:40.859514 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 962) took 2489 ms. Trace:
I20260709 12:25:40.540172 15198 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.860010 10669 rpcz_store.cc:276] 0709 12:25:38.369942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.370049 (+   107us) service_pool.cc:224] Handling call
0709 12:25:38.370353 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:38.370739 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:38.371020 (+   281us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.805019 (+433999us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:38.805779 (+   760us) log.cc:844] Serialized 26 byte log entry
0709 12:25:38.806006 (+   227us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:38.817172 (+ 11166us) log.cc:844] Serialized 14 byte log entry
0709 12:25:38.847868 (+ 30696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:38.847936 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:39.655883 (+807947us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b1800452920. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db
0709 12:25:40.523093 (+867210us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:40.858939 (+335846us) raft_consensus.cc:1685] finished
0709 12:25:40.858988 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.859245 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":133062,"prepare.queue_time_us":707,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":4899,"raft.queue_time_us":260842,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":27671,"spinlock_wait_cycles":48204672,"thread_start_us":244104,"threads_started":1,"wal-append.queue_time_us":423665}
I20260709 12:25:40.473915 15199 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:40.379048 15205 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.455s	user 0.001s	sys 0.001s
W20260709 12:25:40.376868 15207 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.487s	user 0.002s	sys 0.000s
W20260709 12:25:40.383581 15211 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.297s	user 0.001s	sys 0.000s
W20260709 12:25:40.525765 15218 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.614s	user 0.001s	sys 0.000s
I20260709 12:25:40.375370 15212 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:40.869800 15212 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:40.871744 15169 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.873759 15194 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.875353  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2172) took 1419 ms. Trace:
I20260709 12:25:40.870075 15147 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:40.878274 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3248) took 2016 ms. Trace:
I20260709 12:25:40.875804  9945 rpcz_store.cc:276] 0709 12:25:39.455731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.457854 (+  2123us) service_pool.cc:224] Handling call
0709 12:25:39.458325 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:39.499781 (+ 41456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:39.499812 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:39.500269 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:25:39.751233 (+250964us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:39.751289 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:39.751382 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:40.524983 (+773601us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:40.874757 (+349774us) raft_consensus.cc:1685] finished
0709 12:25:40.874805 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.875106 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21921,"raft.queue_time_us":95677,"raft.run_cpu_time_us":2716,"raft.run_wall_time_us":65690,"spinlock_wait_cycles":23563520,"thread_start_us":250607,"threads_started":1,"wal-append.queue_time_us":399075}
I20260709 12:25:40.879528 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:40.880275 10674 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:40.878813 10091 rpcz_store.cc:276] 0709 12:25:38.803326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.448216 (+644890us) service_pool.cc:224] Handling call
0709 12:25:39.448504 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.448762 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:39.724081 (+275319us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.521511 (+797430us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b2400223a18. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a809d 00007f4e090da10e 00007f4e090f188c 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221
0709 12:25:40.801962 (+280451us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.802098 (+   136us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:40.802207 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.812320 (+ 10113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.819536 (+  7216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":766442,"prepare.queue_time_us":189043,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":69,"raft.queue_time_us":751168,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":187,"spinlock_wait_cycles":72018944,"thread_start_us":585584,"threads_started":3,"wal-append.queue_time_us":418017}
I20260709 12:25:40.431221 15217 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:40.889142 15217 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:40.521915 15221 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:25:40.720142 15174 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:40.891067 15174 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:40.860176 15197 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.892879 15174 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.720912 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:40.722231 10019 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:40.742707 15227 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:40.895247 10025 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:40.467774 14971 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.192s
I20260709 12:25:40.765906 15015 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.766562  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:40.767092  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:40.753842 10095 rpcz_store.cc:276] 0709 12:25:38.804231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.445267 (+1641036us) service_pool.cc:224] Handling call
0709 12:25:40.445560 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.750856 (+305296us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.752967 (+  2111us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540002dd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:40.753274 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480872192}
I20260709 12:25:40.770663 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3256) took 1504 ms. Trace:
I20260709 12:25:40.770337 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 856) took 1829 ms. Trace:
I20260709 12:25:40.899070 10094 rpcz_store.cc:276] 0709 12:25:38.940534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.768931 (+1828397us) service_pool.cc:224] Handling call
0709 12:25:40.769532 (+   601us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.769950 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.769978 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.770046 (+    68us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:40.770087 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.770161 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.770261 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17792}
I20260709 12:25:40.899020 10098 rpcz_store.cc:276] 0709 12:25:39.266450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.746326 (+1479876us) service_pool.cc:224] Handling call
0709 12:25:40.769482 (+ 23156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.770277 (+   795us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.770315 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.770360 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:40.770403 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.770462 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.770606 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":773,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":169,"spinlock_wait_cycles":3456}
I20260709 12:25:40.772881 14760 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.901325 14760 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.714526 15055 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.710501 15060 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:40.903035 14904 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:40.904527 14760 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.788237 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 857) took 1821 ms. Trace:
I20260709 12:25:40.905333 10092 rpcz_store.cc:276] 0709 12:25:38.966676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.728901 (+1762225us) service_pool.cc:224] Handling call
0709 12:25:40.729595 (+   694us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.730445 (+   850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.788177 (+ 57732us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:40.788237 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3253) took 1522 ms. Trace:
I20260709 12:25:40.906694 10097 rpcz_store.cc:276] 0709 12:25:39.265530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.738522 (+1472992us) service_pool.cc:224] Handling call
0709 12:25:40.773704 (+ 35182us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:25:40.774049 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.774509 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.774538 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.787985 (+ 13447us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:40.788040 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.788099 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.788190 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27847552}
I20260709 12:25:40.791231 10672 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:40.793502 10096 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
W20260709 12:25:40.798995 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56373.2us
W20260709 12:25:40.790774  9912 connection.cc:256] RPC call timeout handler was delayed by -1.1310133870001664s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
I20260709 12:25:40.909060 15201 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:40.783582 14986 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:40.814421  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 494062us
I20260709 12:25:40.910105 15201 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.911123 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:40.912230 10670 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:40.914156 15015 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:40.830924 10023 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:40.832288 15077 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.833570 15029 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:40.837035 15145 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.838539  9843 catalog_manager.cc:5697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 8, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 8 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:40.846531 10671 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:40.926569 10093 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:40.927809 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:40.928072 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1040) took 1638 ms. Trace:
I20260709 12:25:40.927479  9843 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 10, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 10 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:40.295730 15219 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.272s	user 0.000s	sys 0.000s
I20260709 12:25:40.946211 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:40.955590 10095 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:40.586483 15216 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:40.898295  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 132540us
I20260709 12:25:40.900226 15041 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.901798 10094 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:25:40.772157 15229 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:40.909533  9912 connection.cc:256] RPC call timeout handler was delayed by -1.1301133870001649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
I20260709 12:25:40.735608 10099 rpcz_store.cc:276] 0709 12:25:38.205486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.060718 (+855232us) service_pool.cc:224] Handling call
0709 12:25:39.124458 (+ 63740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:39.124702 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:39.250371 (+125669us) log.cc:844] Serialized 14 byte log entry
0709 12:25:40.733641 (+1483270us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b1800343c00. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:40.734299 (+   658us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:40.734413 (+   114us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:40.734603 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.734701 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:40.734905 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17349,"prepare.queue_time_us":237078,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"spinlock_wait_cycles":4797824,"thread_start_us":806530,"threads_started":2,"wal-append.queue_time_us":727759}
I20260709 12:25:40.814397  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:40.829982 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:40.895279 15227 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:40.915139 10090 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:40.915177 14986 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:40.916229 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:40.898800 15217 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:40.917450 14819 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:40.919552 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:40.877314 15198 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:40.923523 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:40.921097 15015 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:40.993625 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2074) took 1019 ms. Trace:
I20260709 12:25:40.995229 10094 rpcz_store.cc:276] 0709 12:25:39.968224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.900630 (+932406us) service_pool.cc:224] Handling call
0709 12:25:40.901172 (+   542us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.985882 (+ 84710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:40.987963 (+  2081us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:25:40.925047 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.025s	sys 0.015s
W20260709 12:25:40.433331 15196 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.995s	user 0.005s	sys 0.013s
I20260709 12:25:40.928457 10671 rpcz_store.cc:276] 0709 12:25:39.289406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.459022 (+169616us) service_pool.cc:224] Handling call
0709 12:25:40.927851 (+1468829us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:40.928015 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078834048}
I20260709 12:25:40.928551 10093 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:40.932410  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:40.934316 10019 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:41.023571 15227 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.024995 15227 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:41.025908 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:40.896135 15174 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:40.937974 10674 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:40.945044 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2025) took 1482 ms. Trace:
I20260709 12:25:40.946621  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:40.947667 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:40.924185  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:40.960120 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:40.978827 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49870.8us
I20260709 12:25:40.987556  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:40.987530  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0401133870000194s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16140000000000002s for a response
W20260709 12:25:41.036712  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0304127189997416s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1575s for a response
W20260709 12:25:41.037101  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0167133869999816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.164s for a response
I20260709 12:25:41.038018  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:40.986308 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:41.039384 10097 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:41.039434  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:41.040441 10022 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:41.041898 10098 pending_rounds.cc:85] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 5
I20260709 12:25:40.921733 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:41.042371 10098 pending_rounds.cc:107] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 13.6
I20260709 12:25:41.026715 14904 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.042845 10098 raft_consensus.cc:2889] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:41.043138 14904 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.029904 10023 rpcz_store.cc:276] 0709 12:25:39.462414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.462923 (+   509us) service_pool.cc:224] Handling call
0709 12:25:40.944727 (+1481804us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54001383d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:40.944949 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2812305664}
W20260709 12:25:40.999297 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.015s	user 0.026s	sys 0.016s
I20260709 12:25:41.045642 14904 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:41.046083 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.10543e+06us
I20260709 12:25:41.046442 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:41.027308 15216 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:41.049340 14693 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:41.050081 14693 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:41.049441 15216 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.004711  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:41.050577 14693 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:41.051384  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46811.5us
I20260709 12:25:41.005218 14971 raft_consensus.cc:993] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:25:41.007261 10099 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 12:25:41.008332 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3161) took 1386 ms (client timeout 1714 ms). Trace:
I20260709 12:25:41.007869 14986 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.009282  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:41.009418  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 275354us
I20260709 12:25:41.013540 14841 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:40.992996 14819 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:41.016175 15015 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.027158 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:41.027631 10017 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:41.027976 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:41.027653 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:41.028952 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2071) took 1061 ms. Trace:
I20260709 12:25:41.072351 10090 rpcz_store.cc:276] 0709 12:25:39.967056 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.780234 (+813178us) service_pool.cc:224] Handling call
0709 12:25:40.909569 (+129335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:40.993212 (+ 83643us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.027918 (+ 34706us) spinlock_profiling.cc:243] Waited 5.2 ms on lock 0x7b54002fd354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:41.028697 (+   779us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54002fd350. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:41.028862 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":308384384}
I20260709 12:25:41.035517 10096 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.038173 10091 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:40.994058 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81177.3us
I20260709 12:25:41.061534 10093 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
W20260709 12:25:41.055418 10024 rpcz_store.cc:269] 0709 12:25:39.621449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:39.955479 (+334030us) service_pool.cc:224] Handling call
0709 12:25:41.007983 (+1052504us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:41.008150 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2325828480}
I20260709 12:25:41.081365  9945 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:41.069872  9843 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:40.992027 15201 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:41.049223 15041 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:25:41.083451  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:41.026571 15077 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:41.068881 15074 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.089251  9946 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:41.103742 10672 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:41.080957 14971 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.010s
I20260709 12:25:41.109167  9943 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:41.095963 15216 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.139050  9942 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:41.142406 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48896.4us
I20260709 12:25:41.143277 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:41.141587  9940 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:41.147637  9941 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 14 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 12:25:41.150889 15201 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.151371 10095 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:41.151741 15201 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.152384 15191 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:25:41.154506 15201 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:41.155164 14693 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.158622 14693 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:41.160274 14986 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:41.160403 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:41.161897 15201 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:41.163645 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.164741 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:41.165933 15201 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:41.169051  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.169874 15201 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:41.170078  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:41.171166  9941 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:41.178146  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:41.176878 14297 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:41.179965 15155 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:41.186708 10677 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:41.187494 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.187043 15155 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.188531 15063 raft_consensus.cc:1081] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:25:41.187505 14297 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.187053 15191 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:41.198886 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:41.200836  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56634.5us
I20260709 12:25:41.200635 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:41.224948 10675 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:41.198554  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:41.202280 14297 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:41.231596 15251 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.197s	user 0.007s	sys 0.004s
I20260709 12:25:41.190090 14819 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:41.141525 15077 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.202970 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:41.238854  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47344.9us
W20260709 12:25:41.215762 15257 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.079s	user 0.002s	sys 0.002s
I20260709 12:25:41.229825  9843 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 8 to 11, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 11 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:41.247185 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:41.243043 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:41.241763 10676 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:41.136897 15197 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
I20260709 12:25:41.203442 15191 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:41.267062  9941 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:25:41.270748  9912 leader_election.cc:336] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.513s (SENT)
W20260709 12:25:41.279203  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.254583 15147 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.274472 10096 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:41.281862 14819 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.281659 14995 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
W20260709 12:25:41.282754 15259 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.259s	user 0.000s	sys 0.002s
W20260709 12:25:41.283787 15252 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 12:25:41.285763  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 180488us
I20260709 12:25:41.288196 15077 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:41.293058 15199 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.295578 14911 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:25:41.298614 15255 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.316s	user 0.005s	sys 0.000s
W20260709 12:25:41.304704  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.306137 15099 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.269632 14904 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.298656 14841 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:25:41.309338  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:41.314996  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 211983us
W20260709 12:25:41.320410  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58879.3us
I20260709 12:25:41.320609 10675 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
W20260709 12:25:41.324905 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.325369 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.339191  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:41.340304  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.321337 15155 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:41.344602  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:41.344076  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:41.348955  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:41.349882  9940 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:41.350286  9943 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:41.351541 14841 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:41.351933 14841 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:41.352257 15061 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:41.352706 15061 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:41.352996 14841 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.353694 15061 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.349514 14991 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:25:41.327379 14911 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:41.366039 10091 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:41.355057  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:41.374691 15145 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:41.375182 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:41.375411 15145 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:41.375808 10669 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.369534 14693 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:25:41.377460 14693 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:41.377761 15145 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:41.378453 15099 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.378962 14693 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.354532 14841 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.390492 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:41.392601  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:41.395040  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:41.337127 15147 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.418462  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:41.421936 14995 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:41.422658 14995 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:41.423652 14995 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.395749 15077 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:41.442562  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:41.357043 15199 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.449188 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:41.449229  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:41.456215  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:41.458442  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 103679us
I20260709 12:25:41.449007 15061 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.461515 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:41.432626 15201 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
W20260709 12:25:41.467563  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 205461us
I20260709 12:25:41.461556  9843 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 5 to 10, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 10 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:41.458071 15229 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.477905 10669 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:41.481365 14819 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.483083  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:41.485260 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:41.485605 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:41.486156 15227 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:41.486701 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 848) took 1059 ms. Trace:
I20260709 12:25:41.490830  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:41.492240  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:41.492957 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:41.493875 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:41.494690 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:41.495412 10022 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:25:41.495685 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:41.488970 15212 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.618s	user 0.009s	sys 0.005s
W20260709 12:25:41.542647  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50494.8us
I20260709 12:25:41.543690  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:41.496893 10023 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:41.501281  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.495472 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:41.435791 15191 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.547906 10675 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:41.548434 10675 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:41.523909 15199 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:41.529152  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.537249 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:41.540542  9945 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:41.540910  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.514s (SENT) [suppressed 53 similar messages]
I20260709 12:25:41.513639 15077 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:41.552703  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:41.554090 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:41.557636 14760 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:41.531283 15174 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:41.559419 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 192151us
I20260709 12:25:41.538122 10016 rpcz_store.cc:276] 0709 12:25:40.379977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.380104 (+   127us) service_pool.cc:224] Handling call
0709 12:25:40.384027 (+  3923us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:40.384436 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:40.384468 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:40.384997 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:25:40.826546 (+441549us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:41.055072 (+228526us) log.cc:844] Serialized 14 byte log entry
0709 12:25:41.391437 (+336365us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:41.407216 (+ 15779us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:41.407282 (+    66us) raft_consensus.cc:1685] finished
0709 12:25:41.407300 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:41.439050 (+ 31750us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8905,"prepare.queue_time_us":374228,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":59186,"raft.run_cpu_time_us":5926,"raft.run_wall_time_us":44925,"thread_start_us":975869,"threads_started":3,"wal-append.queue_time_us":314160}
I20260709 12:25:41.566735 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:41.563087 10021 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:41.558080  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67502.1us
I20260709 12:25:41.558004 15227 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:25:41.580247 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:41.570377 10675 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:41.570574 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 222112us
I20260709 12:25:41.558081 15060 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:41.602846 10675 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.551244 15212 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:41.571506 15229 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:41.598469 14991 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:41.582167 15227 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:41.596233 14760 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:41.621292  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 68718.8us
I20260709 12:25:41.621544 15060 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:41.613018 15077 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:41.633036 14991 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:41.611910 15265 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:25:41.674189 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:41.683795 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.683529 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:41.684617 10093 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:41.646785 15191 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.644474 15147 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:41.678174 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:41.621716 15029 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:25:41.689198 10091 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: [12.7-12.7]
W20260709 12:25:41.691780  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 159028us
I20260709 12:25:41.694372 10099 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:41.643541 15229 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 12:25:41.707041 10090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:25:41.707846 10090 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:41.708243 10090 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.716465 15077 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:41.719041 14760 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.719134 14991 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.705372 10095 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:41.722455 10096 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:41.722811  9912 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.575s (SENT)
I20260709 12:25:41.739707 10675 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
W20260709 12:25:41.743369 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 169391us
I20260709 12:25:41.741833  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:41.745965  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 122407us
I20260709 12:25:41.730957 15199 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:41.749150 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:41.757761 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 97660.7us
I20260709 12:25:41.757117 10095 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:41.761556  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:41.771914 15061 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:41.773805 15061 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:25:41.782723 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.784459 15061 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:41.785135 15061 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:41.787729 15015 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.703s	user 0.006s	sys 0.006s
I20260709 12:25:41.788965 15015 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.790172 10092 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:41.786506 15061 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:41.790999 15074 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:41.791440 15074 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:41.791751 15061 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.792284 15074 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.793368 15074 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:41.797492 15015 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:41.761512 15139 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:41.799137 15139 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.801832 15139 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:41.802784 15212 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:41.767483 15216 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:41.805425 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:41.806247 15015 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:41.807811 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:41.809279 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:41.809124 15227 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.810194 15199 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:41.811053 15199 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:41.811199  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 53464.3us
I20260709 12:25:41.811666 15199 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:41.812211  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:41.813285 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:41.813665  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:41.814797 15265 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:41.815104 14911 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.816306 14911 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:41.816406 15275 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:41.815646 15265 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:41.811260 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:41.823675 10096 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:25:41.834208  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 111467us
I20260709 12:25:41.835950 15278 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:41.837625 15040 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:41.838040 15040 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:41.838265 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:41.846637 15040 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.848256 15040 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:41.842350 15060 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:41.859596 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:41.862165 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:41.863312 10096 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:41.864952  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.865960  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.866782  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.854300 15099 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:41.877983 10093 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:41.880425  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 66846.2us
I20260709 12:25:41.893733  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:41.842893 15145 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:41.902522 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:41.895150  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:41.905731 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:41.920367 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:41.920367 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:42.119938 10093 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:41.882197 15278 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:41.926383 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:41.932253  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:41.939139  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:41.902640 15145 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:42.099815 10096 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:42.100044 10022 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:25:42.103121 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:42.103899 15217 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:41.915865 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:42.105296 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:41.891235 15197 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
W20260709 12:25:42.106439  9912 leader_election.cc:336] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.693s (SENT)
W20260709 12:25:42.107030 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.135315  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.137833 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:42.118649 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3262) took 1204 ms. Trace:
I20260709 12:25:41.922219 15099 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:25:41.898447 15229 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:42.122756 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 226100us
I20260709 12:25:42.122488 10020 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:42.125095 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:42.142066 10090 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.138826 10092 rpcz_store.cc:276] 0709 12:25:40.914030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:40.921964 (+  7934us) service_pool.cc:224] Handling call
0709 12:25:41.903270 (+981306us) spinlock_profiling.cc:243] Waited 796 ms on lock 0x7b5400305a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:42.118552 (+215282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1831903488}
I20260709 12:25:42.146198 10025 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:42.139341 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.143790 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:42.117486  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.158102 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:42.159739 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:42.161381 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:42.135912 15145 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.165009 10016 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:42.165666 10090 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:42.131268 15278 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:42.167896 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 270965us
I20260709 12:25:42.173331 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:42.173995 10020 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:42.174582 10016 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:42.176014  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.176095 10097 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:42.177922 15284 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:42.175115 15139 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:42.178375 15284 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:42.179594 10090 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:42.179479 15284 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.180193 15227 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:25:42.180794 15284 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:42.136168 15147 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:25:42.182106 10090 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:25:42.171335 15278 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.185390  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:42.186764 10018 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:42.187532  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:42.187285 15229 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.188690 15147 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:42.189597 15229 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:42.189829  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:42.190335 14904 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:42.190904 14904 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:42.175391 15217 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:42.191237 15153 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:42.191701 15153 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:42.191759 15217 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.192314 15265 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:42.192921 15153 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.137564 14995 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:25:42.192027 14904 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.203110 15139 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:42.203684 15139 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:42.207477 10092 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:42.207827 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.208688 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:42.209362 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:42.210078 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:42.210882 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:42.209590 10020 raft_consensus.cc:2410] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 12:25:42.212330 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:42.214331  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:42.214452 10025 raft_consensus.cc:2376] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10. Re-sending same reply.
I20260709 12:25:42.194126 15153 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:42.218012  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 111609us
W20260709 12:25:42.218778  9912 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.714s (SENT)
I20260709 12:25:42.219151  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:42.190517  9942 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:42.220346  9912 leader_election.cc:336] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.733s (SENT)
W20260709 12:25:42.221390  9912 leader_election.cc:336] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.745s (SENT)
W20260709 12:25:42.222666 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.221764  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:42.151424 14819 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.005s
I20260709 12:25:42.226145 10022 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:25:42.227474 10022 pending_rounds.cc:85] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 11
I20260709 12:25:42.227792 10022 pending_rounds.cc:107] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:25:42.228055 10022 raft_consensus.cc:2889] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.231029 10096 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:42.219192 14693 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
W20260709 12:25:42.236460 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 99998.2us
I20260709 12:25:42.249040 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:42.249400 10092 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:42.254343 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:42.231036 14819 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:42.271181  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49827.3us
I20260709 12:25:42.271364 15015 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:42.271962  9912 leader_election.cc:336] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.756s (SENT)
W20260709 12:25:42.272644 15191 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.582s	user 0.009s	sys 0.005s
W20260709 12:25:42.272905  9912 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.835s (SENT)
I20260709 12:25:42.273041  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:42.275601 15275 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:42.226711 15197 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:42.260413 10092 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:42.261868 15199 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.262988 15192 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.263715 10021 raft_consensus.cc:2410] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:25:42.265818 10017 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:42.265610  9942 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:42.266471 10018 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:42.266929 15229 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:42.269109 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 425250us
I20260709 12:25:42.273336  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:42.194448 15217 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:42.277112 15077 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:42.280759  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.250356 14991 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.299522 15061 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:42.304772 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 126963us
I20260709 12:25:42.278723 10095 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:42.257368 14995 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:42.308925 10093 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:42.314025 10098 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:25:42.296495 15074 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:42.301285 15174 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.307889 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:42.314451 10098 pending_rounds.cc:85] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 6
I20260709 12:25:42.317783 10098 pending_rounds.cc:107] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:25:42.318250 10098 raft_consensus.cc:2889] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.306224 15077 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:42.300287 14841 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:25:42.309018 15197 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:42.330402  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:42.341434  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 68578.2us
I20260709 12:25:42.346360 14297 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:42.352098  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:42.352823  9942 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:42.352913 15041 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:42.353219  9942 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:42.353636 15041 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.355978 15041 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:42.357071 15291 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:42.357400 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:42.359316 15294 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.337909 15201 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:42.360586 15201 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:42.361321 15278 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.361654 15201 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.303085 15192 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.363269 14995 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.312886 15229 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.364100 14297 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:42.352656 15077 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.315485 15074 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:42.370816 15263 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:42.372038 15145 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.354128  9942 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:42.400267 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 212511us
I20260709 12:25:42.323381 15174 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.402972  9942 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.404631 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:42.409019  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 133917us
I20260709 12:25:42.401969 15074 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:42.398108 14841 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:42.410121 14297 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:42.414140  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 161662us
W20260709 12:25:42.442725  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.414196 10092 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:42.444996  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:42.446759  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.451017 10094 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:42.454654  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:42.445726 14841 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:42.463223  9946 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 14 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:42.465692 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 97485.5us
W20260709 12:25:42.469441  9912 leader_election.cc:336] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.019s (SENT)
I20260709 12:25:42.471287  9941 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:42.472957 15229 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:42.473803 15229 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:42.476397  9944 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 12:25:42.478926 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79413.8us
I20260709 12:25:42.480667 15269 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:42.480597 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:42.486076 10097 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:42.464653 15029 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.486939 15029 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.488612 10093 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:42.415202 15295 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.507296 15029 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:42.511806  9945 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:42.524816  9942 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:42.467131 14911 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:42.534466  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 89543.8us
I20260709 12:25:42.537765  9942 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:25:42.544401 15145 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:42.545003 15304 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:42.545470 15304 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:42.545877 15304 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.548290 14995 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:42.548875 14995 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:42.549211 14995 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:42.528163 15199 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:42.463747 15155 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:25:42.552795 15305 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:42.511179 15015 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:42.540617 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:42.564647 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:42.566644 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.532826 15063 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:25:42.614329 15273 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:42.550104 14995 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:42.614885  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6c7bd3fa1bfa431c89fc9c9096452a76 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 9 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:42.616182 14995 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:42.574872  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:42.575338  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 105957us
I20260709 12:25:42.578209 15015 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:42.574963 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:42.627487 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:42.579253 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44930.1us
I20260709 12:25:42.631562 10092 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
W20260709 12:25:42.632681 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 65273.5us
I20260709 12:25:42.587949 15077 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:42.595059  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:42.634436 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57562.6us
I20260709 12:25:42.599469 15174 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:42.602674 15265 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:42.603262 15192 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:42.644556  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:42.662786 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 87923.2us
I20260709 12:25:42.650430 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:42.618549 15301 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:25:42.667158 10097 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:42.570806 15155 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:42.671476 15199 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:25:42.677364 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43738us
I20260709 12:25:42.677826 15199 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:42.678331 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:42.562551 15099 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.109s
I20260709 12:25:42.679610 10098 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:42.679173 15199 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.683233 10092 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:42.683403  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.683777  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.684207  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.685860  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:42.686419  9945 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.688510 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:42.619632 15269 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:42.695928 10640 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:42.696870 15297 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:42.698218 15296 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:42.563180 14911 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.651175 15265 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:42.703858  9912 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.094s (SENT)
W20260709 12:25:42.711652  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.704968  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:42.711674 14297 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:42.711880  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:42.701753 10097 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:42.705981  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:42.705904 15060 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:42.672403 15273 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:42.704559 15269 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:42.714378 15139 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
I20260709 12:25:42.687911 15040 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:25:42.706975 15217 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:42.720909 10093 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:25:42.690169 15041 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:25:42.704459 15278 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:42.725082  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.727783  9949 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:42.728919  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:42.714257 15275 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:42.691134 15153 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:25:42.729956  9940 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 8 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
W20260709 12:25:42.731752 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.715420 15147 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:42.691124 15198 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:25:42.733081  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:42.732882 15217 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.733688  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:42.733893 15041 consensus_queue.cc:1243] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:25:42.734663 15265 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.724952 15139 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:42.741856 15299 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.250s	user 0.001s	sys 0.001s
I20260709 12:25:42.710355 15192 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:25:42.743476 15305 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.745186  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:42.747001  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:42.748072 14911 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:42.759557 14693 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:42.716766 15273 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:42.809033 15273 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.809892 15217 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:42.811136 15041 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:42.811555 15041 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:42.812217  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:42.812440 15041 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.813699 15041 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:42.763401  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.766988 15275 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:42.760421 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72723.8us
I20260709 12:25:42.816661 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:42.817150 15273 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.818002 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:42.773075  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.778553  9945 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:25:42.819061  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.819584  9945 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:42.820513  9942 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:42.651161 14841 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:42.792441 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.793983  9941 raft_consensus.cc:2410] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:25:42.791940 14971 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:25:42.829238 14971 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:42.829628 14971 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:42.829702 15269 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:42.830480 14971 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:42.831633  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 139599us
I20260709 12:25:42.832177 14971 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:42.832409  9912 leader_election.cc:336] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.121s (SENT)
I20260709 12:25:42.832932  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:42.794452  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:42.834050 14693 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:42.834126  9944 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:42.834699  9943 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:25:42.835062  9943 pending_rounds.cc:85] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 6
I20260709 12:25:42.835315  9943 pending_rounds.cc:107] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:25:42.835554  9943 raft_consensus.cc:2889] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:42.796034 15265 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:42.797252  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.797152  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:42.797811 15192 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:42.797840 15139 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:42.798213  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:42.797847  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:42.812901  9940 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:42.760414 15304 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:42.794494 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 98683us
I20260709 12:25:42.812230 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:42.780552 15174 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:42.824277  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.846429 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:42.795503  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:42.836386  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:42.837261  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44126.3us
I20260709 12:25:42.786486 15061 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:42.755235 15197 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.081s
I20260709 12:25:42.845243 15198 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:42.863008 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:42.867834 10091 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:42.868641 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:42.869271 10096 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:42.845566 15295 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:42.877810  9912 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.128s (SENT)
W20260709 12:25:42.886694  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 88914.1us
I20260709 12:25:42.887833  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:42.864420 14760 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:42.890691 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72754us
I20260709 12:25:42.876333 14991 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:42.865733 15198 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:42.867007 15309 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:42.896212 10096 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:42.840548 15305 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:42.897881 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 131776us
I20260709 12:25:42.854930 15192 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.897832  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:42.874492 15139 raft_consensus.cc:2764] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:25:42.907682  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:42.912830  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44155.5us
I20260709 12:25:42.914812  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:42.915091 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52160.4us
I20260709 12:25:42.918769  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:42.921814 15319 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:25:42.922663  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.895362 15198 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:42.932854  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:42.939452  9944 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:42.906759 15077 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:42.943365 15074 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:42.912498 14995 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:42.938738 15174 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:42.950548 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:42.963670 15324 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:42.964282 15324 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:42.965310 15324 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:42.984706  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:42.984262 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:42.956161 15314 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.137s	user 0.006s	sys 0.001s
I20260709 12:25:42.896057 14760 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:42.990631  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 117651us
I20260709 12:25:42.961395 15309 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:42.960510 14904 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:43.009572 10096 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
W20260709 12:25:42.994581  9912 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.137s (SENT)
I20260709 12:25:43.019366  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:43.010721  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:43.016021 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.021013 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 103485us
I20260709 12:25:43.027357  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:42.945436 15198 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:43.031843 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43906.3us
I20260709 12:25:43.032133  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:43.015650 15029 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:42.991125 15145 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 12:25:43.008981 14995 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:43.027557 15174 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.022526 15077 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:43.027166 15147 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.054469 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:43.056850  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.067667 15340 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:25:43.071409  9947 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:25:43.040299 15015 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:43.038522 15323 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:43.069253  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81952.3us
I20260709 12:25:43.040208 15029 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:43.030128 15060 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:25:43.049798 15326 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:43.090137 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57348.8us
I20260709 12:25:43.053604 15201 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.093451 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:43.053895 15269 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:43.081250 15198 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.056561 14904 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:43.079725 15155 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:43.076588 15279 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:25:43.106441  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.108282  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:43.108517  9945 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:43.087927 15029 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:43.056756 15077 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:43.114823  9912 leader_election.cc:336] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.186s (SENT)
I20260709 12:25:43.086701 15074 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.053807 15174 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.066357 15147 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.104571 14904 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.098274 15323 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:43.104858 15155 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:43.121950 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:43.155400 10091 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:43.122691 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.121124 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.150017 15192 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:43.151508 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 102721us
I20260709 12:25:43.157533 10094 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:43.119357 15325 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:43.171335 15229 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:43.171581 15337 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:43.152480 14904 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:43.120842 14995 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.174608 15155 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.158715 15323 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.178984 15029 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.180814 15147 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:43.181306 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 90170.8us
I20260709 12:25:43.182966 14819 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:43.186256 10096 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:43.187703 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:43.115401 15201 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.152971 15305 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.193897  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:43.194833 15328 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:25:43.170442 14841 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:43.203300 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.200830 14819 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:43.207275 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.190574 15325 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:43.207732 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.210427 14297 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:43.212030 14297 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:43.214463 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47056.4us
I20260709 12:25:43.190412 15229 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.217108 10669 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
W20260709 12:25:43.219427  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116078us
I20260709 12:25:43.220413 14297 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.223816 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:43.224606 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:43.224875 15315 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.401s	user 0.000s	sys 0.005s
I20260709 12:25:43.225157 15343 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.225368 15325 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.214522 15139 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:43.225773 15343 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.227999 15343 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.228852 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3293) took 1004 ms. Trace:
I20260709 12:25:43.229245 15174 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.229223 10095 rpcz_store.cc:276] 0709 12:25:42.224165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:42.604973 (+380808us) service_pool.cc:224] Handling call
0709 12:25:42.605351 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:42.605730 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:42.605762 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:42.606232 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 12:25:42.709191 (+102959us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:42.709229 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:42.709289 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:43.183615 (+474326us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:43.228537 (+ 44922us) raft_consensus.cc:1685] finished
0709 12:25:43.228560 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:43.228763 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17950,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":35575,"spinlock_wait_cycles":83131136,"thread_start_us":102568,"threads_started":1,"wal-append.queue_time_us":188218}
I20260709 12:25:43.208542 15269 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:43.232589 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:43.233350 10025 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:43.235150 10096 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:43.236454 10025 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:43.237668 14297 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.204545 15297 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.015s
W20260709 12:25:43.224938 15312 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.403s	user 0.004s	sys 0.000s
I20260709 12:25:43.214670 15155 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.233180 10677 raft_consensus.cc:2376] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13. Re-sending same reply.
I20260709 12:25:43.223726 14971 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:43.232334 14760 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:43.253288  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.253052 15197 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.253916 15197 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:43.261092  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.226490 15139 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.248914 14971 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:43.262415 10673 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:43.278733 10669 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 7. Current term is 8. Ops: []
I20260709 12:25:43.285646 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.282475  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:43.292575 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.294807 10091 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
W20260709 12:25:43.301893  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 193692us
W20260709 12:25:43.307485  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.291738 14971 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.314059 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.316797 10669 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:43.315685  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:43.296825 15201 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:43.321038 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:43.321491  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81862.7us
I20260709 12:25:43.321384 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.256072 14819 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.284205 15060 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:25:43.310956 15326 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:25:43.328214  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.328647 10671 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:43.330358  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:43.330727  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.309376 15343 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:25:43.315258 15174 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:25:43.329860 15192 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:43.330284 15326 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:43.356516 10091 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:43.358559 15139 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:43.358881 15348 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:25:43.335350 15029 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.359306 15323 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.363533 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.365078 10090 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:43.366641 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:43.367525 15229 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.356216 15197 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.375012 15354 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:43.378368 15273 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.360847 15326 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.378971 15343 consensus_queue.cc:1243] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:43.367527 10090 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 13 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:43.386349  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:43.391346 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:43.358559 15342 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.395493 14297 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:43.396878 14297 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:43.397395 14297 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:43.401263 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.407157 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:43.411814 10672 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:43.412819 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.413940 10096 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:43.410507 14297 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.419788 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.421152 10675 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:43.424481 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.425410 10671 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:43.427850 14297 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:43.431231 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.360216 15337 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.057s
W20260709 12:25:43.435856  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.441468 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.456279 10022 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:43.360158 14904 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.457294 10098 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:43.458573 10090 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:43.458138 15265 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.613s	user 0.003s	sys 0.010s
W20260709 12:25:43.504212 15349 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:25:43.406672 15342 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.462260 15309 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:43.457881 15275 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:43.696362 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.641152 15343 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:25:43.685282 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.700764 15174 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.650056 15337 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.708961 15174 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.709595 15337 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.712513 15337 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.666620 15273 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:43.717185 15273 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.678619 15354 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.648610 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.657843 14995 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.724932 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.736613 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:43.737488 10090 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:43.737429 10092 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:43.739702  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.740295  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.647548 14904 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.753707  9947 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:43.754103 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:43.755610 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47300us
W20260709 12:25:43.760855  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.747895 15343 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.777139 14995 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.777374 15343 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.779477 14995 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.779714 15343 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.779986 14995 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.780510 14995 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.726451 15354 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:43.783556 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.809952  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:43.810612  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.691303 15304 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.819197  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:43.820626  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.818847  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.827662 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.736985 15342 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.832044  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.833549 10096 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:43.833447 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:43.784687 10090 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:43.837845 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.838774 10091 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:43.753234 15309 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.843345 10096 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:43.843536  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.841707  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.841748  9944 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:43.771878 15029 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.712546 15174 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:43.850989 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:43.699713 14971 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.871479 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.851276  9948 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:43.871874  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:43.854745 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.868948  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.782534 14995 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.873255  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.869846 10094 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:43.783694 15273 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.880023 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:43.880457  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.882095  9949 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:43.872710  9943 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:43.875699 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.831785 15343 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.879495  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:43.890120  9942 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:43.892534 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.893458 10091 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:43.894342 15192 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.003s	sys 0.022s
W20260709 12:25:43.895620 15192 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.003s	sys 0.022s
I20260709 12:25:43.895889 15192 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:43.896883 15192 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.899170  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.901219 15153 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:43.901365 15324 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:43.879374 15174 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:43.902405 15153 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:43.902258 15174 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.871795 15029 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.903347 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.903959 10668 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:43.904736 15174 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.904841 15029 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.905092 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.905709 10671 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:43.902877 15153 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:43.909152 15153 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.910732 15323 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:43.911363 15323 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:43.910996 15153 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:43.912307 15323 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:43.913865 15323 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:43.922477 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.925086 10099 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:43.928045 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.930842 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:43.931648 10675 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:25:43.933331 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:43.929175 15309 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.937142 15309 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:43.938068 10096 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:43.939455  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:43.926370 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.941843 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.942269 10090 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:43.942422 10096 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:43.944303 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.954352 10098 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:43.956144  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:25:43.968899  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.010s	sys 0.028s
W20260709 12:25:43.970822  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.010s	sys 0.028s
I20260709 12:25:43.977782  9942 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:43.924283 15343 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:43.988022 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.988736 10091 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:43.992666 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:43.993492 10090 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
W20260709 12:25:43.997506  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.006912 15153 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.007512 15153 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.010147 15153 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.010391 15339 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:25:44.011618 15339 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:44.014523 15063 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:44.019742 15077 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:44.024942 15326 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.646s	user 0.010s	sys 0.007s
I20260709 12:25:44.063916 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.067106 10090 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:44.058981 15354 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:44.072003  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49319.4us
W20260709 12:25:44.084698 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 182026us
I20260709 12:25:44.086396 10091 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:44.087188 10675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:44.095333 10675 raft_consensus.cc:606] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:44.095706 10675 raft_consensus.cc:614] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:44.073017 14971 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:44.091706  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:44.094319 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.092676 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.096535 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.061687 15326 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:44.090790 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 265627us
I20260709 12:25:44.099992 10092 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
W20260709 12:25:44.100422 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.104660  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:44.102161 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.108457 10091 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:25:44.114799  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:44.114626 15077 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:44.116026  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:44.128181 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.129900  9940 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:44.105808 15354 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.144171 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.119664 15217 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.145118 10099 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:44.150326 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:44.152941  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 68133.8us
I20260709 12:25:44.111742 15306 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:25:44.134943 14971 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:44.160509  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:44.118885 15342 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:25:44.123075 15227 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:25:44.166456 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.168550 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.119403  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:44.176054 10099 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:44.175709  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56414.7us
I20260709 12:25:44.177425  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:44.174865 10096 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:44.179024  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:44.179764  9942 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:44.180821  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:44.182072  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:44.183063 10642 leader_election.cc:336] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.603s (SENT)
I20260709 12:25:44.162876 15343 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.200563 10675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:44.201216 10675 raft_consensus.cc:606] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:44.206494 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:44.155189 15139 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 11.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:25:44.149384 15354 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.213725 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.194914 15309 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:25:44.216461  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56043.8us
W20260709 12:25:44.216572 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 112537us
I20260709 12:25:44.178696 15370 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:44.200243 15077 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:44.222915 15325 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:25:44.227054 15372 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:25:44.199757 15015 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.231750  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:44.170571 15366 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
W20260709 12:25:44.227653 15373 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:25:44.253436 10675 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:44.254091 10675 raft_consensus.cc:606] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:44.254431 10675 raft_consensus.cc:614] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:44.257058 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:44.258718 10017 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:44.259567 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.261514 15192 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:44.261976 15192 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:44.262362 15192 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.265796 10025 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:44.267001 15326 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.236001 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.237753 15354 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:44.238382 15015 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.229422 15077 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.220476 15309 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.277580 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.229617 15370 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.240151 15366 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:25:44.234972 15368 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:25:44.245738 15139 consensus_queue.cc:1243] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 12.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:25:44.232592 15325 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:44.268093 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.269194 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.215544 14971 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:44.272392 10096 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:44.276316  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44655.3us
W20260709 12:25:44.321571 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.322875 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:44.285459 10099 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:44.279053 10094 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:44.289324 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:44.301635 15361 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.425s	user 0.002s	sys 0.000s
I20260709 12:25:44.305757 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:44.310035 15351 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.014s	user 0.000s	sys 0.013s
I20260709 12:25:44.339704 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.341933 10097 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:44.342037 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.343480 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.346371 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.323124 10675 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:44.346818 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.348683 15015 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:44.341213 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3312) took 1155 ms. Trace:
I20260709 12:25:44.350333 10094 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:44.347136 10099 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:44.352792 15217 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:44.292840 15275 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:44.325582  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.363363  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:44.326905 10090 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:44.278621 15077 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.283509 15366 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:44.331220 10021 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:25:44.334199 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:44.370568 10021 pending_rounds.cc:85] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 9
I20260709 12:25:44.338183 10671 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:44.370855 10021 pending_rounds.cc:107] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:44.338969 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:44.371630 10021 raft_consensus.cc:2889] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:44.371927 10668 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:44.341214 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.323977 10095 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:44.353699 10098 raft_consensus.cc:2368] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:25:44.354182 15139 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:25:44.284199 15309 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:44.277240 14991 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:25:44.327270 15304 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.358438 15381 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:25:44.351514 10093 rpcz_store.cc:276] 0709 12:25:43.185475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.186204 (+   729us) service_pool.cc:224] Handling call
0709 12:25:43.186508 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.196586 (+ 10078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.196637 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.207267 (+ 10630us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.296178 (+ 88911us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:43.296239 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.296332 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.309372 (+1013040us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:44.340830 (+ 31458us) raft_consensus.cc:1685] finished
0709 12:25:44.340860 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.341128 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4022,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":4998,"spinlock_wait_cycles":3135360,"thread_start_us":55712,"threads_started":1,"wal-append.queue_time_us":75744}
I20260709 12:25:44.329136 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:44.378468 10096 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:25:44.398368  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:44.401400  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:44.384502  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.284294 15370 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:44.370820 10672 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 8.
I20260709 12:25:44.387249  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:44.387434  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:44.403347  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:44.404034  9947 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:44.371277 10671 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:44.390820 15063 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:44.410820  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.393345 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 859) took 1037 ms. Trace:
I20260709 12:25:44.412686 10097 rpcz_store.cc:276] 0709 12:25:43.355879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.356100 (+   221us) service_pool.cc:224] Handling call
0709 12:25:44.341725 (+985625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:44.342365 (+   640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:44.393098 (+ 50733us) spinlock_profiling.cc:243] Waited 985 ms on lock 0x7b54001928d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:44.393275 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2266080128}
I20260709 12:25:44.397405 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:44.414249 10098 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.388602 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:44.415128 10095 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:44.417279 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.367916 15217 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.367368 15354 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.428768  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:44.429505  9947 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:44.385854 15342 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:44.450448  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:44.451155  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:44.451545  9942 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:44.453127  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:44.444908 10671 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 11.12, Last appended by leader: 12, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.371203 15323 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 12:25:44.422362 15153 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.480074 15153 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.421752 15174 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:44.469373 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 887) took 1264 ms. Trace:
I20260709 12:25:44.452569 15217 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.487556 10018 rpcz_store.cc:276] 0709 12:25:43.204634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.204769 (+   135us) service_pool.cc:224] Handling call
0709 12:25:43.205085 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:43.208863 (+  3778us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:43.208892 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:43.215473 (+  6581us) log.cc:844] Serialized 26 byte log entry
0709 12:25:43.457332 (+241859us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:43.457392 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:43.457490 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:44.300829 (+843339us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:44.457165 (+156336us) raft_consensus.cc:1685] finished
0709 12:25:44.457196 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:44.469270 (+ 12074us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":141,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":3563,"spinlock_wait_cycles":26575104,"thread_start_us":218530,"threads_started":1,"wal-append.queue_time_us":530103}
I20260709 12:25:44.471161 10671 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.422035 15304 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.473814 15366 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:44.395728 15343 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:25:44.407156 14995 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:25:44.397330 15340 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:25:44.408151 15337 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:25:44.444032 15354 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:44.439770 15063 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:44.514706  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.530951  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.534444  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:44.359077 15192 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:44.535535  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 148418us
I20260709 12:25:44.535745 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.536541 10091 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
W20260709 12:25:44.543267  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 141951us
W20260709 12:25:44.545225  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 157895us
I20260709 12:25:44.546537  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:44.555260  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:44.560868 14991 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.561432 14991 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:44.563647 14991 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.565852 15192 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:44.567101 15337 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:44.567723 15337 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:44.570825 15366 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.575122 15343 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:44.575891 15343 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.576097  9991 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:44.576759 15063 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.577366  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:44.578933 14819 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.558238 15174 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:44.583377 15174 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.583776 15174 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:44.587363 15385 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:25:44.551168 15227 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:44.589174 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.558429 15323 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:25:44.590054 10094 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:44.590895 15323 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:44.591357 15323 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:44.591739 15323 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:44.594089 15153 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.595080 15358 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:44.595708 15358 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:44.595930 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.598441 15340 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:44.587484 10093 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:44.609179 10671 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 12. Preceding OpId from leader: term: 12 index: 13. (index mismatch)
I20260709 12:25:44.578526  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:44.600227  9912 connection.cc:256] RPC call timeout handler was delayed by -1.7146265699998366s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16290000000000002s for a response
W20260709 12:25:44.616134  9912 connection.cc:256] RPC call timeout handler was delayed by -1.5292265699999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1835s for a response
W20260709 12:25:44.616627  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4455265699998563s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1928s for a response
W20260709 12:25:44.617050  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2385265699999763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 12:25:44.617419  9912 connection.cc:256] RPC call timeout handler was delayed by -1.1989265699999123s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 12:25:44.623505 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.624341 10091 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:44.624979  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:44.626696  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:44.628073 10095 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:44.629014 10090 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:25:44.639070 15357 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:44.639688 15357 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:44.640758 15357 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:44.596632 15358 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:44.644667 15326 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:44.645393 15326 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.605459 15153 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.651091 15153 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.652238 15304 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:44.653122 15304 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:44.653731 15153 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.654090 15304 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:44.655254 15217 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:44.655844 15217 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:44.656286 15217 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:44.657204 15217 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.667892 15227 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:44.668480 15227 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.669687 15357 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:44.599829 15342 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:44.674494 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.705789 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.698516 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 478868us
W20260709 12:25:44.717846 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.694177 15342 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:44.558694 15325 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.729358  9941 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:44.729988 10098 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
W20260709 12:25:44.724577 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.731568 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.719882 15304 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:44.737141 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:44.721288 14841 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.770779 14841 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:44.716233 15077 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.775596 10019 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:44.776402 15077 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:44.777453 15342 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:44.790508 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.794145  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.795300 15343 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:44.802902 15139 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:44.803395 15139 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:44.804394 15139 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.805895 15139 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:44.807173 15325 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:44.809010 15309 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.727362 15381 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.083s
W20260709 12:25:44.812645 15387 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:25:44.817957 15337 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.820478 15389 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:44.822691 15304 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.825668 15326 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:44.829545 15323 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.850661  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2200) took 1016 ms. Trace:
I20260709 12:25:44.850656 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.856918 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.880813 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.881441 10098 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:44.862005 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:44.850037 15376 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:25:44.848539 15343 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:44.876792 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:44.883378 10094 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:44.863910 15297 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:44.884642 10090 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:44.884722 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:44.880375  9941 rpcz_store.cc:276] 0709 12:25:43.818837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:43.818963 (+   126us) service_pool.cc:224] Handling call
0709 12:25:44.835074 (+1016111us) spinlock_profiling.cc:243] Waited 900 ms on lock 0x7b540003fa54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:44.835232 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2070928128}
I20260709 12:25:44.885656 10093 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:44.882233  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 255712us
I20260709 12:25:44.883045  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:44.893134  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 346648us
I20260709 12:25:44.895107  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:44.921773  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:44.874641 15337 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.927083  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:44.886561  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 255991us
W20260709 12:25:44.918489 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 295540us
W20260709 12:25:44.933961 10062 connection.cc:256] RPC call timeout handler was delayed by -2.6964697759999581s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
I20260709 12:25:44.876672 15391 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:44.882201 15077 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:44.939401 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:44.915666 15387 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:25:44.884358 14841 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:44.935647 15340 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:44.879495 15370 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.942277 15326 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:44.915273 15343 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:44.944398 14995 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:44.940589 15337 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:44.968071  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:44.943886 15389 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:44.930889  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47965.9us
I20260709 12:25:44.985819 10093 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
W20260709 12:25:44.990088 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 124549us
W20260709 12:25:44.988400 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 247182us
I20260709 12:25:44.920830 15297 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:44.996711 10062 connection.cc:256] RPC call timeout handler was delayed by -2.5164697760001218s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
W20260709 12:25:44.969180 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 104494us
I20260709 12:25:44.955567 15174 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:45.028537 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:45.008634 15340 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:45.027853 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 161642us
I20260709 12:25:45.050503 10090 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:45.039330  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.057633  9941 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:45.057735  9948 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:45.059300  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.060781  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.062848  9941 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:45.063005 10090 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:45.063730  9948 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:45.052064  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 123288us
W20260709 12:25:45.052708 10062 connection.cc:256] RPC call timeout handler was delayed by -1.9251697760000752s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2745s for a response
I20260709 12:25:45.054133 15390 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:45.065678  9991 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:45.022243 14995 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.015515 15326 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:25:45.055876 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:44.991381 15227 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:45.077466 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:45.054133 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.067095  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:45.067549  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 99808.1us
I20260709 12:25:45.067480  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:45.031036 14841 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:45.064703 10062 connection.cc:256] RPC call timeout handler was delayed by -1.8261697759999151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28550000000000003s for a response
I20260709 12:25:45.079766 10677 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:45.087085 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.015210 15077 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:45.087728 10672 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:45.085917  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:45.086246 10062 connection.cc:256] RPC call timeout handler was delayed by -1.4832697759998155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32360000000000005s for a response
I20260709 12:25:45.065335 15340 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:45.008968 15391 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:45.094897 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:45.079591  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 152678us
I20260709 12:25:45.076362 14995 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:45.081338  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:45.015723 15399 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:25:45.091110 10677 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:25:45.102516 10677 pending_rounds.cc:85] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 9
I20260709 12:25:45.102922 10677 pending_rounds.cc:107] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:25:45.103276 10677 raft_consensus.cc:2889] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:45.086122 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.095593 10675 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:45.099864 10668 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:25:45.100811  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:45.102000 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.093216  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:45.088755 15309 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:45.076076 15402 raft_consensus.cc:2764] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:45.091331 15357 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:45.087652 15354 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:45.072768 14971 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:25:45.111593 10671 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:25:45.079392 15326 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:45.094838 14991 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:45.086073 14841 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:45.133572 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.135361 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.137104 10676 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:25:45.122885 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.126039 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:45.119939 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.134488 10675 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
W20260709 12:25:45.110613 15388 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:25:45.096882 15304 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:45.132474 10677 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:45.120087 15357 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.152228  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:45.072885 15390 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:45.113204 15339 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:45.140839 10673 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:25:45.164703 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 58146.5us
I20260709 12:25:45.174731  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:45.140580 14841 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:45.181056  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:45.184727  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 91602.6us
I20260709 12:25:45.182628 15410 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:45.182865 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.185595 10673 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:45.144091 15366 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:45.153385 15354 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:45.190851  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.190773  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.192016 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.192574  9944 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:45.183249 15397 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
W20260709 12:25:45.193352 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.192627 10668 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:45.147869 14991 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:45.162366 15357 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:45.208575  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.209180  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.202438 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:45.197683  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.210062 10090 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:45.209444  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.210913  9946 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:45.154289 15415 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:45.171767 15304 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:45.199298  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:45.218405  9940 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:45.220976  9942 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:45.226977  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.233568 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:45.234102 10090 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:45.234532 10099 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:45.235509  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:45.186951 15339 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:45.239503 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:45.240351 10090 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:45.216084 15275 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:25:45.214551 15395 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:25:45.243808 10668 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:45.248342 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:45.212132 15309 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:45.254915 10677 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:45.215610 15279 raft_consensus.cc:476] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Not starting pre-election -- already a leader
I20260709 12:25:45.221318 15304 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:45.265558 15412 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:25:45.258386 15383 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
W20260709 12:25:45.276268 15406 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:25:45.274194 15077 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:45.278124 15405 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:25:45.278318 15412 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:45.278764 15412 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:45.279969 15413 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:25:45.281319 15390 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:45.282493 15419 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:25:45.283018 14991 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:45.283860 15391 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:45.285193 15407 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:25:45.138841 15389 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:45.291198 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:45.292100 10668 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:25:45.294723 15275 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:45.297708 15354 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:45.298439 15354 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:45.299597 15354 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:45.311331 10677 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:45.312530 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:45.313777 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.315455 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.246867 15370 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:45.255964 15415 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:45.256678 15309 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:45.320159 15275 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:45.332769 10090 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:45.212424 15416 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.120s
I20260709 12:25:45.306003 15304 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:45.329823 15309 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:45.376894 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58553.3us
W20260709 12:25:45.378157  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 297111us
W20260709 12:25:45.318773 15396 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:45.379352  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:45.379941 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 197451us
W20260709 12:25:45.380550  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 261352us
I20260709 12:25:45.319036 15337 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:45.415968 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.372385 15227 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:45.418568 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:45.426800  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47560.8us
I20260709 12:25:45.428196  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:45.428999 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.421797 15367 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:45.467408 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48965.7us
I20260709 12:25:45.417409 15400 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 12:25:45.486070 10668 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
W20260709 12:25:45.492960 15139 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.687s	user 0.007s	sys 0.007s
I20260709 12:25:45.464097 15297 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:45.476938 15077 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:45.509655 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.518100 10021 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:45.518226 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.523558 15343 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:45.553709 10019 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:45.536722 10097 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:45.518014  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:45.548048 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:45.551975 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53647.7us
I20260709 12:25:45.528426 10099 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:45.389356 15339 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:45.627466 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:45.628492  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.637097 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.638394 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:45.641566 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:45.643810 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:45.645287 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:45.649829 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.650450 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.651155 10677 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:45.636842 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.656385 10017 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:45.658363 10675 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:45.659732  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.660732  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:45.666038 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.668663 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:45.669468 10674 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:45.673156 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.629227 15227 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:45.705212 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.696666  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:45.664124 15325 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.856s	user 0.012s	sys 0.000s
I20260709 12:25:45.699072 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:45.720532 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:45.666986 10016 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:45.697295 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.695452 15063 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:45.717921  9949 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:45.718662 10017 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:45.727672 10022 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 12: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:45.716809  9842 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:45.735477 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:45.720602 10670 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:45.730754 10672 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:45.731082 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.637461 15077 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:45.740590 10020 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:25:45.741062 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77616.4us
I20260709 12:25:45.745700 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:45.752866 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:45.719895 15227 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:45.753650 10022 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:45.761970 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 127009us
I20260709 12:25:45.764083 10023 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:45.748142 14995 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.636s	user 0.010s	sys 0.002s
I20260709 12:25:45.727641 15375 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.241s
I20260709 12:25:45.753443 15279 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:45.757580 15390 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:45.747514 15427 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:45.782634 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:45.782801 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:45.784696  9940 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.546s	user 0.007s	sys 0.003s
I20260709 12:25:45.794641  9940 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:45.785678 10674 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:45.748777 15063 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:45.813391  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 194470us
I20260709 12:25:45.828083 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:45.829860 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 84259.8us
I20260709 12:25:45.835886 10090 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:45.796759 14971 raft_consensus.cc:2764] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won
I20260709 12:25:45.777480 14995 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:45.853119 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 96812.8us
I20260709 12:25:45.860200 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:45.821799 15198 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:45.822640 15430 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:45.864557 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:45.867319 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:45.853018 15434 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:25:46.023809 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:45.854252 15139 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:46.047946 10024 raft_consensus.cc:2368] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
W20260709 12:25:46.062455 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 299550us
W20260709 12:25:46.065593 15437 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.269s	user 0.001s	sys 0.000s
W20260709 12:25:46.066905 15440 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 12:25:46.056824 15279 raft_consensus.cc:2764] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 12:25:45.849141 15227 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.115667 10670 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:25:46.115701  9912 leader_election.cc:336] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.514s (SENT)
I20260709 12:25:46.119185 10017 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:46.082378 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:46.084602 15432 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.365s	user 0.001s	sys 0.000s
W20260709 12:25:46.091290 15441 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
W20260709 12:25:46.092213 15435 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.319s	user 0.000s	sys 0.000s
W20260709 12:25:46.093019 15439 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 12:25:46.095557 15063 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.102375 15174 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:46.103825 15326 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.956s	user 0.007s	sys 0.005s
I20260709 12:25:46.104456 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:46.104939 14841 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.909s	user 0.006s	sys 0.004s
I20260709 12:25:46.107000 15077 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:45.879377 15198 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.182478 15198 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:46.176157  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 65447.4us
I20260709 12:25:46.186760 15403 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:46.186792  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:46.187448 15403 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.189759 15403 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:46.192682  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:46.193660  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:46.194224 15077 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.194813 15077 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.197476 15077 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:46.206987 15452 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.122s	user 0.000s	sys 0.003s
I20260709 12:25:46.024025 15430 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.126515 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.117396 15306 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:46.210209 15306 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.211630 15430 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.213042 15306 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.076740 15409 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.369s
W20260709 12:25:46.213783 15357 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.977s	user 0.004s	sys 0.005s
I20260709 12:25:46.214145 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.214915 10019 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:46.215411 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:46.215904 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:46.218006 14995 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:46.218750 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 904) took 1324 ms. Trace:
I20260709 12:25:46.219147 10025 rpcz_store.cc:276] 0709 12:25:44.894782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:44.895600 (+   818us) service_pool.cc:224] Handling call
0709 12:25:44.917796 (+ 22196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.102873 (+185077us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:45.244567 (+141694us) log.cc:844] Serialized 14 byte log entry
0709 12:25:45.372228 (+127661us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:45.372282 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:45.372335 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.176437 (+804102us) spinlock_profiling.cc:243] Waited 973 us on lock 0x7b180049c420. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:46.218466 (+ 42029us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54002170d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:46.218527 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.218686 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13025,"prepare.queue_time_us":227720,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":7498,"raft.queue_time_us":53311,"raft.run_cpu_time_us":397,"raft.run_wall_time_us":12021,"spinlock_wait_cycles":427355392,"thread_start_us":264451,"threads_started":2,"wal-append.queue_time_us":172303}
I20260709 12:25:46.221515 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.223335 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.224943 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.226238 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:46.076462 15392 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.367s
W20260709 12:25:46.172183 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94137.3us
I20260709 12:25:46.173974 10675 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.179729 14841 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.186436 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.235620 10022 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
W20260709 12:25:46.240244  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46655.2us
I20260709 12:25:46.254882  9991 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:46.243944 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.173487 15453 raft_consensus.cc:2764] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
W20260709 12:25:46.238276 15340 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.137s	user 0.003s	sys 0.009s
I20260709 12:25:46.262790 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.263592 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:46.240721 15309 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.865s	user 0.003s	sys 0.008s
I20260709 12:25:46.277616 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:46.269568 10018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.001s	sys 0.009s
W20260709 12:25:46.278613 10018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.002s	sys 0.009s
W20260709 12:25:46.270498  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 437834us
W20260709 12:25:46.253583 15304 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.862s	user 0.009s	sys 0.002s
I20260709 12:25:46.237756 15015 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:46.214311 15357 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.232877 15326 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.241074 15343 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:46.282866 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:25:46.319561 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 58320.7us
I20260709 12:25:46.320462 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:46.321915 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:46.269956 10092 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.000s	sys 0.011s
W20260709 12:25:46.324818 10092 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.788s	user 0.000s	sys 0.011s
I20260709 12:25:46.332621 10675 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:46.334290 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:46.252710 15077 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:46.339793 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:46.278364 15403 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:46.353585 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.309230 14841 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.366719 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:46.352929 15306 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:46.326690 15343 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:46.277801 15340 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.283214 15309 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:46.374976 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.000s	sys 0.014s
I20260709 12:25:46.375528 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:46.381038 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.000s	sys 0.014s
I20260709 12:25:46.336663 15445 raft_consensus.cc:2764] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:46.358349 10022 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:46.381903 10675 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:25:46.386214 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 635366us
I20260709 12:25:46.370239 10025 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:46.386989 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:46.383234 15446 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:25:46.424748 10017 raft_consensus.cc:2368] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14.
I20260709 12:25:46.377415 15306 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:46.428105 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:46.383893 10668 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:46.390759 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:46.369273 15403 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.395378 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.380208 15343 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:46.372599 15279 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:46.404724 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:46.349926 15198 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:25:46.407447 15063 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.446960 15063 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.449915 15063 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.409530 15342 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:46.411100 10024 raft_consensus.cc:2368] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
W20260709 12:25:46.411266  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 226820us
I20260709 12:25:46.377799 15425 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:46.450783 15342 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:46.451125 15425 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:46.451718 15425 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:46.452052 15342 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.453922 15342 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.329243 15015 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.456321 15015 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:46.329027 15304 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.459156 10016 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:46.433084  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:46.461723 10022 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:46.435631 10019 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
W20260709 12:25:46.451211  9912 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.614s (SENT)
I20260709 12:25:46.465900  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.436403 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:46.467621  9912 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.640s (SENT)
I20260709 12:25:46.468547  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.468067 10676 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:25:46.472576 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44565.6us
I20260709 12:25:46.399389 14841 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:46.427209 15077 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:46.474344 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.473456 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:46.434960 15029 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:46.444657 15403 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:46.466709 15304 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:46.486688  9941 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 14 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:46.488426 15370 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:46.489919 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 907) took 1179 ms. Trace:
I20260709 12:25:46.521355  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:46.493515 15425 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.481736 14841 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.496042 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:46.526301 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.527719 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.434329 15306 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:46.521412 10018 rpcz_store.cc:276] 0709 12:25:45.310778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.314107 (+  3329us) service_pool.cc:224] Handling call
0709 12:25:45.314396 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.314788 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:25:45.465945 (+151157us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.330244 (+864299us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.330734 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 12:25:46.330875 (+   141us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:46.330905 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.330967 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.489486 (+158519us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b18001ce880. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:46.489567 (+    81us) raft_consensus.cc:1685] finished
0709 12:25:46.489689 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.489849 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61151,"prepare.queue_time_us":109754,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":114382,"raft.run_cpu_time_us":326,"raft.run_wall_time_us":45806,"spinlock_wait_cycles":34323712,"thread_start_us":933729,"threads_started":2,"wal-append.queue_time_us":959023}
I20260709 12:25:46.475986 15139 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:46.476233 15376 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.512573 15297 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:46.434051 15457 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:46.542647 15457 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.543911 15425 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.545037 15457 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.472759 15279 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:46.545974 15279 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:46.539350 15153 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:46.529958 15370 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.526080 15077 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.549893 15370 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:46.530161 15304 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:46.550683 15304 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.551090 15304 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.485903 15366 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:46.519264 15192 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:46.551678 15323 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.530265 15411 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:46.508399 15447 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:46.540231 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.539090 15297 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:46.555542  9946 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:25:46.557731 15354 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.096s	user 0.006s	sys 0.017s
I20260709 12:25:46.555945 15324 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:46.602527 15449 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.517s	user 0.000s	sys 0.001s
I20260709 12:25:46.607447 15309 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:46.608137 15357 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:46.608749 10017 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:46.585325 15077 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.569578 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:46.589217 15354 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.127s	user 0.006s	sys 0.018s
W20260709 12:25:46.627947 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58457.7us
I20260709 12:25:46.628815 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:46.553154 15376 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.630704 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2139) took 1094 ms. Trace:
I20260709 12:25:46.562224 10022 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:46.633179 10642 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:46.634828 10092 rpcz_store.cc:276] 0709 12:25:45.536191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.536491 (+   300us) service_pool.cc:224] Handling call
0709 12:25:45.536732 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:45.536920 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:46.325562 (+788642us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.630246 (+304684us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:46.630302 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:46.630354 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.630444 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.630622 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3081,"prepare.queue_time_us":947897,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"thread_start_us":1082136,"threads_started":2}
I20260709 12:25:46.646294 10022 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:25:46.636204 15275 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.649843 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.611033 15326 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:46.654207 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.657775 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.627617 15269 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:46.636926 15453 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:46.637295 15447 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:46.634644 15389 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:46.666072  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 239601us
I20260709 12:25:46.658907 15324 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:25:46.679917 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:46.680419 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:46.663070 15411 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:46.681458 10092 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:46.684258 10642 leader_election.cc:400] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:25:46.684782 10642 leader_election.cc:403] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:46.664145 15340 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.648356 15275 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.687083 10092 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:25:46.689360  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 256346us
I20260709 12:25:46.690765 10092 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:46.673835 15155 raft_consensus.cc:2764] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:46.693742 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:46.694531 10092 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:46.695766  9912 leader_election.cc:336] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.668s (SENT)
I20260709 12:25:46.696352  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.646857 15297 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.702246 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:46.704090 10099 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.683880 15411 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:46.702246 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:46.708319 10093 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:46.706308  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:46.692056 15326 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.692939 15389 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:25:46.692530 15438 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.680593 15269 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:46.723227 15470 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:25:46.713070 15465 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:46.732354 10673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab118912551248c1985fce40a2e815b0"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:46.718624 15476 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:46.733702 15376 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:46.704692 15324 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:46.736985 15450 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.652s	user 0.001s	sys 0.001s
W20260709 12:25:46.737764 15472 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:25:46.714039 15411 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.741052 15438 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.742475 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1086) took 1094 ms. Trace:
I20260709 12:25:46.742915 10671 rpcz_store.cc:276] 0709 12:25:45.648034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.648185 (+   151us) service_pool.cc:224] Handling call
0709 12:25:45.657451 (+  9266us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:45.692271 (+ 34820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.692323 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:45.706737 (+ 14414us) log.cc:844] Serialized 26 byte log entry
0709 12:25:45.822402 (+115665us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:45.822443 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:45.822511 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.509656 (+687145us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b480002ea58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:46.736408 (+226752us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:46.742221 (+  5813us) raft_consensus.cc:1685] finished
0709 12:25:46.742243 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.742386 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6793,"raft.queue_time_us":7067,"raft.run_cpu_time_us":2765,"raft.run_wall_time_us":833834,"spinlock_wait_cycles":39083776,"thread_start_us":61090,"threads_started":1,"wal-append.queue_time_us":342461}
I20260709 12:25:46.744136 15453 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:46.744714 15453 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.753284 15297 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:46.732519 14841 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.717124 15326 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:46.764035 15433 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.304s	user 0.000s	sys 0.002s
I20260709 12:25:46.796515  9842 catalog_manager.cc:5697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 4 to 11. New cstate: current_term: 11 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:46.800956 10024 pending_rounds.cc:85] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 8
I20260709 12:25:46.801409 10024 pending_rounds.cc:107] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:25:46.801689 10024 raft_consensus.cc:2889] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:46.804069 10021 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: [9.4-14.7]
I20260709 12:25:46.805671 10017 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:46.807056 15412 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.808552 15412 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.798069  9842 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:46.829011  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 122796us
I20260709 12:25:46.831296 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2048) took 1046 ms. Trace:
W20260709 12:25:46.832705 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.833416 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:46.833406  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 142246us
W20260709 12:25:46.834935  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.721s (SENT) [suppressed 19 similar messages]
I20260709 12:25:46.839291 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:46.841131 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:46.841825 10670 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:25:46.842144  9912 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.869s (SENT)
I20260709 12:25:46.846726 10099 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:46.831710 10020 rpcz_store.cc:276] 0709 12:25:45.784777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.785064 (+   287us) service_pool.cc:224] Handling call
0709 12:25:45.814345 (+ 29281us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:45.826895 (+ 12550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:45.826950 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:45.856417 (+ 29467us) log.cc:844] Serialized 52 byte log entry
0709 12:25:46.071408 (+214991us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:46.325537 (+254129us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.325984 (+   447us) log.cc:844] Serialized 14 byte log entry
0709 12:25:46.605913 (+279929us) spinlock_profiling.cc:243] Waited 7.36 ms on lock 0x7b180030dd80. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0910cd70 00007f4e09100293 00007f4e090f5cea 00007f4e090f623a 00007f4e09560a2a 00007f4e095700bf 00007f4e09570038
0709 12:25:46.830766 (+224853us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b180030dd80. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:46.830970 (+   204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:46.831043 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:46.831085 (+    42us) raft_consensus.cc:1685] finished
0709 12:25:46.831098 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:46.831241 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":235271,"prepare.queue_time_us":565377,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":73,"raft.queue_time_us":4734,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":12367,"spinlock_wait_cycles":50668160,"thread_start_us":456274,"threads_started":2,"wal-append.queue_time_us":238565}
I20260709 12:25:46.849776 10671 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:25:46.855798 10671 pending_rounds.cc:85] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 10
I20260709 12:25:46.856083 10671 pending_rounds.cc:107] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:25:46.856333 10671 raft_consensus.cc:2889] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:46.857255 10093 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
W20260709 12:25:46.858815 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 181780us
W20260709 12:25:46.826278 15415 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.406s	user 0.004s	sys 0.012s
I20260709 12:25:46.859186  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:46.859743 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:46.861625 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:46.860272 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:46.864270 10674 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:46.864642 10676 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:46.870380 10670 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:46.849442 15442 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:46.874960 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:46.877758 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:46.878175 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:46.875262 10670 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:25:46.867151 15415 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.450s	user 0.004s	sys 0.012s
I20260709 12:25:46.877821 15468 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:46.881712 10672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 974) took 1775 ms (client timeout 1899 ms). Trace:
I20260709 12:25:46.882678 15416 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.883080 10675 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:46.878360 15479 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:46.884034 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.887972 15326 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:46.888792 15269 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:46.889806 15343 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.891026 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.939668 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.940578 15479 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.940892 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:46.941507 10021 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:46.941176 15479 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:46.865664 15431 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:25:46.897662 10673 raft_consensus.cc:606] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:46.943436 10673 raft_consensus.cc:614] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:46.944175  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:46.945087  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:46.945880 15326 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:46.946167 15358 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:46.866076 15198 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
W20260709 12:25:46.903211  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61093.6us
I20260709 12:25:46.904711 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:46.912626 10676 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:46.913753  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:46.913863  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:46.894800 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.954183 10017 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:46.929023 15416 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.931670 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:46.935415 15370 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:46.939249 15468 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.892414 15279 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:46.879846 15442 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:46.968586  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:46.970323 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2036) took 1425 ms (client timeout 1650 ms). Trace:
W20260709 12:25:46.975531 10023 rpcz_store.cc:269] 0709 12:25:45.545149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.545289 (+   140us) service_pool.cc:224] Handling call
0709 12:25:46.970080 (+1424791us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54001383d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:46.970246 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286551296}
I20260709 12:25:46.849619 15367 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:25:46.957226  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:46.961669  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48009.1us
I20260709 12:25:46.908661 15415 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:46.909206 10672 rpcz_store.cc:269] 0709 12:25:45.107160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:45.107399 (+   239us) service_pool.cc:224] Handling call
0709 12:25:46.881505 (+1774106us) spinlock_profiling.cc:243] Waited 591 ms on lock 0x7b5400611a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:46.881645 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358325120}
W20260709 12:25:46.963366  9912 leader_election.cc:336] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.891s (SENT)
I20260709 12:25:46.961028 15376 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:46.977643 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:47.008728  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:46.981537 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:46.946437 15326 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:46.963897 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:46.949345 15275 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:46.946738 15358 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:47.038314 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 124640us
I20260709 12:25:47.009451  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:47.039611  9948 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:46.983575  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:46.987743  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:46.982987  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:47.019738  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:47.022198 10017 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:46.978240 15442 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.039788  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:47.046103  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:46.916028 15306 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.011081 15324 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:25:47.047495 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:47.048115 15324 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:47.048472 15324 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:47.049149 10672 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:47.049386 15324 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.008316 15192 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:47.050923  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 97164.2us
I20260709 12:25:47.050570 15415 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.051698 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:46.963105 15174 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:46.962960 14995 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:47.042512  9947 raft_consensus.cc:2410] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:47.043036 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:47.040872  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:47.053640 10097 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:46.946084 15403 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.063223  9941 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:47.063638 10017 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:47.011202 15354 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:47.066293 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:47.067207  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:46.983197 15155 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.072s
I20260709 12:25:47.038561 15431 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:47.077019 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:47.077392 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:46.978420 15465 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:47.038470 15343 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:47.078058 10017 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:47.083932 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:47.078397  9912 leader_election.cc:336] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.432s (SENT)
I20260709 12:25:47.085582 10017 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:47.086928 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:47.056249 15324 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.088496 10017 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:47.056996 15306 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:47.093886 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:47.100811 10025 raft_consensus.cc:2368] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:25:47.056185 14819 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.104166 14819 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:47.031915 15376 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.042745 15340 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:25:47.056119 15484 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:47.065728 14971 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.078857 14995 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:47.106071 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3191) took 1016 ms. Trace:
I20260709 12:25:47.095844  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:47.159550  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 177539us
W20260709 12:25:47.159870  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 88360.5us
W20260709 12:25:47.160012 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 299806us
W20260709 12:25:47.160321 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.199600 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:47.161063 10674 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:47.076889 15431 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:47.203598  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:47.176707 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.204650 10099 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:47.050966 15192 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:47.184705  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.058655 15309 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:25:47.190510 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:47.192767  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 85979.4us
I20260709 12:25:47.209220 10091 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.209507  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:47.194368 10020 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:25:47.196239 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.107654 10022 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:25:47.197476 10019 rpcz_store.cc:276] 0709 12:25:46.089166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.225953 (+136787us) service_pool.cc:224] Handling call
0709 12:25:47.105888 (+879935us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b540038d1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:47.106013 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":275588352}
I20260709 12:25:47.203116 10669 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:47.208595  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:47.212328 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.213173 10099 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:47.072383 15358 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:47.248296 15438 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:47.216017 15306 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:47.250474 15306 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.252831 15306 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:47.253953 15494 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:47.160892 15354 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:47.246142 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3201) took 1145 ms. Trace:
I20260709 12:25:47.263121 10025 rpcz_store.cc:276] 0709 12:25:46.100825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.474063 (+373238us) service_pool.cc:224] Handling call
0709 12:25:47.245956 (+771893us) spinlock_profiling.cc:243] Waited 606 ms on lock 0x7b54001c5f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:47.246085 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1392911232}
W20260709 12:25:47.195243 15304 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.638s	user 0.012s	sys 0.009s
I20260709 12:25:47.221909 15477 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:47.221900 15174 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:47.223467 15402 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:47.224149 15482 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:47.093739 15465 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.226780 10096 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:25:47.228763 10020 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:47.229275  9946 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:25:47.229691 15404 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:47.289794 15479 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.292122 15468 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:47.293125  9943 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:25:47.230773 15198 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:47.234773 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.236913 14995 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.202732 14971 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:47.310806 15411 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.571s	user 0.005s	sys 0.008s
I20260709 12:25:47.311242 15411 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.281390 15343 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:47.312433 15343 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:47.313683 14971 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.243884 15366 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:47.215240 15370 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:25:47.244952 15029 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.245441 15430 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:25:47.271317  9944 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:47.272684 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.329424 10095 raft_consensus.cc:2410] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:25:47.187616 15457 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:47.274536 10021 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:47.104909 15326 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:47.224256 14991 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:25:47.278322 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 70490.7us
I20260709 12:25:47.279422 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3199) took 1179 ms. Trace:
I20260709 12:25:47.331379 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:47.280323 10019 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.192736 15484 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:47.283828 10025 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:47.295367 15420 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:47.343917  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:47.206195 15342 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.752s	user 0.025s	sys 0.009s
I20260709 12:25:47.302124 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:47.302441  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:47.305884 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 106396us
I20260709 12:25:47.354218 10096 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:25:47.355324 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:47.307297 15269 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:25:47.308226 15354 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.309509 15465 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:47.320891 10677 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:47.362497 10018 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:47.224167 15192 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:47.320899 10091 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:47.367446  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:47.338335  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:47.333978 10022 rpcz_store.cc:276] 0709 12:25:46.100332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.701237 (+600905us) service_pool.cc:224] Handling call
0709 12:25:47.107314 (+406077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.211097 (+103783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.279189 (+ 68092us) spinlock_profiling.cc:243] Waited 403 ms on lock 0x7b540038d1d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:47.279350 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":927024384}
I20260709 12:25:47.304095 15482 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:47.300675  9941 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:47.331382 15457 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:47.350236 10019 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:47.350945  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:47.389994  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:47.390738  9944 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.281723 15304 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.362661 10673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:47.394187 10673 raft_consensus.cc:606] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:47.348996  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:47.394652 10673 raft_consensus.cc:614] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:47.358527 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:47.332098 15275 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:47.313704 15411 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.395522 10095 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:47.408282 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.341569 15484 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:47.434100 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78854.8us
I20260709 12:25:47.434934 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:47.448069 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:47.452986  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:47.359848 15269 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:47.333110 15029 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.393299 15457 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.391367 15279 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:47.357446 15420 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:47.508651  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:47.500696 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64169.1us
I20260709 12:25:47.525743 10062 leader_election.cc:400] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
I20260709 12:25:47.526240 10062 leader_election.cc:403] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:47.495213 10096 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:47.466224 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.429335 15306 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:47.406857 15343 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:47.501526  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 275301us
I20260709 12:25:47.433856 15342 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:47.451668 15458 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.531996 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
W20260709 12:25:47.491262 15422 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.005s	sys 0.024s
I20260709 12:25:47.550870  9944 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:47.552037 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:47.464233 14819 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.486351 15269 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:47.555691  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:47.613022  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59445.4us
I20260709 12:25:47.484891 15416 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:47.435081 15275 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:47.638129 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 112558us
I20260709 12:25:47.549010 15476 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
I20260709 12:25:47.642442 10095 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:25:47.631260 15422 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.006s	sys 0.027s
I20260709 12:25:47.657136 10096 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.644150  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:47.636132  9949 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:47.553881 15306 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:47.646970 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 327341us
W20260709 12:25:47.617060 15449 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 12:25:47.643535 15153 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.019s	sys 0.066s
I20260709 12:25:47.666889 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:47.667569 10673 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:47.643821  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:47.655259  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:47.671101  9941 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:47.614933 15155 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:47.642858 10670 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:47.639211 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:47.671635 10096 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:47.676151 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:47.659320  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:47.677541  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:47.680464 15297 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.924s	user 0.009s	sys 0.000s
W20260709 12:25:47.682262  9948 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.000s
I20260709 12:25:47.682660  9948 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:47.683856 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:47.688499  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:47.689796 10096 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:47.619858 15342 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:47.672782 15153 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.019s	sys 0.070s
I20260709 12:25:47.698992  9941 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:47.652961 15376 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.665958 15063 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:47.674785 15306 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:47.727394 15491 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:25:47.727885 15430 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:47.728915 15501 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:47.729333 15501 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:47.723250  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:47.653501 14991 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:25:47.673157 15139 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:47.691229 15425 raft_consensus.cc:2764] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
I20260709 12:25:47.739712 10671 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:47.746479  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:47.702267 15198 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:47.654349 14819 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:25:47.702672 15486 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:47.759325  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:47.764760  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:47.723325 15487 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:47.769696 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.770385 10092 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:47.691447 15412 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:25:47.702096 15402 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:25:47.779357 10673 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:47.766993 15192 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:47.711934 15279 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:25:47.775660 15367 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:47.762280 15430 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:47.755527 15358 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:47.782265 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:47.800169 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116603us
I20260709 12:25:47.786051 15479 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:25:47.786756 15404 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:47.775147 15477 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 12:25:47.787186 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:47.809795 10022 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:47.695022 15155 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:47.789441 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.791440  9948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:47.799203 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:47.814397  9948 raft_consensus.cc:606] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:47.661657 15482 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.802251 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:47.815117 10092 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:47.813474 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:47.815299 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:47.777953 15486 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:47.811437 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.009s	sys 0.000s
I20260709 12:25:47.808794 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:47.795629 15367 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.816628 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.795154 15192 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:47.804430 15358 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:47.768096 15198 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:47.761286 15139 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:47.838768 15497 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:25:47.798574 15430 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:47.857376 15499 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:25:47.857549 15430 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:25:47.858979 15490 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.217s	user 0.001s	sys 0.000s
I20260709 12:25:47.859894 15139 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:47.862715 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.816907 10675 raft_consensus.cc:2376] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 14. Re-sending same reply.
I20260709 12:25:47.805150 15304 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:47.828725 15453 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.077s	user 0.008s	sys 0.008s
I20260709 12:25:47.826262 15192 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:47.830049 15342 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.825058 15367 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:47.830689 15442 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.000s
I20260709 12:25:47.842756 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:47.883154 10097 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:47.837232 15420 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.852437 10092 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:47.854828 15155 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.837534 15487 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:47.832453 15358 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:47.843655 15153 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:47.876161 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:47.913652 10675 raft_consensus.cc:2376] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 16. Re-sending same reply.
W20260709 12:25:47.913949 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 105257us
I20260709 12:25:47.824678 15484 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:47.874363 15275 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:47.874363  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:25:47.919286 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:47.818890 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.009s	sys 0.000s
I20260709 12:25:47.915841 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1108) took 1003 ms. Trace:
I20260709 12:25:47.919854 10675 raft_consensus.cc:2376] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 16. Re-sending same reply.
I20260709 12:25:47.920994 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:47.921774 10675 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
W20260709 12:25:47.921803 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 660568us
I20260709 12:25:47.924014 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.920203 10672 rpcz_store.cc:276] 0709 12:25:46.911945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.050004 (+138059us) service_pool.cc:224] Handling call
0709 12:25:47.050306 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.052970 (+  2664us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.052995 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.053441 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 12:25:47.322668 (+269227us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:47.655934 (+333266us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.656210 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.656289 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:47.915568 (+259279us) raft_consensus.cc:1685] finished
0709 12:25:47.915598 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:47.915772 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":368918,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":1005,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":13413,"thread_start_us":539154,"threads_started":2,"wal-append.queue_time_us":468203}
I20260709 12:25:47.932255 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1165) took 1058 ms. Trace:
I20260709 12:25:47.932664 10671 rpcz_store.cc:276] 0709 12:25:46.874160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.874281 (+   121us) service_pool.cc:224] Handling call
0709 12:25:47.932097 (+1057816us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:47.932198 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1844831488}
I20260709 12:25:47.935824 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:47.936610 10025 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:47.937422 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:47.940573 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:47.941116 10021 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:47.944140 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:47.944167 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:47.914553 15153 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:47.947333 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:47.881377 15453 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:47.989889 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:47.991111 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47012us
I20260709 12:25:47.993408 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:47.943897  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:48.000339  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56523.6us
W20260709 12:25:48.000726 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:47.884786 15442 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.002439 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:47.944976 10020 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:48.005982 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.006554 14995 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:48.007014 14995 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.007355 14995 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.015221 10091 raft_consensus.cc:2368] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11.
I20260709 12:25:48.018647 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:48.025620 15518 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:25:48.031746 15431 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.035725 15403 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.035843 15484 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:48.039801 10024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.161s	user 0.026s	sys 0.019s
W20260709 12:25:48.120263 10024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.027s	sys 0.020s
I20260709 12:25:47.911927 15486 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.045203 15442 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.047539 10094 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 12:25:48.049374 15496 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.404s	user 0.001s	sys 0.002s
I20260709 12:25:48.050261 15453 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:47.997275 15447 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:48.050750 10096 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:48.051360  9940 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
W20260709 12:25:48.051719 15513 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:25:48.053004 15029 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.053539 15367 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.055178 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2052) took 1967 ms. Trace:
I20260709 12:25:48.153591 10018 rpcz_store.cc:276] 0709 12:25:46.087976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.539879 (+451903us) service_pool.cc:224] Handling call
0709 12:25:48.054949 (+1515070us) spinlock_profiling.cc:243] Waited 793 ms on lock 0x7b54001c5054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:48.055111 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1823395072}
I20260709 12:25:48.057324 15477 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:47.949786 15198 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.063155 10017 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:48.065689 15376 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:25:48.071357 15457 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.001s
I20260709 12:25:48.073793 15430 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.077121  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:48.077850  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:48.078377  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:48.077795  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:48.168035 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.148169 15431 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.169020 10090 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:48.143150 15447 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:48.090869 10091 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: [12.10-13.11]
I20260709 12:25:48.109599 10092 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:48.115803 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.089855 10025 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:48.002058 15153 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:48.116719 15324 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.994s	user 0.013s	sys 0.000s
I20260709 12:25:48.026335 15192 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.146811 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3206) took 2044 ms. Trace:
I20260709 12:25:48.150265  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:48.150308 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.157166 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:48.184950 10676 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:48.163427 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.168171  9941 raft_consensus.cc:2376] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12. Re-sending same reply.
I20260709 12:25:48.079969  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:48.161679 15376 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:48.175760 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.189270 10095 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:48.179576 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.190217 10092 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:48.182344 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:48.186110  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 106375us
I20260709 12:25:48.191388  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:48.194058  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:48.087361 10097 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 14 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:48.132675 14995 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.198933 10023 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:48.200033 15340 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:48.195031 15269 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.541s	user 0.011s	sys 0.000s
I20260709 12:25:48.253213 15269 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.256511 15269 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.200799 15420 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:48.257205 14995 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:48.258246 15517 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.429s	user 0.000s	sys 0.005s
I20260709 12:25:48.258379 15420 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:48.259016 15420 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:48.258280  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64285.4us
I20260709 12:25:48.260819  9991 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:48.205185 15367 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.183434 10024 rpcz_store.cc:276] 0709 12:25:46.102279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.800333 (+698054us) service_pool.cc:224] Handling call
0709 12:25:46.800719 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:46.802362 (+  1643us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:46.802406 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:46.819407 (+ 17001us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.120765 (+1301358us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:48.146233 (+ 25468us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.146435 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.146495 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.146533 (+    38us) raft_consensus.cc:1685] finished
0709 12:25:48.146551 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.146737 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26509,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":33,"raft.queue_time_us":4894,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7207,"thread_start_us":1241421,"threads_started":1,"wal-append.queue_time_us":618142}
I20260709 12:25:48.171043 15457 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.262921 10093 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:48.263970 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3368) took 1087 ms. Trace:
I20260709 12:25:48.264014 15457 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:48.264412 10093 rpcz_store.cc:276] 0709 12:25:47.176036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.179049 (+  3013us) service_pool.cc:224] Handling call
0709 12:25:47.179323 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.263267 (+1083944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.263759 (+   492us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:48.263915 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2491640064}
I20260709 12:25:48.265458 15442 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:48.259961 15420 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.206142  9948 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:48.188476 15417 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.155731 15306 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.268800 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1109) took 1354 ms. Trace:
I20260709 12:25:48.266952  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:48.173580 15403 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.208639 15458 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.578s	user 0.003s	sys 0.007s
I20260709 12:25:48.207372 15376 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.210414 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:48.209231 15358 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.212126 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:48.176587 15447 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:48.163926 15477 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.207383 15430 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.224117 15453 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:48.227355 15461 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:48.227509 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45247us
I20260709 12:25:48.270462 15458 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.351285 15516 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.310132 15431 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:48.352434 15431 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:48.352949 15431 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.353883  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:48.355265  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:48.356524 15340 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.356909 15275 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.357556 15275 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:48.358777  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:48.244959 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.359685 15417 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:48.269498 15306 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:48.273388 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.361317 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.243963 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.244652 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.245034 15354 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.364501 15354 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.269212 10676 rpcz_store.cc:276] 0709 12:25:46.914392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.098888 (+184496us) service_pool.cc:224] Handling call
0709 12:25:47.170832 (+ 71944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.185285 (+1014453us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.268536 (+ 83251us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400490a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:48.268737 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2331430784}
I20260709 12:25:48.367753 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:48.368180  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:48.369187 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:48.260898 15492 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.598s	user 0.000s	sys 0.002s
I20260709 12:25:48.371490 14995 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:48.372100 14995 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:48.297705 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.374451  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:48.375020 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.375329 15275 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.376653 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.377345 10025 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:48.298022 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:48.267802 15465 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:48.378758 15465 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:48.379637 15465 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.380841 15465 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:48.321888  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.270380 15403 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:48.382524  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60717.5us
I20260709 12:25:48.383044 15442 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.383253  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:48.383577 15403 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.317386 10642 leader_election.cc:400] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11.
I20260709 12:25:48.384459 10642 leader_election.cc:403] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:48.384294  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.384806 15442 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:48.385231 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 68019.1us
I20260709 12:25:48.385461 15367 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:48.386034 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.176823 15409 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:48.339190 15358 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.267220 15501 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.387409 15411 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:25:48.387720 15501 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:48.337704 15139 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:25:48.391124 15139 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:48.391489 15139 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:48.392341 15139 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.393577 15139 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:48.394253 15376 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.395083 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3221) took 1338 ms. Trace:
I20260709 12:25:48.395473 10023 rpcz_store.cc:276] 0709 12:25:47.056878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.274987 (+218109us) service_pool.cc:224] Handling call
0709 12:25:47.303892 (+ 28905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.252696 (+948804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.394825 (+142129us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b54002652d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:48.395018 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2057988480}
I20260709 12:25:48.377322  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:48.396855 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.309679 15416 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.398869 15416 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.399950 10025 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:48.400996 15416 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.401167 15275 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:48.401773 15275 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:48.384297 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3205) took 2282 ms. Trace:
I20260709 12:25:48.402704 15275 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.406910 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:48.407433 15461 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.408267 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:48.348142 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.408922 15515 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.409082 10020 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:48.348896 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.409394 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:48.347471  9947 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:48.409437 15515 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.351994 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1117) took 1034 ms. Trace:
I20260709 12:25:48.351853 15516 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.349581 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.411224 10021 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:48.411839 15515 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.410756 10669 rpcz_store.cc:276] 0709 12:25:47.317654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.317796 (+   142us) service_pool.cc:224] Handling call
0709 12:25:47.318160 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.318569 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.320050 (+  1481us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.076765 (+756715us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b2c000d07e8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:25:48.077178 (+   413us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.146223 (+ 69045us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:48.146255 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.146313 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.351689 (+205376us) raft_consensus.cc:1685] finished
0709 12:25:48.351715 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.351934 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":31635,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":8921,"spinlock_wait_cycles":6640384,"thread_start_us":68648,"threads_started":1,"wal-append.queue_time_us":272823}
I20260709 12:25:48.412184 15516 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.204133 15192 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.362368 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.412714  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2222) took 1445 ms. Trace:
I20260709 12:25:48.412801 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:25:48.208807 15523 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.279s	user 0.001s	sys 0.000s
I20260709 12:25:48.358858  9943 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11. Ops: []
I20260709 12:25:48.373063 14995 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.346212 15453 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.336982 15029 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:48.407939 15461 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.407076 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1112) took 1438 ms. Trace:
I20260709 12:25:48.413228  9942 rpcz_store.cc:276] 0709 12:25:46.967187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.967337 (+   150us) service_pool.cc:224] Handling call
0709 12:25:47.008809 (+ 41472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.310007 (+301198us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:47.310310 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.669388 (+359078us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:47.669439 (+    51us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:47.669479 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.412520 (+743041us) spinlock_profiling.cc:243] Waited 301 ms on lock 0x7b5400208354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:48.412562 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.412666 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14212,"prepare.queue_time_us":585,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":4034,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":16050,"spinlock_wait_cycles":692171008,"thread_start_us":358771,"threads_started":1,"wal-append.queue_time_us":472609}
I20260709 12:25:48.346879 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.386997 15409 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.345709 15477 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.423108 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:48.413933 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:48.385430  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:48.402721 10016 rpcz_store.cc:276] 0709 12:25:46.102019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:46.766894 (+664875us) service_pool.cc:224] Handling call
0709 12:25:46.833012 (+ 66118us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:25:46.879292 (+ 46280us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:46.904680 (+ 25388us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.339822 (+435142us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.340075 (+   253us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:25:47.361713 (+ 21638us) log.cc:844] Serialized 78 byte log entry
0709 12:25:47.383769 (+ 22056us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:47.384187 (+   418us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.408348 (+ 24161us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.434364 (+ 26016us) log.cc:844] Serialized 14 byte log entry
0709 12:25:47.448272 (+ 13908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:47.448341 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:48.038446 (+590105us) spinlock_profiling.cc:243] Waited 9.74 ms on lock 0x7b1800298180. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:48.067268 (+ 28822us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b1800298180. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:48.260173 (+192905us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:48.262526 (+  2353us) spinlock_profiling.cc:243] Waited 7.35 ms on lock 0x7b1800298180. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:48.384073 (+121547us) raft_consensus.cc:1685] finished
0709 12:25:48.384093 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:48.384241 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156549,"prepare.queue_time_us":1917832,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":138,"raft.queue_time_us":120391,"raft.run_cpu_time_us":850,"raft.run_wall_time_us":58394,"spinlock_wait_cycles":186869120,"thread_start_us":427376,"threads_started":1,"wal-append.queue_time_us":599755}
W20260709 12:25:48.440367  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55012.5us
W20260709 12:25:48.438771 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52801.2us
I20260709 12:25:48.430980 10677 rpcz_store.cc:276] 0709 12:25:46.968916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.254930 (+286014us) service_pool.cc:224] Handling call
0709 12:25:47.255516 (+   586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:47.362616 (+107100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.406834 (+1044218us) spinlock_profiling.cc:243] Waited 65.2 ms on lock 0x7b54007f2e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:48.407017 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":149879168}
I20260709 12:25:48.437316 10018 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:48.447954 10676 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:48.451129  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 190521us
I20260709 12:25:48.453444  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:48.450449 10676 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:48.385267 15442 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.461097 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:48.465591  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.484136 10016 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:25:48.435797 15508 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:48.496279  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.447088 15192 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:48.463056 14841 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:48.466101 15442 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:48.464003 15516 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:48.524355 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.525511 15468 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.505510 15343 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:25:48.465008 15417 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:25:48.510545  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.562740 15508 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:48.511329 15192 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:48.513294 15354 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.511152 15477 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:48.570051 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:48.541730 15516 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:48.605036 15465 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.540994 15453 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.599009 15411 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 10. Current term is 11.
W20260709 12:25:48.605377  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94943.3us
I20260709 12:25:48.607189 15515 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:48.622702 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 431121us
W20260709 12:25:48.655015 10062 connection.cc:256] RPC call timeout handler was delayed by -1.2757827760001419s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2847s for a response
I20260709 12:25:48.615427 15192 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:48.608661 15409 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.569566 15153 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.664803 15409 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.665140 10671 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:48.665310 15409 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.580603 15326 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:25:48.609156 15508 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.667474 15465 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:48.667858 15465 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:48.668659 15465 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.669857 15465 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:48.673107 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.674254 15192 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.687772 15192 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:48.700562 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.663753 15306 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.663750 15015 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.700098 15410 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:48.723593 15534 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 12:25:48.724684 15533 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:25:48.663805 15430 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:25:48.709913 15029 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.663990 15515 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.711838 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.705502 15458 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:25:48.733879 15269 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:48.762794 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 51065.6us
W20260709 12:25:48.760354  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 330662us
W20260709 12:25:48.758731  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 344312us
I20260709 12:25:48.773026  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:48.776262 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.761476 15458 consensus_queue.cc:1243] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:48.738338 15306 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.790766 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.791596 10019 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:48.792397 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.793036 10022 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:48.765250 15448 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.738543 15015 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.741107 15410 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.782923  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:48.785300 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:48.781307 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.737042 15431 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.809845 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:48.787456 15501 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:25:48.824343 15410 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.826472 15015 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.781571 15326 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:48.827597 15326 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:48.828290 15306 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.782183 15516 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.831216 15370 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:48.831929 15269 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.834800 14841 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:25:48.836364 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:48.830459 15516 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:48.838510 15431 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:48.845039 15515 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:48.845218 10022 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:48.767413 15275 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:48.852831 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.853114 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.853858 10097 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:25:48.855019 10092 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:48.828783 15326 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.757830 15029 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.847205 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:48.867666 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:48.862722 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.869877 10092 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:48.865924 10022 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:48.801940 15453 raft_consensus.cc:2764] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:25:48.858924 15457 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:48.888689 10018 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:48.870937 15269 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.891634 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.899518 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.874311 15458 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.895874 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.891280 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.866089 15370 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:48.904412 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:48.906252 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.859285 15420 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:25:48.909130 10018 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:48.909374 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.912740 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.914013 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.866710 15275 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.911995 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:48.920634 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:48.896677 15139 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:48.875730 15306 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:25:48.910799 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:48.931957 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:48.933192  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 479812us
I20260709 12:25:48.933995  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:48.932435  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 356482us
I20260709 12:25:48.934430 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.936414 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.936699 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.936986 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:48.937726 10094 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:48.908187 15411 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:48.908056 15410 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:48.908319 15457 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:48.911448 15324 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:48.949730 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.970397 10092 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:48.951244 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:48.951653 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:48.973111 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:48.925810 15458 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:48.994798 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71051.3us
I20260709 12:25:48.975471  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:48.975935 10062 leader_election.cc:400] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:48.971634 10091 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:48.998832 10062 leader_election.cc:403] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:25:49.000445 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:49.009701 10094 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:48.998780 15453 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:49.006422 10673 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:25:49.012089 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:49.020092  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44709.3us
I20260709 12:25:48.969385 15410 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.925457 15370 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:48.939347 15139 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:48.974920 15457 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.029145 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1181) took 1192 ms. Trace:
I20260709 12:25:49.036280 10675 rpcz_store.cc:276] 0709 12:25:47.838375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.922512 (+ 84137us) service_pool.cc:224] Handling call
0709 12:25:47.922811 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.923156 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.923184 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.923622 (+   438us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.170074 (+246452us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:49.028571 (+858497us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.028794 (+   223us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.028876 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.028924 (+    48us) raft_consensus.cc:1685] finished
0709 12:25:49.028940 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.029081 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":858028,"prepare.queue_time_us":858256,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":100,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":3003,"thread_start_us":171229,"threads_started":1,"wal-append.queue_time_us":444978}
I20260709 12:25:49.029650  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:49.040215  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 67436.8us
W20260709 12:25:49.031009  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 120090us
I20260709 12:25:49.042704 10091 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 8. Current term is 9. Ops: []
I20260709 12:25:49.043857 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.043982 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.044603 10092 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:49.044807 10091 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:49.046154 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:49.039873  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:49.048471 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.048540 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.049268 10091 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.049778 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.052649 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1183) took 1207 ms. Trace:
I20260709 12:25:48.994318 15015 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:48.970191 15411 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.019747 15448 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:49.011809 15515 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.053059 10672 rpcz_store.cc:276] 0709 12:25:47.845223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.929041 (+ 83818us) service_pool.cc:224] Handling call
0709 12:25:47.929377 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.949400 (+ 20023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.988639 (+ 39239us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.989299 (+   660us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.278752 (+289453us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:49.052077 (+773325us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.052289 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.052363 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.052408 (+    45us) raft_consensus.cc:1685] finished
0709 12:25:49.052427 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.052565 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":729956,"raft.queue_time_us":252,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":189350,"threads_started":1,"wal-append.queue_time_us":514453}
I20260709 12:25:49.080183 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.087328 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.088543 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:49.081192  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.081705 10098 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:49.071420 15324 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:49.025736 15029 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.032462 15306 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:49.092172 10090 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:49.092710  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:49.053015 15543 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:25:49.054901 15477 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:25:49.097141 10091 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:49.029812 15155 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.098508 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.028910 15358 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:49.217267 15358 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:49.217533 10672 raft_consensus.cc:2410] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:49.217800 15358 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:49.026546 15516 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:49.218402 15139 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:49.218473 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.219408 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.219640 10672 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
W20260709 12:25:49.220052 15537 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 12:25:49.220435 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:49.221573 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.222241 10672 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:49.050356 15438 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.223040 15438 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.223517 15438 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.223898 10090 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:49.225453 10090 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:25:49.225555 15438 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.096751 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.007547 15458 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:49.226940 15531 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 12:25:49.227352 15541 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:25:49.053977 15550 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.175s
I20260709 12:25:49.229136 15458 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.230841 15557 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.231405 15557 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:49.051424 15447 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.008s
I20260709 12:25:49.091631 15410 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.234009 15447 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.234441 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.226962 10092 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:49.236497 10093 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:25:49.218818 15516 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.027289 15442 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.079916 15453 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.164s
I20260709 12:25:49.080206 15551 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:25:49.214730 15528 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:25:49.099290 15306 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:49.288743 15306 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:49.291620 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 371087us
I20260709 12:25:49.292303 15468 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:49.293057 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:49.287542 10675 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:49.294719 15555 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:25:49.274224 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 242349us
I20260709 12:25:49.243284 15431 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:49.279006 15516 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.243212 15029 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:49.267372 15139 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:49.243386 15501 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:49.317431 10092 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:49.311239 15453 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:49.302688 15551 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:49.321798 15549 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 12:25:49.317149 15198 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:49.335255 15457 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.338392 15411 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:49.339197 15559 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:25:49.340327 15442 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.340960 15551 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.319001 15192 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:49.321879 15557 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.316383 15431 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.349472 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:49.319727 15139 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.319550 15029 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.343997 15198 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.321597 15275 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.322078 15501 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.360057 15431 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.361105 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.363791 15269 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:49.365145 15366 raft_consensus.cc:993] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:25:49.371794 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:49.383255 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.384202 10674 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:49.385507 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.387969 10674 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:25:49.390985 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:49.394995 10674 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:49.396991 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:49.397559 10677 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.400403 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:49.401113 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3212) took 2513 ms. Trace:
I20260709 12:25:49.379014 15269 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:49.401558 10017 rpcz_store.cc:276] 0709 12:25:46.885601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.095840 (+210239us) service_pool.cc:224] Handling call
0709 12:25:48.062655 (+966815us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:48.158419 (+ 95764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.398742 (+1240323us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b54001c5050. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:49.398877 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2222716672}
I20260709 12:25:49.373448 15192 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.393190 15366 raft_consensus.cc:1081] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:25:49.404968 10674 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:49.388373 15566 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.408097 15198 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.408846 15447 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:49.423571 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:49.404554 15486 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:49.430419 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:49.431510  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 126499us
I20260709 12:25:49.437321 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.440038 10674 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:49.443629 10674 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:25:49.444590 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1132) took 1005 ms. Trace:
I20260709 12:25:49.430542 15486 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.444890 10674 rpcz_store.cc:276] 0709 12:25:48.438876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.443070 (+1004194us) service_pool.cc:224] Handling call
0709 12:25:49.443360 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.444030 (+   670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.444529 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:49.429126 15015 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:49.430565 15410 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:49.449158 15410 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:49.454020 15501 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.454309 15410 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.455396 15139 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.443040 15409 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.456369 15029 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.454917 15269 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.457947 15269 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:49.459111 15306 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.447194 15486 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.465418 15516 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.467532 15192 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:49.468146 15529 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.131s	user 0.005s	sys 0.000s
I20260709 12:25:49.470197 10677 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:49.432987 15358 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.447896 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:49.475211 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1122) took 1536 ms. Trace:
I20260709 12:25:49.474494 15198 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:49.480887 10677 rpcz_store.cc:276] 0709 12:25:47.938925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.459334 (+520409us) service_pool.cc:224] Handling call
0709 12:25:49.396853 (+937519us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:49.475151 (+ 78298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5993344}
I20260709 12:25:49.481112 15198 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.476739 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1126) took 1070 ms. Trace:
I20260709 12:25:49.482605 10672 rpcz_store.cc:276] 0709 12:25:48.405904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.281342 (+875438us) service_pool.cc:224] Handling call
0709 12:25:49.281682 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.386607 (+104925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.386663 (+    56us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.386730 (+    67us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:49.386788 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.476448 (+ 89660us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54007602d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:49.476498 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.476661 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240553856}
I20260709 12:25:49.484193 15198 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.482707 10677 pending_rounds.cc:85] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 10
W20260709 12:25:49.479076 10671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 989) took 1284 ms (client timeout 1671 ms). Trace:
I20260709 12:25:49.485734 10677 pending_rounds.cc:107] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 13.11
W20260709 12:25:49.485836 10671 rpcz_store.cc:269] 0709 12:25:48.194297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.211864 (+ 17567us) service_pool.cc:224] Handling call
0709 12:25:49.478847 (+1266983us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b54007e4fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:49.479008 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884228864}
I20260709 12:25:49.486629 10677 raft_consensus.cc:2889] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.488209 10062 leader_election.cc:400] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13.
I20260709 12:25:49.489349 15410 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.500379 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1178) took 1685 ms. Trace:
I20260709 12:25:49.490831 15516 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.492956 15358 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.493361 15306 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:49.493558 15457 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:49.494854 10671 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:49.508018 15457 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:49.508818 15457 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:49.511618 15324 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:49.500923 10668 rpcz_store.cc:276] 0709 12:25:47.815226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.815520 (+   294us) service_pool.cc:224] Handling call
0709 12:25:47.815825 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.817951 (+  2126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.817984 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.825513 (+  7529us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.165486 (+339973us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:48.177787 (+ 12301us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.177999 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.178076 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.467832 (+1289756us) spinlock_profiling.cc:243] Waited 80 ms on lock 0x7b2400329798. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a809d 00007f4e090da10e 00007f4e090f188c 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221
0709 12:25:49.499937 (+ 32105us) raft_consensus.cc:1685] finished
0709 12:25:49.500087 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.500267 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1723,"prepare.queue_time_us":862667,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":5868,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":12275,"spinlock_wait_cycles":183959296,"thread_start_us":290681,"threads_started":1,"wal-append.queue_time_us":475631}
I20260709 12:25:49.515654 10668 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:49.525650 15366 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:49.513965 15324 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.528101 15366 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.516824 15501 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:49.532255 15501 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.534475 15501 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.536180 15366 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.536055 15486 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:49.537345 15545 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.526s	user 0.003s	sys 0.000s
W20260709 12:25:49.539656 15548 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.533s	user 0.000s	sys 0.002s
I20260709 12:25:49.541298 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1125) took 1135 ms. Trace:
I20260709 12:25:49.541854 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1123) took 1136 ms. Trace:
I20260709 12:25:49.542241 10673 rpcz_store.cc:276] 0709 12:25:48.405163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.409146 (+  3983us) service_pool.cc:224] Handling call
0709 12:25:49.541707 (+1132561us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54007d3cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:49.541804 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584301440}
I20260709 12:25:49.543447 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3235) took 1177 ms. Trace:
I20260709 12:25:49.543684 15326 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:49.543828 10024 rpcz_store.cc:276] 0709 12:25:48.365858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.413449 (+ 47591us) service_pool.cc:224] Handling call
0709 12:25:48.413723 (+   274us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:48.429118 (+ 15395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.429150 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:48.435325 (+  6175us) log.cc:844] Serialized 52 byte log entry
0709 12:25:48.874778 (+439453us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:48.908016 (+ 33238us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.908344 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.908502 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.908582 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.539057 (+630475us) spinlock_profiling.cc:243] Waited 4.62 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:49.543225 (+  4168us) raft_consensus.cc:1685] finished
0709 12:25:49.543250 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.543388 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31802,"prepare.queue_time_us":117735,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":74,"raft.queue_time_us":21537,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":20571,"spinlock_wait_cycles":10618112,"thread_start_us":363759,"threads_started":1,"wal-append.queue_time_us":449958}
I20260709 12:25:49.537436 10094 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:49.541714 10675 rpcz_store.cc:276] 0709 12:25:48.405670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.064034 (+658364us) service_pool.cc:224] Handling call
0709 12:25:49.071750 (+  7716us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.294235 (+222485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.541078 (+246843us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54006ff754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:49.541240 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":495764736}
I20260709 12:25:49.551909 15501 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:49.552474 15501 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.553750 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2163) took 1309 ms. Trace:
I20260709 12:25:49.554539 15366 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.554167 10095 rpcz_store.cc:276] 0709 12:25:48.244125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.250696 (+  6571us) service_pool.cc:224] Handling call
0709 12:25:48.251090 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.251546 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:48.438211 (+186665us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.765140 (+326929us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.777819 (+ 12679us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.778009 (+   190us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:48.778063 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.778158 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.536791 (+758633us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:49.553482 (+ 16691us) raft_consensus.cc:1685] finished
0709 12:25:49.553580 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.553696 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":591,"prepare.queue_time_us":286507,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":218,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":10059,"spinlock_wait_cycles":28048768,"thread_start_us":445527,"threads_started":2,"wal-append.queue_time_us":459020}
I20260709 12:25:49.555377 15366 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.557761 15366 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.558324 15410 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:49.560030 15556 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.535s	user 0.001s	sys 0.000s
I20260709 12:25:49.560547 15457 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.568786 10092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:25:49.569414 10092 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:25:49.569772 10092 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:49.571678 15324 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.573906 15324 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.574303 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.576305 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.577543 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.578905 10090 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:49.579679 15457 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.579763 10090 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:49.582015 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.582620 10099 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:49.582727 15501 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.583531 10090 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.584784 10090 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:49.588073 10092 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.544108 15326 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:49.591719 15326 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.592964 15326 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.594703  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:49.595616  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:49.598985  9842 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 11, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 11 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:49.600564  9842 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 12, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 12 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:49.603029 15192 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:49.604012 15192 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:25:49.604579 15192 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 8 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:49.605671 15192 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:49.607123  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.607842 15577 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:49.608357  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.608385 15577 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.610093 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.612619 10093 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:49.616129  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:49.616882 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:49.617691 10675 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.617602 15577 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.619599 15577 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:49.619629 14995 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:49.620306 15577 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.621357 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:49.621639 14995 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.622941 15457 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:25:49.623180 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.607745 15192 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:49.627672 10095 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:49.629666  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 695695us
I20260709 12:25:49.630957  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:49.636539 10090 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:25:49.637054 10090 pending_rounds.cc:85] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 7
I20260709 12:25:49.637344 10090 pending_rounds.cc:107] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:25:49.637642 10090 raft_consensus.cc:2889] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:49.651010 15465 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:49.652130  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:49.653646 15198 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:49.648254 10092 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.655198 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1179) took 1837 ms. Trace:
I20260709 12:25:49.652369 15501 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.652091 15577 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.655555  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:49.655556  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:49.657986 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.602159  9842 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 5 to 10, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 10 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:49.658005  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:49.637792 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.655581 10670 rpcz_store.cc:276] 0709 12:25:47.817629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:47.823539 (+  5910us) service_pool.cc:224] Handling call
0709 12:25:47.823874 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:47.833709 (+  9835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:47.833754 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:47.834328 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.115801 (+281473us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:48.145660 (+ 29859us) log.cc:844] Serialized 14 byte log entry
0709 12:25:48.145889 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:48.145955 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.028155 (+882200us) spinlock_profiling.cc:243] Waited 8.55 ms on lock 0x7b18004976e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0d42a58f 00007f4e06469571 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd
0709 12:25:49.625102 (+596947us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b18004976e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:25:49.655009 (+ 29907us) raft_consensus.cc:1685] finished
0709 12:25:49.655028 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.655143 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12989,"prepare.queue_time_us":29579,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":3922,"raft.queue_time_us":10567,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":30016,"spinlock_wait_cycles":35038208,"thread_start_us":236843,"threads_started":1,"wal-append.queue_time_us":318261}
I20260709 12:25:49.662858  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:49.664594  9948 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:49.663820 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.673475  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:49.673568 10099 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:49.678514 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:49.680791 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:49.680506  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49617.7us
I20260709 12:25:49.682337  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:49.682683 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:49.684238  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:49.662304 15358 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:49.691794 10092 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:49.688884  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:49.693717  9991 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:49.694883  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:49.695992 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:49.696236 10090 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 17. Ops: []
I20260709 12:25:49.692662  9842 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 12, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 12 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:49.655241 15198 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:25:49.655725 15465 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 12:25:49.696026 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.698348 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.709259 10092 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:49.699848 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.701740 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:49.701805 10098 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:49.710404 10091 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:49.707284 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:49.706562  9842 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 10, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 10 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:49.688021 15482 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:49.717288  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:49.687970 15410 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.662813 15501 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.721462  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:49.722333  9842 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 11, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 11 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:49.727701  9842 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:49.729265 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:49.701529 15477 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:49.721812  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:49.702263 14995 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:49.681591 15550 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:25:49.707989 15358 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:49.687664 15367 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:25:49.697184 15544 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.723922 10675 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:49.733566  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:49.719342 15370 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:25:49.720185 15482 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.759944  9842 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 10, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 10 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.769304  9842 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 5 to 10, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 10 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:49.764854  9843 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 311) took 37751 ms (client timeout 15000 ms). Trace:
I20260709 12:25:49.765615  9941 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:49.775027  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73290.3us
I20260709 12:25:49.776635 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.778199 10096 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:49.732069 15410 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.778831  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:49.759618 15477 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:49.758565 15569 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:25:49.780371  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:49.783995  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:49.771040 15482 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.782132  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:49.747603 15574 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:25:49.794834  9842 catalog_manager.cc:5697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 8 to 9, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 9 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.796864  9842 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:49.760288 15560 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:25:49.798693 10093 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:49.778071 15275 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.777410 15370 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:49.789285 15550 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:49.804872  9842 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 12, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 12 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:25:49.771085  9843 rpcz_store.cc:269] 0709 12:25:12.007394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:12.007551 (+   157us) service_pool.cc:224] Handling call
0709 12:25:49.758768 (+37751217us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:42.198780 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:42.199550 (+   770us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:42.199609 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:25:42.199650 (+    41us) tablet.cc:662] Decoding operations
0709 12:25:42.201341 (+  1691us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:42.201439 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:42.201469 (+    30us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:25:42.267053 (+ 65584us) tablet.cc:701] Row locks acquired
0709 12:25:42.267108 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:25:42.267300 (+   192us) write_op.cc:270] Start()
0709 12:25:42.267471 (+   171us) write_op.cc:276] Timestamp: P: 1783599942267226 usec, L: 0
0709 12:25:42.267508 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:25:42.268337 (+   829us) log.cc:844] Serialized 6337 byte log entry
0709 12:25:43.008720 (+740383us) op_driver.cc:464] REPLICATION: finished
0709 12:25:43.262470 (+253750us) write_op.cc:301] APPLY: starting
0709 12:25:43.262572 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:43.537137 (+274565us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:43.642876 (+105739us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:43.920008 (+277132us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:43.920402 (+   394us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=32,key_file_lookups=32,delta_file_lookups=32,mrs_lookups=0
0709 12:25:43.920855 (+   453us) write_op.cc:312] APPLY: finished
0709 12:25:43.921345 (+   490us) log.cc:844] Serialized 148 byte log entry
0709 12:25:44.179960 (+258615us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:44.180132 (+   172us) write_op.cc:454] Released schema lock
0709 12:25:44.180220 (+    88us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:44.206381 (+ 26161us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:44.214926 (+  8545us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2236,"reported_tablets":48,"spinlock_wait_cycles":2304,"child_traces":[["op",{"apply.queue_time_us":230061,"apply.run_cpu_time_us":55629,"apply.run_wall_time_us":923232,"cfile_cache_hit":66,"cfile_cache_hit_bytes":71039,"num_ops":16,"prepare.queue_time_us":332015,"prepare.run_cpu_time_us":34396,"prepare.run_wall_time_us":289827,"raft.queue_time_us":290369,"raft.run_cpu_time_us":8999,"raft.run_wall_time_us":285597,"replication_time_us":692139,"spinlock_wait_cycles":2432,"thread_start_us":906171,"threads_started":5,"wal-append.queue_time_us":554129,"wal-append.run_cpu_time_us":26833,"wal-append.run_wall_time_us":2258637}]]}
I20260709 12:25:49.810123 15412 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:49.810726 15412 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.801981 15563 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:49.815491  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82070.2us
I20260709 12:25:49.816147 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1188) took 1499 ms. Trace:
I20260709 12:25:49.817080  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:49.799045 15486 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.810843 15477 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.811882 15550 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.823927 15550 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:49.818552 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1134) took 1044 ms. Trace:
I20260709 12:25:49.822098  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:49.818428  9842 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 11, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 11 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:49.825839  9949 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:25:49.822985  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:49.823174 15367 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:49.827265  9946 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:49.803750 15569 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.824728 10674 rpcz_store.cc:276] 0709 12:25:48.774366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.449605 (+675239us) service_pool.cc:224] Handling call
0709 12:25:49.449903 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:49.450117 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:25:49.450497 (+   380us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.818205 (+367708us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:49.818265 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:49.818323 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.818384 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.818497 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":310553,"prepare.queue_time_us":1991,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":313002,"thread_start_us":56763,"threads_started":1,"wal-append.queue_time_us":70448}
I20260709 12:25:49.816514 10676 rpcz_store.cc:276] 0709 12:25:48.316493 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.458459 (+141966us) service_pool.cc:224] Handling call
0709 12:25:48.462115 (+  3656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.507238 (+ 45123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.507286 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.509290 (+  2004us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.755013 (+245723us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:49.317034 (+562021us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.777270 (+460236us) spinlock_profiling.cc:243] Waited 5.96 ms on lock 0x7b18004b9ac0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:25:49.804464 (+ 27194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.805002 (+   538us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:49.805050 (+    48us) raft_consensus.cc:1685] finished
0709 12:25:49.815573 (+ 10523us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:49.815721 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":395310,"prepare.queue_time_us":547188,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":534012,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"spinlock_wait_cycles":17059072,"thread_start_us":343435,"threads_started":3,"wal-append.queue_time_us":390723}
I20260709 12:25:49.830084 15367 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:49.838289 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 274878us
I20260709 12:25:49.843385 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:49.839974  9941 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:25:49.844022  9941 pending_rounds.cc:85] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 8
I20260709 12:25:49.844285  9941 pending_rounds.cc:107] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:25:49.844671  9941 raft_consensus.cc:2889] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:49.842636 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 152020us
I20260709 12:25:49.848757  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:49.851683  9946 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:49.851828  9842 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 9 to 10, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 10 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.820694 15515 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:49.855520 15501 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:49.855324 15515 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.856570 15367 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.820209 15560 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:49.825778 15465 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:49.854210  9842 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 13, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 13 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.839723 15403 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.823576 15370 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.849370 15486 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:49.865494  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:49.873971 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:49.875605 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:49.881218 10670 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:49.874290 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 556644us
W20260709 12:25:49.879460 15582 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:25:49.803262 15358 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.881225 10674 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:49.863660  9842 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 6 to 10, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 10 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:49.866966 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:49.883234  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:49.890228  9842 catalog_manager.cc:5697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:49.863454 15465 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:49.883564 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.865552 15501 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.886700 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.899061 10091 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:49.887163 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:49.861840 15560 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:49.894181 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.886302 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:49.820719 15563 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.901512 15574 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:49.902029  9842 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 12, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 12 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:49.898746 15486 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:49.873140 15409 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.895772 15465 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:49.896193 15324 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:49.912211 15198 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:49.918107 15550 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.923691 15412 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:49.880879 15403 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:49.928227  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:49.940425 10674 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:25:49.988632 10670 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:49.965025 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.006299 10676 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:49.966609 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.889461 15367 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:49.849208 15585 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:25:49.978529 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.980149 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:49.987413 15409 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.987627 15324 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.008778  9942 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
W20260709 12:25:49.983291  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 71160us
I20260709 12:25:49.943749  9943 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
W20260709 12:25:49.973683 15573 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:25:49.906024 15482 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:49.989703 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:50.019531 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:49.942401 15560 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:49.991395  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:49.994872 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.021483 10668 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:49.996553  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:49.977036 15569 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:49.941730 15516 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 12:25:49.974840 15501 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.001717 15586 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:25:50.023940 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.020428 10670 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 17. Ops: []
I20260709 12:25:50.029645  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:50.027817 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.032948 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.023216 15457 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:50.044528 10674 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:25:50.045514 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.050918 10673 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:50.023015 15458 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:50.036168  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.048290 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.031478 14995 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:50.055873 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:50.055804 15411 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:50.036828 15482 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.055968  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.058281 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.036340 15569 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.034993 15275 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:25:50.098354 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:50.120213  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:50.129478 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71277.5us
I20260709 12:25:50.130251 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.130541  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.131173  9949 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:50.131848  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.132331  9949 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:50.132944  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.054101 15412 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:50.135349  9949 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:50.138389  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.136008 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.058799 15376 raft_consensus.cc:2764] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:25:50.054296 15577 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.145434  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:50.074173 15596 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:25:50.080493 15501 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.077600 15550 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:50.134975 15594 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.052s	user 0.002s	sys 0.000s
I20260709 12:25:50.119328 15411 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:50.156353 15570 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:50.162154 15501 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:50.162966 15501 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.163345 15501 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.163669 10095 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:50.057154 15457 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:50.036943 15486 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.071170 15563 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.054736 15544 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:50.152477  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.152761 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.133998 15567 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.154070 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.148432 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:50.075897 15409 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.137243 15603 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:25:50.182233 15139 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:25:50.182791 15139 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:50.183672 15139 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:50.185343 15597 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:25:50.147850 15482 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:50.187414 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.135648 15389 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:50.137357 15412 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:50.151860 15447 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.137311 15029 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:50.147734 15590 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:50.165755 10675 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:50.166612 10643 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.167665 15508 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.151651 15577 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.186648 15482 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.207044 15577 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.207269 15482 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.169126 15563 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:50.169106  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.207958 15563 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:50.208177  9949 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:50.208453 15563 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.208202 15587 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:50.165331 15457 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.209507 15587 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.209969 15587 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:50.211298 15590 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:50.211735 15590 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.168104 15567 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.176122  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.076788 15403 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.188897 10098 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:50.194104  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49133.5us
I20260709 12:25:50.161202 15570 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:50.209299  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.214056 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.160969 15411 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.223150 10677 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:50.194864 15608 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.235980 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:50.223955  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.225967  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:50.188221 15585 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:50.202137 15412 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.202930 15275 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:50.235440  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.221328 15447 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.244798  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.242578 15605 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.249517 15605 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.250824 15567 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.235378 15508 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:50.251523 15508 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:50.251962 15508 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:50.252187 15605 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.234814 15139 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:50.247609 15477 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:50.254237 15477 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.240039 15516 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:50.255489 15358 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:50.256294 15447 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.256976 15477 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:50.232961 15570 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.265564 15465 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.267983  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.268762  9946 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:50.269629 15586 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:50.270021 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.270906 15447 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:50.270780 15501 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.271610 15447 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.272048 15447 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.272256 15569 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.246717 15577 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:50.265934 15326 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:50.278407  9949 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:50.274236 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:50.269824  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.270990 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1186) took 1955 ms. Trace:
I20260709 12:25:50.284149 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:50.282868 10669 rpcz_store.cc:276] 0709 12:25:48.315997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.455796 (+139799us) service_pool.cc:224] Handling call
0709 12:25:48.456277 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.477038 (+ 20761us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:49.104456 (+627418us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.321434 (+216978us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.328961 (+  7527us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.329197 (+   236us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:49.330008 (+   811us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.789319 (+459311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.270572 (+481253us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b54004d49d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:50.270642 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.270779 (+   137us) raft_consensus.cc:1685] finished
0709 12:25:50.270796 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.270933 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1049740,"prepare.queue_time_us":727936,"prepare.run_cpu_time_us":219,"prepare.run_wall_time_us":1892,"raft.queue_time_us":964783,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":6797,"spinlock_wait_cycles":52763136,"thread_start_us":212155,"threads_started":1,"wal-append.queue_time_us":241583}
I20260709 12:25:50.286984  9946 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:50.290293 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:50.293865  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:50.245864 15324 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.244127 15585 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.241098 15608 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:50.320053 15552 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.294s	user 0.003s	sys 0.003s
I20260709 12:25:50.320778 15608 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.320803 15585 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.321987 15605 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:50.318614 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:50.328208 15409 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:50.329197 15409 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.329550 15409 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:50.330840 15605 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:25:50.334625 15557 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:50.335125 15557 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:50.336122 15557 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.337411 15557 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.288173 15153 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.282940 15326 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:50.299708 15501 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.339301 15326 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.299685 15550 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:50.340582  9940 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:50.340518 15550 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.291802 15567 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:50.289021 15516 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.292928 15606 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:50.285337 15458 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:50.363318  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.364524  9941 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:25:50.250831 15563 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.366551 10091 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:50.247866 15275 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.297397 15482 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:25:50.373345 15550 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.374421 15458 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.351789 15486 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:50.375653 15486 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:50.355960 15447 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.376511 15486 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.354051 15570 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.378311 15447 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.343468 15477 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.379837 15570 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.379940 15560 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.381057 15477 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.358237 15612 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:25:50.382090 15560 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.383508 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.384881 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:50.386026 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.388237 15563 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.388605 15461 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.367831 15516 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.292745 15367 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:25:50.355665 15586 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:50.391769 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.396365 15411 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:50.405840 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:50.406895 10096 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:50.407532 15409 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.407147 15586 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:50.408037 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:50.409215 15586 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.410706 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:50.388157 15465 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.406785 15461 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:50.412911 15411 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:50.413982  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.399051 15139 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:25:50.414175  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 127245us
I20260709 12:25:50.417419 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:50.417860 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.418284 10669 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.418529 10674 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:50.400030 15570 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:50.420346 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.421005 10674 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:50.424369 15516 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.426859 15410 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.423492  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.428818  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.427800 15275 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.428825  9946 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:50.431794 15409 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.433926  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.434638  9946 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.436209  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.436694 10643 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:50.437394  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.440922 14841 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:50.441370 14841 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:50.448565 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:50.450799 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:50.451683 10090 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:50.452204 10669 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:50.454206 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:50.357184 15508 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:50.456338  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.457569 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:50.458053  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:50.458875 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 158757us
I20260709 12:25:50.459612 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:50.459338  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 164237us
I20260709 12:25:50.464612  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:50.464474 15417 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:50.462368 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:50.466461 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:50.469233 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.470216 10021 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:25:50.471977 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.474086 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.476179  9946 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
W20260709 12:25:50.476830 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.478645 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.479277 10021 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:50.479584 15554 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:50.480212 15554 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:50.480278  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.481314  9946 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.481227 15554 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.481693 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.481355 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.483294 10021 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.489914 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.493768  9946 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:25:50.454996 15410 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:50.442181 14841 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.495368 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.505875 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.513090  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.513481 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.513873  9946 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:50.501973  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.514504  9940 raft_consensus.cc:2368] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:25:50.506108 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:50.505178  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47174.7us
I20260709 12:25:50.510258  9949 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:50.515570 10025 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:50.515877  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.516330  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.516961  9949 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:50.454550 15409 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:50.519979  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.520695  9940 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:25:50.520020  9944 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:50.521873 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.522522 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.520907 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.513640 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.519673  9991 leader_election.cc:400] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:25:50.528484  9949 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:50.524612 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:50.524168  9991 leader_election.cc:403] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:50.526548 10021 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:50.527716  9940 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:50.524942 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.501480 15468 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:50.500026 15412 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.531826 15619 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:50.532284 15619 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.532630 15619 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.532995 15412 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.499650 15417 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:50.534031 15417 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:50.534858 14841 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:50.500384 15457 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:50.495656 15516 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:50.527809  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.499411 15587 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.542047 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:50.543120  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:50.520426 15139 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:50.544914  9946 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:25:50.545017 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.527104 15608 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:50.504599 15590 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.523553 15609 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:50.543875 15340 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.555804 15608 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.563301 15608 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:50.562831 15619 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.546941 10677 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 12: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:50.519434 15410 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.564764  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.566277  9942 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:50.541132 15366 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:50.566893 15366 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:50.565986 15619 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.567276 15366 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.567332 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.569379 15486 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:50.551839 15139 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.571193 15139 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:50.571736  9945 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:50.573097  9945 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:50.547667 15587 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.548188 15461 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:50.575628 15461 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:50.548630 15563 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:25:50.576772 15461 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.577741 15563 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:50.578116 15563 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:50.541728 15516 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.578792 15516 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.501624 15606 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:25:50.580418 15619 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:50.580976 15619 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:50.580751 15563 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.581997 15619 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.583303 15619 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.544297 15412 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:50.552650  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.584072 15412 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:50.554729  9949 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:50.585011 15412 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:50.585958 15567 consensus_queue.cc:1175] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 12 status { last_received { term: 12 index: 10 } last_committed_idx: 10 last_received_current_leader { term: 12 index: 10 } }
I20260709 12:25:50.589159  9947 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:50.539022 15411 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:50.591722 15411 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:50.593317 15411 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:50.584528 15448 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
W20260709 12:25:50.596433 15623 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.066s	user 0.002s	sys 0.000s
I20260709 12:25:50.550906 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2060) took 2205 ms. Trace:
I20260709 12:25:50.545904 15508 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:50.556808 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.559309 15590 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.560401 15458 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:50.582294 15586 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:50.574390  9945 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:25:50.556414  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.612429 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.557456 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.613265 10021 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:50.614674 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.613194 10025 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:50.617233 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.534888 15417 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.610631 15410 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.619307 15544 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:50.599228 10023 rpcz_store.cc:276] 0709 12:25:48.345214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:48.397792 (+ 52578us) service_pool.cc:224] Handling call
0709 12:25:48.398121 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:48.439186 (+ 41065us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:48.439221 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:48.459232 (+ 20011us) log.cc:844] Serialized 26 byte log entry
0709 12:25:48.970236 (+511004us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:48.995910 (+ 25674us) log.cc:844] Serialized 14 byte log entry
0709 12:25:49.052865 (+ 56955us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b4800002958. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:25:49.559466 (+506601us) spinlock_profiling.cc:243] Waited 4.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:49.624815 (+ 65349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.550547 (+925732us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b54001379d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:50.550636 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.550689 (+    53us) raft_consensus.cc:1685] finished
0709 12:25:50.550702 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.550845 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1283186,"prepare.queue_time_us":24081,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":700522,"raft.run_cpu_time_us":5312,"raft.run_wall_time_us":593488,"spinlock_wait_cycles":57596800,"thread_start_us":457203,"threads_started":1,"wal-append.queue_time_us":519615,"wal-append.run_cpu_time_us":4781,"wal-append.run_wall_time_us":1536889}
I20260709 12:25:50.620869 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.631958 10022 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:50.602118 15563 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:50.629316 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.556857 15370 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:50.638841 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 113954us
I20260709 12:25:50.639981 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.642230 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.628911 15577 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.648877 10025 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:50.650656  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:50.652208  9945 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:50.655809 10023 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:25:50.657444 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.632831 15275 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:25:50.647804 15410 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:50.634780 14995 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.033s
I20260709 12:25:50.620690 15458 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.632874 15420 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:25:50.662317 15417 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:50.651178 15567 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:50.663971 15611 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:50.663846 15609 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.687748 10023 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:50.636482 15569 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:25:50.689217 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.682396 15410 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:50.727497 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.664759 15606 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:50.719583 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.725654 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:50.730855 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.732758 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:50.733593 10092 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:50.733959 15588 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:50.734586 15621 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.107s
I20260709 12:25:50.735015 10091 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:50.735246 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.662040 15577 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.692476 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3403) took 1031 ms. Trace:
I20260709 12:25:50.728641 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.725885  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.737715 10093 rpcz_store.cc:276] 0709 12:25:49.660701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.663381 (+  2680us) service_pool.cc:224] Handling call
0709 12:25:50.692291 (+1028910us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54003ac0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:50.692414 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":297015040}
I20260709 12:25:50.734527 15588 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.739923 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:50.740784 10093 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:50.682257 15555 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:50.694284 15611 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:50.736011 15431 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:50.724102 15448 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:50.691362  9945 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:50.740839 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.728471 15569 consensus_queue.cc:1243] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:50.764978  9942 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:25:50.766793  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:50.730053 15410 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:50.768301 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.781198 10023 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:50.781697 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.783589 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:50.778287 10096 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:50.778930 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:50.777348 15608 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.787317 10093 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:50.748663 15420 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:50.739754 15630 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:50.794505 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:50.762494 15431 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:50.807763 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:50.809191 10669 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:50.809319 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.810434 10669 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:50.810719 10668 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:25:50.811856 10676 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:25:50.781543 15621 consensus_queue.cc:1243] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:25:50.832202  9940 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:25:50.786625 15567 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:50.736441 15606 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.814350 15588 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.793803 15420 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:50.799685 15340 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:25:50.852567 10097 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:50.854363 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.854919 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.780872 15448 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.862946 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.863554 10025 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:50.837006 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:50.846544 10674 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:50.847313 10673 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:50.862255 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.763389 15555 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.869124 10018 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:50.869959  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.890045 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.864266 15611 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.835412 15619 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:50.874181  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:50.903414 10097 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:50.898628  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.887316 10673 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:50.897413 10019 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:25:50.895416 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.900605  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:50.838516  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:50.856603 15420 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:50.918102 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:50.830994 15551 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
W20260709 12:25:50.923971  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:50.924278  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:50.878537 15632 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:25:50.747718 15609 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.804765 15630 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.949038 15630 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.950739 15570 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:50.951335 15570 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.953414 15570 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.956463 15609 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.895529 15577 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:50.958947 15626 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.232s	user 0.002s	sys 0.000s
I20260709 12:25:50.959980 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.901284 15442 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:25:50.891080 15624 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:25:50.981004 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2172) took 1290 ms. Trace:
I20260709 12:25:50.981487 10099 rpcz_store.cc:276] 0709 12:25:49.690438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.690628 (+   190us) service_pool.cc:224] Handling call
0709 12:25:49.713063 (+ 22435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.713438 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:49.730277 (+ 16839us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.343981 (+613704us) spinlock_profiling.cc:243] Waited 1.45 ms on lock 0x7b180046db60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:25:50.611237 (+267256us) log.cc:844] Serialized 14 byte log entry
0709 12:25:50.611462 (+   225us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.628853 (+ 17391us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.629002 (+   149us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:50.629035 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.629105 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:50.980726 (+351621us) raft_consensus.cc:1685] finished
0709 12:25:50.980759 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:50.980932 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7297,"prepare.queue_time_us":22810,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":13089,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":16479,"spinlock_wait_cycles":15862400,"thread_start_us":259378,"threads_started":1,"wal-append.queue_time_us":911792}
I20260709 12:25:50.984293 10021 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:50.932734 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.984696 15516 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.986109 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:50.937230 15551 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:50.937629 15608 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.937857 15430 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:50.988503 15430 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:50.989464 15430 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.989913 10022 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:50.939870 15606 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.941500 15555 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.942189 15567 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.995955 15370 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:50.881114 15458 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.945551 15515 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:50.946612 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:50.946718 15619 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:50.935672 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:50.920944 15366 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.970125 15448 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:50.972995 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:50.886580 15431 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:50.978363 15611 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.005571 10094 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:50.984349 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:51.031152 10673 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:51.012045 10676 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:50.923954 15468 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:51.016255 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69928.5us
I20260709 12:25:51.007030 15430 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.020983 15619 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:51.024454 15515 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.038136 15431 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.034936 15410 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:51.064967 15139 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:51.086301 15592 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.075s	user 0.005s	sys 0.000s
I20260709 12:25:51.079209 15420 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:51.169519  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 6c7bd3fa1bfa431c89fc9c9096452a76 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:51.193176  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:51.194471  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.195334  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:51.195717  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.196460  9947 raft_consensus.cc:2368] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:51.195363  9945 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:25:50.963613  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 89720.1us
I20260709 12:25:50.929926 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:51.190264  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 265896us
I20260709 12:25:50.997949 10023 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:50.984755 10021 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:51.201417  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:51.204694 10021 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:50.969494 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:51.205338  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:51.213946  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:51.214510  9949 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:51.214929  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:51.201452  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:51.218001  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:51.219429 10021 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:51.222960  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.026144 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:51.225353 10092 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:25:51.227401 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:51.228394 10092 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.022667 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:51.249377  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48439us
I20260709 12:25:51.258562 15457 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:51.259963 15457 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:51.263254 15420 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:51.263513 15551 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:51.264284 15551 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:51.264940 15551 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:51.265130 15571 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:51.265790 15571 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:51.266260 15571 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:51.266570  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2259) took 1173 ms. Trace:
I20260709 12:25:51.281173 15198 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:51.281121  9940 rpcz_store.cc:276] 0709 12:25:50.093209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.535853 (+442644us) service_pool.cc:224] Handling call
0709 12:25:50.536162 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.835938 (+299776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.265881 (+429943us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b54001a31d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:51.266339 (+   458us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":674694912}
I20260709 12:25:51.267910 15577 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:51.284965 14841 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:51.269696  9946 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:25:51.269698  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2063) took 1575 ms (client timeout 1906 ms). Trace:
W20260709 12:25:51.299281  9948 rpcz_store.cc:269] 0709 12:25:49.694448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.694578 (+   130us) service_pool.cc:224] Handling call
0709 12:25:51.269203 (+1574625us) spinlock_profiling.cc:243] Waited 355 ms on lock 0x7b540042e854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:51.269509 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":815688064}
W20260709 12:25:51.268661  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2072) took 1423 ms (client timeout 1756 ms). Trace:
I20260709 12:25:51.268204 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:51.271857 15410 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:51.273744 15457 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:51.312951  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:51.313447  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:51.313802  9949 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:51.313024 15403 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:51.315179 15403 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:51.315400 15417 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:51.317435 10096 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
W20260709 12:25:51.306324  9941 rpcz_store.cc:269] 0709 12:25:49.845230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.860775 (+ 15545us) service_pool.cc:224] Handling call
0709 12:25:51.268186 (+1407411us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54000168d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:51.268417 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":911583616}
I20260709 12:25:51.308089  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:51.268800 15608 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:25:51.367669 10092 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
W20260709 12:25:51.369418  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.268317 15583 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:51.369138 15461 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.266805 15358 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.372454 15326 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.373383 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:51.292865 14995 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.086s
I20260709 12:25:51.378732 15515 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:51.269479 15477 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 3, Time since last communication: 0.113s
I20260709 12:25:51.382850 15457 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.404214 15477 consensus_queue.cc:1243] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:25:51.404659 15457 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.407433  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:51.408171 15642 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.409066 15642 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:51.411218  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.411550 15430 raft_consensus.cc:2764] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:51.412336 15430 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:51.413154 15430 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.414250 10643 leader_election.cc:400] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:51.414657 10643 leader_election.cc:403] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:25:51.384922 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 117149us
I20260709 12:25:51.415167 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:51.417376 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:51.419100 15411 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:51.419211 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:51.419937 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1136) took 1733 ms. Trace:
I20260709 12:25:51.420496 10025 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 8.
I20260709 12:25:51.421671 15411 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.422546 15389 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:51.425478 15389 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:51.425798 14995 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.426538 14995 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.428467 15621 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:51.429025 15621 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:51.443852  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:51.443619 15461 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.444638  9948 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.450341 14995 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:51.396262 15139 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:51.469226 10095 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:51.420430 10671 rpcz_store.cc:276] 0709 12:25:49.658976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.659512 (+   536us) service_pool.cc:224] Handling call
0709 12:25:49.671801 (+ 12289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:49.824888 (+153087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:49.824917 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:49.825390 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 12:25:49.993210 (+167820us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:49.993245 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:49.993292 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.063198 (+1069906us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:51.279733 (+216535us) raft_consensus.cc:1685] finished
0709 12:25:51.286858 (+  7125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.391948 (+105090us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":175766,"raft.queue_time_us":157655,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":10566,"spinlock_wait_cycles":8878848,"thread_start_us":167514,"threads_started":1,"wal-append.queue_time_us":185808}
W20260709 12:25:51.476254 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 103258us
I20260709 12:25:51.400503 15516 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:51.400800 10093 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:25:51.401626 10090 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:51.403551 15366 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:51.430518 15621 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.317147 15403 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.446393 15461 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:51.478546 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:51.482903 15457 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:51.488859 15430 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:51.498648  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.508364 15403 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:51.510560 10095 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:51.519304 15650 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:51.566479 15461 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:51.573829 15649 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:51.574239  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 268191us
I20260709 12:25:51.574626 15658 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:51.576226 15658 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:51.577996 15658 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:51.517022 10093 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:51.631148  9945 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:25:51.528939 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67832.2us
W20260709 12:25:51.637094 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71087.3us
I20260709 12:25:51.545639  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:51.638108 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 159752us
I20260709 12:25:51.652354 15611 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:51.654932  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:51.657826  9945 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:51.658445 15461 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.659756 15461 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.658915 15430 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.658988 15608 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:51.648518 15551 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.504786 15642 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:51.654276 15611 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:51.673774  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:51.663051 15430 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.664451 15461 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:51.683082  9945 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:25:51.683576 15657 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:51.669553 15650 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:51.673065 15611 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:51.679042 10097 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:51.685640 15457 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:51.689778  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.690241 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:51.692087  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54105.7us
I20260709 12:25:51.687512 10021 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:25:51.694249 10021 pending_rounds.cc:85] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 8
I20260709 12:25:51.696121 15608 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:51.701661 15608 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.701465 15457 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.702277 15608 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:51.678089 14841 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:25:51.704412 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:51.698024 10021 pending_rounds.cc:107] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:25:51.706950 10021 raft_consensus.cc:2889] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:51.686877 15657 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:51.709595 10093 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:51.712324  9945 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:51.713860 10095 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:51.671977 15358 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.687057 15571 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:51.717260  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 468758us
I20260709 12:25:51.704949 15411 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:51.698572 15461 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:25:51.709230 15657 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:51.723027 10099 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
W20260709 12:25:51.723016 15275 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:25:51.696276 15650 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:51.732584  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 187122us
I20260709 12:25:51.739895 15430 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:51.740118 15650 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:51.745232  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:51.739193 15611 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.752269 15366 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:51.755220 15477 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:51.755919 15477 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:51.755163  9948 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:51.759400 15477 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.738261 15457 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:51.763988 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:51.731137 15516 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:25:51.782935  9944 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:51.755677 15551 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:51.790567 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60439.4us
I20260709 12:25:51.791905 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:51.792841 15457 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:51.794567 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:25:51.805850 15634 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.907s	user 0.001s	sys 0.010s
I20260709 12:25:51.807137 15366 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:51.764456 14841 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:25:51.811816 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47939.4us
I20260709 12:25:51.831990 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3246) took 1318 ms. Trace:
I20260709 12:25:51.818655 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:51.832391 10017 rpcz_store.cc:276] 0709 12:25:50.513425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.513552 (+   127us) service_pool.cc:224] Handling call
0709 12:25:50.513860 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:50.514079 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.514102 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:50.519751 (+  5649us) log.cc:844] Serialized 26 byte log entry
0709 12:25:50.673250 (+153499us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:50.673302 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:50.673393 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.804991 (+1131598us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:51.831549 (+ 26558us) raft_consensus.cc:1685] finished
0709 12:25:51.831570 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.831830 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15866240,"thread_start_us":132806,"threads_started":1,"wal-append.queue_time_us":252260}
I20260709 12:25:51.819842 15461 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:51.833630 15544 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:51.834244 15461 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:51.834312 15544 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:51.835639 15461 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.836480 15029 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:51.837061 15029 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.819859 10674 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:51.832005 14841 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:25:51.840703 15029 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:51.815850 15358 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:51.823953  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.842923 15457 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:51.843514 15457 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:51.844529 15457 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:51.845319 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50950.2us
I20260709 12:25:51.846078 14995 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:51.846701 14995 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:51.828504 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3270) took 1134 ms. Trace:
I20260709 12:25:51.830003 15571 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:51.867097 10090 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:25:51.848646 10024 rpcz_store.cc:276] 0709 12:25:50.693503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.693656 (+   153us) service_pool.cc:224] Handling call
0709 12:25:50.694050 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.780427 (+ 86377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.780457 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.780503 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:50.780536 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.827598 (+1047062us) spinlock_profiling.cc:243] Waited 60.6 ms on lock 0x7b540011ba54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:51.827733 (+   135us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.828078 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":139291904}
I20260709 12:25:51.851034 15326 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:51.851600 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:51.851799 15608 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:51.855953  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 99892.6us
W20260709 12:25:51.859674  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64069.6us
I20260709 12:25:51.861624 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2186) took 1073 ms. Trace:
I20260709 12:25:51.862692 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2189) took 1063 ms. Trace:
I20260709 12:25:51.865883 15431 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:25:51.867648 10090 pending_rounds.cc:85] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 10
I20260709 12:25:51.875825 10090 pending_rounds.cc:107] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 11.11
I20260709 12:25:51.876178 10090 raft_consensus.cc:2889] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:25:51.844497  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.872385 10096 rpcz_store.cc:276] 0709 12:25:50.787821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.788621 (+   800us) service_pool.cc:224] Handling call
0709 12:25:50.789059 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:51.321319 (+532260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.861230 (+539911us) spinlock_profiling.cc:243] Waited 504 ms on lock 0x7b5400164c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:51.861518 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1159082240}
I20260709 12:25:51.886530  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:51.888294 15544 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.889776 15544 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:51.892860 15569 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:25:51.896879  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:51.905192  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 120474us
I20260709 12:25:51.848330 14995 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:51.905886 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 228080us
I20260709 12:25:51.915819 15403 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:51.916769 15403 consensus_queue.cc:1243] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:51.917277 15403 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:51.918872 15354 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:51.919319 15354 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:51.920666 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:51.920428 15354 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:51.921470 10677 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:51.920579 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:51.921926 15354 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:51.872813 10098 rpcz_store.cc:276] 0709 12:25:50.799410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.799588 (+   178us) service_pool.cc:224] Handling call
0709 12:25:50.800048 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:51.027575 (+227527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:51.027605 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:51.028070 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:25:51.572868 (+544798us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:51.573284 (+   416us) log.cc:844] Serialized 14 byte log entry
0709 12:25:51.573494 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.573597 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:51.858692 (+285095us) spinlock_profiling.cc:243] Waited 57.5 ms on lock 0x7b540023a620. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cbdbe 00007f4e094d6930 00007f4e094d68d8 00007f4e094d6858 00007f4e094d6813 00007f4e094d5a9c 00007f4e0f3c12f6 00007f4e0f3bbaa8 00007f4e095127cb 00007f4e095152c0
0709 12:25:51.862132 (+  3440us) raft_consensus.cc:1685] finished
0709 12:25:51.862270 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:51.862466 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":172,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":99371,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":8760,"spinlock_wait_cycles":132251008,"thread_start_us":257653,"threads_started":2,"wal-append.queue_time_us":296096}
I20260709 12:25:51.918002 15657 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:51.927479 15589 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:51.918147 15403 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:25:51.931931 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:51.931880 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:51.934232 10675 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:51.934963 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:51.939913 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:51.940845 10025 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:51.941201 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:51.941982 10016 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
W20260709 12:25:51.936443  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56571.4us
I20260709 12:25:51.936583 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:51.945142 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:51.928999 15431 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:51.951977 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:51.937527 15589 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:51.946770 10065 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.906s (SENT) [suppressed 14 similar messages]
I20260709 12:25:51.945919 14995 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.952831 10677 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:51.957000 15409 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:51.957492 15409 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.957855 15409 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:51.948179 15275 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:25:51.970779 10019 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:51.971385 10019 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:51.972188  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:51.972667 10096 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:51.973980  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:51.974483 10668 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:25:51.977391 10024 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:51.969343 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:51.984750 10090 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:51.985754 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:51.981622 15516 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:25:51.989018 15366 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:51.989226 15516 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:51.989691 15516 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:51.990250 10674 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:51.992635 15649 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:51.993249 15649 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:51.993680  9943 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:51.994686 15343 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:51.995450 15649 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:51.995395 15343 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:51.954355 15410 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:51.997787 15343 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:51.972467 10019 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.001989  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2273) took 1185 ms. Trace:
W20260709 12:25:52.018132  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:51.999536 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:52.019249  9944 rpcz_store.cc:276] 0709 12:25:50.816443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.816864 (+   421us) service_pool.cc:224] Handling call
0709 12:25:52.001766 (+1184902us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b540042e354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:52.001936 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003038080}
W20260709 12:25:52.026855  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52968.8us
I20260709 12:25:52.029284 10024 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:52.031591 10019 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:51.977380 15650 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.029803 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:51.999294 15465 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:52.022487 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:51.998392 15343 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:52.042738 10677 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:52.024868 15326 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.044902 10674 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:52.002866 15139 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:52.003093 15411 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.039s
I20260709 12:25:52.042804 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.044862  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:51.998317 15410 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.055898 10670 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:25:52.056783 15589 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.057293 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:52.057957 15589 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.059274 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:52.060263 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:52.069139 10669 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:52.062705 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2182) took 1258 ms. Trace:
I20260709 12:25:52.066821  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:52.071455 15671 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.070796 10091 rpcz_store.cc:276] 0709 12:25:50.749310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.749500 (+   190us) service_pool.cc:224] Handling call
0709 12:25:50.784125 (+ 34625us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:50.962392 (+178267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:50.962437 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:50.962497 (+    60us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:25:50.962546 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:51.960082 (+997536us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b5400305f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:51.967625 (+  7543us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:52.007596 (+ 39971us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":409372032}
I20260709 12:25:52.040037 15465 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:52.076006 15586 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:52.075634 15465 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.076382 10675 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:52.076673 15567 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.077658 15567 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.078056 15567 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.045059 15029 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.079864 15029 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.082252 15139 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.076418 15586 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:52.084614 15139 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.053711 10096 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:52.053774  9941 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:52.023990 15420 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:25:52.025573 15516 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.084950 15586 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.046531 15461 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:52.097966 10668 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:25:52.100318 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:52.108202  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56720.6us
I20260709 12:25:52.047290 15343 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.118942 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.123224 10093 raft_consensus.cc:2368] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:52.112737 15642 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.012s
I20260709 12:25:52.124598 15465 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.096493 15650 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.154541 15650 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.157070 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.166188 10096 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 9: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:52.151016 15586 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.155495 15608 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:52.205068  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 212651us
I20260709 12:25:52.208330 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.119787 15409 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.180617 15657 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.168876 15343 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:52.218858  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.124240 15461 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.230356  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:52.231386 15343 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.232239 15343 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.234965 15657 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:52.235507 15340 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:52.235948 15340 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:52.237082 15340 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.239815 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:52.240530 10096 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.239696 10673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:52.243064 10673 raft_consensus.cc:606] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:52.243435 10673 raft_consensus.cc:614] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:52.243592 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:52.247771  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 123603us
W20260709 12:25:52.248787  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.250445 10671 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:52.152168 15659 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.243s
I20260709 12:25:52.256433 10098 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:52.256711 10090 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.267163 10673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c"
dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59508
I20260709 12:25:52.267627 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.278260 10096 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:52.235548 15657 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:52.280220 10673 raft_consensus.cc:606] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:52.283573 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.285992 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.251235 15650 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:52.250831 15139 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:52.280671 10093 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:52.286685 10098 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:52.256187 15403 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:52.293095 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.286806 15657 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.295130 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:52.301203 10096 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
W20260709 12:25:52.301180 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 93489.8us
I20260709 12:25:52.296595 15516 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:52.312326 15674 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:25:52.313643 15621 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.313771 15672 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:25:52.315495 15410 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.315780 15306 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:52.316340 15306 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.323982 15668 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:52.324720 15668 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.331007 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:52.331650 10098 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:52.332768 15676 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.333051 15567 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.333317 15676 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.334165 10097 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:52.335703 15676 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.339591 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.307953 15343 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.322899 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2188) took 1416 ms. Trace:
I20260709 12:25:52.319497 15650 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:52.309244 15139 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.349148 15679 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.349846 15679 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.352106 15679 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:52.355377  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.350329 15139 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.359274 10093 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 12:25:52.360335 15611 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.360611 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:52.361892 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.362761 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.345047 10094 rpcz_store.cc:276] 0709 12:25:50.794158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:50.794286 (+   128us) service_pool.cc:224] Handling call
0709 12:25:52.182147 (+1387861us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b5400162954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:52.210192 (+ 28045us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416586624}
I20260709 12:25:52.365922 10019 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:52.374835 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.320848 15403 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.381404 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.383061 15668 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.377449 15642 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:25:52.383612 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.394698  9947 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:52.394723 10099 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:52.398128 15029 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:52.388165 15680 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:52.399065 15680 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.401396 15680 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.402020 15409 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.402580 15677 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:52.405231  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.368021 15139 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.369542 15516 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.392273 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:52.371469 15678 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:52.404237 15544 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.504s	user 0.006s	sys 0.006s
I20260709 12:25:52.425321 15411 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:25:52.420578  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:52.432852 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:52.423060 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48978us
I20260709 12:25:52.406131 15642 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.423521 15677 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.442729 15416 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:25:52.443516  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:52.443859 15416 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:52.444451 15416 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:52.444919 15677 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.445529 15677 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:52.445418 15416 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.426569 15411 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:25:52.446066  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.452025 15672 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:25:52.455108  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.455612  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.455435 15672 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.457530  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.448947  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.448633  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.458738  9946 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:52.450125  9947 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:25:52.449355  9943 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:25:52.446461 15540 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:52.461520 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:52.461922 15540 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:52.462366 15540 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:52.451822 15029 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:52.463136 15029 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:52.463492 15403 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.464079 15029 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:52.465677 15354 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.543s	user 0.007s	sys 0.006s
I20260709 12:25:52.465952 15029 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.422457 15516 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.467196  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.459138  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.457772 15672 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.459039  9945 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:52.451327  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:52.450941  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.473839 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.405475 15567 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.451716  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.427455 15409 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.486546  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:52.430493 15410 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:52.488958 10017 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 12:25:52.509426  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.495203 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59547.9us
I20260709 12:25:52.511811 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.512598 10096 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:52.513722 10098 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:52.439076 15551 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.515931  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:52.510907 15544 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:52.529831 15544 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.530216 15544 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:52.477013 15461 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.445997 15677 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:52.534694  9946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:52.535344  9946 raft_consensus.cc:606] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:52.535706  9946 raft_consensus.cc:614] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:52.535442 15677 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.507071 15649 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:25:52.536747 15677 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:25:52.538275 14995 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.003s
I20260709 12:25:52.453100 15457 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 12:25:52.419144 15139 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.540052 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.511113 15354 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 12:25:52.540791 10019 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:52.541241 15139 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.507818 15678 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.544157 15678 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.544843  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.511902 15410 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.546257 15410 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.519507  9948 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:52.522609 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.512900 15642 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.523352 15409 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.511091 15668 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.524466 15672 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.528831  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:52.529409 15671 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.551276  9945 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:25:52.551276 10023 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:25:52.582089 15029 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.551612 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.583431 10016 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 8.
I20260709 12:25:52.565037 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.518118 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.556337 10022 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 7. (term mismatch)
I20260709 12:25:52.582271 10023 pending_rounds.cc:85] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 9
I20260709 12:25:52.570374 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.587566 10019 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:52.587013 10023 pending_rounds.cc:107] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:25:52.552081 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.570874 15671 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.588958 10023 raft_consensus.cc:2889] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.592381 10023 pending_rounds.cc:107] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:25:52.592684 10023 raft_consensus.cc:2889] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.554133 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.620309  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.621181  9948 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 9.
I20260709 12:25:52.595882  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:52.605247 10090 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:52.575368 15465 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:25:52.586351 10022 pending_rounds.cc:85] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 6
I20260709 12:25:52.626449 10022 pending_rounds.cc:107] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 12:25:52.626925 10022 raft_consensus.cc:2889] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.627103 10024 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:52.589426 10020 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:25:52.629521  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:52.546569  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.625555 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:52.632249 10675 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:25:52.632025  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.633024 10090 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: [12.11-14.12]
W20260709 12:25:52.626180 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 205224us
I20260709 12:25:52.634068  9941 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:52.633316  9945 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:52.635322  9946 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:25:52.634780  9945 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:52.641058 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.631153 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:52.643246  9948 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:52.567159 15642 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:52.590231 15516 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.620126 14841 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.631641  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.645179 10096 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:52.581748 15403 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:52.569641 15409 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:52.650892  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55093.5us
I20260709 12:25:52.648690 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:52.584190 15306 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:52.671094 15671 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.636231 15688 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:25:52.649609 15679 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:52.628477 15608 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.675631 15608 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.677980 15608 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.678211 14841 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.560055 15551 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.680910 15551 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.575433 15430 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:25:52.682328 15430 consensus_queue.cc:1243] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 13.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:52.682580 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.546897 15544 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.665472  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:52.667585 10090 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:25:52.621827 15029 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.700615 10016 pending_rounds.cc:85] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 3
I20260709 12:25:52.664814  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2086) took 1296 ms. Trace:
I20260709 12:25:52.668108 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:52.570029 15672 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.692448 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:52.704028 10671 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:52.692473 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:52.704550 10019 raft_consensus.cc:2368] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14.
I20260709 12:25:52.706130  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.693081 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.569947 15668 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.576447 15540 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.707799 10023 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:25:52.656996 15403 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:52.701105 10016 pending_rounds.cc:107] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:25:52.682742 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:52.701193  9949 rpcz_store.cc:276] 0709 12:25:51.368042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.625573 (+1257531us) service_pool.cc:224] Handling call
0709 12:25:52.664725 (+ 39152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:25:52.703168 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:52.713351 10096 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:52.712707 10676 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:25:52.713732  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.657070 15139 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:52.710764  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:52.712430 10016 raft_consensus.cc:2889] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:52.681188 14841 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.719746  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.720709  9941 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:52.722049 10022 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:52.722672 15679 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.723186 15679 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.664866  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:52.725224  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60424.5us
I20260709 12:25:52.726800 15687 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:52.727017 15416 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:52.727538 15416 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:52.727950 15416 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:52.728581  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.729214  9941 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:25:52.727950 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66243.2us
I20260709 12:25:52.731492 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.732087 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:52.733817 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:52.740621 15340 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.727538 15687 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.744534 15540 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.745354  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:52.746802 15668 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.749962 15029 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.752483 15544 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.753196  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:52.753821  9940 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:25:52.756479 15684 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
W20260709 12:25:52.759522 15675 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.466s	user 0.000s	sys 0.003s
I20260709 12:25:52.727964  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.740636 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:52.740041 15650 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:52.716049 15403 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:25:52.774920  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:52.771731 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.772181 15411 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.774660 14841 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.779134  9945 raft_consensus.cc:2376] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17. Re-sending same reply.
I20260709 12:25:52.739326 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.790259 10022 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:52.781989  9941 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
W20260709 12:25:52.790819 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57100.5us
I20260709 12:25:52.732322 10099 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:52.702039 15551 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:52.784646  9940 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:25:52.791628 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:52.792652 14841 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:52.792858 15430 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:52.793713 15677 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:25:52.794099 15677 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:52.794981 14841 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.795157 15677 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.796082  9948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:25:52.796694 15677 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:52.798285 15306 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:52.798868 15657 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.799595  9944 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:52.796666  9948 raft_consensus.cc:606] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:52.800627  9948 raft_consensus.cc:614] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:25:52.801295 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:52.802376 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.804159 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.803079 10090 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:52.805811 10096 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:52.804800 10023 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 8.
I20260709 12:25:52.805087  9949 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:52.809273 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.812734  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.813361  9945 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:52.819119  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:52.831849 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:52.839428  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.840076  9944 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:25:52.841174 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:52.803575 15551 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:52.843030 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:52.820294 15621 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.506s	user 0.005s	sys 0.006s
I20260709 12:25:52.828315 15306 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:52.852526 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.856707 10093 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:52.856889 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:52.857826  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.858121 10092 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:52.853518 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.854113 10025 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:52.804527 15668 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.040s
I20260709 12:25:52.833576 15544 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:52.860740  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.847371 15642 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:52.843719 15457 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:52.870931 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.861006 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.885484 10095 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:52.846669 15679 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.820785 15139 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.812294 15411 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.851527 15687 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:52.859301 15275 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.887265 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.894183 10091 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 12:25:52.901144 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 97222us
W20260709 12:25:52.900388  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 122616us
I20260709 12:25:52.902221  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.903981 10017 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:52.904701  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:52.905339 10018 raft_consensus.cc:2368] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:25:52.881094 15642 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:52.918315 15642 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:52.917517  9944 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:52.878729 15676 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:52.888847 15544 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:52.927604 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:52.944746  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:52.923049  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:52.957695 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.958441 10021 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:25:52.958901  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:52.959415  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:52.961069  9941 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:52.931165 15679 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:52.969012  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:52.937134 15657 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.943513 15672 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:52.974753 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:52.901584 15457 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.970328 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:52.974118  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:52.914222 15275 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:52.978076 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:52.989075 10099 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:52.991259  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.971128 15621 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:52.995208  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.033104  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.034693  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.035522  9940 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:53.035461 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:53.036157 10025 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.036969 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.037711 10018 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:53.039810 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.040635 10023 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 12:25:53.041394 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.997396  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:52.979100 15687 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:25:52.999089 15700 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:25:52.999361 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.999744 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:53.045604 10021 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:52.987682 15410 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.046341  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.047212  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:53.004002 15551 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.011967  9941 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:52.982779 15586 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:53.053000  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.055106 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:53.053659  9943 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:52.987223 15689 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.065203 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:53.018867 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 103730us
I20260709 12:25:53.022480 15465 raft_consensus.cc:993] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:25:53.025065  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.064812  9943 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:53.066636  9947 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:25:53.030026 15411 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:52.972831 14841 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:53.018921 10094 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.043799 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:52.947064 15676 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.048317  9942 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:53.051822  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.012864  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.047972 15679 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:53.064620  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:53.069448  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56665.1us
I20260709 12:25:53.071686 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:53.064738 10090 raft_consensus.cc:2376] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16. Re-sending same reply.
I20260709 12:25:53.072340 15679 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.072750 15679 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.073146 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.073606 15688 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:53.073796 10023 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:53.075336 10023 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:53.068101 10018 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:53.080634 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:53.081287 10019 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:25:53.071796 15551 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.083875 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:53.093555 10097 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
W20260709 12:25:53.096589  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44851us
I20260709 12:25:53.100234 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:53.102042 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:53.059981 15420 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:53.117293 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.116045 15586 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:25:53.132493 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47976.1us
I20260709 12:25:53.143342 15340 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.144290 15340 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.146742 15586 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:53.094735 15430 raft_consensus.cc:1081] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:25:53.152230  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:53.120661 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:53.152009 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51853.4us
I20260709 12:25:53.161345 10025 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:53.163384 10018 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:53.164242  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.098886 15567 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.115912 15029 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:53.117925 15420 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.098018 15672 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:53.184639 10642 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:53.164958  9945 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:53.153399 15411 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:25:53.194521  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.198203  9948 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:53.156467 15461 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:53.159221 15516 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:53.119134 15687 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:25:53.123015 15416 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.205582 15370 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:53.198910 14841 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:53.206085  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:25:53.207834 15694 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.215s	user 0.001s	sys 0.000s
I20260709 12:25:53.178342 15029 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:53.212949  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:53.213498  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78690.3us
I20260709 12:25:53.217075 10019 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:53.193111 15420 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.092535 15689 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.103564 15544 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:53.244081 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 77014us
I20260709 12:25:53.211205 15461 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:25:53.235692 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51182.3us
W20260709 12:25:53.234884 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 202779us
I20260709 12:25:53.264261 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.213908 15340 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.211722 15516 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:53.183923 15567 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.232498 15370 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.234511 15410 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.241989 15029 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.274103 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:53.243337 15540 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:53.266505 15461 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.268545 15678 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:25:53.283150 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:53.288231 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.293569 10019 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:53.269770 15139 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.269837 14841 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:53.282560 15370 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:53.264665 15409 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:25:53.289188 15540 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.306226 15540 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.272378 15354 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:25:53.291595 15461 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:53.310076  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.314581  9941 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:25:53.286091 15586 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.300951 15410 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:25:53.315991 15457 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.314370 15685 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:25:53.334197 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 213634us
W20260709 12:25:53.334864 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71549us
I20260709 12:25:53.343127  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.343820  9941 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:53.301986 15657 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:25:53.348460 15657 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:53.348915 15657 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:53.349237 15716 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:25:53.350160 15711 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:25:53.350886 15676 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:53.357452 15650 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.000s
I20260709 12:25:53.357960 15650 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:53.359163 15689 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:53.359423 15567 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:53.367046 14841 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.347481 15416 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.373416 10094 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
W20260709 12:25:53.380182 15677 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.583s	user 0.006s	sys 0.006s
I20260709 12:25:53.386047 15275 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:53.386943 15712 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.181s	user 0.000s	sys 0.004s
I20260709 12:25:53.387693 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:53.338627 15621 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:25:53.389192 15516 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.437826 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:53.438874 10098 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:25:53.391558  9941 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:25:53.399775 15642 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.349889 15657 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.340265 15306 raft_consensus.cc:476] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Not starting pre-election -- already a leader
I20260709 12:25:53.401233 15679 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.406117 10090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
W20260709 12:25:53.409801 15403 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.598s	user 0.009s	sys 0.000s
I20260709 12:25:53.425513 15688 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:53.426271  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:25:53.341995 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.427786 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:53.441458  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.438452 10091 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:53.481961 10022 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:25:53.483561 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55871.4us
W20260709 12:25:53.485589 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 240323us
I20260709 12:25:53.486402 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:53.489737  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48391.6us
I20260709 12:25:53.501561 15554 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:53.502064 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.503223 15725 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:53.503473 10017 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:53.503769 15725 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:53.504878 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:53.505023 15725 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.506724 15725 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.506863 15737 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:53.507393 15737 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.507738 15737 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.510468 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:53.511142 10677 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.512005  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.513396  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.514006  9945 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:53.517560 15689 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.518885 10677 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:53.505764 15685 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:53.519572 15685 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.519979 15685 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.521835 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:53.518097 15689 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.527686 15689 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.431928 15409 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.432505 15611 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:53.533603 10093 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:53.535729  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.541278 15420 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.441315 15198 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.443372 15678 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:25:53.451082 15410 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.549872 10090 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
W20260709 12:25:53.552412 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 210494us
I20260709 12:25:53.556221 10090 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:53.556612 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 248950us
I20260709 12:25:53.558775 10021 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:53.532992 15676 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.544517 15687 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:53.570086 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:53.449486 15461 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:53.552412  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 233617us
I20260709 12:25:53.557725 15275 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:53.482517 15029 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.531178 15403 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.560906 15730 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:53.572060 15657 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.563887 15416 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:53.586488 15730 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.614992 15730 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:53.616995 15340 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:53.617456 15340 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:53.595007 15717 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:25:53.598764 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:53.620713 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:53.621995 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:25:53.599743  9842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 322) took 21599 ms (client timeout 15000 ms). Trace:
I20260709 12:25:53.600502 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:53.602236 15737 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.588348 15687 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:53.624698 15737 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:53.624146 15687 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.625375 15649 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:53.626144 15649 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:53.629062  9840 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 4 to 11, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 11 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:25:53.630788 15551 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.531s	user 0.011s	sys 0.000s
I20260709 12:25:53.569754 15410 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.631163 15717 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:53.557835 15409 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.632810 15420 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.633860 15409 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:53.638691 10065 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:53.639596  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:53.640305 15642 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:53.640538 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:53.640818 15649 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:53.641211 10024 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:53.598210 15685 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.604240 15461 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.642383 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.604633  9912 leader_election.cc:400] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14.
I20260709 12:25:53.654043  9912 leader_election.cc:403] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
W20260709 12:25:53.654583  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50170.4us
I20260709 12:25:53.655005 10668 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:53.572232 15676 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.607446  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:53.556947 15198 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:53.643405 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45961.5us
W20260709 12:25:53.651747 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51338.8us
I20260709 12:25:53.658991 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:53.662232 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.663344 10021 raft_consensus.cc:2410] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:53.663964 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:53.664705 10022 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 13: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:53.664842 10098 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:25:53.660063 10090 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.602703 15679 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.640486 15717 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:25:53.705711  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66200.5us
I20260709 12:25:53.661914 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:53.708763 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:53.713359  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:53.710237  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.570391 15718 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 12:25:53.715058 15709 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:53.715422 15709 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:53.716213 15709 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.717428 15709 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.710115 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.719238  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.719889 10023 raft_consensus.cc:2368] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14.
I20260709 12:25:53.719856  9940 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:25:53.713608 10675 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:53.709275  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.722947 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:53.724141  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.647029 15275 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:53.725064 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:53.725802 10673 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:53.720745  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.723012 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:53.726542  9946 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.660667 15409 raft_consensus.cc:2764] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:25:53.655049 15457 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:53.728307  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.729072 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.740202 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.729893 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.723995 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.729127  9943 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
W20260709 12:25:53.733287 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74352.4us
I20260709 12:25:53.661062 15411 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:53.740108  9945 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:53.703526 15430 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14.
I20260709 12:25:53.746191 15577 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.746377 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:53.747025 10672 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:53.746752 15577 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.747989  9840 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 2 to 10. New cstate: current_term: 10 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:53.749640 15577 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.615013 15416 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.753317 15370 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.745379 15457 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.754992 15457 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:53.755774 15370 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.757162 15029 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.758369 10017 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:53.759706 15540 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.761754 15540 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.765479 10025 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:53.767490 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:53.774536 10677 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:25:53.776683 10024 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:25:53.739611 10643 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.720881 15672 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.720726 15420 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:53.618461 15340 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:53.783157  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 73978.7us
I20260709 12:25:53.784030 10670 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:53.733968 15679 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.738163 15725 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:53.810954 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:53.789799  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.788040  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.811919  9945 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.796456 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:53.812557 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 369293us
W20260709 12:25:53.802932 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63392.4us
I20260709 12:25:53.803689 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:53.812592  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.781373 15611 raft_consensus.cc:2764] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:25:53.818969 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:53.820739 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:53.820888  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:53.822822 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:53.823408 10670 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.811862 10091 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:53.829142  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.830066 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:53.830411 10672 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:53.830819  9912 leader_election.cc:400] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:25:53.831166  9912 leader_election.cc:403] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:53.831059 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:53.832858 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:53.833231 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.827967 15551 raft_consensus.cc:2764] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:25:53.839074 15430 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:53.839395 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:53.839761 15198 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.840175 10672 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:53.840217 15430 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:53.840790 15430 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:53.841730 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:53.841581 15430 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.842326 10677 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:53.811156 15420 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:53.842866 15431 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:53.843307 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:53.843961  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:53.843611 15420 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.844517 15442 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:25:53.845222 15442 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:53.845468  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.847087 15577 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:53.847445 15725 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:53.847707 15577 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:53.848189 15577 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.848546 15725 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.852847 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:53.853997  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:53.828336 15139 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:53.854651 15718 raft_consensus.cc:2764] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:25:53.828543  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.856424  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.857013  9942 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:53.845569 15442 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:53.857774 15442 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:53.858662 15708 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:53.859288 15718 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:53.859244 15708 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.861593 15708 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.861896 15718 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:25:53.781091 15730 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.863879 15650 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:25:53.864269 15370 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.865927 15730 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.865851 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:53.821080 15029 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:53.867010 15029 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.867348 15029 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:53.867223 15730 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:53.867899 15430 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:53.868530 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:53.814433 15409 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:25:53.871383 15730 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:53.871764 15730 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:53.872134 15730 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.874007 15659 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:53.874579 15659 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.874178  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:53.875747 15431 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:53.876168 15431 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:53.876572 15431 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:53.878774 15354 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:53.879438 15354 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.882020 15354 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:53.882899 15737 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:53.883701 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.784603 15672 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.893877 15730 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:53.907622 15730 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.894974 15275 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:53.897853  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.899401 15676 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.897133 15416 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:53.898216  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.900200 10062 leader_election.cc:400] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14.
I20260709 12:25:53.913403  9940 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:53.914757 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:53.918460 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:53.920776 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:53.921927 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:53.923499  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.924872 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:53.925719 10670 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:53.925783 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:53.909590 10090 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:53.891554 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55781.6us
I20260709 12:25:53.927596  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:53.928228 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:53.935537 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:53.622828  9842 rpcz_store.cc:269] 0709 12:25:32.000544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:32.001230 (+   686us) service_pool.cc:224] Handling call
0709 12:25:53.599500 (+21598270us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:50.159252 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:50.169269 (+ 10017us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:50.169323 (+    54us) write_op.cc:435] Acquired schema lock
0709 12:25:50.169341 (+    18us) tablet.cc:662] Decoding operations
0709 12:25:50.171545 (+  2204us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:50.171636 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:50.171669 (+    33us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:25:50.173312 (+  1643us) tablet.cc:701] Row locks acquired
0709 12:25:50.173334 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:25:50.173507 (+   173us) write_op.cc:270] Start()
0709 12:25:50.173631 (+   124us) write_op.cc:276] Timestamp: P: 1783599950173450 usec, L: 0
0709 12:25:50.173653 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:25:50.174435 (+   782us) log.cc:844] Serialized 8305 byte log entry
0709 12:25:50.502774 (+328339us) op_driver.cc:464] REPLICATION: finished
0709 12:25:50.739037 (+236263us) write_op.cc:301] APPLY: starting
0709 12:25:50.747521 (+  8484us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:51.022967 (+275446us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:51.023113 (+   146us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:51.129386 (+106273us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:51.129894 (+   508us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 12:25:51.129969 (+    75us) write_op.cc:312] APPLY: finished
0709 12:25:51.130644 (+   675us) log.cc:844] Serialized 188 byte log entry
0709 12:25:51.130930 (+   286us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:51.131203 (+   273us) write_op.cc:454] Released schema lock
0709 12:25:51.131286 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:51.131873 (+   587us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:51.131920 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4541841,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":223903,"apply.run_cpu_time_us":33493,"apply.run_wall_time_us":402936,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91949,"num_ops":21,"prepare.queue_time_us":84219,"prepare.run_cpu_time_us":23562,"prepare.run_wall_time_us":82889,"raft.queue_time_us":183048,"raft.run_cpu_time_us":43062,"raft.run_wall_time_us":182323,"replication_time_us":328946,"thread_start_us":350890,"threads_started":4,"wal-append.queue_time_us":53181,"wal-append.run_cpu_time_us":46412,"wal-append.run_wall_time_us":2056479}]]}
I20260709 12:25:53.933271  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:53.941172  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:53.949025  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:53.795018 15403 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:54.014991  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:54.022251  9947 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:53.847025 15723 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:53.875177 15659 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:53.932650 15730 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:25:54.031831  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 82925.8us
I20260709 12:25:53.932467 15677 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.022702  9947 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:53.821512 15678 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.204s
I20260709 12:25:53.947217 15416 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:25:53.955988 15737 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:54.047394  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:53.949767 15642 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:53.946555 15718 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:54.053022 15718 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.053505 15718 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:53.820389 15679 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.026186 15567 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:25:54.064143  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:54.059087 15731 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:25:54.029066 15723 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:54.028277 15540 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:54.091993  9945 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:54.097266 15745 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.052449 15677 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.060976 15642 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.121920 15742 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:54.113678 15708 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:54.151701 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 204254us
I20260709 12:25:54.141479  9946 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:54.051357  9947 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:54.155777 15747 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 12:25:54.142503 15723 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.142575 15540 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:54.308800  9945 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:54.156072 15642 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:54.326226 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 231347us
I20260709 12:25:54.315696  9947 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.135412 15416 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.309074 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:25:54.339799 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:54.340783 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 412618us
W20260709 12:25:54.317819 15657 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.707s	user 0.012s	sys 0.006s
W20260709 12:25:54.345240  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.308449 15366 raft_consensus.cc:2764] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:54.328198 15540 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:54.339532 15457 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.579s	user 0.005s	sys 0.011s
I20260709 12:25:54.337503 15409 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.232085 15577 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.395027 10016 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:54.395875  9940 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:54.407778 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3297) took 1147 ms. Trace:
I20260709 12:25:54.408313 10025 rpcz_store.cc:276] 0709 12:25:53.231341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.233175 (+  1834us) service_pool.cc:224] Handling call
0709 12:25:54.378473 (+1145298us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54002666d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:54.378719 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150630144}
I20260709 12:25:54.419884  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:54.422188 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.428815 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.429490 10673 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:54.429044 10675 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:54.432673 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:54.436157 10022 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.335577 15708 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.347986 15416 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:54.456404  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.363559 15457 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.436950 15730 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:25:54.426270 15737 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:54.438357 15420 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:54.457648 15732 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:25:54.431408 15758 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:54.464898 15736 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.105s	user 0.001s	sys 0.001s
I20260709 12:25:54.506945 10094 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
W20260709 12:25:54.507893 15753 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:25:54.496589 15403 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:54.506791 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.513047 15737 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.508564 15403 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.514925 15586 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.515486 15403 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:54.513679 15751 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:25:54.520661  9942 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.655s	user 0.014s	sys 0.000s
I20260709 12:25:54.521396  9942 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:54.523319  9946 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 12:25:54.523388 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 183684us
I20260709 12:25:54.497857 15679 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:54.519526 15431 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.005s
I20260709 12:25:54.508726 15735 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.542120 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.516691 15758 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:54.526738 15442 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.668s	user 0.004s	sys 0.007s
W20260709 12:25:54.542102 15430 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.674s	user 0.008s	sys 0.003s
I20260709 12:25:54.557600 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:54.559433 10019 raft_consensus.cc:2368] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
I20260709 12:25:54.560199 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:54.564942  9947 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:25:54.571419 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:54.573679 10019 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 9. Current term is 10. Ops: []
I20260709 12:25:54.571419 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:54.575448 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.576160 10675 raft_consensus.cc:2368] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:25:54.500140 15420 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.548942 15677 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.515498 15586 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:54.595323  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 175503us
I20260709 12:25:54.597064  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:54.598026  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:54.598878  9945 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:54.550209 15679 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:54.588505 15718 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.534s	user 0.013s	sys 0.000s
I20260709 12:25:54.587816  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:54.590457  9944 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:54.607070  9947 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:54.556964 15657 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.544709 15431 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:54.583528  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 219683us
I20260709 12:25:54.605710  9942 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:25:54.612182 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.578539 15430 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:54.547005 15709 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:54.615216  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:54.612828 10675 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 9.
I20260709 12:25:54.616806 10021 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.622120 10095 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:25:54.555549 15442 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.546185 15735 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.633838  9991 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:54.639837 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.642725 10098 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:54.640975 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.644202  9949 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:54.646766 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.616945 15769 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:54.652909 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.653378 15769 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.663965  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:54.670878 15762 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:25:54.644205 15772 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.673015 15611 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:54.675130 15420 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.674533 15611 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:54.675750 15420 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:54.676115 15420 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:54.676128 15611 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:54.676692 15420 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:54.677299 15611 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:54.680852 15672 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.782s	user 0.010s	sys 0.000s
I20260709 12:25:54.681020 15735 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:54.685185 15442 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:54.685825 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.686921 10672 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:54.687592 10677 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:25:54.636063 15709 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.689034 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.690057 10669 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:54.690137 15709 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.691260 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.692260 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:54.692721 15776 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.693010 15730 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.640924 15677 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.672842 15772 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.707667 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.704689 10673 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:25:54.644367 15751 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:54.623411 15430 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:54.665539 15370 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.801s	user 0.011s	sys 0.000s
I20260709 12:25:54.717645 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:54.718668  9912 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.000s	sys 0.012s
W20260709 12:25:54.719242  9912 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.000s	sys 0.012s
W20260709 12:25:54.719540  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 653535us
I20260709 12:25:54.715396 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.611732 15718 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.709903 15773 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.666774 15577 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:54.698809 15029 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.831s	user 0.011s	sys 0.000s
I20260709 12:25:54.719676 10673 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:54.623709 15657 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.610879 15679 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.681576 15672 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.718415 15709 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:54.722860 15740 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.210s	user 0.001s	sys 0.001s
I20260709 12:25:54.700311 15416 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.703495 10021 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
W20260709 12:25:54.740799  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:54.765071 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:54.746474  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 112718us
I20260709 12:25:54.740478 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.750275 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:54.766681 10669 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:54.749505 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.745100 10670 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:54.751132 10097 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:54.761725 10022 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:54.768712 10023 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:54.732456 15370 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:54.771009  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2108) took 1072 ms. Trace:
I20260709 12:25:54.771605  9941 rpcz_store.cc:276] 0709 12:25:53.682064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.682892 (+   828us) service_pool.cc:224] Handling call
0709 12:25:53.683201 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:53.704554 (+ 21353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:53.704583 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:53.720547 (+ 15964us) log.cc:844] Serialized 26 byte log entry
0709 12:25:54.050800 (+330253us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:25:54.050844 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:54.050916 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:54.754674 (+703758us) raft_consensus.cc:1685] finished
0709 12:25:54.754697 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:54.754860 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27322,"raft.queue_time_us":25126,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":14091,"thread_start_us":287922,"threads_started":1,"wal-append.queue_time_us":875563}
I20260709 12:25:54.735209 15621 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.774449  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:54.774899 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.762207 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:54.739578 15403 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:54.764053 15717 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.042s	user 0.012s	sys 0.000s
I20260709 12:25:54.732548 15735 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.770150 15775 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.739421 15677 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.780432 10020 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:54.716156 15730 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:54.775533  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 118751us
I20260709 12:25:54.770431 15430 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.747030 15029 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.788556 15370 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:25:54.785681  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 133807us
I20260709 12:25:54.634049 15758 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.801222 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.795377 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:54.801887 10098 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:54.780758 15577 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:54.789716 15737 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:54.790198 14841 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:54.831270 15430 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:54.831920 14841 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:54.832026 15430 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.832983 14841 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.832048 15411 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:54.835687 15411 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:54.836515 15411 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.837698 15411 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:54.837904 15769 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:54.795203 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.838990 15789 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:54.839385 15685 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:54.806164 15770 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:54.806675 15765 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.161s
I20260709 12:25:54.842988 15770 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:54.839612 15789 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.843711 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.844563 10676 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:54.843976 15770 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.845489 15431 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.840778 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:54.842339 15551 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:54.847352 15551 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.847810 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:54.848062 15772 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.795800 15677 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.849336 15757 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:54.849720 15340 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:54.849869 15757 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:54.850865 15757 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.851223 15642 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.739173 15773 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.808266 15029 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:54.810861 15410 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:54.784536 15621 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.794019 15708 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.803539 15775 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.791908 15717 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:25:54.819162 15764 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.292s	user 0.003s	sys 0.000s
I20260709 12:25:54.883785  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:54.723954 15751 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.819221 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:54.907428 15657 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:54.907737 10090 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:54.908797 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.909425 10090 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:25:54.824159 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:54.912497 15540 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.545s	user 0.010s	sys 0.000s
I20260709 12:25:54.912987 15540 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.792791 15735 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.831153 15787 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:54.839502 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.853376 10675 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:54.910442 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.853902 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 10.
I20260709 12:25:54.837715 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:54.921895 10098 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:54.839666 15659 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.780s	user 0.008s	sys 0.003s
I20260709 12:25:54.813376 15730 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:54.850577 15677 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:54.763226 15709 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.884979 15785 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:54.893287 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.855453 15577 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.873617 15737 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:54.899250 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.900353 10093 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:54.953174 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:54.901546 15762 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:54.944779 15659 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.961439 15737 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:54.963303 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:54.724557 15776 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:54.918982  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:54.972486 10093 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:54.923134 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:54.926167 10095 raft_consensus.cc:2368] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:54.972556  9948 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 10: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:54.942824  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:54.900586 15139 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:54.899587 15586 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.894999 15416 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
W20260709 12:25:54.955538 15783 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:25:54.944100 15751 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.946801 15730 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:54.965371 10672 raft_consensus.cc:2368] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:25:54.965912  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:54.924682 15773 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.901738 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:54.995929 10677 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:25:54.974211 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 125264us
I20260709 12:25:54.998432 15540 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:54.984943 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:54.988601 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:25:54.991513  9940 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:25:55.001461  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.001618 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.002214  9940 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:55.002354 10090 raft_consensus.cc:2410] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:25:54.950462 15718 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:54.984257  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:54.973596 15723 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.625s	user 0.010s	sys 0.001s
I20260709 12:25:54.953922 15735 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:54.982252 10022 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:55.020016 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:54.972398 15430 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.019614  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.024482  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:55.024848 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:55.002945 15718 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.026197  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:55.026669  9945 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:54.994969 15762 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:55.017378 15366 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:25:55.018383 15773 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:55.018193 15611 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:55.038208 15430 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:55.044885  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:55.045156  9940 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:25:55.045745  9945 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.007946 15751 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.037950  9912 leader_election.cc:400] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:25:55.040397 10022 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:25:55.049278 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:55.049943 15782 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:25:55.052142 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:55.055177 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:55.056443 10022 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:55.052798 10017 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:55.058753 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:54.989739 15139 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.003891 15730 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.067000 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:54.947261 15787 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.067793 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:55.069386 10025 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:55.071412 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:55.072479  9945 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:55.076592 15457 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:55.042135 15717 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:55.020817 15723 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.087527  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:55.051373 15751 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.065955 15642 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.084079 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:55.033046 15718 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.077767 10670 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:25:55.100857 15775 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.101071 15761 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:55.101316 15730 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:55.101467 15761 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:55.088512  9945 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:55.104507 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.105515 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:55.106225 10025 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:55.095285 15717 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.109094 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:55.063222 15420 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.110033 10674 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:55.054116 15773 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.125609  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.126852  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.113186 10091 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:55.057953 15611 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.115115 10017 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.102442 15791 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.135558 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:55.136270 10025 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:55.121605  9945 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:25:55.124100 15776 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.105139 15621 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.129398 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1034) took 1608 ms. Trace:
I20260709 12:25:55.141206 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.139822 10668 rpcz_store.cc:276] 0709 12:25:53.521441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.524706 (+  3265us) service_pool.cc:224] Handling call
0709 12:25:53.525036 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:53.655459 (+130423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.129161 (+1473702us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540050a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:55.129336 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298477312}
I20260709 12:25:55.142235 10022 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:55.141880 10091 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 12.
I20260709 12:25:55.130630 15679 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.145125 10016 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:55.151558 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.152230 10091 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:25:55.144490 15709 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:55.158454 15796 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:55.158470 15798 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:55.161005 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:55.161839 10096 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3478) took 1241 ms (client timeout 1621 ms). Trace:
W20260709 12:25:55.162175 10096 rpcz_store.cc:269] 0709 12:25:53.920495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.921415 (+   920us) service_pool.cc:224] Handling call
0709 12:25:55.160852 (+1239437us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:55.161778 (+   926us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27395328}
I20260709 12:25:55.167554 15139 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:55.167884 15420 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:55.168780 15790 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:25:55.169176 15611 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.171406 15787 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.171872 15717 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:25:55.172979 15793 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:25:55.173806 15723 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.174227 15796 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:55.174669 15718 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.147054 15791 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.175346 15796 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.177603 15791 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.178071 15798 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:55.178457 10025 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:25:55.179486 10024 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:55.187633  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:55.189374 15756 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:55.150259 15366 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:25:55.159926 15799 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 12:25:55.191996 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.179126 15798 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.195001 10091 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:55.196060 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:55.196630 10091 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.191761  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:55.197898 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3477) took 1277 ms. Trace:
W20260709 12:25:55.199995 15803 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:25:55.198278 10092 rpcz_store.cc:276] 0709 12:25:53.920203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.921347 (+  1144us) service_pool.cc:224] Handling call
0709 12:25:55.197171 (+1275824us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:25:55.197381 (+   210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.197573 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.197602 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:55.197645 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:55.197673 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.197726 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.197846 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41949056}
I20260709 12:25:55.202240 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.203639 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.204487 10092 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:55.206772 15751 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.208630 15370 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:55.209105 15370 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:55.214049 10017 raft_consensus.cc:2410] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:25:55.216145 15773 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.217206 15340 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:55.217743 15340 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.218170 15340 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:55.179914 15765 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.220495 15765 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.222548 15544 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.223264 15544 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.224880 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:55.225541 10023 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:25:55.225889 15544 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.215338 15723 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.228006 15723 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.210001 15370 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.233736 10091 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:55.236967 15370 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:55.243037 10091 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:25:55.249603 15723 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.250618 10091 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:25:55.253650 10091 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:55.258102 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:55.259590 10016 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:55.241416 15430 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:55.266014 15306 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:55.278082 15761 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.279327 15761 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.279773 15340 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.282774 15430 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.286267  9990 leader_election.cc:400] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:55.286541  9990 leader_election.cc:403] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.287092  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.288264  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.288379  9945 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 13 index: 12. (term mismatch)
I20260709 12:25:55.288838  9945 pending_rounds.cc:85] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 11
I20260709 12:25:55.288879  9943 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:25:55.289230  9945 pending_rounds.cc:107] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:25:55.289644  9945 raft_consensus.cc:2889] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:55.290385  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:55.291078  9947 raft_consensus.cc:2368] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:55.294899  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:55.295531  9941 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.291687  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:55.298511 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:55.299921 15742 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:55.300340 15742 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.300678 15742 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:55.300876 14841 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:55.301410 15139 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:55.302786 15751 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:55.303637 15751 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:55.303951 15751 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:55.304492 15752 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.815s	user 0.002s	sys 0.001s
I20260709 12:25:55.304801 15751 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.305953 15751 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.286486 15340 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.296983 15370 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:55.302739 15544 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:25:55.326812 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.327289  9941 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:55.333017 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.333060 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.342800 10669 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 12:25:55.342993 10670 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:55.334636 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.343899 10673 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:25:55.346171 10668 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:25:55.349119 10643 leader_election.cc:400] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:55.349504 10643 leader_election.cc:403] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.354110 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3479) took 1433 ms. Trace:
I20260709 12:25:55.355324 10099 rpcz_store.cc:276] 0709 12:25:53.920751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.921482 (+   731us) service_pool.cc:224] Handling call
0709 12:25:55.311544 (+1390062us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:55.354019 (+ 42475us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25637248}
I20260709 12:25:55.267323 15366 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.357326 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.358400 10673 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:25:55.357149 10643 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:55.366504 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:55.375521  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.376724 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.377472 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:55.378228 10673 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.359369 15723 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.380580 10676 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:25:55.380601 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.382755 10668 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:55.376415 15411 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.008s
I20260709 12:25:55.327760 14841 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:55.388317 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.388485 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.392292 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.378496 15737 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:55.393519 15737 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:25:55.396572 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.396624  9945 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:55.400420 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.403348 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.378655 15742 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.416896 10673 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:55.401203 15411 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.014s
I20260709 12:25:55.417641 14841 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.418213 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.418783 10671 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:55.421404 10670 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:55.422017 15742 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:55.422736  9987 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:55.424491 10016 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:55.425457 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.426064 10674 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:55.425483  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:55.427896  9947 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:55.422945 10675 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:55.429875  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:55.432034 15411 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:55.432399 15786 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.432116 15756 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.432747 15411 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.432899 15786 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.433459 15756 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.434723 15791 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:55.435518 15756 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.435266 15791 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.435951 15544 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:55.437464 15791 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.437915 15787 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:55.438347 15787 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.442797 15787 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:55.446902 15306 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:55.447479 15306 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:55.437274 15544 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:55.463573  9987 leader_election.cc:400] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:55.465962  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:55.468114  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:55.467974 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:55.474100 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:55.474723 10016 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:55.461053 15723 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:55.499019  9947 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:25:55.487015 15411 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.470898 15340 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:25:55.478335 15370 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:55.492120 15306 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.473398 15275 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:25:55.473251 15791 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:55.500362 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:55.399109 15737 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.507354 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:55.513373 15577 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.514983 15786 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.515064 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.522428 10669 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:55.522770 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.533735 10091 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:55.523393 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.530213 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:55.519440 15370 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.532444 10016 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:25:55.543030 15765 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.540118 15725 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.007s
I20260709 12:25:55.540238 15689 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:55.549093 15689 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.549464 15689 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.524127 15745 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:55.529837 15685 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:55.525971 15791 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.560063  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:55.540525 15370 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:55.567263 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.526976 15411 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:25:55.570957 15650 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:55.578658 15650 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:55.579659 15650 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.580773 15650 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:55.569981 15685 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:55.584641 15685 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.565056 15791 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:55.592432 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.558869 15139 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:25:55.571478 15765 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:55.598681 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:55.599429 10091 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:55.594239 10099 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:55.602344 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.603156 10099 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:55.565115 15745 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.564015 15787 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:55.689198  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.620069 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.597740 15765 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:55.756822 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.753664 10099 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:55.761937 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:55.762611 10670 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
W20260709 12:25:55.764567  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.610932 15306 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:55.773681  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:55.755487 15758 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.760330 15765 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:55.823624 10091 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:25:55.799644 15742 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.835510 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3306) took 1012 ms. Trace:
I20260709 12:25:55.836125 10022 rpcz_store.cc:276] 0709 12:25:54.823059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.057999 (+234940us) service_pool.cc:224] Handling call
0709 12:25:55.835256 (+777257us) spinlock_profiling.cc:243] Waited 65.7 ms on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:55.835425 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151123840}
I20260709 12:25:55.849330 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:55.842993 15611 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:55.820175 15758 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:55.879637 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:55.881381 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:25:55.860184 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 315230us
I20260709 12:25:55.860877 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.919948 15650 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:55.920622 10062 leader_election.cc:400] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
I20260709 12:25:55.920961 10062 leader_election.cc:403] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:25:55.920544 15650 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.921490 15796 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:25:55.922726 15306 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
I20260709 12:25:55.923283 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:55.862085 10099 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:25:55.926028 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.901547 10642 leader_election.cc:400] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
I20260709 12:25:55.878441 15611 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:55.927953 15611 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:25:55.929124  9949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1174) took 2037 ms (client timeout 1676 ms). Trace:
W20260709 12:25:55.929559  9949 rpcz_store.cc:269] 0709 12:25:53.892069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.897533 (+  5464us) service_pool.cc:224] Handling call
0709 12:25:55.928887 (+2031354us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b540003eb54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:55.929050 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679644928}
I20260709 12:25:55.931538 15551 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.908066 15672 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.915474 15731 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.933027 15659 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.916754 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3303) took 1156 ms. Trace:
I20260709 12:25:55.937199 10019 rpcz_store.cc:276] 0709 12:25:54.760708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.760863 (+   155us) service_pool.cc:224] Handling call
0709 12:25:54.761197 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:54.918012 (+156815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:54.918043 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:54.918082 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:54.918116 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.916493 (+998377us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54001383d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:55.916541 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.916670 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360177920}
I20260709 12:25:55.942035 15771 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:55.942342 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:55.942698 15771 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.933033 15731 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.944937 15771 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.945502 15771 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:55.945660 15731 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.946656 15830 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:25:55.947321 15823 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:25:55.954133  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.956475 15789 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:55.957595  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:55.957978 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.958354  9944 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:55.961526  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:55.964186 15787 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:55.964931 15787 raft_consensus.cc:2764] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:25:55.965575 15787 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:25:55.947469 15825 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:25:55.966318  9944 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:25:55.971657 10095 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:55.840480 15742 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.973425 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3486) took 1167 ms. Trace:
W20260709 12:25:55.974505 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73197.6us
I20260709 12:25:55.974864  9947 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:55.975785 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:55.976773  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.977522  9947 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:55.973757 10095 rpcz_store.cc:276] 0709 12:25:54.806111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.990066 (+183955us) service_pool.cc:224] Handling call
0709 12:25:55.957813 (+967747us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:25:55.973366 (+ 15553us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8212864}
I20260709 12:25:55.978825 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:55.979480 10095 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:55.980700 10095 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:55.982757 15789 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:55.983289 15789 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:55.985401 15789 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:55.993942 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2221) took 2351 ms (client timeout 2005 ms). Trace:
W20260709 12:25:55.994535 10093 rpcz_store.cc:269] 0709 12:25:53.642194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.642687 (+   493us) service_pool.cc:224] Handling call
0709 12:25:55.993676 (+2350989us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54002fd854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:55.993827 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2729764096}
I20260709 12:25:55.996201 10093 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:55.998049 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3481) took 1198 ms. Trace:
I20260709 12:25:55.999388 10091 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:55.899469 15725 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:55.898908 15139 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:56.000561  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.020187  9941 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
W20260709 12:25:56.021857  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.001667 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3482) took 1201 ms. Trace:
I20260709 12:25:56.011519  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.023065  9947 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
W20260709 12:25:56.023564  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2116) took 2100 ms (client timeout 1887 ms). Trace:
W20260709 12:25:56.023967  9948 rpcz_store.cc:269] 0709 12:25:53.922678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.063859 (+141181us) service_pool.cc:224] Handling call
0709 12:25:56.023406 (+1959547us) spinlock_profiling.cc:243] Waited 901 ms on lock 0x7b54000b6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:56.023512 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2071517056}
W20260709 12:25:56.002408  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 392765us
I20260709 12:25:56.108340  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.021708  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:55.998584 10090 rpcz_store.cc:276] 0709 12:25:54.800103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.002963 (+202860us) service_pool.cc:224] Handling call
0709 12:25:55.018347 (+ 15384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.095359 (+ 77012us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:55.095388 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:55.095437 (+    49us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:55.095472 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:55.997766 (+902294us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b540014a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:55.997820 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:55.997978 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":176506240}
I20260709 12:25:56.022280 10098 rpcz_store.cc:276] 0709 12:25:54.800415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.005035 (+204620us) service_pool.cc:224] Handling call
0709 12:25:56.000926 (+995891us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:25:56.001126 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:56.001335 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.001359 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:56.001400 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:56.001437 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.001491 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:56.001596 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34683776}
W20260709 12:25:56.072705 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2110) took 2328 ms (client timeout 3055 ms). Trace:
W20260709 12:25:56.127466 10024 rpcz_store.cc:269] 0709 12:25:53.671802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:53.783124 (+111322us) service_pool.cc:224] Handling call
0709 12:25:55.999982 (+2216858us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54001f6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:56.000127 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2883439872}
I20260709 12:25:56.102241 15577 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.103174 15758 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:55.946439 15731 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.132355  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.141870  9941 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:25:56.142838  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.143625  9949 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:25:56.143760  9945 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:25:56.144191  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:25:56.144784  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.145747 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:56.146762  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.129782  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:56.134011 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50451.3us
I20260709 12:25:56.135632  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:56.136873 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 161140us
I20260709 12:25:56.139079  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.148960  9947 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:56.150421 10097 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:56.203967 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:56.204541 10018 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:25:56.198027  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 68338.2us
I20260709 12:25:56.141034 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:56.022701 15763 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:56.198769  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.206537 10099 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:56.070031 15139 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:56.208729  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.019701 15650 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.210330  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:56.223030  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.111299 15354 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:56.207715 15769 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.234197 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:56.234920 10675 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
W20260709 12:25:56.235088  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 461477us
I20260709 12:25:56.132373 15772 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:56.142239 15830 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:56.255844 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:56.240753  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.256145 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:56.256904  9947 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:56.260829 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:56.262040 10025 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:25:56.267714 10677 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 10: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:56.145140 15758 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:56.272320 10099 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:56.274745 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:56.208107 15366 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:56.277686 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:56.285233 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:56.215200 15139 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:56.289985 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:56.290408 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:56.290899 10021 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:25:56.294332 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:56.294765 10017 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:25:56.294780  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.231774 15650 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.223146 15731 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.234334 15769 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.139683 15577 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:56.315289 10669 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:56.314035  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.332794 15731 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.334865 10675 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:56.335098 15759 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:56.335685 15759 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.336356 15769 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.337774 15759 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.339982 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:25:56.345254 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.215335 15411 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:25:56.245510 15810 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:56.346942 10099 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 12:25:56.347291 10099 pending_rounds.cc:85] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 9
I20260709 12:25:56.347810 10099 pending_rounds.cc:107] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 12:25:56.347259 15810 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:56.348093 10099 raft_consensus.cc:2889] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:56.349529 10099 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:25:56.349800 10091 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:56.315586 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:56.355751 10674 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.356948 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:56.357527 10090 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.358894 10672 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:56.296252 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:56.359655 10018 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:56.366137  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.367555  9947 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:56.370784 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:56.274202 15737 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:56.374413 15737 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.377485 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:56.329593 15689 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.000s
I20260709 12:25:56.383024  9944 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:25:56.383488 10016 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:56.368438 15370 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.000s
I20260709 12:25:56.386004  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:56.327520 15742 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.390314  9947 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:25:56.369647 15765 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.545s	user 0.007s	sys 0.002s
I20260709 12:25:56.370582 15544 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.243013 15354 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:56.404524 10671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.037s
I20260709 12:25:56.404206  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:56.405460  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:56.292810 15758 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:56.439213 10671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.006s	sys 0.038s
I20260709 12:25:56.278051 15772 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:56.441473  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:56.228420 15763 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.443375  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:56.458882  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:25:56.439343 15791 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.843s	user 0.010s	sys 0.000s
I20260709 12:25:56.467959  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.451385 15544 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:56.389745 15685 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.007s	sys 0.020s
I20260709 12:25:56.385893 15689 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.476609 15787 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:56.422200 15746 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.371161 15611 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.376453 15650 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.389967 15370 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.487223 15586 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 12:25:56.482659 15672 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:56.458338 15366 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:56.480510 15685 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.638s	user 0.008s	sys 0.021s
I20260709 12:25:56.492527 15672 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:56.496547 15672 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:56.464581 14841 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:56.494446  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.495110 10093 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:25:56.503718 10025 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:56.507344 10090 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:25:56.504292 10063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.001s	sys 0.013s
I20260709 12:25:56.511373  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:56.511744 10063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.001s	sys 0.013s
W20260709 12:25:56.514418  9942 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.007s	sys 0.008s
W20260709 12:25:56.514897  9942 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.007s	sys 0.008s
I20260709 12:25:56.495114 15366 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:56.506450 15340 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:56.482856 15791 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.385231 15810 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.423853 15765 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:56.537442  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:56.539330  9944 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:56.491600 15746 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:56.568368 15340 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:56.568838 15544 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.568997 15340 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:56.569998 15544 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.571382 15577 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.519109 15366 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:56.577924  9944 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:25:56.583559 10023 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:56.592363 10022 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:56.593392 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.541095 15737 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.367990  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:25:56.622390  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.648223  9947 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:56.570484 15544 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:56.653791  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.659416  9947 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:25:56.635732 15768 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:56.665417  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:56.667759 10668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.008s	sys 0.015s
W20260709 12:25:56.668143 10668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.008s	sys 0.015s
I20260709 12:25:56.675354  9949 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:25:56.675946  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 465664us
I20260709 12:25:56.677359  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:56.678941  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:56.680377  9945 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:25:56.683283 15772 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:56.689499 15772 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:56.690856 15772 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.696668 15772 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.680305  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:56.699193 15611 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.703009 15659 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:56.703670 15659 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:56.703706 15611 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:56.704298 15659 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:56.704609 15611 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.713541 10674 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:56.724082  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:56.728250  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.735134  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.736807  9945 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:56.679620 15577 raft_consensus.cc:2764] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election decision vote started in defunct term 17: won
I20260709 12:25:56.681493 15758 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:56.736039  9949 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:56.739841  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:25:56.767997 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:56.719906 15789 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:25:56.776028 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:56.718106 15763 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.771579 10023 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:56.723172 15796 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.005s	sys 0.011s
I20260709 12:25:56.778149  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.779418 15745 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.771569 15354 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.788470 15370 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:56.791657 15839 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:25:56.792968 15611 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.676941 15768 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:56.768116 15650 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.799428  9949 raft_consensus.cc:2376] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18. Re-sending same reply.
I20260709 12:25:56.773417 15789 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:56.739061 15139 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.828861  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:56.800839 15689 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.722167 15672 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:56.770586 15430 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:56.853073  9991 leader_election.cc:400] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 9 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 9. Current term is 10.
I20260709 12:25:56.854547  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:56.811340 15745 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:56.873726  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:56.764617 15762 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:56.806789  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.824182 10675 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:56.829224  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.877101  9945 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:25:56.829520  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:56.829659  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.758239 15787 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:56.812561 15354 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:56.801971 15796 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.006s	sys 0.011s
I20260709 12:25:56.811734  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:56.880882 10676 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:56.831490 15789 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:56.830974 15791 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.884104 15746 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:56.885493  9840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 330) took 35568 ms (client timeout 15000 ms). Trace:
W20260709 12:25:56.900352  9840 rpcz_store.cc:269] 0709 12:25:21.317544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:21.317818 (+   274us) service_pool.cc:224] Handling call
0709 12:25:56.885364 (+35567546us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:54.423548 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:54.506493 (+ 82945us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:54.506546 (+    53us) write_op.cc:435] Acquired schema lock
0709 12:25:54.506558 (+    12us) tablet.cc:662] Decoding operations
0709 12:25:54.507094 (+   536us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:54.507190 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:54.507216 (+    26us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:25:54.507675 (+   459us) tablet.cc:701] Row locks acquired
0709 12:25:54.507716 (+    41us) write_op.cc:260] PREPARE: finished
0709 12:25:54.507893 (+   177us) write_op.cc:270] Start()
0709 12:25:54.508034 (+   141us) write_op.cc:276] Timestamp: P: 1783599954507825 usec, L: 0
0709 12:25:54.508065 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:25:54.513064 (+  4999us) log.cc:844] Serialized 929 byte log entry
0709 12:25:55.065473 (+552409us) op_driver.cc:464] REPLICATION: finished
0709 12:25:55.193321 (+127848us) write_op.cc:301] APPLY: starting
0709 12:25:55.193438 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:25:55.199454 (+  6016us) tablet.cc:1370] finished BulkCheckPresence
0709 12:25:55.207113 (+  7659us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:25:55.210262 (+  3149us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:25:55.210812 (+   550us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 12:25:55.210988 (+   176us) write_op.cc:312] APPLY: finished
0709 12:25:55.211519 (+   531us) log.cc:844] Serialized 33 byte log entry
0709 12:25:55.234311 (+ 22792us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:55.234446 (+   135us) write_op.cc:454] Released schema lock
0709 12:25:55.234530 (+    84us) write_op.cc:341] FINISH: Updating metrics
0709 12:25:55.235150 (+   620us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:25:55.235191 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1259750,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":110351,"apply.run_cpu_time_us":7138,"apply.run_wall_time_us":44779,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":509063,"prepare.run_cpu_time_us":36380,"prepare.run_wall_time_us":289868,"raft.queue_time_us":213099,"raft.run_cpu_time_us":9877,"raft.run_wall_time_us":117822,"replication_time_us":531632,"thread_start_us":466117,"threads_started":4,"wal-append.queue_time_us":249918,"wal-append.run_cpu_time_us":28646,"wal-append.run_wall_time_us":1453151}]]}
I20260709 12:25:56.912281 15768 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.897271  9944 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:56.896508 15354 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.922928 15791 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:56.886686 15745 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:56.936647 15306 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:56.937279 15306 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:56.825460 15785 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.061s
I20260709 12:25:56.848357 15370 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.881953 15611 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:56.942730 10095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.005s	sys 0.015s
I20260709 12:25:56.856799 15551 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:56.876348 15430 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:56.875948 15577 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:56.864714 15765 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.926489 15745 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.941529 15354 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:56.953791 10095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.613s	user 0.006s	sys 0.015s
W20260709 12:25:56.954658 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.001s	sys 0.022s
W20260709 12:25:56.961861 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.001s	sys 0.022s
W20260709 12:25:56.962327 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 677701us
W20260709 12:25:56.965711 10643 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.047s	user 0.018s	sys 0.004s
I20260709 12:25:56.841292 15340 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.962574 15544 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:56.982923 10643 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.064s	user 0.018s	sys 0.005s
W20260709 12:25:56.987565 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.06869e+06us
I20260709 12:25:56.954735 15370 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:56.983891 15416 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.017s	sys 0.006s
I20260709 12:25:56.959214 15577 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:56.991580 15416 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.018s	sys 0.006s
I20260709 12:25:56.993175 15577 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.965986 15139 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:56.957259 15551 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:56.995913 15551 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:56.998390 15370 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:57.000404 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1197) took 1241 ms. Trace:
I20260709 12:25:56.980968 15354 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.009928 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:57.011054 10675 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:56.897318  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:56.938337 15306 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:56.985961  9946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.711s	user 0.015s	sys 0.010s
W20260709 12:25:57.028182  9946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.015s	sys 0.010s
I20260709 12:25:56.957674 15689 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:57.033806 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:57.035852  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.012s	sys 0.019s
W20260709 12:25:57.046931  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.012s	sys 0.019s
I20260709 12:25:57.056982 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.058547 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:57.059307 10676 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:57.060431 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:57.065296 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.000801 10668 rpcz_store.cc:276] 0709 12:25:55.759013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.759132 (+   119us) service_pool.cc:224] Handling call
0709 12:25:55.804979 (+ 45847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.131128 (+326149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.131157 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.139095 (+  7938us) log.cc:844] Serialized 26 byte log entry
0709 12:25:56.668468 (+529373us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:56.668507 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.810382 (+141875us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54007e4fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:56.810430 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.000099 (+189669us) raft_consensus.cc:1685] finished
0709 12:25:57.000118 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.000344 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3834,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":4479,"spinlock_wait_cycles":749214464,"thread_start_us":529009,"threads_started":1,"wal-append.queue_time_us":859570}
W20260709 12:25:57.066380 10092 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.016s	sys 0.023s
W20260709 12:25:57.066725 10092 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.016s	sys 0.024s
I20260709 12:25:57.068598 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:57.056095 15768 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:57.075011 15768 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:57.036840 15354 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.075852 15768 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.076406 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:57.077008 10098 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.077250 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:57.077008 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.152s	user 0.024s	sys 0.100s
I20260709 12:25:57.077903 10096 raft_consensus.cc:2376] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14. Re-sending same reply.
I20260709 12:25:56.984489 15763 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
W20260709 12:25:57.078982 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.154s	user 0.024s	sys 0.100s
W20260709 12:25:57.087271 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 1.16266e+06us
I20260709 12:25:57.086944 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.091447 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:57.078675 15611 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:57.000221 15745 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:25:57.065625 15544 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:57.096499 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.105178 15650 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:57.105640 15650 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:57.106036 15650 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:25:57.110136  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.010s	sys 0.018s
W20260709 12:25:57.110646  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.011s	sys 0.018s
W20260709 12:25:57.111071  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 622826us
I20260709 12:25:57.112179  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:57.065194 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.917s	user 0.023s	sys 0.041s
W20260709 12:25:57.113500 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.023s	sys 0.041s
W20260709 12:25:57.114267 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 966053us
I20260709 12:25:56.985030 15787 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.123238 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3484) took 2318 ms. Trace:
W20260709 12:25:57.135229 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49790.3us
W20260709 12:25:57.139837 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48494.6us
W20260709 12:25:57.140179  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 242914us
I20260709 12:25:57.142032  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:57.143847  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:57.145493 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.180217 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:57.180716 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:57.157166  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.181669  9944 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:57.157631 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:57.182760 10675 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.153460 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:57.158887 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:57.162067  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.164390  9949 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:25:57.186125 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.185479  9948 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:57.186515 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.123827 10092 rpcz_store.cc:276] 0709 12:25:54.805263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.206369 (+401106us) service_pool.cc:224] Handling call
0709 12:25:55.212495 (+  6126us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.212785 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:55.239749 (+ 26964us) log.cc:844] Serialized 14 byte log entry
0709 12:25:56.378714 (+1138965us) spinlock_profiling.cc:243] Waited 918 us on lock 0x7b1800476200. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:25:57.122739 (+744025us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.122826 (+    87us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:57.122896 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.122987 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.123159 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55481,"prepare.queue_time_us":26349,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":38,"raft.queue_time_us":56982,"spinlock_wait_cycles":2111232,"thread_start_us":688154,"threads_started":1}
I20260709 12:25:57.165719  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:57.115159 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:57.159551 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:57.169449  9941 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:25:57.086864 15785 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:57.180857 10671 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:57.158116  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.194561 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:57.195533  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:25:57.197214 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82162.6us
I20260709 12:25:57.152972 15306 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:57.152714 15551 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:25:57.204643 10094 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:25:57.205962 10022 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:25:57.216725 10098 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:57.198633 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:57.208989  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67051.2us
I20260709 12:25:57.197450 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:57.218663 10669 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:57.154325 15848 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:57.225329 10024 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:25:57.225853 10024 pending_rounds.cc:85] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 8
I20260709 12:25:57.226148 10024 pending_rounds.cc:107] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:25:57.226454 10024 raft_consensus.cc:2889] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:25:57.228375 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.220292  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:57.090252 15689 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.105232 15841 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 12:25:57.198141 15672 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:57.194958 15544 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.170437 15796 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:25:57.208142 15772 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:25:57.235599 10025 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:25:57.238649 10675 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:57.078424 15430 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.246562 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.109714 15611 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.252360 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:57.253213 10024 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:57.242153 10019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s	user 0.016s	sys 0.018s
I20260709 12:25:57.222779 15843 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:57.221194 15746 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:25:57.223242 15765 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:57.236006 15857 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:57.208046 15785 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:57.253953 10019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.016s	sys 0.019s
W20260709 12:25:57.241112 14841 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.063s	sys 0.117s
W20260709 12:25:57.245460 15586 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.009s	sys 0.030s
I20260709 12:25:57.243880 15772 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:57.244079 15544 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.259109 15672 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:57.259882 15765 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:57.284936 14841 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.063s	sys 0.118s
I20260709 12:25:57.334568 14841 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:57.334533 15765 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.335816 14841 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.338210 15551 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:25:57.279080 15659 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.008s
I20260709 12:25:57.338820 15551 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.339727 15659 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.341475 15551 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:57.341957 15659 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:57.333899 15672 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:57.233407 15848 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.342767 15551 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.317448  9839 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 9 to 13, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 13 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
W20260709 12:25:57.284549 15810 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.045s	sys 0.044s
W20260709 12:25:57.317713 15586 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.010s	sys 0.031s
I20260709 12:25:57.376080 15611 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:57.362969 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 173761us
I20260709 12:25:57.284520 15843 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:57.328716 15772 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:57.380580 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:57.376129 15810 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.045s	sys 0.044s
I20260709 12:25:57.400447 15842 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:25:57.400983 15842 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:57.401440 15544 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:57.401885 15842 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.403435 15842 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:57.406102 15672 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.417502 15785 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:57.417819 15768 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:57.419005  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2315) took 1293 ms. Trace:
I20260709 12:25:57.419847 15789 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.420090 15848 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:57.419420  9946 rpcz_store.cc:276] 0709 12:25:56.125354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.128632 (+  3278us) service_pool.cc:224] Handling call
0709 12:25:56.145592 (+ 16960us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.146319 (+   727us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.146349 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.207395 (+ 61046us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.028738 (+821343us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:25:57.028799 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.028912 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.418754 (+389842us) raft_consensus.cc:1685] finished
0709 12:25:57.418785 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.418940 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4105,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":19878,"thread_start_us":760283,"threads_started":1,"wal-append.queue_time_us":1191402}
I20260709 12:25:57.421573 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2252) took 1917 ms. Trace:
I20260709 12:25:57.422356 15789 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:57.326990 15857 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:25:57.394025  9839 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 9 to 12, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 12 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:57.429659 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:25:57.442935  9942 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.003s	sys 0.016s
I20260709 12:25:57.421988 10095 rpcz_store.cc:276] 0709 12:25:55.504152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.981424 (+477272us) service_pool.cc:224] Handling call
0709 12:25:55.981732 (+   308us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:56.135822 (+154090us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.135853 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:56.215135 (+ 79282us) log.cc:844] Serialized 52 byte log entry
0709 12:25:56.961842 (+746707us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:56.961897 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:56.961995 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:57.421354 (+459359us) raft_consensus.cc:1685] finished
0709 12:25:57.421376 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.421516 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141230,"raft.queue_time_us":351332,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":82,"thread_start_us":620484,"threads_started":1,"wal-append.queue_time_us":1048219}
W20260709 12:25:57.445523  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.004s	sys 0.010s
I20260709 12:25:57.445751  9839 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a reported cstate change: term changed from 8 to 9, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 9 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:57.460633  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.004s	sys 0.010s
I20260709 12:25:57.378062 15551 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:57.449795 15554 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.015s	sys 0.020s
W20260709 12:25:57.464361  9942 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.004s	sys 0.016s
I20260709 12:25:57.400506 15725 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:57.388957 15856 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
W20260709 12:25:57.446748  9827 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.017s	sys 0.013s
I20260709 12:25:57.477970  9839 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:57.508113 10671 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
W20260709 12:25:57.508064  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 364270us
I20260709 12:25:57.510021  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2136) took 1555 ms. Trace:
W20260709 12:25:57.507776 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 287033us
I20260709 12:25:57.464402 15789 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:57.494978 15554 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.015s	sys 0.020s
W20260709 12:25:57.510749  9827 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.017s	sys 0.014s
I20260709 12:25:57.538502  9839 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:57.536603 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 332338us
I20260709 12:25:57.536274  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:57.450069 15586 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 12:25:57.540201  9839 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 3 to 12. New cstate: current_term: 12 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:57.542284 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:57.545025  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 289273us
W20260709 12:25:57.545077 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 382284us
I20260709 12:25:57.526719 15846 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:57.531663 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:57.556900  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:57.559322 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:57.553397 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:57.534114 15689 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:57.562809  9839 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 9 to 10, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 10 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:57.566455 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.569062 10092 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:25:57.567286  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:57.573735  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:57.526824 15725 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.535772 15789 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.579072 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.580479 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:57.578146 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:57.591055 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:57.591850 10675 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:57.593099 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.588996  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:57.565238 15354 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:57.583473 15796 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:25:57.577010 15846 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:57.586524 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.564352 15577 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:57.532931 15843 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.601624 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:57.588893 10090 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:57.581234  9839 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 10 to 11, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 11 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:25:57.584282 15430 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:25:57.562849 15544 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:25:57.601552 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:57.572428 15611 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:25:57.630235 10093 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.587198 15689 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:57.567669 15861 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:25:57.600083 15737 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
W20260709 12:25:57.630647 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 201068us
I20260709 12:25:57.610075 15659 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:25:57.620806  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 366909us
W20260709 12:25:57.655644 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 102352us
I20260709 12:25:57.668197  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:57.588402 15731 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:25:57.619757 15746 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.617764 15354 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:25:57.672158 14841 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:25:57.682087  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.636837  9839 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:57.674576  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.677084  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.686241  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:25:57.656669 15854 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
I20260709 12:25:57.617962 15879 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:57.695709  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.614286 15650 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:57.702673  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 136751us
I20260709 12:25:57.645839 15689 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.706842  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.711951  9946 raft_consensus.cc:2410] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:25:57.707609  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.712620  9945 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:57.650540 15554 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.649763 15370 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:57.709779  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:57.713056 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:25:57.714035 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 155419us
I20260709 12:25:57.656562 15796 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:25:57.718541 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:57.722512  9839 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 11 to 13, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 13 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:57.725629 10093 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:25:57.702086 15868 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.717387 15689 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:57.706773 15879 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:57.714026 15354 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:57.743844  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:57.719849 15370 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.680917 15659 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.753701 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:57.744529  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:57.732972 15787 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:57.732215 15416 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:57.702441 15772 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.695338 15877 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:25:57.765583  9839 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 11 to 13, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 13 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:57.770494  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:57.770815  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82460.9us
I20260709 12:25:57.771960 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:57.775557  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:57.740233 15881 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:57.741024 15551 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:57.727872 15894 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:25:57.755719 15774 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:25:57.778803 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:25:57.779387  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:25:57.792748 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:57.793324  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:57.760067 15370 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:57.783195  9839 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 9 to 12, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 12 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:57.727622 15554 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.514052  9942 rpcz_store.cc:276] 0709 12:25:55.954303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:55.955212 (+   909us) service_pool.cc:224] Handling call
0709 12:25:55.955523 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:55.955726 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:25:56.515286 (+559560us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.509538 (+994252us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.509594 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:25:57.509648 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.509791 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:57.509961 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14446,"prepare.queue_time_us":36875,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":16113,"spinlock_wait_cycles":847232,"thread_start_us":1538141,"threads_started":2,"wal-append.queue_time_us":991225}
I20260709 12:25:57.758246 15859 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:57.758842 15879 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:57.786940 15796 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:57.796367  9947 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:57.797534 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:57.767343 15887 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:57.801927  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.804127  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.805903  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.823633  9948 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 9.
I20260709 12:25:57.822854  9949 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:25:57.824738  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.825313 15731 raft_consensus.cc:2764] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:25:57.825404  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.825871  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.826166 15731 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:57.777726 15861 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:25:57.773471 15416 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.783463 15881 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:57.819833 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:57.826478  9948 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:57.842661  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.843390  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.785125 15611 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:25:57.849244 15611 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:57.849654 15611 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:57.848910  9839 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 10 to 11, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 11 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:25:57.843531  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.888921  9947 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:57.800345 15787 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:57.837239 15874 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:25:57.835774  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.826684  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.844534 15898 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:25:57.847376 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:57.816358 15789 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:57.813915 15855 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:57.800220 15846 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:25:57.848459 10094 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:25:57.835963 15659 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:57.848490 10093 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:25:57.827543 15763 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:57.813918 15742 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 12:25:57.791134 15837 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:57.835001 15887 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:25:57.850270  9948 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:57.865810 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.866317 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.929258 10099 raft_consensus.cc:2410] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 12:25:57.929644 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.867033 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.799870 15872 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:57.797261 15878 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:25:57.843899  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.936671  9942 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:57.814038 15551 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:57.898339 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:57.831357 15888 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:25:57.756183 15868 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.907226 10090 raft_consensus.cc:2368] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
W20260709 12:25:57.847728 15862 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.305s	user 0.002s	sys 0.000s
I20260709 12:25:57.837189 15765 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:57.910295  9839 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 10 to 12, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 12 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:57.869112 15842 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:57.837528 15871 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:57.867784 15881 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:57.919575  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:57.966219 10090 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:57.926291 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:57.922986  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 334054us
I20260709 12:25:57.967638  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:57.935060 15872 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:57.954752 15650 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:57.818717 15859 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:57.932013 10097 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:25:57.932722 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:57.916999 15855 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:57.899322 15354 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:57.943116 15865 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.330s	user 0.000s	sys 0.005s
I20260709 12:25:57.939134  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:57.956667 10096 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:57.927798 15837 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:57.958635  9839 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:57.937350 15762 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.051s	user 0.014s	sys 0.009s
I20260709 12:25:57.932179 15869 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:57.898613 15877 consensus_queue.cc:1243] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:25:57.931592 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:57.945698 15551 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:57.971374 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:58.027022 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:58.027529 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:57.929100 15763 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:57.959002 15889 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:25:58.005638 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.007279 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:58.012185  9941 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:25:57.955672 15878 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:58.013947  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 74940.8us
I20260709 12:25:58.003086 15855 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:25:58.014052 15768 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.596s	user 0.012s	sys 0.003s
I20260709 12:25:58.075105  9839 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:58.030944 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:58.078399 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:57.969266 15891 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:57.964505 15685 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.074963 10097 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:25:57.974139 15765 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:58.081872  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.028046 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:58.083070 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.031018 15551 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:57.971696 15725 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:25:58.032169 15890 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:25:58.096045  9991 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.002171 15872 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:58.084594 15763 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:58.035660 15762 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:25:58.087271 15865 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:25:58.089591 15340 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:58.115965  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:58.103293  9839 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:58.095184 15765 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:57.971311 15871 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.133628  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.022779 15908 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:25:58.080209 15842 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.117512  9947 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:25:58.118135 15912 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:25:58.120332 15909 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:25:58.099583 15685 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.126930 15745 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:58.099033 15894 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:25:58.131305  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 370337us
I20260709 12:25:58.099330 15791 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:58.131783 14841 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:25:58.150780 14841 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:58.151198 14841 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:58.147179 10671 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:58.138795 15846 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.100716 15891 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.156745 15868 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:58.157579 15891 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.157338 15846 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.158938 15689 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.159546 15846 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:58.161160 15689 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.147991 15871 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.148425 15685 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.114513 15554 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.162684 15659 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:25:58.163968 15860 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.620s	user 0.002s	sys 0.000s
I20260709 12:25:58.166850 15765 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 12.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.169466 10669 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:25:58.144625 15340 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:58.133275 15725 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:58.032336 15869 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.154716 15762 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:58.140758  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:58.234032 14841 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.180128 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1204) took 1023 ms. Trace:
I20260709 12:25:58.182282  9839 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:58.184934 10092 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:58.185545 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.186825 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.193557 10671 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
W20260709 12:25:58.236022 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44012.3us
I20260709 12:25:58.195003 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.199134 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.239522 10093 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:25:58.201028 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:58.104285 15837 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.211041 15846 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:58.144822 15842 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.139463 15908 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.254765 15762 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:25:58.254837 15908 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:58.255326 14841 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.255118 15370 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.257608 15370 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.176719 15772 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:58.258867 15859 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:58.260792 15746 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.559s	user 0.005s	sys 0.005s
I20260709 12:25:58.261415  9839 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 7 to 14, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 14 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:58.179973 15829 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:58.227350 15893 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.226658  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:58.163980 15774 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:58.230623  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:58.181212 15878 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.235800 10668 rpcz_store.cc:276] 0709 12:25:57.156727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.156871 (+   144us) service_pool.cc:224] Handling call
0709 12:25:57.201402 (+ 44531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.201596 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:57.539347 (+337751us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.179655 (+640308us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b180030cc40. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287
0709 12:25:58.179751 (+    96us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.179797 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:58.179840 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.179927 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.180059 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5419,"prepare.queue_time_us":144862,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":5602,"raft.queue_time_us":91103,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":4898,"spinlock_wait_cycles":15243520,"thread_start_us":549859,"threads_started":3,"wal-append.queue_time_us":187050}
I20260709 12:25:58.248900 10677 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:25:58.267472  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 126800us
W20260709 12:25:58.233008 15306 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.020s	user 0.010s	sys 0.003s
I20260709 12:25:58.270583  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:58.272579 14841 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:58.272876  9839 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 11 to 14, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:58.275182 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:58.275251 15772 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:25:58.268898 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.277029 15370 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:58.254374 15842 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:58.279913 15370 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.280349 15370 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:58.280447 15785 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:25:58.280953 15785 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.281301 15785 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:58.266966 15746 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:25:58.254859 15366 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.622s	user 0.014s	sys 0.011s
I20260709 12:25:58.242681 15900 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:58.285221 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.266726 15762 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 12 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:58.307794  9839 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a reported cstate change: term changed from 10 to 14, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:25:58.306082 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:58.284493 15906 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:25:58.268870 15774 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:58.310250 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.311384  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:58.242818 15725 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.318715  9941 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:25:58.320101 15725 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.320842 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.322052 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:58.312378 15306 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:58.313906 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.322573 10024 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:58.283800 15768 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:25:58.282977 15846 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 15.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:25:58.314792 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:58.330919 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.314734 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.316027 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.316566 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:58.332044 10020 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:58.332587 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.333736 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.323985  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:58.335199  9946 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:58.268632 15139 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.325129  9839 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:58.335811 15139 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.325145  9942 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:25:58.298234 15842 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:58.298430 15837 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:58.335141 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.328320 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.339214 10025 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:25:58.307634 15672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.009s	sys 0.020s
I20260709 12:25:58.336735  9839 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 10 to 12, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 12 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:25:58.300670 15900 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:25:58.359242  9946 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:58.339193  9941 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:25:58.309144 15787 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:25:58.298270 15772 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:25:58.369526  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:58.373607  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:58.375188 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.341665 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.326843 15731 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:25:58.364439 15672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.011s	sys 0.020s
I20260709 12:25:58.378091 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.376228 10018 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:25:58.279032 15829 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.278169 15893 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.381558 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:58.318557 15577 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:58.356176 15875 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.305008 15366 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:58.312199  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:25:58.388765 15765 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:58.380489 15731 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:58.366144 15354 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:25:58.431969 10024 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
W20260709 12:25:58.418403  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 147979us
W20260709 12:25:58.433739 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 123570us
I20260709 12:25:58.421180 10020 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:25:58.415256 15577 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:25:58.379905 15861 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:25:58.372605 15787 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:58.437041 10021 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:25:58.426137 15875 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:58.432935  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 120809us
I20260709 12:25:58.514031 15340 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:25:58.386016 15774 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.567368 15893 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:58.569218 10095 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:25:58.611183 10095 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:58.482409 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47491.1us
I20260709 12:25:58.508601 15861 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:58.496907  9944 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:58.547786 15881 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.574s	user 0.005s	sys 0.004s
I20260709 12:25:58.482901 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.433104 15772 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:58.550051 15787 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:58.527980 15611 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.665s	user 0.010s	sys 0.001s
I20260709 12:25:58.562624 15370 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.485396 10020 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:25:58.354212 15917 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.113s
I20260709 12:25:58.521124 15872 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.435593 15731 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:58.496042 15366 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:58.488758 15921 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:25:58.372304 15551 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.568045 15758 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:25:58.607340  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.438241 15765 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:25:58.657092  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49839.3us
I20260709 12:25:58.608775 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:25:58.607163  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 172817us
I20260709 12:25:58.609931 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.657941  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:25:58.586310 15785 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.499444 15900 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.631388 10020 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:58.431120 15879 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.001s
I20260709 12:25:58.671032 15879 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.671500 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.672163 10094 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:25:58.630141 10095 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.673241 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.656140 10024 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:25:58.676306 15858 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.170s	user 0.001s	sys 0.002s
I20260709 12:25:58.489144 15139 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:58.677433 10024 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:25:58.677776 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1007) took 2548 ms. Trace:
I20260709 12:25:58.657877 10668 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 10.
I20260709 12:25:58.678555 15875 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:58.648708 15787 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:58.678737 15861 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:58.680929 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.703s (SENT) [suppressed 20 similar messages]
I20260709 12:25:58.681202 15904 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:58.656255 15758 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:25:58.661737 15867 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.071s	user 0.000s	sys 0.007s
I20260709 12:25:58.661598 15774 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:58.662411 15765 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:58.663321 15855 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.001s
I20260709 12:25:58.665207  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:58.681779 15904 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:58.668323 15551 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:25:58.668762  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 699966us
I20260709 12:25:58.676577 15908 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.617005 15881 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.629873 15611 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:58.682826 10024 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
W20260709 12:25:59.155875 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.156009  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:59.088462 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 407537us
I20260709 12:25:58.955636  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2357) took 1037 ms. Trace:
I20260709 12:25:58.686061 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.731432 10676 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:25:58.973251 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.018195 10020 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:25:58.678153 10019 rpcz_store.cc:276] 0709 12:25:56.129030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:56.129171 (+   141us) service_pool.cc:224] Handling call
0709 12:25:56.129490 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:56.138335 (+  8845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:56.138366 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:56.391955 (+253589us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b2c00046bc8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:25:56.409924 (+ 17969us) log.cc:844] Serialized 26 byte log entry
0709 12:25:57.279651 (+869727us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:25:57.279738 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:57.279833 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:58.675692 (+1395859us) spinlock_profiling.cc:243] Waited 188 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:25:58.677537 (+  1845us) raft_consensus.cc:1685] finished
0709 12:25:58.677558 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:58.677696 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2177,"raft.queue_time_us":5580,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":11587,"spinlock_wait_cycles":442989824,"thread_start_us":790428,"threads_started":1,"wal-append.queue_time_us":963932}
I20260709 12:25:59.239169 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:58.731945 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:25:59.078964 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.240618 10093 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:58.686766 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:58.844452 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:59.244019 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:58.687412 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:59.157831  9947 rpcz_store.cc:276] 0709 12:25:57.919215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.919330 (+   115us) service_pool.cc:224] Handling call
0709 12:25:58.955345 (+1036015us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b5400016dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:58.955552 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":342930176}
I20260709 12:25:59.252668 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:59.254701 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:59.193508 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:59.274514 15139 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:59.275048 15139 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.275324 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:59.276180 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:59.276803 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:59.277712 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:59.277989 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:59.279729 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:59.285279 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:59.285878  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:59.286525  9948 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:25:59.289532 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:58.685782 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:25:59.292467  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 136614us
I20260709 12:25:58.648995 15917 consensus_queue.cc:1243] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:25:59.291060 10099 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:59.298202 15139 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.015378 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1016) took 1774 ms. Trace:
I20260709 12:25:58.962139  9941 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:25:59.299460 10022 raft_consensus.cc:2368] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:25:59.291975 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:59.294205  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:59.296519 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:59.297217 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:59.300110  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:59.167496  9942 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:25:58.853361 15857 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:59.300674 10019 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:59.299124  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:59.301913 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:59.091719 15370 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.345041 14841 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:25:59.346041 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:59.298712 10016 rpcz_store.cc:276] 0709 12:25:57.240550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.240666 (+   116us) service_pool.cc:224] Handling call
0709 12:25:57.249413 (+  8747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:57.289552 (+ 40139us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:25:57.550024 (+260472us) log.cc:844] Serialized 14 byte log entry
0709 12:25:57.655192 (+105168us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:57.655248 (+    56us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:57.655298 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.015088 (+1359790us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b5400137ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:25:59.015132 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.015298 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":51493,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":467,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":91736192,"thread_start_us":313906,"threads_started":2,"wal-append.queue_time_us":166542,"wal-append.run_cpu_time_us":5972,"wal-append.run_wall_time_us":1372156}
I20260709 12:25:59.345134 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:25:59.305286  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 640114us
I20260709 12:25:59.305217 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:59.346762 10019 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:25:59.311053 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:59.349962 10099 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:25:59.305756 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:59.304745 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:59.184834 15868 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.682080 15354 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:59.284844 15772 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.367652  9948 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:25:58.951807 15894 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:58.851655 15901 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.286549 15689 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.351974  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:59.192986 15796 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:58.673259 15879 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:59.356843 10092 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 11.
I20260709 12:25:59.352705 10017 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:25:59.359299 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:25:59.296890 15856 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:25:59.361420 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:25:58.731581 15577 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.356189 15917 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:59.363862 15758 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.286079 15737 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:59.291961 15878 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.368717  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:58.875480 15900 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:58.848299 15869 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:59.187012 15855 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.373879 10025 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:59.375344  9942 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:25:59.353134 15859 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:25:59.394204 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2146) took 1018 ms. Trace:
I20260709 12:25:59.394949 10025 rpcz_store.cc:276] 0709 12:25:58.376072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.383381 (+  7309us) service_pool.cc:224] Handling call
0709 12:25:59.361353 (+977972us) spinlock_profiling.cc:243] Waited 5.3 ms on lock 0x7b5c00173e0c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0cec0161 00007f4e0d061a79 00007f4e0e51b9b6 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:25:59.394129 (+ 32776us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12183808}
I20260709 12:25:59.384677 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:25:59.386575 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:25:59.386724 10668 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:25:59.366918 15872 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.358502 15551 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.398085  9948 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:25:59.406793 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.384228 15785 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.420734  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:59.422942 15872 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:25:59.407464 15551 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.427462 10023 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:25:59.428198 15855 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.393827 15869 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.393399 15737 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.377660 15894 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.394367 15859 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:59.377462 15901 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.377295 15689 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.402683 15758 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:25:59.445180  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2360) took 1135 ms. Trace:
I20260709 12:25:59.390964 15917 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:59.465634  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.467476  9948 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:25:59.470579 15551 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.368237 15868 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.474411 10016 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:25:59.373612 15772 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.373760 15929 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 12:25:59.419219 15879 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.486438 10097 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:25:59.477079 15731 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.818s	user 0.008s	sys 0.003s
I20260709 12:25:59.377754 15796 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:59.485874 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 244035us
I20260709 12:25:59.486451  9940 rpcz_store.cc:276] 0709 12:25:58.310257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.322658 (+ 12401us) service_pool.cc:224] Handling call
0709 12:25:58.323020 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:25:58.608432 (+285412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.608465 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:25:58.609091 (+   626us) log.cc:844] Serialized 52 byte log entry
0709 12:25:58.962840 (+353749us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:58.962891 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:58.962984 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:25:59.444865 (+481881us) raft_consensus.cc:1685] finished
0709 12:25:59.444896 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:25:59.445096 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26052,"raft.queue_time_us":57718,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":4385,"thread_start_us":617068,"threads_started":2,"wal-append.queue_time_us":771709}
I20260709 12:25:59.394172 15878 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.526448 10091 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:25:59.528136  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:59.548970 10093 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:25:59.549201  9948 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:59.483534 15859 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:25:59.553963  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:59.535764 10672 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:25:59.532652 15787 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.853s	user 0.006s	sys 0.006s
I20260709 12:25:59.482826 15869 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.556833  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:59.566768  9948 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:25:59.566918 10016 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:25:59.568048  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:59.569080  9948 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:59.544462 15731 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.581002 15904 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:59.580057  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:59.533257 15855 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:25:59.593065  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:59.586202  9941 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:25:59.599848  9940 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:25:59.602466  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:59.604511 15901 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:59.606312 15917 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:59.606953 15586 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:59.607672 15855 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:25:59.608294  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:59.697746  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89529.2us
I20260709 12:25:59.609292 15857 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:25:59.699726 15857 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:25:59.616406 15869 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.553640 15742 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:25:59.619823 15737 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.628729  9948 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:25:59.546547 15879 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.622982 15881 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.586038 15841 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:59.618330 15772 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:59.694047  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:25:59.707213  9940 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:25:59.710382  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:59.715359  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:25:59.724921 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 348648us
I20260709 12:25:59.664809 15929 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.700166 15857 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:59.697993 15586 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:25:59.697384 15855 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:59.700187 15354 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:25:59.698173 15901 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:59.743575  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 439961us
I20260709 12:25:59.761135  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:25:59.749679 14841 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:59.726498 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:25:59.759150  9940 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
W20260709 12:25:59.769506 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.763499 14841 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.789724  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:25:59.791491  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:25:59.739409 15861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.057s	user 0.004s	sys 0.020s
I20260709 12:25:59.795358 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:25:59.796218 10098 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:59.729522 15875 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:25:59.799470 10090 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.013s	sys 0.013s
W20260709 12:25:59.800206 10090 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.514s	user 0.014s	sys 0.013s
I20260709 12:25:59.743155 15586 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:25:59.712638 15904 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:59.801975  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:59.760679 15772 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:25:59.800802 15861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.004s	sys 0.021s
I20260709 12:25:59.725286 15841 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.790169 15869 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.798956 15881 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:25:59.796012 15937 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:59.819299 15796 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:59.797616 15354 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:59.820416 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:25:59.828908 10098 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:25:59.741539 15929 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.832922  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:25:59.815402 15689 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.796226 15901 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:25:59.853720 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.802959 15936 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:25:59.831351 15861 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:25:59.817836 15737 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:25:59.837041 15796 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.871089  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:25:59.819964  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:25:59.869670 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 323399us
I20260709 12:25:59.822441 15772 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:25:59.811080 15904 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.825693 15742 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:25:59.864125 15689 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:25:59.831475 15881 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:25:59.856416 15863 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.915045 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:25:59.909744 10098 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:25:59.867118 15878 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:59.821385 15855 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.861943 15917 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:25:59.827315 15869 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:59.994237 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:25:59.994266  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:25:59.831140 15937 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:25:59.997538 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 271139us
W20260709 12:25:59.993782  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 202393us
I20260709 12:25:59.851037 15787 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:25:59.992688 14841 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:25:59.841120 15868 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:25:59.897496 15861 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:25:59.904186 15731 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:25:59.992049 15875 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:00.063905 15875 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.067087 15787 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:00.020453 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:00.025274  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116595us
W20260709 12:26:00.079698 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59345.6us
I20260709 12:26:00.080775 10642 leader_election.cc:400] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14.
W20260709 12:25:59.928169 15935 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.353s	user 0.001s	sys 0.000s
I20260709 12:26:00.084393  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2150) took 1125 ms. Trace:
I20260709 12:26:00.084875  9946 rpcz_store.cc:276] 0709 12:25:58.958546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.958918 (+   372us) service_pool.cc:224] Handling call
0709 12:25:58.976983 (+ 18065us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:58.977447 (+   464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:58.977490 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:58.998164 (+ 20674us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.394747 (+396583us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:59.394785 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.394852 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.084137 (+689285us) raft_consensus.cc:1685] finished
0709 12:26:00.084166 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.084308 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":5682,"thread_start_us":235829,"threads_started":1,"wal-append.queue_time_us":500778}
I20260709 12:26:00.087391  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:00.087942 15787 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.089483  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:00.090147 15787 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:00.078367  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56233.4us
I20260709 12:26:00.101305  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:00.107017 15787 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.107766 15787 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.110584 15787 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:00.099479 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 184514us
I20260709 12:25:59.924379 15366 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:00.023221 14841 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:25:59.928328 15944 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.993250 15416 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:25:59.927160 15940 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:25:59.921093  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 101186us
I20260709 12:26:00.152551  9940 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 10.
I20260709 12:26:00.157742  9949 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:26:00.208004  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2154) took 1218 ms. Trace:
I20260709 12:26:00.208402  9949 rpcz_store.cc:276] 0709 12:25:58.989862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.991044 (+  1182us) service_pool.cc:224] Handling call
0709 12:26:00.150202 (+1159158us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:26:00.150499 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.207649 (+ 57150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.207937 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5625600}
I20260709 12:26:00.210865 15787 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:00.211488 15787 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.211913 15787 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:25:59.931528 15894 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:00.029228 15861 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:00.202767 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:00.205547 15875 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:25:59.914165 15796 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.033730 15879 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:00.003536 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:00.263382 14841 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:00.157645 15855 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.268676 10098 raft_consensus.cc:2368] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17.
I20260709 12:26:00.272615 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:00.152153 15366 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:00.280544 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.229684 15737 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.027070 15917 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.003212 15863 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.002336 15354 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.269937 15942 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:00.307591 15950 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.138s	user 0.001s	sys 0.001s
W20260709 12:26:00.295120  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.383900 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.387862  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 286651us
W20260709 12:26:00.328025 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.430949 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:00.454005 10098 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:00.282526 15366 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:00.460757 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:00.359606 15958 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:26:00.299367 15737 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:00.470973 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 1049) took 2447 ms. Trace:
I20260709 12:26:00.269164 15875 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:00.475723 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 394150us
I20260709 12:26:00.199469 15944 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.485939 10674 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:26:00.360122 15841 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:00.228482 15940 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.573133 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:00.499996 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2281) took 1293 ms. Trace:
I20260709 12:26:00.478492 10669 rpcz_store.cc:276] 0709 12:25:58.006216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.006378 (+   162us) service_pool.cc:224] Handling call
0709 12:26:00.453139 (+2446761us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b5400610dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:00.453399 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188109696}
I20260709 12:26:00.580230 10098 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:00.574656 10090 rpcz_store.cc:276] 0709 12:25:59.207001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.285028 (+ 78027us) service_pool.cc:224] Handling call
0709 12:25:59.285343 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.800766 (+515423us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:59.800812 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.811513 (+ 10701us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.918457 (+106944us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:59.918523 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.918662 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.499608 (+580946us) raft_consensus.cc:1685] finished
0709 12:26:00.499641 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.499906 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91859,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":17718,"thread_start_us":601428,"threads_started":2,"wal-append.queue_time_us":413450}
I20260709 12:26:00.309820 15964 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.284s
I20260709 12:26:00.595798  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1206) took 2829 ms. Trace:
I20260709 12:26:00.597034 15937 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:00.597275 15931 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.745s	user 0.032s	sys 0.012s
I20260709 12:26:00.600584 15611 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:00.602280 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:00.603080 10094 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
W20260709 12:26:00.603274 15957 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:26:00.604072 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:00.595827 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:00.604643 10094 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:00.604759 10098 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:00.605654 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:00.606202 10098 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:00.607209 15917 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:00.607319 10098 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:00.607928 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2284) took 1395 ms. Trace:
I20260709 12:26:00.608253 10098 rpcz_store.cc:276] 0709 12:25:59.212398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.606780 (+1394382us) service_pool.cc:224] Handling call
0709 12:26:00.607049 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.607703 (+   654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.607884 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:00.609298 15900 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.218s	user 0.010s	sys 0.000s
I20260709 12:26:00.607928 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2285) took 1393 ms. Trace:
I20260709 12:26:00.611439 15796 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:00.611176 10090 rpcz_store.cc:276] 0709 12:25:59.214486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.607261 (+1392775us) service_pool.cc:224] Handling call
0709 12:26:00.607511 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.607670 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.607693 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.607732 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:00.607760 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.607807 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.607877 (+    70us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.613619 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2289) took 1376 ms. Trace:
I20260709 12:26:00.614063 10090 rpcz_store.cc:276] 0709 12:25:59.236874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.612637 (+1375763us) service_pool.cc:224] Handling call
0709 12:26:00.612983 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.613201 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.613238 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.613308 (+    70us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:00.613354 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.613425 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.613553 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.616060 10090 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 13. Preceding OpId from leader: term: 14 index: 14. (index mismatch)
I20260709 12:26:00.616137 15940 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:00.616883 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2290) took 1331 ms. Trace:
I20260709 12:26:00.617327 10090 rpcz_store.cc:276] 0709 12:25:59.285823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.615307 (+1329484us) service_pool.cc:224] Handling call
0709 12:26:00.615644 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.616587 (+   943us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.616820 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.619164 15965 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.627565 15965 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.628224 15869 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:00.628893 15904 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:00.629478 10090 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:00.629813 15965 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:00.630514 15956 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:00.596354  9945 rpcz_store.cc:276] 0709 12:25:57.766408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.766558 (+   150us) service_pool.cc:224] Handling call
0709 12:25:57.777193 (+ 10635us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.029354 (+252161us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:25:58.089192 (+ 59838us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.151784 (+ 62592us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.226916 (+ 75132us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.227263 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 12:25:58.227440 (+   177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:25:58.227488 (+    48us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:25:58.227537 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.595465 (+2367928us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b540042e854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:00.595595 (+   130us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.595718 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95319,"prepare.queue_time_us":79686,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":10723,"raft.queue_time_us":7321,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":208,"spinlock_wait_cycles":579267456,"thread_start_us":46197,"threads_started":1,"wal-append.queue_time_us":56994,"wal-append.run_cpu_time_us":2564,"wal-append.run_wall_time_us":1141380}
I20260709 12:26:00.632414 10675 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:00.632675 15875 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:00.632992 15936 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:00.633543 15774 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.447s	user 0.008s	sys 0.006s
I20260709 12:26:00.633460 15936 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.635118  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1215) took 1153 ms. Trace:
I20260709 12:26:00.635764  9945 rpcz_store.cc:276] 0709 12:25:59.481360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.633120 (+1151760us) service_pool.cc:224] Handling call
0709 12:26:00.633551 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.634650 (+  1099us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.634698 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.634759 (+    61us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:00.634813 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.634888 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.635044 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.637295 15577 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.637646 15863 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:00.637874 15577 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.638798 15929 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:00.278946 15881 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.639117  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1218) took 1052 ms. Trace:
I20260709 12:26:00.639650  9945 rpcz_store.cc:276] 0709 12:25:59.586669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.637556 (+1050887us) service_pool.cc:224] Handling call
0709 12:26:00.637859 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.638673 (+   814us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.638718 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.638781 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:00.638824 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.638904 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.639048 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.640419 15577 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:00.619964 15930 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.960s	user 0.023s	sys 0.045s
I20260709 12:26:00.641603  9945 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:00.618558 15758 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.095s	user 0.006s	sys 0.002s
I20260709 12:26:00.641969 15941 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:00.642630 15611 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.643015 15941 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.643127 15611 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.643790 10094 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:26:00.644548 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2293) took 1275 ms. Trace:
I20260709 12:26:00.645166 15941 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:00.646999 15689 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:00.647224 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74463us
I20260709 12:26:00.636838 15936 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:00.648423 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:00.502454 15943 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.501s	user 0.000s	sys 0.000s
W20260709 12:26:00.501921 15948 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.459s	user 0.000s	sys 0.001s
W20260709 12:26:00.621596 15857 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.870s	user 0.007s	sys 0.002s
I20260709 12:26:00.642401  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1221) took 1030 ms. Trace:
I20260709 12:26:00.672111  9945 rpcz_store.cc:276] 0709 12:25:59.611639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.640927 (+1029288us) service_pool.cc:224] Handling call
0709 12:26:00.641358 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.642097 (+   739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.642337 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.630638 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2291) took 1343 ms. Trace:
I20260709 12:26:00.609325 10098 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:00.673167 10090 rpcz_store.cc:276] 0709 12:25:59.286890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.618229 (+1331339us) service_pool.cc:224] Handling call
0709 12:26:00.629089 (+ 10860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.630260 (+  1171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.630553 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.644941 10094 rpcz_store.cc:276] 0709 12:25:59.369165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.603836 (+1234671us) service_pool.cc:224] Handling call
0709 12:26:00.644470 (+ 40634us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.575906 15908 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:00.676308 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2288) took 1440 ms. Trace:
I20260709 12:26:00.499500 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:00.676936 10098 rpcz_store.cc:276] 0709 12:25:59.236178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.608925 (+1372747us) service_pool.cc:224] Handling call
0709 12:26:00.609170 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.673848 (+ 64678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.676231 (+  2383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.475732 15879 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:00.645175 15611 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:00.558126 15859 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.980s	user 0.007s	sys 0.007s
W20260709 12:26:00.501214 15961 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 12:26:00.513514 15901 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:00.671423 10675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1263) took 1389 ms (client timeout 1726 ms). Trace:
I20260709 12:26:00.564299 15942 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:00.629578 15772 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:00.500339 15951 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 12:26:00.675194 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3520) took 1285 ms. Trace:
I20260709 12:26:00.727759 10090 rpcz_store.cc:276] 0709 12:25:59.389933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.674273 (+1284340us) service_pool.cc:224] Handling call
0709 12:26:00.674665 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.674854 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.674878 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.674918 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:00.674956 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.675015 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.675137 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.680853 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:00.730214 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49460.4us
I20260709 12:26:00.731720 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:00.733510 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:00.491874 15737 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:00.709662 15586 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.895s	user 0.010s	sys 0.000s
I20260709 12:26:00.683954 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3524) took 1293 ms. Trace:
I20260709 12:26:00.684402 15900 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:26:00.507534 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.674580 15857 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.708379  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1216) took 1177 ms. Trace:
I20260709 12:26:00.747725  9940 rpcz_store.cc:276] 0709 12:25:59.531119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.821700 (+290581us) service_pool.cc:224] Handling call
0709 12:26:00.708184 (+886484us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54000b6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:00.708313 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":733168512}
I20260709 12:26:00.708848 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2279) took 1505 ms. Trace:
W20260709 12:26:00.719040 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3334) took 2401 ms (client timeout 1995 ms). Trace:
W20260709 12:26:00.749891 10022 rpcz_store.cc:269] 0709 12:25:58.317894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.330687 (+ 12793us) service_pool.cc:224] Handling call
0709 12:26:00.718801 (+2388114us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54001f7554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:00.718940 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2225591040}
W20260709 12:26:00.720078 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:00.724848 10675 rpcz_store.cc:269] 0709 12:25:59.276119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.278982 (+  2863us) service_pool.cc:224] Handling call
0709 12:26:00.665207 (+1386225us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:00.665338 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2900692224}
W20260709 12:26:00.735039  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.000s	sys 0.012s
I20260709 12:26:00.683573  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1222) took 1071 ms. Trace:
W20260709 12:26:00.753461  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.931s	user 0.000s	sys 0.012s
I20260709 12:26:00.753623  9945 rpcz_store.cc:276] 0709 12:25:59.611946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.672978 (+1061032us) service_pool.cc:224] Handling call
0709 12:26:00.682901 (+  9923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.683120 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.683154 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.683210 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:00.683273 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.683344 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.683495 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:00.720228 15861 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.505s	user 0.008s	sys 0.001s
I20260709 12:26:00.741138 10098 rpcz_store.cc:276] 0709 12:25:59.390790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.678059 (+1287269us) service_pool.cc:224] Handling call
0709 12:26:00.678469 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.683478 (+  5009us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.683520 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.683583 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:00.683648 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.683732 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.683877 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.749130 10097 rpcz_store.cc:276] 0709 12:25:59.203887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.254360 (+ 50473us) service_pool.cc:224] Handling call
0709 12:26:00.708668 (+1454308us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:00.708791 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":530404096}
I20260709 12:26:00.698552 15944 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:00.752892 10065 connection.cc:256] RPC call timeout handler was delayed by -1.1904381409999587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15860000000000002s for a response
I20260709 12:26:00.799705 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:00.802874 10098 raft_consensus.cc:2368] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15.
I20260709 12:26:00.804039 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:00.800558 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:00.805686 10097 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:00.806756 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:00.815830  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.816718 10065 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:00.739257 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:00.819191 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79993us
I20260709 12:26:00.832885 15900 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:00.824622 10019 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:26:00.827471 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:00.742969 15964 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:00.721402 15859 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.722543 15879 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.736035 15967 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:00.820806 15961 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:26:00.791971 15772 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:00.843844 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2283) took 1633 ms. Trace:
I20260709 12:26:00.735976 15366 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:00.848805 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.831986 15904 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:00.736207 15942 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.852720 15900 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:00.814736 15949 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:00.823272 15908 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:00.851898 14841 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.116s	sys 0.210s
I20260709 12:26:00.747982 15586 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.855290 10097 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:00.840957 15954 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:00.841177 15969 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:00.852479 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:26:00.863101 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.707798 15762 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.799000 15861 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.862462 15964 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:00.823125 15758 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:26:00.822971 15846 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:26:00.868177 10096 rpcz_store.cc:276] 0709 12:25:59.210627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.360122 (+149495us) service_pool.cc:224] Handling call
0709 12:25:59.360459 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.522957 (+162498us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:25:59.522997 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.550793 (+ 27796us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.835668 (+284875us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:25:59.835722 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:25:59.835814 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:00.843598 (+1007784us) raft_consensus.cc:1685] finished
0709 12:26:00.843628 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.843791 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":195025,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":14127,"thread_start_us":410227,"threads_started":2,"wal-append.queue_time_us":461535}
I20260709 12:26:00.881511 15900 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:00.823139 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:00.876339 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 297635us
I20260709 12:26:00.902155 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:00.902568 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.902993 10098 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:00.904027 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:00.887960 14841 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.116s	sys 0.211s
I20260709 12:26:00.899211 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:00.874738  9948 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:00.880970 15949 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:00.906148 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:00.905676 10096 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:00.872140 15796 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:26:00.909708 10022 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:26:00.894701 15787 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.682s	user 0.004s	sys 0.009s
I20260709 12:26:00.911123 10098 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:00.914016  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:00.914630 10098 raft_consensus.cc:606] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:26:00.915030 10098 raft_consensus.cc:614] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:00.896574 15742 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:00.916121 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:00.920917 10669 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:00.873786 15772 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.915426 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3526) took 1509 ms. Trace:
I20260709 12:26:00.927268 15139 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:00.926769 10096 rpcz_store.cc:276] 0709 12:25:59.406565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.914800 (+1508235us) service_pool.cc:224] Handling call
0709 12:26:00.915097 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.915212 (+   115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.915227 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.915255 (+    28us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:00.915274 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.915307 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.915384 (+    77us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.917371 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:00.930619 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:00.931356 10022 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:00.931850  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1204) took 3765 ms. Trace:
I20260709 12:26:00.932219 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:00.917603 10673 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:00.932912 10022 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:00.898023 15969 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:00.919868 15974 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:00.919926 15829 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:00.916411 10095 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:00.899221 15964 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:00.923019  9948 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:26:00.917129 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:00.933719 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:00.933954 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:00.941972 10022 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:00.927034 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:00.932237  9944 rpcz_store.cc:276] 0709 12:25:57.166837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.182428 (+ 15591us) service_pool.cc:224] Handling call
0709 12:25:57.182722 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.612033 (+1429311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.931641 (+2319608us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540018a1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:00.931800 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3021767040}
I20260709 12:26:00.943033 15139 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:00.942828 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:00.912462 15949 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:00.894237 15908 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.938544 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3528) took 1072 ms. Trace:
I20260709 12:26:00.939332 10021 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:00.920686 15953 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:00.941201 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3530) took 1028 ms. Trace:
I20260709 12:26:00.948333 10096 rpcz_store.cc:276] 0709 12:25:59.865932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.937673 (+1071741us) service_pool.cc:224] Handling call
0709 12:26:00.938053 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.938229 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.938254 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.938296 (+    42us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:00.938331 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.938386 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.938487 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.897328 15954 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:00.935758 15742 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:00.951771 10098 rpcz_store.cc:276] 0709 12:25:59.913140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.940370 (+1027230us) service_pool.cc:224] Handling call
0709 12:26:00.940716 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.940902 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.940924 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.940964 (+    40us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:00.940993 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.941048 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.941144 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.936457 15872 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:00.955457 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3532) took 1026 ms. Trace:
I20260709 12:26:00.961184 10096 rpcz_store.cc:276] 0709 12:25:59.928680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.954375 (+1025695us) service_pool.cc:224] Handling call
0709 12:26:00.954823 (+   448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.955051 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.955089 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.955147 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:00.955186 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.955258 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:00.955385 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:00.960301 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1057) took 2765 ms. Trace:
I20260709 12:26:00.963198 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:00.962489 10676 rpcz_store.cc:276] 0709 12:25:58.191659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.191855 (+   196us) service_pool.cc:224] Handling call
0709 12:25:58.231124 (+ 39269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.187518 (+956394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.956858 (+1769340us) spinlock_profiling.cc:243] Waited 453 ms on lock 0x7b54006dfe54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:00.957040 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1040902912}
I20260709 12:26:00.933159 15354 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:26:00.924482 15787 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:00.972807 10098 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:00.975231 10098 raft_consensus.cc:606] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:00.975719 10098 raft_consensus.cc:614] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:00.960809 15857 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:00.959093 15370 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:26:00.961006 15872 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:00.997426  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.001992 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:01.004441 10016 raft_consensus.cc:2368] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
W20260709 12:26:01.005398  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 332764us
I20260709 12:26:00.957315 15953 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.012907 10098 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:01.013444 10098 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:01.015995 15970 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:01.016604 15970 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:01.010219 15857 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.023883 15855 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:01.025625 10017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2138) took 2692 ms (client timeout 1936 ms). Trace:
W20260709 12:26:01.026057 10017 rpcz_store.cc:269] 0709 12:25:58.333596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.377860 (+ 44264us) service_pool.cc:224] Handling call
0709 12:26:01.025464 (+2647604us) spinlock_profiling.cc:243] Waited 967 ms on lock 0x7b540038d1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.025558 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2224699648}
I20260709 12:26:01.028066 15966 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:01.028525  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:01.030102 15855 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:26:01.030699 15855 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:01.031687 15855 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:01.032823  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2364) took 1581 ms. Trace:
I20260709 12:26:01.035104  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:01.036039 14841 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:01.036510  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:26:01.039064  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:01.040201  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:01.051584 10643 connection.cc:256] RPC call timeout handler was delayed by -1.4342689399995834s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16710000000000003s for a response
I20260709 12:26:01.014415 10098 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:01.033352  9942 rpcz_store.cc:276] 0709 12:25:59.451298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.451678 (+   380us) service_pool.cc:224] Handling call
0709 12:25:59.452146 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.452471 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:25:59.801945 (+349474us) log.cc:844] Serialized 14 byte log entry
0709 12:25:59.929124 (+127179us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.032147 (+1103023us) spinlock_profiling.cc:243] Waited 36.7 ms on lock 0x7b18003d4200. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:26:01.032338 (+   191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.032411 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:01.032484 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.032579 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.032748 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28498,"prepare.queue_time_us":5615,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":29281,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":33429,"spinlock_wait_cycles":106324608,"thread_start_us":470910,"threads_started":2,"wal-append.queue_time_us":557996}
I20260709 12:26:01.055517 15765 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:01.056454  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.06221e+06us
I20260709 12:26:01.056753  9990 leader_election.cc:400] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15.
I20260709 12:26:01.057060 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.057968  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:01.058677 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.058903  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:01.041430 15942 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:01.062855 15981 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:26:01.063529 15829 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.066431  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2376) took 1366 ms. Trace:
I20260709 12:26:01.067183  9942 rpcz_store.cc:276] 0709 12:25:59.699547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.065497 (+1365950us) service_pool.cc:224] Handling call
0709 12:26:01.065797 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.065994 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.066032 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.066092 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:01.066137 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.066209 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.066360 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:01.069430 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 227200us
I20260709 12:26:01.001967 15879 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:01.007802 15859 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:01.072417 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.072597 10668 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:01.073367 14841 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:01.074234  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:01.077020 14841 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:01.077383 14841 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.077710 14841 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.080155  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:01.064167 15829 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.038201 15366 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:01.086796  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.070345 15966 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:01.089689 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2282) took 1881 ms. Trace:
I20260709 12:26:01.090792 15908 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:01.097960 10099 rpcz_store.cc:276] 0709 12:25:59.208373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.350762 (+142389us) service_pool.cc:224] Handling call
0709 12:25:59.352630 (+  1868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:25:59.575266 (+222636us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:25:59.575293 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:25:59.575758 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:25:59.831139 (+255381us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:25:59.831195 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.081807 (+1250612us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54002fc954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:01.081856 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.081905 (+    49us) raft_consensus.cc:1685] finished
0709 12:26:01.081920 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.089612 (+  7692us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6541,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":511189248,"thread_start_us":255027,"threads_started":1,"wal-append.queue_time_us":391164}
I20260709 12:26:01.097429 15966 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:01.104615 15966 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:01.107009 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:01.109174 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:01.097658  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.120748  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.121415 10097 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:01.121694  9942 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:01.095414 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.122529 10673 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 10.
I20260709 12:26:01.096225 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.018505 15857 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.112982  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2375) took 1413 ms. Trace:
W20260709 12:26:01.115355 15986 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:01.099263  9943 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:01.119544 10099 raft_consensus.cc:2368] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:26:01.137986 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.142087 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:01.149125 10097 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.151078 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.154430 10669 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:26:01.145795  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.167658  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:01.140821  9944 rpcz_store.cc:276] 0709 12:25:59.699233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.943003 (+1243770us) service_pool.cc:224] Handling call
0709 12:26:00.943259 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.943446 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:00.993573 (+ 50127us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.104642 (+111069us) log.cc:844] Serialized 14 byte log entry
0709 12:26:01.112626 (+  7984us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.112679 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:01.112725 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.112810 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.112928 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33553,"prepare.queue_time_us":150333,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":71,"raft.queue_time_us":39684,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":156,"thread_start_us":115982,"threads_started":2,"wal-append.queue_time_us":107324}
I20260709 12:26:01.146894 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:01.146984 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.078642 15551 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:01.168761 10675 raft_consensus.cc:2376] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 17. Re-sending same reply.
I20260709 12:26:01.168871  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.146534 14841 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:01.169972  9944 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:01.170778 10097 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:01.170930  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.172219 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:01.174402 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.211542  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:01.176261  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2153) took 2163 ms (client timeout 2101 ms). Trace:
I20260709 12:26:01.174479  9914 leader_election.cc:400] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17.
I20260709 12:26:01.189828  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.186937 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:01.213758  9942 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:01.078500 15842 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:26:01.211542 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.219651 10674 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:01.215392  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.181926 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:01.213049  9914 leader_election.cc:403] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:01.119572 15945 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:01.221390  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.222311 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:01.212488  9943 rpcz_store.cc:269] 0709 12:25:58.985159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.985321 (+   162us) service_pool.cc:224] Handling call
0709 12:26:00.913850 (+1928529us) spinlock_profiling.cc:243] Waited 3.88 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:26:01.148119 (+234269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8933248}
I20260709 12:26:01.241223 10099 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:01.222846 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:01.094879 15973 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:01.222980 15945 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.225186 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:01.247061 10095 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:01.247020  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:01.099335 15879 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.116s
I20260709 12:26:01.041864 15742 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.231266 15953 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:01.232748  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58401.3us
I20260709 12:26:01.233486 15366 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17.
I20260709 12:26:01.235018 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.017711 15970 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:01.259958 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 172272us
I20260709 12:26:01.260092 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:01.260439 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.260466 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 964) took 3384 ms. Trace:
I20260709 12:26:01.261277 10095 rpcz_store.cc:276] 0709 12:25:57.864718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:57.866018 (+  1300us) service_pool.cc:224] Handling call
0709 12:25:57.904097 (+ 38079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:58.569105 (+665008us) spinlock_profiling.cc:243] Waited 22.9 ms on lock 0x7b54001d77a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:26:00.915999 (+2346894us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b580051e3d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:26:01.249128 (+333129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.249481 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519894144}
I20260709 12:26:01.118387 15861 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:01.235769 15944 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:01.276188  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2163) took 1984 ms (client timeout 1953 ms). Trace:
W20260709 12:26:01.276855  9941 rpcz_store.cc:269] 0709 12:25:59.291355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.299885 (+  8530us) service_pool.cc:224] Handling call
0709 12:26:01.275917 (+1976032us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b5400096354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.276087 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":564251008}
I20260709 12:26:01.278407  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.280345  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.242393  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.282337  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.258571  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.219746 15772 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:01.073004 15765 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.213034 15859 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:01.271293 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.082098 15872 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:01.260056 15771 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:26:01.244474 15551 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:01.262379 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.282648  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:01.283954  9941 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:26:01.299561 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3522) took 1909 ms. Trace:
I20260709 12:26:01.246075 15370 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:01.306900 15772 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:01.307619 15772 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.308089 15772 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.290182  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:01.291774  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 138676us
I20260709 12:26:01.309962 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:01.311443  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:01.293298  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.251595 15973 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:01.251693  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.313115  9942 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:01.300467  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.314998  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.300122 10094 rpcz_store.cc:276] 0709 12:25:59.390358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.674810 (+1284452us) service_pool.cc:224] Handling call
0709 12:26:00.675203 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.675490 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.675527 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:00.683165 (+  7638us) log.cc:844] Serialized 26 byte log entry
0709 12:26:00.957743 (+274578us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:00.957777 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:00.957912 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.299247 (+341335us) raft_consensus.cc:1685] finished
0709 12:26:01.299277 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.299479 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":136531,"threads_started":1,"wal-append.queue_time_us":269790}
I20260709 12:26:01.301630 15551 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.314324  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.325886  9942 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:01.326929  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.328092  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.315006  9941 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:01.330698  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.331804  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:01.332248 10094 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:01.324023 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.332369  9941 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.329116  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:01.298934 15366 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:01.343437  9942 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:01.337633 15937 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:01.345145 15937 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:01.345610 15937 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:01.346973  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53725.1us
I20260709 12:26:01.348052  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.349325  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.298657 15859 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:01.356935 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.356756  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1223) took 1744 ms. Trace:
I20260709 12:26:01.357864  9940 rpcz_store.cc:276] 0709 12:25:59.612284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.748946 (+1136662us) service_pool.cc:224] Handling call
0709 12:26:00.816832 (+ 67886us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.102214 (+285382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.102257 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.102691 (+   434us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.215715 (+113024us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:01.215761 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.215825 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.356508 (+140683us) raft_consensus.cc:1685] finished
0709 12:26:01.356535 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.356683 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":285020,"raft.queue_time_us":286449,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":64,"thread_start_us":112596,"threads_started":1,"wal-append.queue_time_us":169210}
I20260709 12:26:01.359695  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.360199 15900 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.360939  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.362067  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:01.356756 15992 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:01.364066  9940 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:01.365542  9940 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:01.366739  9940 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:01.367941  9940 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:01.372819  9942 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:01.383121 10018 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:01.383750 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.384368  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54606.3us
I20260709 12:26:01.377164  9941 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:01.385515  9940 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:01.316154  9944 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
W20260709 12:26:01.379945 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3320) took 3066 ms (client timeout 2022 ms). Trace:
I20260709 12:26:01.292376 15954 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:01.386447 10023 rpcz_store.cc:269] 0709 12:25:58.313535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.313664 (+   129us) service_pool.cc:224] Handling call
0709 12:26:01.379632 (+3065968us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54001f8e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.379844 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2521944832}
I20260709 12:26:01.388253 15992 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:01.388152 15879 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:01.388990 15366 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:01.389187 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.390908 10025 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:01.393950  9944 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:01.394459 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2153) took 2787 ms. Trace:
I20260709 12:26:01.394834 10025 rpcz_store.cc:276] 0709 12:25:58.607017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.403341 (+796324us) service_pool.cc:224] Handling call
0709 12:26:01.390357 (+1987016us) spinlock_profiling.cc:243] Waited 51 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:26:01.390557 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.394211 (+  3654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.394404 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117278464}
W20260709 12:26:01.393988  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 226412us
I20260709 12:26:01.299880 15901 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.398262  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:01.405582 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.503664  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:01.504649  9944 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:26:01.505862  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 98242.9us
I20260709 12:26:01.508496  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:01.510243 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 978) took 2825 ms (client timeout 1929 ms). Trace:
W20260709 12:26:01.510977 10092 rpcz_store.cc:269] 0709 12:25:58.684403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.687194 (+  2791us) service_pool.cc:224] Handling call
0709 12:26:01.509200 (+2822006us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b540014a0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.510126 (+   926us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256536832}
I20260709 12:26:01.390584 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.513257  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:01.515008  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:01.321612 15758 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.516714  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:01.328589 15973 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:01.522318 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.531320 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.532308  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:01.521793  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.519982  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.540325 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:01.544445  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2371) took 2078 ms (client timeout 2663 ms). Trace:
W20260709 12:26:01.545023  9948 rpcz_store.cc:269] 0709 12:25:59.465926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.714979 (+249053us) service_pool.cc:224] Handling call
0709 12:26:01.544206 (+1829227us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000b6154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.544365 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2663593728}
I20260709 12:26:01.385070 15872 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.547260 10023 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:01.548302 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:01.549211 10023 raft_consensus.cc:2376] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 17. Re-sending same reply.
I20260709 12:26:01.550546 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.551433 10023 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:01.552639 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:01.554100 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:01.560499  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 345674us
I20260709 12:26:01.561884  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:01.507920 15869 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:01.293459 15944 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.390879 15992 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.517302 15370 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:01.574529  9940 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.598011 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.536396 15354 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:01.517419 15965 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:01.531710 15861 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:01.554896 10023 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:01.596921 15942 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:01.605576  9944 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:26:01.513032 15577 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.084s
I20260709 12:26:01.613173 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.615121  9943 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:01.617000  9944 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:01.601975 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 362350us
I20260709 12:26:01.627030 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:01.629912 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:01.617859 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 228759us
I20260709 12:26:01.579020 15672 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:01.522837 15901 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.596009 15992 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:01.541781 15758 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.516855 15949 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.641490 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:01.565007 15774 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:26:01.611227 15863 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:01.613214 15829 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:01.642292 10023 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:01.611977 15354 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:01.613062 15861 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:01.623915 15787 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:01.683775 10671 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:01.639156 15857 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:01.644245 15965 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 12:26:01.651492 15953 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:01.683837 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2278) took 2481 ms (client timeout 1509 ms). Trace:
W20260709 12:26:01.688078 10093 rpcz_store.cc:269] 0709 12:25:59.202058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.241319 (+ 39261us) service_pool.cc:224] Handling call
0709 12:26:01.683639 (+2442320us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54003ac5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.683768 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621772544}
I20260709 12:26:01.651274 15742 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:01.620883 15972 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:26:01.645457 16000 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:01.672144 15354 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:01.672343 15861 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:01.625590 15942 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.632995 15999 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.076s
W20260709 12:26:01.698083 15964 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.745s	user 0.007s	sys 0.005s
I20260709 12:26:01.633682 15366 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:01.699713 15900 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:01.705368 15765 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:01.692102 15937 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.716804 15863 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:01.727248 15863 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.717372 15659 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:01.737265 15659 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:01.738494 15659 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:01.739307 15949 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:01.706737 16000 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:01.712718 15859 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:01.707289 16009 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:01.740339 15659 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:01.971158 15879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.016s	sys 0.054s
I20260709 12:26:01.974738 10093 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:01.975337 10093 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:01.976044 15758 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:01.977293 10093 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:01.980505 10098 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:01.980873 15737 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.190s	user 0.013s	sys 0.003s
W20260709 12:26:01.893496  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 373348us
I20260709 12:26:01.981492 15737 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:01.900261  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 360506us
W20260709 12:26:01.910877  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.013s	sys 0.029s
W20260709 12:26:01.984045  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.014s	sys 0.030s
I20260709 12:26:01.985199  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:01.931514 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3344) took 1111 ms. Trace:
I20260709 12:26:01.987110 10023 rpcz_store.cc:276] 0709 12:26:00.819939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.612712 (+792773us) service_pool.cc:224] Handling call
0709 12:26:01.931145 (+318433us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:01.988790 15551 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:01.931715 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2152) took 3325 ms. Trace:
I20260709 12:26:01.989611 15551 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:01.989753 10018 rpcz_store.cc:276] 0709 12:25:58.605995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.279060 (+673065us) service_pool.cc:224] Handling call
0709 12:26:01.379790 (+2100730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.383633 (+  3843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.931163 (+547530us) spinlock_profiling.cc:243] Waited 2.09 s on lock 0x7b54001f8e50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.931486 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4816660224}
I20260709 12:26:01.660830 10025 raft_consensus.cc:2410] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:26:01.936265 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.937016 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.939913 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1032) took 1769 ms. Trace:
I20260709 12:26:01.940300 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.941075 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.941978 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2143) took 3606 ms. Trace:
I20260709 12:26:01.942061 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:01.962428 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.963487 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:01.965027 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.965386 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3525) took 2567 ms. Trace:
I20260709 12:26:01.967198 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:01.967607 10091 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 983) took 3281 ms (client timeout 4293 ms). Trace:
I20260709 12:26:01.967284 15370 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:01.989037 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:01.980499 10093 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:01.831672 15942 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:02.071318 10016 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:02.072037 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1035) took 1052 ms. Trace:
I20260709 12:26:02.072412 10016 rpcz_store.cc:276] 0709 12:26:01.019613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.933819 (+914206us) service_pool.cc:224] Handling call
0709 12:26:02.071955 (+138136us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.073475  9914 leader_election.cc:400] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:26:02.073895  9914 leader_election.cc:403] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:26:01.780182 15944 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:02.074622  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56988.1us
I20260709 12:26:02.075460 15944 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 10. Current term is 11.
I20260709 12:26:01.853889 15945 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:02.076153  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:02.078122 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1275) took 2702 ms. Trace:
I20260709 12:26:01.716467 15964 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.078645 10672 rpcz_store.cc:276] 0709 12:25:59.375891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.376051 (+   160us) service_pool.cc:224] Handling call
0709 12:25:59.376399 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:25:59.555238 (+178839us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.077439 (+2522201us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54005b95d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:02.077882 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363260160}
W20260709 12:26:02.079417 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.080169 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.081000 15964 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:01.777712 15966 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.673s	user 0.002s	sys 0.011s
I20260709 12:26:02.082629 15966 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.074004 10016 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:02.085655 15966 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:01.846264 15969 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:02.086261 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:01.849874 15973 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:01.855644 15985 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.875s	user 0.000s	sys 0.004s
I20260709 12:26:01.857923 15992 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.091553 15904 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:02.092106 15904 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:02.093590 15992 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:02.049913 15551 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.093458 15904 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.095293 10676 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:01.851686 15995 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.679s	user 0.000s	sys 0.017s
I20260709 12:26:01.763144 16000 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:01.859580 16001 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.149s	user 0.000s	sys 0.002s
I20260709 12:26:02.114953 15973 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:02.115053  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2366) took 2650 ms. Trace:
I20260709 12:26:02.115746  9949 rpcz_store.cc:276] 0709 12:25:59.464299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.209397 (+745098us) service_pool.cc:224] Handling call
0709 12:26:00.209788 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.266710 (+1056922us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.266740 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.267233 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 12:26:01.567464 (+300231us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:01.567516 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.567613 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.859064 (+291451us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:02.114480 (+255416us) raft_consensus.cc:1685] finished
0709 12:26:02.114522 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.114844 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":954606,"raft.queue_time_us":888030,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":20967,"spinlock_wait_cycles":6235520,"thread_start_us":480125,"threads_started":2,"wal-append.queue_time_us":416751}
I20260709 12:26:01.840504 16009 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:01.853811 16007 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:26:01.784870 16016 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 12:26:02.009172 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 356828us
I20260709 12:26:01.970386 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2305) took 1120 ms. Trace:
I20260709 12:26:02.122759 10094 rpcz_store.cc:276] 0709 12:26:00.849510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.333814 (+484304us) service_pool.cc:224] Handling call
0709 12:26:01.509374 (+175560us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.509655 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.509697 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:01.509762 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:01.509819 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.509902 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.969768 (+459866us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540014a0d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.970071 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402812288}
I20260709 12:26:01.992254 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:02.125303 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3355) took 1007 ms. Trace:
I20260709 12:26:02.125453 15762 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.125705 10018 rpcz_store.cc:276] 0709 12:26:01.117923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.991520 (+873597us) service_pool.cc:224] Handling call
0709 12:26:02.125224 (+133704us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.126320 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1234) took 1142 ms. Trace:
I20260709 12:26:02.021368 10019 rpcz_store.cc:276] 0709 12:26:00.170188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.718520 (+548332us) service_pool.cc:224] Handling call
0709 12:26:00.722409 (+  3889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:00.853142 (+130733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.939102 (+1085960us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54001f7a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:01.939673 (+   571us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234288896}
I20260709 12:26:02.126763 10676 rpcz_store.cc:276] 0709 12:26:00.984430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.989196 (+  4766us) service_pool.cc:224] Handling call
0709 12:26:01.010116 (+ 20920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.110042 (+1099926us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.125936 (+ 15894us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400490a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:02.126234 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2493865728}
I20260709 12:26:02.023622 10090 rpcz_store.cc:276] 0709 12:25:59.397628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.728875 (+1331247us) service_pool.cc:224] Handling call
0709 12:26:00.729273 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.873627 (+144354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:00.873682 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:00.873747 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:00.873794 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.964608 (+1090814us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540022ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:01.964709 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:01.965065 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331373824}
W20260709 12:26:02.024140 10091 rpcz_store.cc:269] 0709 12:25:58.685961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.686502 (+   541us) service_pool.cc:224] Handling call
0709 12:26:01.967048 (+3280546us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b5400094054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:01.967293 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":653800320}
I20260709 12:26:02.022482 10021 rpcz_store.cc:276] 0709 12:25:58.335109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.659729 (+324620us) service_pool.cc:224] Handling call
0709 12:25:58.660037 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:00.948641 (+2288604us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.941350 (+992709us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b540038d6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:01.941742 (+   392us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5241296640}
I20260709 12:26:02.130216 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1039) took 1044 ms. Trace:
I20260709 12:26:02.130725 10021 rpcz_store.cc:276] 0709 12:26:01.085940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.128925 (+1042985us) service_pool.cc:224] Handling call
0709 12:26:02.129359 (+   434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.129766 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.129806 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.129871 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:02.129912 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.129977 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.130126 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.034722 10017 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:02.134474 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3337) took 1316 ms. Trace:
W20260709 12:26:02.037740 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.039566 10092 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:02.040475 10024 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:02.040764  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:02.042719  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.134958 10017 rpcz_store.cc:276] 0709 12:26:00.817869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.936444 (+1118575us) service_pool.cc:224] Handling call
0709 12:26:02.134390 (+197946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.043821 15370 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:02.044943 10023 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 12:26:02.135625  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59713us
I20260709 12:26:02.048712 14841 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:02.051218 10020 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:02.056164 15586 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:02.060120 15672 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:02.068619  9940 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:02.070127 15901 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:01.844400 15996 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.671s	user 0.007s	sys 0.026s
I20260709 12:26:02.105053 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2155) took 1274 ms. Trace:
I20260709 12:26:02.112609  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1213) took 2714 ms. Trace:
I20260709 12:26:02.120528 16009 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.127794 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1231) took 1209 ms. Trace:
I20260709 12:26:02.043207 10670 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:26:02.139112 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3348) took 1045 ms. Trace:
I20260709 12:26:02.047185 15737 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:02.130462 10643 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:02.107048 15904 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.156327 10024 rpcz_store.cc:276] 0709 12:26:01.093902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.939713 (+845811us) service_pool.cc:224] Handling call
0709 12:26:02.139035 (+199322us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.143151 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2159) took 1307 ms. Trace:
I20260709 12:26:02.168917 10023 rpcz_store.cc:276] 0709 12:26:00.835764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.988600 (+1152836us) service_pool.cc:224] Handling call
0709 12:26:02.143083 (+154483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.155759 10671 rpcz_store.cc:276] 0709 12:26:00.917951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.918072 (+   121us) service_pool.cc:224] Handling call
0709 12:26:00.918389 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.684649 (+766260us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.127334 (+442685us) spinlock_profiling.cc:243] Waited 765 ms on lock 0x7b5400723654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:02.127612 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1759523072}
I20260709 12:26:02.144572 15672 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:02.153321  9947 rpcz_store.cc:276] 0709 12:25:59.389397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:59.389531 (+   134us) service_pool.cc:224] Handling call
0709 12:26:00.739092 (+1349561us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:26:00.739309 (+   217us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:01.101020 (+361711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.101051 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:01.101656 (+   605us) log.cc:844] Serialized 52 byte log entry
0709 12:26:01.173529 (+ 71873us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:01.173573 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:01.173641 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:01.851082 (+677441us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:02.103880 (+252798us) raft_consensus.cc:1685] finished
0709 12:26:02.103935 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.104220 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":361296,"raft.queue_time_us":363643,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"spinlock_wait_cycles":107137664,"thread_start_us":71504,"threads_started":1,"wal-append.queue_time_us":70864}
I20260709 12:26:02.153317 10016 rpcz_store.cc:276] 0709 12:26:00.825467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.073223 (+1247756us) service_pool.cc:224] Handling call
0709 12:26:02.073719 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.093146 (+ 19427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.100124 (+  6978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.154867 10098 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
W20260709 12:26:02.156911  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 558760us
W20260709 12:26:02.157696  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 122718us
I20260709 12:26:02.158764 15869 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.189496 15869 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:02.190057  9943 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.568s	user 0.017s	sys 0.006s
I20260709 12:26:02.190532  9943 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:02.191098 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1223) took 1720 ms. Trace:
I20260709 12:26:02.191541 10670 rpcz_store.cc:276] 0709 12:26:00.460289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.460402 (+   113us) service_pool.cc:224] Handling call
0709 12:26:01.966751 (+1506349us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431eee 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:26:02.181110 (+214359us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3668864}
I20260709 12:26:02.192533 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2297) took 1419 ms. Trace:
I20260709 12:26:02.192986 10095 rpcz_store.cc:276] 0709 12:26:00.772610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.262217 (+489607us) service_pool.cc:224] Handling call
0709 12:26:01.262532 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.262817 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:01.681184 (+418367us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.191683 (+510499us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.191907 (+   224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.191974 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:02.192067 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.192151 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.192339 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":459913,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":7257,"thread_start_us":816911,"threads_started":2,"wal-append.queue_time_us":410022}
I20260709 12:26:02.158418 15789 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:02.155896 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3340) took 1323 ms. Trace:
I20260709 12:26:02.141307 15863 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:02.185421 10098 pending_rounds.cc:85] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 11
I20260709 12:26:02.195477 15737 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:02.195966 10098 pending_rounds.cc:107] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 12:26:02.196280 10098 raft_consensus.cc:2889] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:02.197165 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2296) took 1424 ms. Trace:
I20260709 12:26:02.197212  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:02.197551 10098 rpcz_store.cc:276] 0709 12:26:00.772340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.002464 (+230124us) service_pool.cc:224] Handling call
0709 12:26:01.012749 (+ 10285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.979752 (+967003us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b58006551d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:26:02.196896 (+217144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.197112 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34323328}
I20260709 12:26:02.190605 15846 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:02.198688 15846 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:02.198750  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:02.200410 15846 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:02.201045 15772 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.892s	user 0.014s	sys 0.014s
I20260709 12:26:02.201539 15772 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.195147 10020 rpcz_store.cc:276] 0709 12:26:00.818815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.050079 (+1231264us) service_pool.cc:224] Handling call
0709 12:26:02.050656 (+   577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.140079 (+ 89423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.142023 (+  1944us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:02.195498 15863 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:02.203423 15863 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.203516 15370 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.205211  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:02.206188 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2309) took 1054 ms. Trace:
I20260709 12:26:02.206758  9949 raft_consensus.cc:2368] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:02.207056  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:02.209378  9942 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 13. (term mismatch)
W20260709 12:26:02.210664 15841 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:26:02.210532 14841 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:02.202092 15846 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.214293  9942 pending_rounds.cc:85] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 12
W20260709 12:26:02.184402  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.006s	sys 0.020s
I20260709 12:26:02.217556 10093 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:26:02.220567  9942 pending_rounds.cc:107] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 14.13
I20260709 12:26:02.221194  9942 raft_consensus.cc:2889] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:02.226876  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.006s	sys 0.021s
I20260709 12:26:02.235466  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:02.241694  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63176.3us
I20260709 12:26:02.244827 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2308) took 1161 ms. Trace:
I20260709 12:26:02.213552 10096 rpcz_store.cc:276] 0709 12:26:01.083620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.978686 (+895066us) service_pool.cc:224] Handling call
0709 12:26:01.979543 (+   857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.980290 (+   747us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:01.980872 (+   582us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.106476 (+125604us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.106705 (+   229us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.106755 (+    50us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:02.132815 (+ 26060us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b18001ee140. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:02.137860 (+  5045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.137942 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.138067 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":31860,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":3489,"raft.queue_time_us":67602,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":2810,"spinlock_wait_cycles":3183360,"thread_start_us":53237,"threads_started":1}
I20260709 12:26:02.245782 10093 rpcz_store.cc:276] 0709 12:26:01.082977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.689332 (+606355us) service_pool.cc:224] Handling call
0709 12:26:01.692761 (+  3429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:01.974389 (+281628us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b540023c420. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:26:02.220931 (+246542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.244753 (+ 23822us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643201920}
I20260709 12:26:02.254761 10019 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:02.257153 15829 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:26:02.257748 15829 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:02.257979 15842 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:26:02.258169 15829 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:02.258886 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2311) took 1075 ms. Trace:
I20260709 12:26:02.259297 10099 rpcz_store.cc:276] 0709 12:26:01.141348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.983473 (+842125us) service_pool.cc:224] Handling call
0709 12:26:01.984016 (+   543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:01.984859 (+   843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:01.984910 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:01.985689 (+   779us) log.cc:844] Serialized 26 byte log entry
0709 12:26:02.016468 (+ 30779us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:02.016535 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.016639 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.213219 (+196580us) raft_consensus.cc:1685] finished
0709 12:26:02.213242 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.216733 (+  3491us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32516,"raft.queue_time_us":8213,"raft.run_cpu_time_us":426,"raft.run_wall_time_us":29253,"thread_start_us":30204,"threads_started":1,"wal-append.queue_time_us":225196}
W20260709 12:26:02.262737 10062 connection.cc:256] RPC call timeout handler was delayed by -2.3357880919998024s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1936s for a response
W20260709 12:26:02.263368 10062 connection.cc:256] RPC call timeout handler was delayed by -2.3330880919997981s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
I20260709 12:26:02.264299 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2306) took 1402 ms. Trace:
W20260709 12:26:02.265239 10062 connection.cc:256] RPC call timeout handler was delayed by -2.1872880920000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
W20260709 12:26:02.265950 10062 connection.cc:256] RPC call timeout handler was delayed by -1.9403209740003149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21680000000000002s for a response
I20260709 12:26:02.266741 15366 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:02.264835 10092 rpcz_store.cc:276] 0709 12:26:00.861627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.512935 (+651308us) service_pool.cc:224] Handling call
0709 12:26:02.024966 (+512031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.135935 (+110969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.264005 (+128070us) spinlock_profiling.cc:243] Waited 479 ms on lock 0x7b5400094050. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:02.264228 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1102431616}
I20260709 12:26:02.270968 15868 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:02.274423 15366 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:02.274783 15937 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:02.275430 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 145186us
I20260709 12:26:02.275929 15366 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.277700 15737 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:02.278242 10095 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:26:02.277505 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:02.257347 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.265290  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 703453us
I20260709 12:26:02.294024 10018 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: []
I20260709 12:26:02.300673 15970 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.310523  9946 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:26:02.311451 15771 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.311952  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1220) took 2700 ms. Trace:
I20260709 12:26:02.312206 15771 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.312427  9946 rpcz_store.cc:276] 0709 12:25:59.611281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.089219 (+477938us) service_pool.cc:224] Handling call
0709 12:26:02.311662 (+2222443us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:02.311856 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5105176448}
I20260709 12:26:02.313292 15893 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.313941 15893 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.315431 15771 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:02.320684 10062 connection.cc:256] RPC call timeout handler was delayed by -1.6489105739997285s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19530000000000003s for a response
W20260709 12:26:02.321302 10062 connection.cc:256] RPC call timeout handler was delayed by -1.340342585000144s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27490000000000003s for a response
I20260709 12:26:02.255811 15901 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:02.284461 15737 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:02.318475 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.296989 15937 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:02.311980 15796 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.339938 15737 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:02.367488  9948 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:26:02.377034  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:02.366654 10065 leader_election.cc:400] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:02.378911 10065 leader_election.cc:403] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:02.318392 15758 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:02.375387  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:02.395547 16020 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.396410 16020 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.395462  9945 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:26:02.405679  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 113482us
I20260709 12:26:02.402872  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:02.409162 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:02.424077 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.429816 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:02.435343 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 157972us
I20260709 12:26:02.451884 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1040) took 1334 ms. Trace:
W20260709 12:26:02.454476 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.455288  9942 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:02.456748  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:02.458947 10670 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:26:02.451143 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 94466.5us
I20260709 12:26:02.454385 15904 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:02.463963  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:02.468904  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:02.471242  9940 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:02.471153 10098 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:02.395411 15863 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.470563 15965 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.452332 10021 rpcz_store.cc:276] 0709 12:26:01.097096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.132150 (+1035054us) service_pool.cc:224] Handling call
0709 12:26:02.132768 (+   618us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.133230 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:02.222453 (+ 89223us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.425410 (+202957us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.430987 (+  5577us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.431070 (+    83us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:02.431139 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.431229 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.431405 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":166121,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":16946,"thread_start_us":253060,"threads_started":2}
I20260709 12:26:02.450706 15689 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:02.404631 15937 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.477839 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:02.432278 15772 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:02.419800 15901 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.412156 15796 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.504011 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:02.505005  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:02.506300  9944 raft_consensus.cc:2368] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:02.511243 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:02.516888 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1041) took 1364 ms. Trace:
I20260709 12:26:02.523937 10095 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:26:02.525928 16027 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:26:02.526374 16016 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.475368 15758 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.529081 15758 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:02.533246 10676 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:02.534718 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:02.536254 10674 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:02.536522 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.537389 15737 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.538630 10096 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:02.538861 15737 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.540915 15973 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:02.541332 15973 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:02.518428 16020 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:02.542384 15973 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.540503 16013 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:02.543920 15973 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:02.544054 15861 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.831s	user 0.016s	sys 0.003s
I20260709 12:26:02.542726 16014 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:02.544636 15861 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.534802 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:02.546865  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1225) took 1440 ms. Trace:
I20260709 12:26:02.547334  9943 rpcz_store.cc:276] 0709 12:26:01.105939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.282286 (+176347us) service_pool.cc:224] Handling call
0709 12:26:02.546611 (+1264325us) spinlock_profiling.cc:243] Waited 310 ms on lock 0x7b540009c4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:02.546780 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":713450752}
W20260709 12:26:02.548161 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 58353.8us
I20260709 12:26:02.548831  9941 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:02.549868 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:02.550164  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2179) took 1288 ms. Trace:
I20260709 12:26:02.551856 15689 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:02.552380 15689 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:02.552806 15689 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:02.508214 15868 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:02.547443 10669 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
W20260709 12:26:02.568810  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 178234us
I20260709 12:26:02.570359 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:02.557461 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47126.3us
I20260709 12:26:02.550530  9941 rpcz_store.cc:276] 0709 12:26:01.261693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.394770 (+133077us) service_pool.cc:224] Handling call
0709 12:26:02.548543 (+1153773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.549266 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.549984 (+   718us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540009c4d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:02.550109 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2652312960}
I20260709 12:26:02.518709 10017 rpcz_store.cc:276] 0709 12:26:01.152262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.141591 (+989329us) service_pool.cc:224] Handling call
0709 12:26:02.203823 (+ 62232us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.204366 (+   543us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:02.281442 (+ 77076us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.498000 (+216558us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.510547 (+ 12547us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:02.510616 (+    69us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:02.516508 (+  5892us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.516601 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.516828 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9417,"prepare.queue_time_us":95130,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":2236,"raft.queue_time_us":15058,"thread_start_us":248753,"threads_started":2}
I20260709 12:26:02.556844 15586 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.592609 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:02.608475 10674 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:26:02.600551 15861 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:02.602558 15967 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:02.603186  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 139319us
W20260709 12:26:02.607069 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 241212us
W20260709 12:26:02.617164 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.617914 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.618453 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.621852 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.606757 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:02.571097 15868 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:02.637790 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.641225 10099 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:02.636833  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:02.644526 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 127762us
I20260709 12:26:02.643187  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:02.654186 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.631624 15868 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:02.659572 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.645643 16020 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:02.615377 15771 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:26:02.691068 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:02.701536  9940 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:02.692616  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:02.645650 15965 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:02.711205  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64507us
I20260709 12:26:02.711674  9943 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:02.712411 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:02.710958 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:02.718531 10020 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:02.636243 15586 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.719453 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:02.722808 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:02.724718  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.725234 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 67135.1us
I20260709 12:26:02.728760  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:02.730789 16040 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 12:26:02.731276 16035 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:02.733330 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2310) took 1634 ms. Trace:
I20260709 12:26:02.733723 10097 rpcz_store.cc:276] 0709 12:26:01.099144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.982051 (+882907us) service_pool.cc:224] Handling call
0709 12:26:02.011395 (+ 29344us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:02.102639 (+ 91244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.102679 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:02.103677 (+   998us) log.cc:844] Serialized 52 byte log entry
0709 12:26:02.467832 (+364155us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:02.503425 (+ 35593us) log.cc:844] Serialized 14 byte log entry
0709 12:26:02.504972 (+  1547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.505047 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:02.732959 (+227912us) raft_consensus.cc:1685] finished
0709 12:26:02.732982 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:02.733147 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45792,"prepare.queue_time_us":33678,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4624,"raft.queue_time_us":48170,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":3769,"thread_start_us":363620,"threads_started":1,"wal-append.queue_time_us":481508}
I20260709 12:26:02.705997 16020 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.742789 15861 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:02.721913 15354 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.851s	user 0.008s	sys 0.012s
I20260709 12:26:02.728250 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:02.654580 15689 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.704114 15970 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:02.752414 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:02.743055 16034 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:26:02.757349 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.757534 16000 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.759508 15861 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:02.765470  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 139436us
I20260709 12:26:02.765902 10097 raft_consensus.cc:2376] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17. Re-sending same reply.
I20260709 12:26:02.766996 16014 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:02.767666 15966 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:02.767845 15970 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.768548 15966 consensus_queue.cc:1243] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
W20260709 12:26:02.752043 10642 leader_election.cc:336] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.703s (SENT)
I20260709 12:26:02.743422 15758 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:26:02.766271 15861 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.759662 15965 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.770584 15970 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:02.758033 15942 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:02.772775 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56158.7us
I20260709 12:26:02.758706 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:02.770860 10024 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:02.773020 15758 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:02.773097 15689 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:02.782279 15758 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.774793 15901 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:02.772791 16023 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:26:02.783914 15758 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:26:02.775704 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.774374 15771 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:26:02.786335 10091 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:02.771390 15969 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:26:02.787186 15901 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:02.787520 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:02.787675 15901 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:02.788305 10091 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.788940 15901 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.770876 15354 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.776647 10023 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:02.776069 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.757715  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:02.783281 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:02.860356 10022 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:26:02.868988  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 176556us
W20260709 12:26:02.775404 16041 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:26:02.884543  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 126959us
I20260709 12:26:02.900543 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.899418 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:02.904024 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.905519 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:02.906847 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.907969 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:02.908617 10098 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:02.775151 15937 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:02.912808 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:02.914562 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.919957 10097 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:02.921350 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:02.922202 16023 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:02.925524 16023 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:26:02.929306 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:02.942651 15586 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:02.722500 15829 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:02.947364 10668 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 11: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:02.948534 15672 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.711s	user 0.011s	sys 0.004s
I20260709 12:26:02.949100 15672 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:02.950466  9941 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:02.929481 15354 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:02.956101 15689 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:02.957234 15969 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:02.958034 15969 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:02.958662 15901 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:02.994395 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:02.990851 15869 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.778s	user 0.014s	sys 0.000s
I20260709 12:26:02.999542 15869 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.004551 15970 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:02.950237 15829 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:02.998778 10091 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:02.958906 10098 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:02.901294 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:02.967406  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:02.973212 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 70735.7us
I20260709 12:26:03.028462 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:03.030230 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:03.030762 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:02.990782 15969 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.039772 10021 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:26:03.040217 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.042851 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.042963 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.042996 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.041357 15354 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.059706 15989 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:03.060217 15354 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:03.066008 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 168389us
I20260709 12:26:03.067206 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:03.068145 15989 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:03.070112 15989 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:03.046312 15672 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.082260 10097 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:03.083494 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:03.084261 10097 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.088956  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2374) took 3464 ms. Trace:
I20260709 12:26:03.089440  9945 rpcz_store.cc:276] 0709 12:25:59.624295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:00.754959 (+1130664us) service_pool.cc:224] Handling call
0709 12:26:02.314084 (+1559125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.399678 (+ 85594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.088207 (+688529us) spinlock_profiling.cc:243] Waited 1.96 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:03.088761 (+   554us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54000b5c50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:03.088890 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3446599936}
I20260709 12:26:03.035368 10098 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.036633 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.005330 15970 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.037979  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:03.094134 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.103585  9943 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
W20260709 12:26:03.108807 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.075969 15354 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.123286 10097 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:03.124112 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:03.124830 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:03.097734 15737 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.001s
I20260709 12:26:03.127066  9949 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 13 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:03.128010 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1036) took 2101 ms. Trace:
I20260709 12:26:03.128444 10019 rpcz_store.cc:276] 0709 12:26:01.019905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.127524 (+1107619us) service_pool.cc:224] Handling call
0709 12:26:02.127953 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.257084 (+129131us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.121260 (+864176us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54002d5d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:03.121455 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":262736128}
I20260709 12:26:03.119302 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:03.132390 10094 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:03.104014 16023 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.134783 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.139501 15964 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:03.139931 15964 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.140422 15868 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.140785 15964 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.132275 15354 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:03.141820 15973 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.597s	user 0.013s	sys 0.000s
I20260709 12:26:03.140973 15967 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.150458 15737 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.138211 15672 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:03.152135 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:03.154748 10093 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:03.155807  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 162003us
I20260709 12:26:03.156448 16023 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:03.163935 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.220775 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.166189 10016 raft_consensus.cc:2410] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:26:03.221619 16059 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.221798 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.222609 10675 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:03.222357 16059 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.223066  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.223691 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.192312 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.222407  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.226804 15762 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:03.226897 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.202773 15737 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.229296 15737 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.207337 15354 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.191124 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.230324 10668 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:03.206784 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.231266 10671 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
W20260709 12:26:03.232352 10642 leader_election.cc:336] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.929s (SENT)
I20260709 12:26:03.232823 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.232904 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:03.192999 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.195565 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:03.192188 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:03.201116 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.194888 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:40768 (request call id 2) took 2430 ms. Trace:
I20260709 12:26:03.225777 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.188216 15872 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:03.189685 15970 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.210484 16057 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.205401  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:03.223909 16052 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:03.224526 10674 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:03.243294 10097 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:03.190483 15936 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:03.210475 15366 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.166509 15967 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:03.235092 10025 rpcz_store.cc:276] 0709 12:26:00.736969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.033389 (+1296420us) service_pool.cc:224] Handling call
0709 12:26:03.166778 (+1133389us) spinlock_profiling.cc:243] Waited 46.4 ms on lock 0x7b540038dbd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955c735 00007f4e095549fa 00007f4e0e51c4c9 00007f4e07a98c54 00007f4e07a98be2 00007f4e07a98b11 00007f4e07a98a8c 00007f4e07a97d42 00007f4e0789148c 00007f4e078908c6
0709 12:26:03.166963 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":106758528}
I20260709 12:26:03.237751 15829 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:26:03.237859  9947 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:03.238900 15872 consensus_queue.cc:1243] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:26:03.239075 16023 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.249782  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.241941  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:03.242700 10098 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:03.245543 10091 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:26:03.191435 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:03.247093 14841 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:03.246887  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:03.246884 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.248883  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:03.255084  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:03.256801  9940 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:03.250494 10676 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:03.259994  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:03.262439 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:03.263084 10021 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.265252 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:03.265734 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.267203 16000 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:03.267777  9940 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:03.268842  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:03.269248 15965 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.269487  9940 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:03.267241  9943 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.273101 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:03.246148 16063 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:03.276808  9947 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:03.278132 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:03.273934 10019 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:03.261722 15992 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:03.283118 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:03.284001 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.278787 14841 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:03.261664 15366 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.282225 15992 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:03.252089 16057 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.293403 16056 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:03.294778  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:03.293359 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:03.296826 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:03.304833  9943 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:03.307878  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:03.311173 14841 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:03.306321  9943 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:03.315495  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:03.318768 16059 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.320760 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.323525 10098 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:26:03.326848  9945 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:03.332345 16000 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:03.333328 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.334039 10098 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:26:03.335494 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.337360 10098 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:03.338840  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2184) took 1160 ms. Trace:
I20260709 12:26:03.339304  9944 rpcz_store.cc:276] 0709 12:26:02.178162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.180972 (+  2810us) service_pool.cc:224] Handling call
0709 12:26:03.338620 (+1157648us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54000b7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:03.338776 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":706756480}
I20260709 12:26:03.343959 16057 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.347323 15366 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.347997 15966 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:03.348410 15966 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.348768 15966 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.352487 15893 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.297731 15992 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.358309 10094 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
W20260709 12:26:03.359211 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62472.6us
I20260709 12:26:03.273984 15936 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.361325  9942 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:03.295008 16063 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.363607 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:03.370143 10025 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 11: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:03.371181 10093 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
W20260709 12:26:03.371515  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69558.5us
I20260709 12:26:03.375500  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.376880  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:03.377507  9943 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.379004  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.379961  9942 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:03.380623 10025 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:03.380767  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.383129 16063 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.383858 15863 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.384516 16059 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:03.383857  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.385239  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.385633 16059 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:03.385998 16059 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.386474  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.388993 10095 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:03.390246 10095 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:03.391111 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3345) took 2302 ms (client timeout 2732 ms). Trace:
W20260709 12:26:03.391553 10022 rpcz_store.cc:269] 0709 12:26:01.088363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:01.940497 (+852134us) service_pool.cc:224] Handling call
0709 12:26:03.390895 (+1450398us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400216bd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:03.391022 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752294656}
I20260709 12:26:03.392900 15942 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:26:03.396027  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:03.393569  9940 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:03.392233 15872 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:03.394177 15829 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:03.394200 15868 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.386991 16059 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.397473 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:03.322258 16056 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.398720 10097 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: [12.11-15.13]
I20260709 12:26:03.399560 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:03.401729 10094 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:03.405565 10095 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:03.406929  9942 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:03.407697 10021 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:03.408327 10669 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:26:03.409674 10065 leader_election.cc:400] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:03.412894 10065 leader_election.cc:403] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:03.410593 10062 leader_election.cc:400] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:03.418632  9943 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:03.415048  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:03.419925 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:03.424854  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.426198 10097 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:03.431206 10094 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:03.409185 15758 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:26:03.435111  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.437356  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.442356  9943 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:03.439116 14841 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:03.440531 10021 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:26:03.427516 15366 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:03.410537 15970 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:26:03.434167 15829 consensus_queue.cc:1243] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 16.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:03.454790 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.455371 10019 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:03.428124 15796 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.439335 15863 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.458544  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:03.442282 15689 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:03.472383 15689 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:03.472826 15689 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.447250 15366 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:03.475569 15366 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:03.476173 16058 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:26:03.478123 10090 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:03.479511 10090 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:03.480751 10090 pending_rounds.cc:85] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 10
I20260709 12:26:03.481002 10090 pending_rounds.cc:107] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:26:03.481218 15869 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.483753 15863 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.484622 16020 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:03.486579 15737 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:26:03.487003  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55984.2us
I20260709 12:26:03.487036 15737 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.487303  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:03.487495 15354 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.487934  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:03.488765  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.488250 15737 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.490046  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.490334 15945 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:03.490768 15945 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:03.490159 15737 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.491168 15945 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.492025  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.493398  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.448719 16016 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:03.428531 15893 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:26:03.471545 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:03.460397 16059 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:03.494565  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.497788 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:03.496842 10092 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:03.481215 10090 raft_consensus.cc:2889] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:03.499014 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:03.502797 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 132031us
I20260709 12:26:03.463016 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.524062 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2156) took 2689 ms. Trace:
I20260709 12:26:03.524679 10018 rpcz_store.cc:276] 0709 12:26:00.834727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.126615 (+1291888us) service_pool.cc:224] Handling call
0709 12:26:02.141275 (+ 14660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:02.294732 (+153457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:02.649687 (+354955us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b540038dbd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:03.523820 (+874133us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b540038dbd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:03.523984 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":379076608}
W20260709 12:26:03.526232 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.547811 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.552763 10097 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:26:03.469959 16057 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:03.514096 14841 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:03.555680  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84654.4us
I20260709 12:26:03.495042 15861 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:03.492066 15863 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.558813 15861 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.496996 15904 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 12:26:03.574129 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:03.542174 15796 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:03.482347 15967 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.586962 10098 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:03.494910 15772 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:26:03.587005  9948 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:03.595479 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.561858 15861 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.544782 16009 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:26:03.585739 15354 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:26:03.617062 15965 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:03.617472 15965 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:03.618800 16021 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:26:03.619683  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 129617us
I20260709 12:26:03.603701 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:03.624848 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 127738us
W20260709 12:26:03.629076 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 62197.1us
I20260709 12:26:03.632330 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.520833 15969 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.617735 15945 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:03.643425 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.618480 15965 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.626621 16021 consensus_queue.cc:1243] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:03.626044 15796 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:03.627004 15962 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:03.627192 15904 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:03.653554 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105875us
I20260709 12:26:03.648976  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.655596 10020 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.662111 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.663679 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:03.665580  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:03.648303 16016 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.638305 16020 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:03.557224 16056 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:03.683861  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:03.684520 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 80897.8us
I20260709 12:26:03.685804 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.687202 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:26:03.689958 10095 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:03.654371 16066 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:03.696960 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.709273 10098 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:03.666338 15904 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.711833 15965 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.684201 15366 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:03.654215 16063 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:26:03.719550 15796 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.695981 16016 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.722982 15969 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.724566 15904 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:03.644826 15354 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 12:26:03.726022 16063 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:03.726423 16063 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:03.713927 15966 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.727341 16063 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.728646 16063 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.729276 15829 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:03.729784 15829 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.729861 15966 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.731097 10021 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:03.730834 15829 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:03.732342 16068 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:26:03.733281 16051 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.509s	user 0.000s	sys 0.007s
I20260709 12:26:03.733083 15829 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.665839 15936 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.685499 15872 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:03.653625 15370 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:26:03.686664 16020 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:03.658355 15689 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.745735 15964 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.752216 15942 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:03.752684 15942 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:03.760607 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72982.8us
I20260709 12:26:03.760970 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.761412 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:03.763981  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.766002  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:03.770931 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 138857us
I20260709 12:26:03.775899 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3553) took 1227 ms. Trace:
I20260709 12:26:03.779445 10096 rpcz_store.cc:276] 0709 12:26:02.548769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.548951 (+   182us) service_pool.cc:224] Handling call
0709 12:26:02.549323 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:02.583166 (+ 33843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.583208 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:02.628520 (+ 45312us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.075516 (+446996us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:03.075571 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.075665 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.732718 (+657053us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:03.763679 (+ 30961us) raft_consensus.cc:1685] finished
0709 12:26:03.763696 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.775808 (+ 12112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49825,"raft.queue_time_us":44590,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":25361,"spinlock_wait_cycles":28132096,"thread_start_us":382264,"threads_started":1,"wal-append.queue_time_us":570604}
I20260709 12:26:03.716266 16066 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.795284 16066 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.795899 15689 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:03.784489 16081 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:03.787746 15945 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:03.781530 15962 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.797801 15962 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.772727 15872 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.791307  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:03.801301  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:03.802314  9943 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:03.803768 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54995us
I20260709 12:26:03.763206 16072 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:03.781474 15366 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:03.812335  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2187) took 1564 ms. Trace:
I20260709 12:26:03.812484 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:03.774804 16079 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:03.817909  9949 rpcz_store.cc:276] 0709 12:26:02.247659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.256292 (+  8633us) service_pool.cc:224] Handling call
0709 12:26:02.289448 (+ 33156us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:03.132733 (+843285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.812063 (+679330us) spinlock_profiling.cc:243] Waited 837 ms on lock 0x7b5400206f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:03.812269 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1925783424}
I20260709 12:26:03.799846 15872 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.811915 16066 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:03.834141 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:03.814499 16072 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:03.815249 15967 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:03.839535  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85214us
I20260709 12:26:03.842653 15771 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:03.819366 16078 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:03.847476 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 150626us
W20260709 12:26:03.845449  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54421.8us
I20260709 12:26:03.850091 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.851778  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:03.844398 16072 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:03.846483 15771 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:03.870640  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:03.873059 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:03.874943 15771 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:03.878358 15859 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:03.879077 16074 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:26:03.879004 15859 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.871018 15992 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.881814 15859 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.883347 15992 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:03.886744 15945 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:03.890082  9940 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.910211 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:03.911192 10095 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 11.
I20260709 12:26:03.914251 10095 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:03.915514 10095 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:03.915594 16072 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.917842 10095 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:03.896663 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:03.889108 16066 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:26:03.929145  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 62699.2us
I20260709 12:26:03.893813 15758 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:03.933010 14841 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
W20260709 12:26:03.943112  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 94063.4us
I20260709 12:26:03.888865 15965 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:26:03.926442 16017 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:03.917842 16072 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:03.956736 15942 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.957741 15942 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:03.967192 16055 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.779s	user 0.003s	sys 0.000s
I20260709 12:26:03.968905  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2398) took 1610 ms. Trace:
I20260709 12:26:03.969285  9946 rpcz_store.cc:276] 0709 12:26:02.358759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.358934 (+   175us) service_pool.cc:224] Handling call
0709 12:26:02.379437 (+ 20503us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:02.471321 (+ 91884us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:02.471354 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:02.509947 (+ 38593us) log.cc:844] Serialized 52 byte log entry
0709 12:26:03.025353 (+515406us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:03.025398 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.025479 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.966445 (+940966us) spinlock_profiling.cc:243] Waited 8.25 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:03.968555 (+  2110us) raft_consensus.cc:1685] finished
0709 12:26:03.968582 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:03.968843 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2919,"raft.queue_time_us":17181,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":17424,"spinlock_wait_cycles":18965248,"thread_start_us":417790,"threads_started":1,"wal-append.queue_time_us":572567}
W20260709 12:26:03.976028 16071 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.229s	user 0.000s	sys 0.001s
W20260709 12:26:03.980233 15901 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.986s	user 0.012s	sys 0.000s
I20260709 12:26:03.981555 15936 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:03.982132 15936 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.983034 15936 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:03.985453 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 199823us
I20260709 12:26:03.986774 16082 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:03.987280 16082 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.987406  9941 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:03.988224 16082 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:03.992311 16079 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:03.998371 10099 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:03.992745 16016 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:03.951354 16085 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:26:04.017277  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:04.020622  9940 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:04.020512 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 147576us
I20260709 12:26:04.024279  9947 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:04.026428 10097 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:04.028486 15859 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:04.029795 15859 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:04.030122 15967 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:04.030196 15859 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:03.904240 10092 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:33250
W20260709 12:26:04.029570 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 179562us
I20260709 12:26:04.043926 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:04.045931 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59831.1us
I20260709 12:26:03.944594 15370 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 12:26:04.050908  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:03.943920 15758 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:03.953604 16017 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:04.055681 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 283989us
I20260709 12:26:04.062882 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:04.063468 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:04.065430  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:04.064604  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:04.076015  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:04.077790 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.078228 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.009845 15771 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:04.081840  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 131766us
I20260709 12:26:04.018064 16057 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:26:04.012779 16021 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:26:04.066000 16017 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.081039 10095 pending_rounds.cc:85] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 9
I20260709 12:26:04.075366  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:04.062647 16009 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:04.083283  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:04.084735  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:04.077626 16016 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:04.101107 10095 pending_rounds.cc:107] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 12:26:04.104792 10095 raft_consensus.cc:2889] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:04.106771 10092 raft_consensus.cc:606] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:04.107283 10092 raft_consensus.cc:614] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:04.111050 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:04.111322 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56272.7us
I20260709 12:26:04.082449 15970 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.054420 15861 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:04.115329 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.118820 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:04.116815 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.119674 10669 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:04.116930 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.119230 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.111956 10676 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:26:04.043289 15859 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:04.121119  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56598.3us
I20260709 12:26:04.099823 15992 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:04.112687 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.059716 15863 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 12:26:04.104533 16016 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:04.186697 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:04.077193 15962 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:26:04.188694 14841 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:04.200373 14841 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:04.200940 14841 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:04.126055  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43532.7us
W20260709 12:26:04.193670 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73698.9us
I20260709 12:26:04.122879  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:04.181015 10672 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:04.202039 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:04.191586  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 145802us
W20260709 12:26:04.197548  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 161561us
I20260709 12:26:04.125305 16099 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.206271  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 83469.1us
I20260709 12:26:04.203421  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:04.178334 15936 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:04.191092 16098 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:26:04.186856 15992 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:04.199579 16016 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.217519 15366 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:04.206377 15859 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.212684 15945 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:04.091995 16079 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.201324 15901 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:04.245322 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.243294 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:04.247846  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45889.9us
I20260709 12:26:04.179622 16066 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 12:26:04.252099 16104 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:04.253485 16076 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 12:26:04.246796 16082 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:04.254961 16089 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:04.255381 15901 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:04.219652 16009 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:04.259016 10674 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:04.267117 16009 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:04.267647 16009 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:04.250739 16059 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.730s	user 0.009s	sys 0.003s
I20260709 12:26:04.248482 15945 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:04.278221  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71387.6us
W20260709 12:26:04.287645  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.295760 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.262732 16066 consensus_queue.cc:1243] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:04.224139 15936 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.264760 15796 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:04.259682 15771 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:04.272852 15901 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:04.208050 16099 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.365024 10676 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: [15.14-15.14]
W20260709 12:26:04.352083 16091 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
W20260709 12:26:04.353317 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 986) took 1579 ms (client timeout 1951 ms). Trace:
W20260709 12:26:04.354082 16096 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:26:04.277830 16082 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.348551 15796 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:04.286165 15366 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.071s
I20260709 12:26:04.356660 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.357267 16009 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:04.359383 16083 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.572s	user 0.004s	sys 0.000s
W20260709 12:26:04.361126 16100 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:26:04.361450 16063 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.632s	user 0.013s	sys 0.000s
I20260709 12:26:04.278251 16089 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:04.365100  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 161703us
I20260709 12:26:04.286536 15945 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:04.436291 15936 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:04.437613 15901 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.440060 15901 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:04.368106 10099 rpcz_store.cc:269] 0709 12:26:02.774390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:02.782992 (+  8602us) service_pool.cc:224] Handling call
0709 12:26:04.353099 (+1570107us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400305a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:04.353234 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2670714752}
I20260709 12:26:04.445140 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.446246 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.447285 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.448129 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:04.448367 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.448764 10669 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:04.449030 10099 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:04.258673 15992 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:04.452354 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.454851 10671 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 12:26:04.457010 10668 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:04.458648 10669 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:04.459030 15966 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:04.459407 15966 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:04.460284 15966 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.461721 15966 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:04.346941 15758 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:04.380713 10677 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:04.390525 16009 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:04.392607 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1000) took 1610 ms. Trace:
I20260709 12:26:04.465828 10093 rpcz_store.cc:276] 0709 12:26:02.781976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.406091 (+624115us) service_pool.cc:224] Handling call
0709 12:26:03.406444 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.447901 (+ 41457us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.447953 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:03.470475 (+ 22522us) log.cc:844] Serialized 26 byte log entry
0709 12:26:03.649982 (+179507us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:03.658848 (+  8866us) log.cc:844] Serialized 14 byte log entry
0709 12:26:03.718419 (+ 59571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:03.718522 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:03.961318 (+242796us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b18000bc560. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:04.252763 (+291445us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:04.392369 (+139606us) raft_consensus.cc:1685] finished
0709 12:26:04.392391 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.392529 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2374,"prepare.queue_time_us":8760,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":7607,"raft.queue_time_us":31653,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":9877,"spinlock_wait_cycles":123717888,"thread_start_us":131286,"threads_started":2,"wal-append.queue_time_us":200882}
W20260709 12:26:04.393821 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:04.469072 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.394321 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:04.472185 16093 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.480s	user 0.003s	sys 0.000s
W20260709 12:26:04.474344 16094 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.520s	user 0.002s	sys 0.000s
I20260709 12:26:04.401998 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:04.404246  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:04.404708 16089 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:04.405071  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:04.405827  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:04.408452 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.411862 15970 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:04.417188 16082 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:04.373530 16020 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:04.379348 15796 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:04.421068 16058 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.478s	user 0.000s	sys 0.001s
I20260709 12:26:04.421301 16099 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:04.422243 10096 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:04.427656 15861 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:04.431669 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:04.546522 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.367651 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.440339 15829 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:04.431469 16016 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.442052 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.553030 10676 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:26:04.397236 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:04.495489 10097 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:26:04.501708  9945 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:04.501760  9946 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 12:26:04.549837  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 160479us
I20260709 12:26:04.502033  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:04.502782  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:04.503472 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:04.504837 10674 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:04.529259 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.556784 10669 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:04.490492 14841 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.513257 10093 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:04.506313 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:04.538412 16111 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:04.538358 15868 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:04.539495 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:04.602020 10671 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:04.545647 10096 raft_consensus.cc:606] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:04.602792 10096 raft_consensus.cc:614] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:04.529464 15366 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.562729 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:04.553943 10092 raft_consensus.cc:2376] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15. Re-sending same reply.
W20260709 12:26:04.611428 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48825us
I20260709 12:26:04.612044 10098 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:04.560588  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 274064us
I20260709 12:26:04.548844 15829 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:04.554865 15772 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.606287 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:04.559893  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4773635759997887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
I20260709 12:26:04.612124 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:04.562778  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:04.503432 16101 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
I20260709 12:26:04.602109 15904 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.613456 14841 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:04.634266  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2928635759999452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
W20260709 12:26:04.637497  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0840635760000623s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2122s for a response
W20260709 12:26:04.638074  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0579635760000201s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21509999999999999s for a response
I20260709 12:26:04.635200  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:04.615283 15964 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.864s	user 0.014s	sys 0.000s
I20260709 12:26:04.640102 15967 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:04.645027 16105 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.395s	user 0.003s	sys 0.000s
W20260709 12:26:04.646241 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 140021us
I20260709 12:26:04.657374 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:04.651940 10675 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:04.632141 15354 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:04.617268 16111 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:04.660012  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:04.670806 15861 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:26:04.679557  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 116855us
I20260709 12:26:04.680466  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:04.681664  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46585.6us
I20260709 12:26:04.684182 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.654893 16009 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:04.698253 10671 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
W20260709 12:26:04.701700 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44399.4us
I20260709 12:26:04.702867 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:04.632479 15366 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.715003 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:04.640400 15904 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.720239 10675 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:04.719652 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.647464 16066 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:26:04.637153 15772 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.746176 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:04.747538 10675 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:04.671114 15970 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:04.749279 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46524.7us
I20260709 12:26:04.753237 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:04.746332 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:04.681332 15758 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:04.754946 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.756064 10669 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:04.760654  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:04.764148 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:04.764536 10675 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:26:04.717788 16063 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:26:04.768373  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 87979.4us
I20260709 12:26:04.774144  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:04.665751 15354 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:04.754530 15872 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:04.748916 15869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.014s	sys 0.021s
I20260709 12:26:04.787529 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2342) took 1060 ms. Trace:
I20260709 12:26:04.759557 15758 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:04.767889 16082 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:04.722072 16009 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:04.722211 15945 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:04.785355 16056 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.017s	sys 0.014s
I20260709 12:26:04.791577  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:26:04.792595 10090 rpcz_store.cc:276] 0709 12:26:03.726829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.931959 (+205130us) service_pool.cc:224] Handling call
0709 12:26:03.932241 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:03.949184 (+ 16943us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:03.949234 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.013892 (+ 64658us) log.cc:844] Serialized 26 byte log entry
0709 12:26:04.106615 (+ 92723us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:26:04.106654 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.106744 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:04.644509 (+537765us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b2400223868. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a809d 00007f4e090da10e 00007f4e090f188c 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221
0709 12:26:04.773533 (+129024us) raft_consensus.cc:1685] finished
0709 12:26:04.773564 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:04.787441 (+ 13877us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5047,"raft.queue_time_us":8603,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":42102,"spinlock_wait_cycles":63378176,"thread_start_us":32992,"threads_started":1,"wal-append.queue_time_us":148042}
W20260709 12:26:04.813220 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 308622us
W20260709 12:26:04.816774 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52724us
I20260709 12:26:04.817538 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:04.704244 15829 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:04.821768  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.781678 15861 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:04.783839 16057 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:04.795285 15869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.014s	sys 0.021s
W20260709 12:26:04.823606  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49581.7us
I20260709 12:26:04.834615  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:04.822364 15829 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:04.836076 15859 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.002s
W20260709 12:26:04.814343 16056 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.018s	sys 0.014s
I20260709 12:26:04.795375 16079 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:04.839797 16128 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:04.840581 16128 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:04.796434 15872 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.843230 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.843497 15904 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:04.844069 10669 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:04.844765 15872 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:04.841691 16128 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:04.845207 15942 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.887s	user 0.014s	sys 0.000s
I20260709 12:26:04.830717 16111 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:04.815654 15758 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:04.853113 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 241035us
W20260709 12:26:04.850373  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 127642us
I20260709 12:26:04.857028 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.857466 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:04.867991 10668 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:04.872885 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:04.862804 10062 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.621s (SENT) [suppressed 30 similar messages]
I20260709 12:26:04.814620 16106 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:26:04.828748 16016 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:04.833211 15970 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:04.920141 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:04.924193 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:04.926770 10675 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:04.930680 16123 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:26:04.931746 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:04.932098 16124 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:26:04.933872  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:04.940749 10672 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:04.951172 10676 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:26:04.855155 16114 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:04.961933  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:05.001040 10090 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:05.005219 15945 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:05.008491 16009 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:05.009286 16017 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.902s	user 0.009s	sys 0.000s
I20260709 12:26:05.009800 16017 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:05.013442 16118 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
W20260709 12:26:05.015324 16121 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:26:05.016611 16130 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:26:05.017707 16009 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:05.018085 16009 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:05.018293 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2344) took 1244 ms. Trace:
I20260709 12:26:05.019438 16009 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:05.020789 16009 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.024169 15772 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.026424 16106 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:05.026930 16106 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:05.027805 16106 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.029214 16106 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.031180 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:05.022426 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.033483 10668 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:05.037961 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.033890 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:05.039256  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 167930us
I20260709 12:26:04.855373 15904 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
I20260709 12:26:05.043747  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.047253  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.048199  9940 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:05.049710  9948 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:05.050227  9948 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:04.963056 15861 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.018694 10095 rpcz_store.cc:276] 0709 12:26:03.773609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:03.928711 (+155102us) service_pool.cc:224] Handling call
0709 12:26:03.942308 (+ 13597us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.181968 (+239660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.182017 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:04.190571 (+  8554us) log.cc:844] Serialized 52 byte log entry
0709 12:26:04.527382 (+336811us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:04.527425 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:04.758200 (+230775us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54002fce54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:04.758272 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.017961 (+259689us) raft_consensus.cc:1685] finished
0709 12:26:05.017984 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.018230 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2789,"raft.queue_time_us":55013,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":18568,"spinlock_wait_cycles":318405376,"thread_start_us":294201,"threads_started":1,"wal-append.queue_time_us":536018}
I20260709 12:26:05.060935  9940 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:05.066646 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:05.067498 10676 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:05.068445 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.072808  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.073557  9943 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:26:05.074555 10094 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:05.077767 10099 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.015s	sys 0.011s
W20260709 12:26:05.068164 15966 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.606s	user 0.011s	sys 0.003s
W20260709 12:26:05.088862 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 123324us
W20260709 12:26:05.091856 10099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.015s	sys 0.011s
I20260709 12:26:05.092497 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:05.092698  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 258204us
W20260709 12:26:05.098806  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 274836us
I20260709 12:26:05.086094  9948 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.085326 15366 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.157586  9948 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.156278 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.114058 16056 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.136406 16020 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.163987 10021 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
W20260709 12:26:05.149931 16096 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:26:05.172134 10021 pending_rounds.cc:85] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 9
I20260709 12:26:05.176122 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:05.181613 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.177464 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:05.146063 16083 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:05.178287 10017 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:05.182704  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.135493 15861 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:05.178934 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.179066 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.188830 10024 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:05.181558 10021 pending_rounds.cc:107] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:26:05.189603 10021 raft_consensus.cc:2889] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:05.183744 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 320969us
I20260709 12:26:05.183893 10098 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:05.173055 10670 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.556s	user 0.011s	sys 0.000s
I20260709 12:26:05.182354 10023 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
W20260709 12:26:05.187536  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.017s	sys 0.013s
W20260709 12:26:05.190740  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.017s	sys 0.013s
I20260709 12:26:05.190896 10670 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
W20260709 12:26:05.189265  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.192531  9947 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:05.194330  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:05.161219 15796 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.611s	user 0.004s	sys 0.007s
I20260709 12:26:05.193437  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:05.203796  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:05.191167  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 905899us
I20260709 12:26:05.207623  9949 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:05.208930 10016 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:26:05.210230 10016 pending_rounds.cc:85] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 9
I20260709 12:26:05.210536 10016 pending_rounds.cc:107] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:26:05.211097 10016 raft_consensus.cc:2889] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.208881  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:05.159379 15962 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:05.219612  9942 pending_rounds.cc:85] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 9
I20260709 12:26:05.222110  9942 pending_rounds.cc:107] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:26:05.222486  9942 raft_consensus.cc:2889] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.205451 15967 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.224757 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.226706 15861 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.228917 15861 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.229840 10019 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:05.211619 15796 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.232939  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:05.234190 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.235360 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.239442  9943 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:05.145334 16116 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:26:05.253221 10097 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:05.185051 15354 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:05.256845 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.258239 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.258976 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.259413 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.259460 16136 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.260068 10019 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:05.260413 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.260133 16136 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.261641 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.263173 10021 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:05.163311 16021 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:26:05.264833 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.281183  9948 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 12:26:05.281529  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.271633 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.282553  9943 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:05.276293  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 90915.1us
I20260709 12:26:05.282578 10023 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.275882 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.271003 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.260140 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.287456  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:05.214375 16146 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.285594 10018 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:05.290109 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:05.290772 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.281584  9948 pending_rounds.cc:85] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 10
I20260709 12:26:05.306798  9948 pending_rounds.cc:107] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:26:05.293910 10019 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:05.309065 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.309794 10019 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.307487 10017 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.311424  9948 raft_consensus.cc:2889] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.315444 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.315969 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:05.262930 16135 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:05.318881 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.324445 10016 raft_consensus.cc:2410] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:26:05.324787 10021 raft_consensus.cc:2368] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17.
I20260709 12:26:05.325970 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:05.327785 16130 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.328934 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:05.330278 16130 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.326560  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.275008 15366 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:05.339531  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49474.4us
I20260709 12:26:05.341286 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:05.262567 16136 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.264477 15370 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:05.351648 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.333220  9948 raft_consensus.cc:2410] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:26:05.355317 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.272047 15861 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:26:05.360297  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:05.364799 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.319519 16056 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:05.340462 16117 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.468s	user 0.000s	sys 0.001s
I20260709 12:26:05.359099 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.362490 10021 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:05.363773 10019 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:05.316993 15772 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:05.372274  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.346021 15796 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.254045 15962 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.386157 10021 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:26:05.386613 10021 pending_rounds.cc:85] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 8
I20260709 12:26:05.319074 16116 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:26:05.397650 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.400142  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:05.381085 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50571.3us
I20260709 12:26:05.378396 10024 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:05.386862 10021 pending_rounds.cc:107] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:26:05.422281 10021 raft_consensus.cc:2889] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.373385 16016 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.375097 10020 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:26:05.388610  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:05.316478 16114 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:26:05.323304 15945 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.056s
I20260709 12:26:05.381322 15772 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:05.378811 16017 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.428791 10016 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:26:05.429883 10020 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:05.433503  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45131.7us
I20260709 12:26:05.361676 15366 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:05.437638 10018 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:05.322187 16146 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:05.421103 15969 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.425295 15859 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.446544 10024 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:05.447407  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:05.377889 15370 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.383136 16056 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.455272  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:05.455950 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.457321 10022 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:05.455539 15893 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:05.458837 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.459378 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:05.450400 16140 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.203s	user 0.000s	sys 0.001s
I20260709 12:26:05.402503 16135 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
W20260709 12:26:05.457827 16126 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.293s	user 0.000s	sys 0.001s
I20260709 12:26:05.467435 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.468878 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.466526 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.474308 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.475060 10020 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:05.459316 15868 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
W20260709 12:26:05.477136 16139 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:26:05.478574 10021 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:05.480223 10021 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:05.481315 16148 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:05.481805 15868 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:26:05.482162 16145 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:26:05.482285 15868 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:05.482990 15868 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.460724 15969 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:05.492269 16056 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:05.492168 15969 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.492235 15868 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.448549 16141 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:26:05.454075 15859 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:05.463867 16119 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.109s
I20260709 12:26:05.494535 15969 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.509699 15868 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:05.517437 15970 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.562s	user 0.010s	sys 0.000s
I20260709 12:26:05.519649 15970 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:05.520097 15859 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:05.520797 15829 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.000s
W20260709 12:26:05.523298 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.524299 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:05.524857 10092 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.527772 10016 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:05.461640 16082 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.543186 10017 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:05.546962 10019 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:05.533417 10020 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:05.546876 10669 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:05.551716 10017 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:05.531701 10676 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:05.570864 10671 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
W20260709 12:26:05.515957 16153 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:26:05.498390 16138 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 12:26:05.587366 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:05.621649  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.626338  9946 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:26:05.577638 16082 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.629490 15967 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.630798 15970 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:05.631372 15970 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.632505  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.513816 15962 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:05.633682 15970 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:05.636379  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.636766 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:05.637576 15970 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.638196 15970 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.639670 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.640081 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:05.496826 16146 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:05.641647 16056 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:05.511945 15366 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:05.508057 15772 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:05.500067 15370 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.645311 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:05.646718 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.647559 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.648129 10018 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:05.641353 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:05.652398 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:05.654460  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 199281us
I20260709 12:26:05.669700  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.587143 15859 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:05.683583 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:05.647022 15970 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:05.688344 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:05.689550  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 289517us
I20260709 12:26:05.690287 10675 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.654920 16056 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:05.690563  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:05.656957 15901 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.695076  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:05.695751 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:05.682278 16080 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:05.702471 10670 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:26:05.657200 16084 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:26:05.686570 15859 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:05.700428 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:05.704737 10090 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:05.657912 15962 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:26:05.708732 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.724741 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.694114 16056 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:05.710000 16009 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.689s	user 0.009s	sys 0.005s
I20260709 12:26:05.709991 10018 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: [13.11-13.11]
W20260709 12:26:05.715250 16137 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.458s	user 0.005s	sys 0.000s
I20260709 12:26:05.736761 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3578) took 1221 ms. Trace:
I20260709 12:26:05.719404 10676 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:26:05.722688 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 135409us
I20260709 12:26:05.736982 15859 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 11, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.698196 15370 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:05.738351 15859 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:05.700059 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:05.739307 10094 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:05.699980 15970 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:26:05.742033 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:05.682008 15366 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:26:05.731578 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.750346 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:05.751982 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:05.753356 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:05.754503 16128 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.727797 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:05.755872 10017 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.708743 10674 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:05.737301 10091 rpcz_store.cc:276] 0709 12:26:04.515557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.545193 (+ 29636us) service_pool.cc:224] Handling call
0709 12:26:04.545604 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:04.789722 (+244118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.789758 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:04.794128 (+  4370us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.093666 (+299538us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:05.093699 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.093762 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.714635 (+620873us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:05.736381 (+ 21746us) raft_consensus.cc:1685] finished
0709 12:26:05.736413 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.736684 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":71645,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":20535,"spinlock_wait_cycles":45441792,"thread_start_us":518501,"threads_started":2,"wal-append.queue_time_us":438716}
W20260709 12:26:05.691314 15758 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.818s	user 0.013s	sys 0.008s
I20260709 12:26:05.745932 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.759925  9942 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:26:05.760675 10062 leader_election.cc:400] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17.
I20260709 12:26:05.748449 10021 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:05.761438 10021 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:05.760640 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1254) took 1053 ms. Trace:
I20260709 12:26:05.763461 10671 rpcz_store.cc:276] 0709 12:26:04.707378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.714746 (+  7368us) service_pool.cc:224] Handling call
0709 12:26:05.760429 (+1045683us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b5400690d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:05.760565 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1845520768}
I20260709 12:26:05.765332 10062 leader_election.cc:403] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:05.770015 10098 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:26:05.772027 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:05.762493 10021 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:05.779424 10021 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:05.781713 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:05.748005 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.699299 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3576) took 1411 ms. Trace:
I20260709 12:26:05.700600 16146 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:26:05.796063 10098 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:05.734943 16056 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:05.801283 10098 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:05.802338  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:05.796505 10675 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:05.754766 16132 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:05.815442  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:05.798830 10096 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.028s	sys 0.030s
W20260709 12:26:05.817353 10096 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.028s	sys 0.030s
I20260709 12:26:05.813501 10017 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:05.755555 15772 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.816934  9947 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:05.786379 10099 rpcz_store.cc:276] 0709 12:26:04.287544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.449542 (+161998us) service_pool.cc:224] Handling call
0709 12:26:04.449927 (+   385us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:26:04.492157 (+ 42230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.492188 (+    31us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:26:04.493264 (+  1076us) log.cc:844] Serialized 104 byte log entry
0709 12:26:05.094210 (+600946us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:05.333534 (+239324us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.352255 (+ 18721us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.697819 (+345564us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.698892 (+  1073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.698978 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:05.699033 (+    55us) raft_consensus.cc:1685] finished
0709 12:26:05.699051 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:05.699232 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57524,"prepare.queue_time_us":343533,"prepare.run_cpu_time_us":139,"prepare.run_wall_time_us":2642,"raft.queue_time_us":58534,"raft.run_cpu_time_us":607,"raft.run_wall_time_us":40505,"thread_start_us":1082591,"threads_started":3,"wal-append.queue_time_us":684785}
W20260709 12:26:05.779548 16139 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 12:26:05.824232  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:05.825970 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:05.827865 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:05.829697 16017 raft_consensus.cc:2764] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:26:05.830461 16017 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:05.778690 15758 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.831485 15796 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:05.831854 16017 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:05.832357 15796 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:05.832715 15796 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:05.832885 16017 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:05.833248 15758 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.833472 10095 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:05.833709 15796 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:05.835026 16148 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:05.836685 16148 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:05.838063 15796 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:05.782476 15893 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:05.833282 16017 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:05.759351 15904 raft_consensus.cc:2764] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:26:05.777912 16009 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:05.867588 10023 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.579s	user 0.012s	sys 0.000s
I20260709 12:26:05.795964 15901 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:05.873202 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:05.873226  9947 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:05.877751 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 195198us
I20260709 12:26:05.878113 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.886468 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:05.815193 16128 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:05.882884 10023 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:05.882675 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:05.896988 10675 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
W20260709 12:26:05.896260  9944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.025s	sys 0.038s
I20260709 12:26:05.894191 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:05.897711  9944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.025s	sys 0.038s
I20260709 12:26:05.841337 15861 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:05.884554 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:05.823123 16160 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.889009 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.967120 10674 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:05.965248  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.967585 10017 raft_consensus.cc:2410] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
W20260709 12:26:05.967339  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 189636us
I20260709 12:26:05.891315 10021 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:26:05.968677 10021 pending_rounds.cc:85] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 8
I20260709 12:26:05.968941 10021 pending_rounds.cc:107] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:26:05.969179 10021 raft_consensus.cc:2889] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:05.970346 16160 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.971385 15904 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:05.971995 15904 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:05.973363 16160 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.974015 16063 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:05.974444 16063 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:05.893210 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.922645 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.967947  9947 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:05.976509 10019 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:26:05.898118 10022 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:05.849773 16130 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:05.978736 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.979575 10017 raft_consensus.cc:2368] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17.
I20260709 12:26:05.980404 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:05.982667 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:05.987011  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.949378 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:05.923775 16157 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.233s	user 0.000s	sys 0.000s
I20260709 12:26:05.839570 15772 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:05.871094 15758 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:26:05.886086 15901 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:05.848989 16112 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.897897 16141 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.954748 16082 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:05.960127 16021 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:05.962318 16084 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:05.891652 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:06.037042 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54491.7us
I20260709 12:26:05.965811 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:06.037868 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:05.974310 15904 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:05.964872 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:06.007419 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:06.046846 10092 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
W20260709 12:26:06.049654 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 84897.9us
I20260709 12:26:06.032907  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:06.051220 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 85525.4us
I20260709 12:26:06.049763 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:06.039619 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:05.993063 16016 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:06.006654 16162 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 12:26:06.058188  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:05.998999 16103 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.060612 10024 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:06.065135 16129 raft_consensus.cc:2764] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:26:06.081267 16103 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:06.065675 16167 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:26:06.065699  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.067263 15893 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.083824 16103 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:06.070748 16170 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:06.036672 16128 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:06.007218 15901 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.048807 16123 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:06.074308 16056 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.057569 16063 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.056756 15904 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.072629 15772 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:06.036902 16021 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:06.088024 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:06.096827 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 131775us
I20260709 12:26:06.147472 16057 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:06.148070 16057 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:06.149108 16057 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:06.158187 14841 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
W20260709 12:26:06.132175  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66574.7us
W20260709 12:26:06.070749 16165 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 12:26:06.087466 16135 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:06.164917 15872 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:06.168607 15872 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:05.999469 16130 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.172869 10093 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:06.173368 10093 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:06.177904 16143 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.720s	user 0.004s	sys 0.009s
I20260709 12:26:06.174350 10093 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.056483 16082 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 12:26:06.184100 16111 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:06.184537 16111 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:06.185357 16111 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.186668 16168 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:06.187243 16111 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:06.187263 16168 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.192920 16168 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:06.195415 16174 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:26:06.198031 16009 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17.
I20260709 12:26:06.200443 16056 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:06.144163 16106 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:06.202913 15772 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:06.203363 15772 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:06.204015 15772 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.203235 16009 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:06.205368 16009 consensus_queue.cc:1243] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 18.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:06.204777 16132 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:06.206319 10099 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:06.208871 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1258) took 1495 ms. Trace:
I20260709 12:26:06.209369 15962 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:06.209798 15962 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:06.210736  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.210830 15962 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.209280 10673 rpcz_store.cc:276] 0709 12:26:04.713469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.763118 (+ 49649us) service_pool.cc:224] Handling call
0709 12:26:04.781514 (+ 18396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:04.781918 (+   404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:04.781949 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:04.796565 (+ 14616us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.149050 (+352485us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:05.149090 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.149155 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.177278 (+1028123us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:06.208560 (+ 31282us) raft_consensus.cc:1685] finished
0709 12:26:06.208581 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.208813 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2775,"raft.queue_time_us":993,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":5680,"spinlock_wait_cycles":4696832,"thread_start_us":228406,"threads_started":1,"wal-append.queue_time_us":558164}
I20260709 12:26:06.211154 16021 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.212383 15962 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.217326 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1016) took 1363 ms. Trace:
I20260709 12:26:06.217782 10096 rpcz_store.cc:276] 0709 12:26:04.854485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.855984 (+  1499us) service_pool.cc:224] Handling call
0709 12:26:04.932646 (+ 76662us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:05.135022 (+202376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.135054 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:05.171486 (+ 36432us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.817756 (+646270us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:05.868251 (+ 50495us) log.cc:844] Serialized 14 byte log entry
0709 12:26:05.868514 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.868615 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.152889 (+284274us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b18003513a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:06.216978 (+ 64089us) raft_consensus.cc:1685] finished
0709 12:26:06.217092 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.217258 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":665,"prepare.queue_time_us":52633,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":4217,"raft.queue_time_us":84569,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":50615,"spinlock_wait_cycles":4731264,"thread_start_us":764155,"threads_started":2,"wal-append.queue_time_us":963874}
I20260709 12:26:06.180065 16099 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:06.221266 16173 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 12:26:06.100041 16128 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:06.222388 16128 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:06.225411 16017 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.100912 16123 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.228734 16123 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:06.062564 16084 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.118s
I20260709 12:26:06.162336  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:06.237926 10674 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:06.150391 16057 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:06.241046 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 178071us
I20260709 12:26:06.242527 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:06.240331 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 186941us
I20260709 12:26:06.243814 16132 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:06.244457 16020 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:26:06.241189 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 291954us
I20260709 12:26:06.244756 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:06.245631 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:06.246646  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 84364.5us
I20260709 12:26:06.252635 15970 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:06.253330 16176 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.267s	user 0.002s	sys 0.001s
I20260709 12:26:06.254717  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1262) took 1220 ms. Trace:
I20260709 12:26:06.255095  9941 rpcz_store.cc:276] 0709 12:26:05.033900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.035028 (+  1128us) service_pool.cc:224] Handling call
0709 12:26:05.067712 (+ 32684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:05.067965 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.068002 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:05.632545 (+564543us) spinlock_profiling.cc:243] Waited 5.23 ms on lock 0x7b2c00012928. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:26:05.633047 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:26:05.846007 (+212960us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:05.846042 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.846185 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:06.230767 (+384582us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:06.254470 (+ 23703us) raft_consensus.cc:1685] finished
0709 12:26:06.254493 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:06.254646 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51265024,"thread_start_us":212553,"threads_started":1,"wal-append.queue_time_us":341715}
I20260709 12:26:06.250984 15992 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:26:06.242677 16123 raft_consensus.cc:2764] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:26:06.286188 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:06.250808 16084 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.291558  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:06.295423  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:06.296888 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:06.287503 16132 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:26:06.303062  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50795.3us
I20260709 12:26:06.305100  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.288110 16130 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:06.306407 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.308638 15901 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.315855 16084 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:06.319268 16184 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:06.351847 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:06.325440  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:06.265865 16020 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:26:06.319315 16169 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:06.357868 16130 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:06.358645 16132 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:06.359542 16021 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.330471 16135 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:06.373497 16185 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:06.373957 15772 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.352440 16184 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:06.384577  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79556.3us
I20260709 12:26:06.385627  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.383564  9941 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:06.376556 16132 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.360502 16123 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:06.344204 16192 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.081s
I20260709 12:26:06.355695 16128 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:06.403492  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.402364  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.388228 16184 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:06.354545 16084 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:06.412467 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 267286us
I20260709 12:26:06.413488 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:06.401494 16135 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:06.417960  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.421655 16185 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:06.420921 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.416633 15869 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.431162 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:06.404189 15868 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:26:06.414768 16170 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:06.433576 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.434743 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.434430 16170 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.420881 16190 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:26:06.437755 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1297) took 1943 ms. Trace:
I20260709 12:26:06.438169 10672 rpcz_store.cc:276] 0709 12:26:04.493888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.494013 (+   125us) service_pool.cc:224] Handling call
0709 12:26:04.934684 (+440671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:04.959593 (+ 24909us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.437526 (+1477933us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540073afd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:06.437699 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":993070336}
I20260709 12:26:06.438788 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.435503 10671 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:06.431536  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.440716 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:06.441829 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 145028us
W20260709 12:26:06.444610 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.447947 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:06.452661  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:06.479823  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48497.1us
I20260709 12:26:06.482700  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:06.463716  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:06.485056  9941 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.489346  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:06.491021 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.443200 15796 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.493110 15796 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.448283 16099 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.494522 16099 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.497210 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.497860 10672 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:06.432724 16169 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:06.442826 16192 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:06.439095 16185 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:06.508045 10676 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:06.441116 16193 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:06.513512 15901 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:06.514249 15796 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:06.482445 16057 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.505398  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:06.505400 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57541.3us
I20260709 12:26:06.519953 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:06.518034  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:06.520421 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.521503 16113 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:06.521605 16200 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:06.522128 16113 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:06.522289 16200 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:06.509936 16185 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.522944 16185 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:06.523355 16200 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.528405 10093 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:06.510088 15904 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:26:06.530967 15904 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.531328 15904 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:26:06.533669  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.509763 16192 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.534915 16192 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.535750 10099 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:06.535898 10091 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:06.536144 10097 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:26:06.536242 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1059) took 1082 ms. Trace:
I20260709 12:26:06.536652 10024 rpcz_store.cc:276] 0709 12:26:05.453979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.454107 (+   128us) service_pool.cc:224] Handling call
0709 12:26:05.898502 (+444395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.081249 (+182747us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.535342 (+454093us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400217ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:06.536047 (+   705us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400217ad0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:06.536188 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1291263616}
I20260709 12:26:06.491719 16123 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:06.499478 16106 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:06.538172 10672 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:26:06.439793 15893 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.069s
I20260709 12:26:06.538985 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:06.539997 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:06.551062  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:06.503440 16021 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.554411 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:06.556821 10021 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:06.558180 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:06.517702 16128 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:06.559330 10090 raft_consensus.cc:2410] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 17 because replica has last-logged OpId of term: 16 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:26:06.563681  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:06.567320  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.567130 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.570241 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:06.573067  9949 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:06.548274 15868 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:06.575335 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.583110 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.584168 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:06.589356  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.589919 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.591156 10668 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:06.592868 10062 leader_election.cc:400] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17.
W20260709 12:26:06.593127  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.555998 16123 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:06.593848 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.559828 15772 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:06.595997 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.598009 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:06.534480 16184 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.560281 16084 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:06.553333 16169 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 12:26:06.595577 15901 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:06.604182 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:06.611570 10668 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:06.591707  9949 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:26:06.612380 10675 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:06.576804 16106 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:06.586117 16168 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:26:06.610612  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47059.9us
I20260709 12:26:06.558280 16021 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.614461 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:06.617151 16195 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:06.619266 10674 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:06.595860 16016 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.621513  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:06.624243 10672 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.625591  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:06.597852 16080 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:06.642236 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:06.648089 10093 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:06.631857 10096 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:06.629700 10021 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:06.565377 16185 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.656929 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1020) took 1797 ms. Trace:
I20260709 12:26:06.674585 10093 rpcz_store.cc:276] 0709 12:26:04.859846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:04.933968 (+ 74122us) service_pool.cc:224] Handling call
0709 12:26:04.934259 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:06.172740 (+1238481us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54001a0520. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:26:06.527784 (+355044us) spinlock_profiling.cc:243] Waited 3.43 ms on lock 0x7b58004bb6d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:26:06.656591 (+128807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.656861 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":337462784}
I20260709 12:26:06.608379 16056 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
W20260709 12:26:06.623396 16199 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:26:06.679949 15771 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:26:06.596285 16135 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
W20260709 12:26:06.681651  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56157.2us
I20260709 12:26:06.690851  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:06.618305 16182 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.621057 16168 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:06.612242 15970 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:26:06.698441  9947 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:26:06.612123 16082 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 12:26:06.701823 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:06.696447 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 92064.3us
I20260709 12:26:06.677084 16101 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:06.680610 15901 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:06.706710 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.709609 10021 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:06.680979 15772 raft_consensus.cc:2764] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:06.693387 16099 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:06.705003 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:06.747208 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:06.749986 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 122669us
I20260709 12:26:06.750774 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:06.712718 16146 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:06.709003 16187 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:06.618144 16113 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.749680 16193 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:06.762825 16204 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:26:06.756886 16016 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:26:06.716365 15901 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.772843 16187 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:06.768695 16099 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:06.788401 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.786546  9941 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:06.834201 16021 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:06.835290 16210 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:06.759872 16106 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.752336 16123 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:06.814314  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.838193  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 147480us
W20260709 12:26:06.834448 16211 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:26:06.757632 16168 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:06.857806 16215 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:26:06.858202 16168 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:26:06.865597 16214 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:26:06.845232 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56917us
I20260709 12:26:06.836261 16198 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:06.869670 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:06.871263  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.879395  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:06.848493 16184 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:06.851468 16200 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:06.876031 16187 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.885562 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.885980 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.891373 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.892172 10019 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:06.892671 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:06.894363 15962 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.681s	user 0.005s	sys 0.005s
I20260709 12:26:06.895963 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.842047 16164 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:06.918848 16164 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:06.919273 16164 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:06.898566 10093 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:06.900729 15869 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:06.921545  9941 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:06.901295 15901 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:06.901646 15945 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:06.907371 16135 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:26:06.911180 16150 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.473s	user 0.002s	sys 0.002s
I20260709 12:26:06.911420 16198 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:06.913523 16203 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.387s	user 0.000s	sys 0.002s
I20260709 12:26:06.917512 16192 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:06.939157 10669 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:06.922585 15945 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:06.958370 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.960049 10019 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:06.961261 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:06.961959 10019 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:06.965616 10672 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:06.906296 15904 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:06.968849 15893 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:26:06.984447 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:06.963652 16217 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:26:06.984273 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:06.948783 15901 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:06.986708  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:06.988232 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 118679us
I20260709 12:26:06.993991 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:06.999135 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:07.000385 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:07.002844 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:06.969211 16182 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.008507 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:06.956738 15868 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:26:07.010610 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2345) took 1767 ms. Trace:
I20260709 12:26:07.015493 10095 rpcz_store.cc:276] 0709 12:26:05.243356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.251155 (+  7799us) service_pool.cc:224] Handling call
0709 12:26:05.256374 (+  5219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:05.834034 (+577660us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.010336 (+1176302us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b5400340b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:07.010525 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326768512}
I20260709 12:26:07.022894 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:07.023775 10024 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:07.002074 15772 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:06.985077 16146 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:26:07.014667 15962 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:26:07.015269 16179 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:26:07.047271  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 90130.5us
I20260709 12:26:07.018282 16182 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.046469 16135 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:26:07.053671 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 50941.6us
I20260709 12:26:07.051424  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:07.068384 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:07.072031 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:07.042215 15772 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.082813 16200 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:07.083256 16200 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.083626 16200 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.084120 15796 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.084674 15796 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.090010  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:07.031617 15945 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.062763 16135 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:07.070957 15901 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:07.087445 16201 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:07.092685 16127 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.947s	user 0.007s	sys 0.002s
I20260709 12:26:07.087049 16192 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:07.091746 16106 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:07.102775 15758 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:07.106979 15758 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.082453 16164 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:07.108176 16154 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.659s	user 0.006s	sys 0.000s
I20260709 12:26:07.095407 16147 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:07.111935 16135 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.113180 16135 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.094447 16230 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:07.102928 16222 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.145s	user 0.001s	sys 0.001s
I20260709 12:26:07.109601 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3365) took 1949 ms. Trace:
I20260709 12:26:07.109534 16164 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.125821 15904 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.105330 15945 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.119586 10025 rpcz_store.cc:276] 0709 12:26:05.159886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.160039 (+   153us) service_pool.cc:224] Handling call
0709 12:26:05.182387 (+ 22348us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:05.184363 (+  1976us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:05.184392 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:05.209975 (+ 25583us) log.cc:844] Serialized 52 byte log entry
0709 12:26:05.345819 (+135844us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:05.345865 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:05.345933 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.056326 (+1710393us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:07.109289 (+ 52963us) raft_consensus.cc:1685] finished
0709 12:26:07.109310 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.109533 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26982,"raft.queue_time_us":540,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":21400,"spinlock_wait_cycles":5117696,"thread_start_us":117671,"threads_started":1,"wal-append.queue_time_us":205203}
I20260709 12:26:07.103516 15796 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:07.121589 16233 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 12:26:07.135468 15772 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.136864 16182 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:07.146101 16200 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:07.148860 16209 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.397s	user 0.002s	sys 0.000s
I20260709 12:26:07.161301 16200 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:07.162940 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:07.121851 16198 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.119320 16230 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:07.138729  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.139473  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45106.6us
I20260709 12:26:07.140188 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:07.167548 10674 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.148952 10676 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:07.151502 15758 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:07.170608  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:07.177955 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:07.178272 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:07.179286 10091 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:26:07.185940 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:07.190279 10090 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:07.192214 10674 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:07.192601 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:07.198050 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:07.198130 10097 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:26:07.171599 16230 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.184732 15945 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:07.179013 16179 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:07.221451  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 83989us
I20260709 12:26:07.223125 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:07.223737 10021 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.226621 16200 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:07.235409 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:07.237939 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:07.238761 10021 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:07.240523 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:07.240897 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:07.241492 10016 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:07.239781 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:07.241170 10025 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:07.243357 10024 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 16. Ops: []
I20260709 12:26:07.202791 16198 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:26:07.246287 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:07.246914 10021 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:07.250903  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.242691 16160 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:07.253034 16160 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.220971 16179 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.257755 16179 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.261576 16017 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:07.269519  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.229938 16200 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.270843 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:07.249287 15945 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:07.279251 16200 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.277258 15904 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:07.299896 16179 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.302412 10021 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:07.303838 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:07.311337  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:07.312881  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.315233  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:07.315881  9941 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.288620 15796 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:07.322844 10016 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:26:07.324748  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:07.325606  9948 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:07.327636  9941 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:07.329375  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.332743  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.333246  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.335603  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:07.336019  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.336653  9941 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:07.339774  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:07.341886 15945 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.343276  9944 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:07.343684  9944 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 13 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:07.344497  9944 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.345674  9944 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:07.328599 15992 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.351033 15992 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.353322 15992 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.361729 16160 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.363549  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:07.365204  9948 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
W20260709 12:26:07.372308 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.378191  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:07.380908  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.381932  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.383055 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:07.322629 15796 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.393757 16179 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:07.396608  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.397578  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.398053 16200 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.401144 16221 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:07.401765 16221 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.404052 16221 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:07.404357 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:07.404644 16221 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:07.404989 10016 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.405189 16221 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:07.415155 16229 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.325s	user 0.002s	sys 0.001s
I20260709 12:26:07.417027 16063 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:07.418241 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:07.419283 16113 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.419852 16113 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.422490 10098 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:07.423055  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.423882  9949 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 12:26:07.424636 16207 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.821s	user 0.006s	sys 0.000s
I20260709 12:26:07.398492  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:07.434345 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:07.389434 15796 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:07.435210 10094 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:07.433322 10098 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:07.438282 10016 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:07.441771  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.431949 16107 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:07.460052 16187 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.465721 10094 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:07.436841 16113 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.455569 16200 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:07.431813 15893 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 12:26:07.465128 16187 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.443845 16021 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:26:07.385751 16230 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.488462 15872 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.472474 15992 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:26:07.502063 15992 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:07.502486 15992 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:07.503829 16230 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:07.503439 15992 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.505645 16187 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:07.505522 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:07.505788 15992 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.485080 16146 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 12:26:07.526346 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:07.527246 10094 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:07.533087 16099 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.533969 16099 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.534861 16185 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:07.536629 16099 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:07.536965 16206 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.963s	user 0.002s	sys 0.000s
I20260709 12:26:07.515836 16221 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:07.543102  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1273) took 1098 ms. Trace:
W20260709 12:26:07.585841 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.593926 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:07.550616  9945 rpcz_store.cc:276] 0709 12:26:06.444794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.445427 (+   633us) service_pool.cc:224] Handling call
0709 12:26:06.456723 (+ 11296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.457177 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.457219 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.457817 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.566336 (+108519us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:06.566374 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:06.566437 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.536247 (+969810us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:07.542773 (+  6526us) raft_consensus.cc:1685] finished
0709 12:26:07.542799 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.543018 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5647,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":73023872,"thread_start_us":108114,"threads_started":1,"wal-append.queue_time_us":116089}
I20260709 12:26:07.518687 15872 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.606554 15872 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:07.607738 16182 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.608337 16182 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.596350 16113 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 12:26:07.611824 16113 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:07.612196 16113 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:07.613040 16113 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.614279 16113 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:07.618055  9944 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:26:07.617049 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64511.5us
I20260709 12:26:07.634084 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:07.634404  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 115264us
I20260709 12:26:07.638042  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:07.627097 16099 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:07.647640  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 127014us
I20260709 12:26:07.650470 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:07.636348 16166 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:26:07.643013 16021 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.644832 15992 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:07.652415 16099 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.689029 16168 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:07.689182 16021 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.689904 16168 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:07.671187  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer 9aa345bcd43b44088114b7b6f2830b17 from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 13 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:07.672662 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:07.689066 16099 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.691303 10673 raft_consensus.cc:2376] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 18. Re-sending same reply.
I20260709 12:26:07.689077 15992 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:07.695390 15992 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:07.702915 15967 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:07.703510 15967 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.704644 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:07.705703 15967 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:07.705855  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:07.706440  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.707108  9947 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:07.707226  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:07.708045 16258 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:07.708675  9942 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:07.709461  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.707126  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:07.675839 10091 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:07.680236 15904 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:07.653966 16233 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 12:26:07.721282 15904 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:07.682283 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 299300us
I20260709 12:26:07.722718 16184 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:07.723073 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:07.724666 16184 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:07.682897 16128 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:07.685458  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.656638 16169 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:26:07.710032  9949 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:07.725687 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:07.716076  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:07.643745 15893 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 12:26:07.698444 16228 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.603s	user 0.000s	sys 0.000s
I20260709 12:26:07.729233  9944 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:07.730438  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.732533  9941 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:07.727952  9943 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:07.740285 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:07.743583 10021 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:07.744553  9949 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:07.746867 10021 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:07.749431 10021 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:07.751294 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:07.753899 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:07.719265 16254 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:07.755976 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:07.756290  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.756434  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:07.760830 16200 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:07.761901 10673 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:07.761808  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:07.765549  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:07.760015 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:07.767238  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:07.764438 16212 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:07.727437 16185 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.778961 10098 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 11. Current term is 12. Ops: []
I20260709 12:26:07.777846 10018 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:26:07.769228 16212 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.764597 16190 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 12:26:07.755761 16221 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:07.787083 16193 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:07.786923 16221 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.787506 16193 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:07.787865 16193 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.725579 15796 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.771072 16200 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:07.783365 16017 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:07.782569 16212 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:07.813146  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52703.1us
I20260709 12:26:07.816068 16257 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:07.816720 16257 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:07.817363 16262 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:07.817636 16257 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.798031 16233 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:07.725587 16184 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.828197  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:07.790079 16198 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:26:07.809324 16200 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:07.806174 16221 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.813272 15859 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:07.835448  9941 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:07.846194 16182 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:07.790503 16185 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.792577  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:07.868896 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 112971us
W20260709 12:26:07.862367  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 96878.5us
I20260709 12:26:07.880116  9944 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:07.881244 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:07.884249 10674 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 16. Ops: []
I20260709 12:26:07.885100 10671 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:07.887266 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:07.887524  9948 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:26:07.888677 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:07.865271 15893 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:07.859215 15859 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:07.887966 10673 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:07.887043 16169 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:07.875051 16171 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:07.893040 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:07.899483 10676 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:07.910800 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:07.893839 16021 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:07.935400 16260 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:26:07.943886 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1082) took 1395 ms. Trace:
I20260709 12:26:07.944291 10017 rpcz_store.cc:276] 0709 12:26:06.549141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.549345 (+   204us) service_pool.cc:224] Handling call
0709 12:26:06.549703 (+   358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.553042 (+  3339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.553088 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.553670 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 12:26:07.549227 (+995557us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18004a07a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:26:07.654844 (+105617us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:07.654908 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.655079 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.938392 (+283313us) raft_consensus.cc:1685] finished
0709 12:26:07.938412 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.943816 (+  5404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15532,"raft.queue_time_us":8051,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":14305,"spinlock_wait_cycles":6086016,"thread_start_us":105287,"threads_started":1,"wal-append.queue_time_us":1175625}
I20260709 12:26:07.947423 15796 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:07.948971  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:07.949636  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:07.951761 16200 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:07.952992 16185 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:07.963639 16123 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:07.965423 16200 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:07.965684 16123 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:07.964268 16169 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:07.967396 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:07.969511  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 176995us
I20260709 12:26:07.971292 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:07.966297 16123 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:07.980093 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:07.973785  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 145697us
I20260709 12:26:07.975139 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2197) took 1994 ms. Trace:
I20260709 12:26:07.980967 10023 rpcz_store.cc:276] 0709 12:26:05.980297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:05.980440 (+   143us) service_pool.cc:224] Handling call
0709 12:26:05.980737 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:06.169670 (+188933us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:06.169702 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:06.170172 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 12:26:06.413206 (+243034us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:06.421203 (+  7997us) log.cc:844] Serialized 14 byte log entry
0709 12:26:06.421786 (+   583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:07.960910 (+1539124us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540038d1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:07.960979 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:07.961067 (+    88us) raft_consensus.cc:1685] finished
0709 12:26:07.974927 (+ 13860us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:07.975084 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7936,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":1238,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":189,"spinlock_wait_cycles":434793472,"thread_start_us":242661,"threads_started":1,"wal-append.queue_time_us":266617,"wal-append.run_cpu_time_us":2035,"wal-append.run_wall_time_us":1006673}
I20260709 12:26:07.983235  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:07.976136 10671 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:26:07.957633 16168 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.987099 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:07.991351 16099 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.994128 16128 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:26:07.971968 15992 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:07.997813 15992 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.001510 16230 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:08.002084 16230 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.003240 16231 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:08.003857 16231 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.005004  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.005867  9948 raft_consensus.cc:2410] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 16 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:26:07.989830 16198 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:08.007371 16231 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:07.987792 15796 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:08.004280 16230 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.008046 15796 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:08.008479 15796 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:08.013339  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:07.971972 16200 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:08.017717  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.013756 16259 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:26:08.019742  9943 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:26:08.020047 16057 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:08.020435 16057 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:08.021299 16057 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.021965 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:08.022645 10093 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:08.023783 10676 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:26:08.017781 16221 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.025583 16221 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.027832 16221 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.028373 16221 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:08.028852 16221 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.029449 10674 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:07.988294 16063 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:08.018669 16128 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:08.031775 16128 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:08.032328 10669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1265) took 1599 ms (client timeout 1785 ms). Trace:
W20260709 12:26:08.032721 10669 rpcz_store.cc:269] 0709 12:26:06.433209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:06.433333 (+   124us) service_pool.cc:224] Handling call
0709 12:26:08.032145 (+1598812us) spinlock_profiling.cc:243] Waited 505 ms on lock 0x7b54006126d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:08.032262 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1161033472}
I20260709 12:26:08.036144 16164 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:08.036558 16164 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:08.037424 16164 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.040248 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.040879 10672 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:08.022364 16057 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:08.042969 16080 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.043529 16080 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.046700 16099 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.046957 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.049013 16230 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:08.049602 16230 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.052670 16230 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.053018 16201 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.019634 16268 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.054464 16258 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:08.054431 16268 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.055270 16258 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.056064 16141 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:08.057078 16268 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.057641 16268 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.057790 16141 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:26:08.058279 16141 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:08.059870 16141 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.061144 16141 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:26:08.053596 16201 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.062950 10097 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:08.058394 16258 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.067312 10095 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:08.067463 16104 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:08.067894 16104 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:08.068675 16104 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.065215 16201 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:08.072831 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:08.074259 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:08.074898 10017 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:08.075060  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:08.075610  9949 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:08.077939 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:08.065322  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.079975  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.080567  9948 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:08.081681  9948 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:08.085853 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:08.087472 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.090423 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.094949 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.096269 10093 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:08.101239 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.101774 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:08.092677 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:08.107064 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.110391 10674 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:08.120214 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.121206 10672 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:08.123762 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.092854 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.134158 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:08.134455 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:08.135423 10023 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:08.137498 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:08.139626  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 120649us
I20260709 12:26:08.140056 10092 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:08.069859 16104 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:08.143857 10090 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:08.151804 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.158329 10093 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:26:08.151494 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.159389 10099 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:08.160751 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:08.162458 10677 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:08.163808 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:08.148116  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:08.142778  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:08.046419 16080 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.164450 10024 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
W20260709 12:26:08.170437 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 77872.4us
I20260709 12:26:08.058140 16268 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.171705  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:08.185531 10023 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:08.191744 10022 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:08.191989 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.173728 10017 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:08.200470 10675 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:08.134755 15893 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.233042 10025 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:08.212546 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:08.222240 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:08.239735 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:08.205029 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.248581 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:08.248802  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.250541 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:08.251900 10024 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 17 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:26:08.244297  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:08.174453 16200 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:08.261863  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 90274.7us
I20260709 12:26:08.145427 16009 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:08.261018  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61147.8us
I20260709 12:26:08.174721 16017 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.268965  9940 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:08.269299 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56889.6us
I20260709 12:26:08.273337 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.235843 16168 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:08.283882 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:08.284145 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.233335 16221 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.200742 16099 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:08.284307 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:08.290403 10022 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:08.153369 16169 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.151s
I20260709 12:26:08.190160 16231 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.116s
I20260709 12:26:08.207962 10090 raft_consensus.cc:2410] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:08.189985 15869 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:08.309157 15869 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.310021 15869 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.314693 10022 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:08.326629  9948 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:08.234424 16264 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:08.077283 16123 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.339037 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:08.339205  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 72867.4us
I20260709 12:26:08.275847 16187 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:08.190836 16259 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
W20260709 12:26:08.352900  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 84644.3us
I20260709 12:26:08.354064  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:08.286036 15970 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:08.287556 15758 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.261919 15893 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.283470 16266 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:08.349056 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 126841us
I20260709 12:26:08.287233 16200 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:08.288120 15992 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.374845  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.283845 16184 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
W20260709 12:26:08.401762  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61711.3us
I20260709 12:26:08.338841 16009 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:08.288282 16017 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.334559 16255 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:26:08.339704 16185 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:26:08.353763 16264 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:08.413808  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59823.9us
I20260709 12:26:08.415014  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:08.342012 16099 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.421082  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.434989  9941 raft_consensus.cc:2410] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 12:26:08.422874 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:08.417969 16212 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.001s
I20260709 12:26:08.431398  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.379459 16187 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:08.445933  9940 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:08.448238 16113 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:08.448343 16264 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.449455 16164 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.451406  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.379241 15966 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:26:08.454994 16017 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.456931 16009 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.457777 16212 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.459022  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.375296 16123 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:08.407431 15970 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:08.455652  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:08.507189 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44697.6us
I20260709 12:26:08.507907  9948 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:08.458418 16268 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:08.444542 16193 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.656s	user 0.012s	sys 0.000s
I20260709 12:26:08.518646 16200 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.518901 16193 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.521245 10676 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:08.521610 16193 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:08.522044 16212 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.523288 15893 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.525040 10672 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:08.471843 16113 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.526945 16113 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:08.472146 16272 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
W20260709 12:26:08.500161  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 249299us
I20260709 12:26:08.470112  9949 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:08.532485 10669 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:08.533663  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.534139  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.381537 15758 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.423630 16266 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.472289 15861 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:08.412442 15992 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.514560 15970 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.516350 16017 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:08.526711 16160 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:08.531800 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.535851 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:08.500970 16190 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:08.478407  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:08.540288 10673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.006s	sys 0.011s
I20260709 12:26:08.476184 16255 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:08.496809 16021 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:08.550887 10673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.007s	sys 0.011s
I20260709 12:26:08.562713 16166 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:08.563648 16166 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:08.564056 16166 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:08.564905 16166 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.558779  9838 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:08.568558 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:08.546219 16266 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:08.570254 15992 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.556586 16185 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.530982 16187 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.557313 16184 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:08.547457 15861 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.634527 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:08.626653 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:08.606734 16193 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.561075 16021 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.684038  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.687489  9949 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 12.
I20260709 12:26:08.680523  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:08.688951  9991 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:08.678278 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:08.608121 16179 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:08.709692 16280 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.110s	user 0.001s	sys 0.001s
I20260709 12:26:08.640690 16056 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.682448 15992 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.742959 15992 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.745169 15992 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.746165  9838 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 9 to 14, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 14 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:08.748023  9838 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:08.749291  9838 catalog_manager.cc:5697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 14, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:08.680924 16160 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:08.751395 16160 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:08.752585 16160 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.756587 10672 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:08.617913 16184 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.758877 16128 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:08.761051 16184 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:08.761493 15861 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.631124 15859 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:08.726974 15796 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.718s	user 0.012s	sys 0.000s
I20260709 12:26:08.762879 15859 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.762743 15796 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.763980 10091 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:08.765134 15796 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.701858 15904 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:08.766009 15904 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:08.766223 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:08.715322 16179 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.701047 16255 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:08.735617 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:08.786842 10673 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:08.736238 15758 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.792949  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:08.793231  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:08.694757 16193 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.702207 15771 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:26:08.687649 16080 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:08.741225 16104 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.569s	user 0.014s	sys 0.000s
I20260709 12:26:08.614499 16185 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.768221 15869 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.810073 16185 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.773574  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:08.810861  9949 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:08.811468 15869 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.812355  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:08.773820  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.779492  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:08.814674 16274 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.199s	user 0.002s	sys 0.000s
I20260709 12:26:08.815027  9948 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:08.795532  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:08.816949 15370 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:08.817376 15370 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.817706 15370 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:08.820106 15861 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:08.820489 10021 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:08.822254 16179 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:08.798880 16193 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:08.800352 16266 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.805239  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:08.805932 16169 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:08.827723 15370 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:08.796757  9943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.009s	sys 0.015s
W20260709 12:26:08.829020  9943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.010s	sys 0.015s
I20260709 12:26:08.819788  9949 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:08.830931  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.833279 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:08.829682 10018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.010s	sys 0.006s
W20260709 12:26:08.833957 10018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.011s	sys 0.006s
W20260709 12:26:08.824451 10023 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.008s	sys 0.016s
W20260709 12:26:08.834780 10023 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.009s	sys 0.016s
I20260709 12:26:08.834463 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:08.835160  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.836079  9949 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:08.837602 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.838497 10094 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:08.786396 16255 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:08.785887 16128 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:08.849361  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44228.6us
I20260709 12:26:08.853593  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:08.849092 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:08.850764 10096 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:08.854386  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:08.852334  9949 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:08.849018 16171 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 12:26:08.857529 10099 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:26:08.857630  9949 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:26:08.859069  9949 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:26:08.858920 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.859700 10093 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 11.
I20260709 12:26:08.823170 16286 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:26:08.861445 16286 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.861539 16285 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:08.862159 16286 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.863922 16021 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.865335  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:08.767596 16056 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.825665 16099 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.830802 16201 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:08.861478  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:08.870514 15370 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.850283 16017 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.855809 16255 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:08.853279 16254 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:08.855500 15758 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:08.890133 10095 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:08.896081 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:08.897202 10099 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:08.894065  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:08.796185 16113 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.898300  9947 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:08.891909  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 447044us
I20260709 12:26:08.873188 16259 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:08.867161 16293 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:08.787854 15904 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.885291 15966 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:08.873458 16290 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:08.943846  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:08.946926 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:08.947636 10093 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:26:08.947991 16056 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:08.952414 16266 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:08.961655 15966 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.963047 16286 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.964184 15966 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:08.965513 15869 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:08.965919 15869 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:08.966295 15869 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:08.968832  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:08.971969 15370 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:08.869630 16128 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.977418  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:08.978139 16285 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:08.981365 16182 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:08.982122 15967 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:08.983203 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 273775us
I20260709 12:26:08.986238 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:08.987767 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:08.988957 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:08.898437 16017 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:08.990374 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:08.992156 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 216719us
I20260709 12:26:08.988891 16063 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:08.997478 10021 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:08.888909 16201 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:08.996307 10093 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:26:08.998564 15370 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:09.000205 10019 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:08.991881 16182 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:09.009276 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.015635 10091 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:09.017154  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2245) took 1003 ms. Trace:
I20260709 12:26:08.941658 16259 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.017704  9838 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 14, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 14 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:09.023988  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:09.019009 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:09.019577 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:09.020495  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:09.025645 10673 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:09.025900 10095 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:09.034140 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:09.034847 10673 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.038321 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:09.038914 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.044283 10016 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:09.045414 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.039184 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.047405 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.047717 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:09.040623 10017 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:26:09.049016 10019 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:26:09.039851 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.049746 10022 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:09.038712 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.050381 10019 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:26:09.053399  9945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.000s	sys 0.044s
I20260709 12:26:09.037381  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:09.055011  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:09.056623  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:09.027922 16168 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.693s	user 0.013s	sys 0.002s
I20260709 12:26:09.061699  9942 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:08.998003 16293 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:26:09.058573 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:09.060163  9945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.000s	sys 0.045s
I20260709 12:26:09.065161 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:09.069056 10025 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.069380 10670 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:09.025141 15966 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:09.066224 10020 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:09.070240 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.069443 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.045140 15796 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:09.092283 16113 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.058418 16169 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:09.094295 16169 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.094709 16169 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:09.083806 16009 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:09.096530 16009 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:09.096925 16009 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.097200 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1086) took 1027 ms. Trace:
I20260709 12:26:09.097568 10018 rpcz_store.cc:276] 0709 12:26:08.069969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.070095 (+   126us) service_pool.cc:224] Handling call
0709 12:26:08.070415 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.070808 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.070844 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.157099 (+ 86255us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.834388 (+677289us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:08.834426 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.834489 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.096970 (+262481us) raft_consensus.cc:1685] finished
0709 12:26:09.096991 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.097142 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3127,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78,"thread_start_us":567610,"threads_started":1,"wal-append.queue_time_us":857466}
I20260709 12:26:09.042186 16056 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:09.101640 10090 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:09.054287 16185 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:09.106981 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53865.1us
I20260709 12:26:09.112200 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:09.035627 16292 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:09.100979 15796 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.112557 16296 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:09.120783 16296 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.121587 16169 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.115550 16182 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.124119 16182 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:09.124549 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:09.125137 10018 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:09.127452 16201 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.071631 16168 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:26:09.118247 15758 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 12:26:09.064378 16298 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:09.118701 16063 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:09.129980 16298 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:09.130319 16063 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.130910 16298 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.132968 16063 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:09.134637 16128 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:09.118847 15992 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.136099 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.099550 16285 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:09.136328 15992 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.137015 16285 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:09.137856 16285 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.139014 16285 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:09.140702 10018 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:09.141886  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:09.142882 16230 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:26:09.143129 16289 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.202s	user 0.001s	sys 0.001s
I20260709 12:26:09.143409 16230 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:09.144285 16230 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.118959 15796 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.146745 16168 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.147240 16168 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.087280 16212 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.068632 16286 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:26:09.150981  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.108063 10025 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:09.152606  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:09.067226 16290 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:09.115613 16056 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.161218 10095 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:09.162693 10673 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:09.163760  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:09.164603 16268 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:09.164741 16017 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:09.165196 16268 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:09.160161 16290 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:09.166103 16268 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.166986 16290 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:09.168392 16123 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.658s	user 0.002s	sys 0.009s
I20260709 12:26:09.108184 16293 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
I20260709 12:26:09.157835  9947 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:26:09.170475  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2249) took 1109 ms. Trace:
I20260709 12:26:09.173035  9943 rpcz_store.cc:276] 0709 12:26:08.061103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.061235 (+   132us) service_pool.cc:224] Handling call
0709 12:26:08.061570 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.062851 (+  1281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.062891 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.176443 (+113552us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.832314 (+655871us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:08.832358 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.832431 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.170238 (+337807us) raft_consensus.cc:1685] finished
0709 12:26:09.170262 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.170417 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3840,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":544232,"threads_started":1,"wal-append.queue_time_us":901116}
I20260709 12:26:09.101027  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2244) took 1087 ms. Trace:
I20260709 12:26:09.159897 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.175468  9945 rpcz_store.cc:276] 0709 12:26:08.013255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.013478 (+   223us) service_pool.cc:224] Handling call
0709 12:26:08.021219 (+  7741us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.021407 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:08.512756 (+491349us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.100471 (+587715us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.100546 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:09.100650 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.100782 (+   132us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.100963 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":160890,"prepare.queue_time_us":389781,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":29956,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":9343,"thread_start_us":868910,"threads_started":2}
I20260709 12:26:09.181629  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.119091 16170 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 12:26:09.188328  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:09.170552 16301 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:26:09.154090 16292 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.194072  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:09.235754  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.199092 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.236644  9991 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:09.236913 10020 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:09.165925 16255 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.170296 15859 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:09.169731 15869 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.239261 16292 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.239275  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:09.228291  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.167042 16169 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.103835 15966 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.269218 15966 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:09.184652 16304 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:09.150981 16212 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.273494 16212 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:09.275115  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2461) took 1175 ms. Trace:
I20260709 12:26:09.275538  9941 rpcz_store.cc:276] 0709 12:26:08.100220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.474351 (+374131us) service_pool.cc:224] Handling call
0709 12:26:08.474704 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.475073 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.475113 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.501338 (+ 26225us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.887793 (+386455us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:08.887833 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.887898 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.274492 (+386594us) raft_consensus.cc:1685] finished
0709 12:26:09.274519 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.274872 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":52778,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":77,"thread_start_us":356778,"threads_started":1,"wal-append.queue_time_us":532733}
I20260709 12:26:09.277292  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.275146 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.262091  9943 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:09.181770 16259 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:09.283824  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.285059 16169 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:09.289655 16255 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.266328 15869 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:09.278434  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.304392  9941 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:09.160385 16185 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.235445 16123 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.314193  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.322472  9943 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:09.259635 16164 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:09.330437  9945 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:26:09.332520  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:09.306770  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:09.263067 15992 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.286077 16169 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.340063  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.264505 16296 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.397677 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.399806  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:09.401234 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 349331us
W20260709 12:26:09.403288  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.342041  9941 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:09.350745 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 501704us
W20260709 12:26:09.395929 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 284419us
I20260709 12:26:09.409310 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:09.420372  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.421977 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:09.422173  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.427976 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:09.341531 16200 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:09.429982 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:09.434736  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:09.443468 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:09.396700 16164 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:09.331264 16123 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.454259 15771 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:09.511750 15893 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:09.512203 15893 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:09.512534 16221 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:09.512908 16221 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:09.512980 15893 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.514076 15893 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.513779 16221 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.514810 16221 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:09.455164 16082 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:09.518819 16082 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:09.519248 16082 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:09.526916 16315 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:09.527446 16099 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:09.528045 16099 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:09.528201 16315 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:09.528717 16315 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.528990 16099 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.531281 16315 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.532178 16169 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:09.537042 16316 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:26:09.505232 16295 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:26:09.441926 16084 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:09.541533 16212 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:09.542155 16212 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.544536 16212 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:09.457705 16309 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:26:09.447628 16200 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:09.549021 16303 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.415s	user 0.000s	sys 0.001s
I20260709 12:26:09.550791 16200 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.551857 16179 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:09.552552 15992 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:09.552451 16179 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.553153 15992 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:09.555765 16308 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.433s	user 0.001s	sys 0.001s
I20260709 12:26:09.556114 16009 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:09.456730 16311 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:09.542198 15796 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:09.491148 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63237.9us
W20260709 12:26:09.487774 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78573.1us
W20260709 12:26:09.495553 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 480785us
I20260709 12:26:09.445024 16063 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:09.441619 16056 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:09.501745  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.562201  9838 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 17, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 17 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:09.505519 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62142.2us
I20260709 12:26:09.583539  9944 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:09.510887 16123 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:09.526952 16332 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:09.532510  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:09.595533  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 160887us
I20260709 12:26:09.478790 16057 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:09.598985  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2247) took 1549 ms. Trace:
I20260709 12:26:09.506660  9943 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:09.600389  9941 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:09.587551 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.588711 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.631127 10025 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:09.589923 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:09.590657 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.588907 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.589184 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:09.632755 10019 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:09.633271 10672 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:09.634068 10091 raft_consensus.cc:2368] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:26:09.584856 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:09.615352  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.614250 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.611442  9942 rpcz_store.cc:276] 0709 12:26:08.049134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.049266 (+   132us) service_pool.cc:224] Handling call
0709 12:26:08.049556 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.062091 (+1012535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.593335 (+531244us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540003f554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:09.598905 (+  5570us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2326945024}
I20260709 12:26:09.618840 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.640515 10020 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:09.619504 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:09.620152  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.492496 15859 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:09.493994 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1321) took 1479 ms. Trace:
I20260709 12:26:09.623448 10024 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:26:09.563573 16179 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:09.564371 16305 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.429s	user 0.000s	sys 0.001s
I20260709 12:26:09.614400 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.598642 16021 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:09.636437  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:09.617673 16201 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.642414 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.645678 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:09.591902 16063 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:09.610718 16200 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:09.648463 16285 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.000s
W20260709 12:26:09.635851  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 483296us
W20260709 12:26:09.648831 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64068.3us
I20260709 12:26:09.675092 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:09.638425 16123 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:09.650133 16319 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:09.678905 16123 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.679548 16319 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:09.680012 16319 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:09.681366 16123 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.682053 15859 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.683822 16009 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:09.685549 15796 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.562165 16185 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:09.616884 16323 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:09.686918 16323 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:09.642467  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.689227 16082 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.690071  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:09.625022 16111 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:09.655426  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.692843 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1286) took 1585 ms. Trace:
I20260709 12:26:09.692559 16111 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.693225 10669 rpcz_store.cc:276] 0709 12:26:08.107918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.108099 (+   181us) service_pool.cc:224] Handling call
0709 12:26:08.108454 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.533056 (+424602us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.692623 (+1159567us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b5400690d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:09.692790 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":974596992}
I20260709 12:26:09.693843 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:09.694940 16082 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:09.695552 16101 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:09.696017  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:09.019913  9946 rpcz_store.cc:276] 0709 12:26:08.013521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.018995 (+  5474us) service_pool.cc:224] Handling call
0709 12:26:08.019260 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:08.019447 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:08.642419 (+622972us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.016803 (+374384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.016862 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:09.016922 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.016987 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.017100 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57976,"prepare.queue_time_us":331111,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"raft.queue_time_us":353,"thread_start_us":865869,"threads_started":2}
I20260709 12:26:09.701025 16326 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:09.701768 10016 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:09.701581 16326 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.702781 16200 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.703776 16326 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:09.703994 10017 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:09.704789 16200 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:09.704967 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.710439 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.712251 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:09.712973 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:09.705555 10022 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:09.716049 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:09.717826  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:09.719271 10019 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:09.655921 10097 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:09.721403 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.722030 10022 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:09.646893  9944 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:09.659459  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:09.723642  9942 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:09.715426 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:09.726531 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 51527us
I20260709 12:26:09.712680 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:09.726699 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.732007 10019 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:26:09.642481 10671 rpcz_store.cc:276] 0709 12:26:08.014809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.015054 (+   245us) service_pool.cc:224] Handling call
0709 12:26:08.015345 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:08.015733 (+   388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.015762 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:08.154930 (+139168us) log.cc:844] Serialized 26 byte log entry
0709 12:26:08.632355 (+477425us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:09.031370 (+399015us) log.cc:844] Serialized 14 byte log entry
0709 12:26:09.493436 (+462066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.493575 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.493635 (+    60us) raft_consensus.cc:1685] finished
0709 12:26:09.493654 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:09.493893 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52318,"prepare.queue_time_us":84036,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":77002,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":5787,"thread_start_us":1111638,"threads_started":3,"wal-append.queue_time_us":665689}
I20260709 12:26:09.733985 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.734696 10022 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:09.737406  9946 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:26:09.737818  9946 pending_rounds.cc:85] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 8
I20260709 12:26:09.738032  9946 pending_rounds.cc:107] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:26:09.738253  9946 raft_consensus.cc:2889] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:09.625269 15904 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:26:09.742332 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:09.747000 10669 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:09.650285 16329 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.651661 16113 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:09.656841 16063 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:09.742062 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:09.626171 16057 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.750811 10017 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:09.727438 16017 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.689275 16323 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:09.756680 16290 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 12:26:09.732262 16285 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.767558  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:09.804076 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48896.4us
I20260709 12:26:09.810027 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:09.740442 15869 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.744877 16185 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:09.656550 16201 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.811072  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.778432 10669 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:09.810783 10017 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:09.756367 16009 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:09.816131  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:09.757946 16082 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:09.785805 16113 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:09.968667  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:09.766646 15992 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:09.819744 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48752.8us
I20260709 12:26:09.757633 15796 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.213s
W20260709 12:26:09.765268 16298 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.018s	sys 0.009s
I20260709 12:26:10.035687  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:10.108227 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:10.092051 16099 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.038s	sys 0.051s
I20260709 12:26:09.764793 16111 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:09.994854 16182 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.870s	user 0.011s	sys 0.000s
I20260709 12:26:09.805832 16231 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:10.127463 10094 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:10.095155 16305 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 12:26:09.770068 16330 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:09.819355 16293 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:09.813898 16200 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:10.143541 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:10.109477 16255 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.031s	sys 0.057s
I20260709 12:26:09.991453 16009 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:10.124356 16221 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.609s	user 0.012s	sys 0.004s
W20260709 12:26:10.123027 16298 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.989s	user 0.022s	sys 0.011s
W20260709 12:26:10.124691 16301 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.306s	user 0.000s	sys 0.000s
I20260709 12:26:09.804056 15771 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.329s
I20260709 12:26:10.036305 15962 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:09.775916 16160 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.346s
I20260709 12:26:10.089921 16082 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:09.759809 16329 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:09.962502 16146 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.171s
W20260709 12:26:10.217809 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 450793us
I20260709 12:26:10.179927 16268 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:09.964722 15869 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:09.815414 16017 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:10.184968 16230 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.033s	user 0.056s	sys 0.072s
I20260709 12:26:09.817086 16290 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:10.156447 16099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.038s	sys 0.052s
I20260709 12:26:10.125128 15992 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.150961 16292 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.889s	user 0.012s	sys 0.000s
W20260709 12:26:10.225446 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 462396us
I20260709 12:26:09.966002 16285 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:10.218863 15859 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.052s	sys 0.096s
I20260709 12:26:10.235721 10672 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:26:10.220279 16255 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.032s	sys 0.057s
I20260709 12:26:10.221262 10668 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:26:10.415517 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:10.416602 16328 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:26:10.416715 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:10.290158 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:10.372527 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:26:10.418267 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45845.5us
I20260709 12:26:10.419039 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:10.419929 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3623) took 1357 ms. Trace:
I20260709 12:26:10.420225 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:10.420356 10093 rpcz_store.cc:276] 0709 12:26:09.062548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.067606 (+  5058us) service_pool.cc:224] Handling call
0709 12:26:09.067968 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.153522 (+ 85554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.153570 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.154105 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:26:09.709398 (+555293us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:09.709440 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.709523 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.419684 (+710161us) raft_consensus.cc:1685] finished
0709 12:26:10.419709 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.419865 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38728,"raft.queue_time_us":39920,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":476596,"threads_started":1,"wal-append.queue_time_us":1003482}
I20260709 12:26:10.421963 16184 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:10.422550 16184 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:10.423501 16184 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.424072 16017 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:10.424438 16285 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.424751 16017 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:10.222317 16330 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.425544 16017 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:10.425899 16017 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:10.402845 15992 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:10.428520 15992 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:10.428989 16330 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.430054 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:10.430370 15869 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.293617 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1119) took 1730 ms. Trace:
I20260709 12:26:10.250566 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:10.151876 16326 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:10.281388 16113 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.229739 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:10.218228 16200 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:10.217900 16293 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:10.392230 10063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.030s	sys 0.018s
I20260709 12:26:10.211290 16231 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:10.385172 16230 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.233s	user 0.059s	sys 0.081s
I20260709 12:26:10.384965 16268 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:10.225659 16009 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:10.449852 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:10.451488 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:10.426303 16017 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.403376 16057 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:10.298900 16331 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 12:26:10.295390 16221 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:10.412946 15859 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.056s	sys 0.097s
I20260709 12:26:10.413934 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:10.433107 10675 raft_consensus.cc:2368] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:10.443691  9942 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
W20260709 12:26:10.446049 10063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.031s	sys 0.019s
I20260709 12:26:10.433485 10674 rpcz_store.cc:276] 0709 12:26:08.563208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:08.563351 (+   143us) service_pool.cc:224] Handling call
0709 12:26:08.563720 (+   369us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:08.564078 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:08.564109 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:08.564681 (+   572us) log.cc:844] Serialized 52 byte log entry
0709 12:26:08.883795 (+319114us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:08.895024 (+ 11229us) log.cc:844] Serialized 14 byte log entry
0709 12:26:08.995874 (+100850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:08.995952 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:09.563534 (+567582us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:10.272431 (+708897us) raft_consensus.cc:1685] finished
0709 12:26:10.293322 (+ 20891us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.293534 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":126189,"prepare.queue_time_us":15705,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":102046,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":7617,"spinlock_wait_cycles":99813760,"thread_start_us":318672,"threads_started":1,"wal-append.queue_time_us":569972}
I20260709 12:26:10.435292 15758 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:10.489890 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:10.436466 16255 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:10.490756 10095 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:10.424451 15970 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:10.454968 16296 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:10.448204 16200 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.447964 16293 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:10.563145 10672 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:10.375211 16082 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:10.567595 10095 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:10.571553 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:10.597518 10022 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 12.
I20260709 12:26:10.439088 15796 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 12:26:10.577623  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:10.407492 16292 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.566394 16255 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:10.585045 16336 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:26:10.457011 16185 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.122s
I20260709 12:26:10.506950 16298 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
W20260709 12:26:10.580300 16063 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.804s	user 0.004s	sys 0.006s
I20260709 12:26:10.596298 10095 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:10.612810  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.617072 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:10.465149 16326 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.620745 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2411) took 1509 ms. Trace:
I20260709 12:26:10.621596 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:10.638428  9940 raft_consensus.cc:2368] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:26:10.589927 16341 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.390513 15962 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.634552 10095 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:10.644073  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:10.636257  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:10.637472 10090 rpcz_store.cc:276] 0709 12:26:09.111426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.155122 (+ 43696us) service_pool.cc:224] Handling call
0709 12:26:09.179990 (+ 24868us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.184292 (+  4302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.184329 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.185199 (+   870us) log.cc:844] Serialized 26 byte log entry
0709 12:26:09.644298 (+459099us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:09.644355 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:09.644603 (+   248us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.620490 (+975887us) raft_consensus.cc:1685] finished
0709 12:26:10.620515 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.620679 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3790,"raft.queue_time_us":4229,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":103,"thread_start_us":348893,"threads_started":1,"wal-append.queue_time_us":519531}
I20260709 12:26:10.577318 15970 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.590802 15859 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:10.649125 15992 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:10.567315 16221 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.649518 16255 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.650635 16255 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.651384 16221 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.616070 16182 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:10.604929 16342 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:10.652544 16342 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:10.654806 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1124) took 1233 ms. Trace:
I20260709 12:26:10.641120 16063 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:10.648481 16341 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.655611 10022 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:10.657233  9987 leader_election.cc:400] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:10.659842  9987 leader_election.cc:403] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:10.655772 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:10.616524 16230 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:10.681921 16230 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.683295 16268 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:10.684178 16230 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.655253 10670 rpcz_store.cc:276] 0709 12:26:09.421624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.421749 (+   125us) service_pool.cc:224] Handling call
0709 12:26:09.449836 (+ 28087us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:09.450327 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.450372 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:09.479671 (+ 29299us) log.cc:844] Serialized 52 byte log entry
0709 12:26:10.036699 (+557028us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:10.132061 (+ 95362us) log.cc:844] Serialized 14 byte log entry
0709 12:26:10.152916 (+ 20855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.153025 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.654501 (+501476us) raft_consensus.cc:1685] finished
0709 12:26:10.654524 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.654732 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77469,"prepare.queue_time_us":90651,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":26425,"raft.queue_time_us":32948,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":29887,"spinlock_wait_cycles":16490880,"thread_start_us":466802,"threads_started":1,"wal-append.queue_time_us":1144324}
I20260709 12:26:10.660900 10019 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:10.686983 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:10.675231 16335 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:10.679301 16185 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:10.679829 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2419) took 1146 ms. Trace:
W20260709 12:26:10.636839 16169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.011s
I20260709 12:26:10.684089 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:10.704819 10016 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:10.667474 16231 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:10.701512 10096 rpcz_store.cc:276] 0709 12:26:09.533878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.535240 (+  1362us) service_pool.cc:224] Handling call
0709 12:26:09.535640 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.652378 (+116738us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.652409 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.652452 (+    43us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:09.652486 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.679586 (+1027100us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b5400305554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:10.679637 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.679760 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268009856}
I20260709 12:26:10.582721 16296 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.707580 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:10.712380 10017 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:10.658938 16182 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:10.688234 16221 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:10.718432 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:10.719370 10016 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:10.700707 16230 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:10.649556 15859 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:10.697459 10672 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
W20260709 12:26:10.708062 16169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.005s	sys 0.012s
I20260709 12:26:10.722255 16342 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:10.711988 16185 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.749799 16290 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:10.784823 10019 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:10.786882 10019 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:26:10.771126 16332 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.008s	sys 0.011s
I20260709 12:26:10.775609 16350 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:10.760305 16230 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:10.795953 16332 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.008s	sys 0.011s
I20260709 12:26:10.819846 16292 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.824182 15970 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:10.832614 16348 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:26:10.842021 16319 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.161s	user 0.009s	sys 0.005s
W20260709 12:26:10.851857 16295 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:26:10.844211 16296 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.845417 15859 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:10.847285 16341 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.848243 16326 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.848853 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3413) took 1149 ms. Trace:
I20260709 12:26:10.849647 16185 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:10.807399 16350 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.857508 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:10.856521 10023 rpcz_store.cc:276] 0709 12:26:09.699223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.705037 (+  5814us) service_pool.cc:224] Handling call
0709 12:26:09.705423 (+   386us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.737594 (+ 32171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.737639 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:09.763943 (+ 26304us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.449319 (+685376us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:10.449359 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.449422 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:10.848555 (+399133us) raft_consensus.cc:1685] finished
0709 12:26:10.848579 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.848771 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31695,"raft.queue_time_us":32492,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":15056,"thread_start_us":343387,"threads_started":1,"wal-append.queue_time_us":908469}
I20260709 12:26:10.822065 16168 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:10.820091 16169 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.854982 16063 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:10.851670 16319 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.909049 16082 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:10.914021 16333 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.543s	user 0.002s	sys 0.000s
I20260709 12:26:10.920855 10669 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:10.923476  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:10.924463 16082 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:10.927731 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.928640 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2233) took 1329 ms. Trace:
I20260709 12:26:10.929219 10021 rpcz_store.cc:276] 0709 12:26:09.599126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.599259 (+   133us) service_pool.cc:224] Handling call
0709 12:26:09.599535 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.654785 (+ 55250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.654814 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.654854 (+    40us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:09.654892 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.928293 (+1273401us) spinlock_profiling.cc:243] Waited 55.1 ms on lock 0x7b540038d1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:10.928363 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.928557 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126622464}
I20260709 12:26:10.932575 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1070) took 1222 ms. Trace:
W20260709 12:26:10.932796 10091 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1059) took 1880 ms (client timeout 2099 ms). Trace:
W20260709 12:26:10.933351 10091 rpcz_store.cc:269] 0709 12:26:09.052498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.052642 (+   144us) service_pool.cc:224] Handling call
0709 12:26:10.932568 (+1879926us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540022ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:10.932728 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1243538944}
I20260709 12:26:10.936645 16221 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.937317 16221 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.937780 16182 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:10.938585 16182 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.939723 16221 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.940330 16221 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:10.941368 16182 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.945751 16170 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:10.946432 16170 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.948747 16170 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:10.949739  9948 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:10.933190 10094 rpcz_store.cc:276] 0709 12:26:09.710053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.710191 (+   138us) service_pool.cc:224] Handling call
0709 12:26:10.932287 (+1222096us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b540014a0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:10.932490 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":951580032}
W20260709 12:26:10.951956  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1307) took 1260 ms (client timeout 1662 ms). Trace:
I20260709 12:26:10.952247 16286 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:10.952453  9940 rpcz_store.cc:269] 0709 12:26:09.691698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.695753 (+  4055us) service_pool.cc:224] Handling call
0709 12:26:10.951726 (+1255973us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b54000b5254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:10.951892 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2165458688}
I20260709 12:26:10.952855 16286 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.955391 10643 leader_election.cc:400] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:26:10.955760 10643 leader_election.cc:403] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:10.956729 16357 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:26:10.959594 15962 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:10.965552 16166 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:10.966226 16350 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:10.967765  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2263) took 1548 ms (client timeout 1708 ms). Trace:
W20260709 12:26:10.968227  9943 rpcz_store.cc:269] 0709 12:26:09.418915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.421881 (+  2966us) service_pool.cc:224] Handling call
0709 12:26:10.967560 (+1545679us) spinlock_profiling.cc:243] Waited 83.4 ms on lock 0x7b5400207e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:10.967695 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191814400}
I20260709 12:26:10.969483  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:10.971127  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2483) took 1659 ms. Trace:
I20260709 12:26:10.971578  9941 rpcz_store.cc:276] 0709 12:26:09.311195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.629161 (+317966us) service_pool.cc:224] Handling call
0709 12:26:09.629475 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:09.813760 (+184285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.813789 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:09.813834 (+    45us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:09.813864 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.943869 (+1130005us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:10.970925 (+ 27056us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.971070 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423307520}
I20260709 12:26:10.973097  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:10.976498  9947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2469) took 2115 ms (client timeout 2140 ms). Trace:
W20260709 12:26:10.977000  9947 rpcz_store.cc:269] 0709 12:26:08.860826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.235422 (+374596us) service_pool.cc:224] Handling call
0709 12:26:10.944480 (+1709058us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540003eb54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:10.976431 (+ 31951us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3071985024}
I20260709 12:26:10.978969 15872 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:10.979668 15872 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:10.981514  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2273) took 1270 ms. Trace:
I20260709 12:26:10.981947  9942 rpcz_store.cc:276] 0709 12:26:09.711536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.992501 (+280965us) service_pool.cc:224] Handling call
0709 12:26:10.981339 (+988838us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b5400096354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:10.981452 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764670848}
I20260709 12:26:10.983587 16230 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:10.987568  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.988077  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.988395  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:10.988616  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1301) took 1396 ms. Trace:
I20260709 12:26:10.988996  9947 rpcz_store.cc:276] 0709 12:26:09.591757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.977955 (+1386198us) service_pool.cc:224] Handling call
0709 12:26:10.978294 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.978518 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.978543 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.988282 (+  9739us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:10.988360 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.988435 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.988560 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:10.991853  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:10.996398 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 597636us
I20260709 12:26:10.884264 16169 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.004185  9838 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 14, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 14 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:11.005867  9838 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 13, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 13 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:11.006831 10673 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:11.010993 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:11.012271 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.013764 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:11.014672 10095 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:11.015556  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1300) took 1424 ms. Trace:
I20260709 12:26:10.983726  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1302) took 1391 ms. Trace:
I20260709 12:26:11.016031 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:11.017059 10099 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:11.017582  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1303) took 1404 ms. Trace:
I20260709 12:26:11.014683 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.018487 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:11.018837 10093 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:11.017994  9947 rpcz_store.cc:276] 0709 12:26:09.612726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.989959 (+1377233us) service_pool.cc:224] Handling call
0709 12:26:10.990298 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.990494 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.990545 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.990589 (+    44us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:11.017354 (+ 26765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.017422 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.017528 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.021714  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.016089  9942 rpcz_store.cc:276] 0709 12:26:09.592020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.982842 (+1390822us) service_pool.cc:224] Handling call
0709 12:26:10.983204 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:10.983390 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.983421 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:10.983467 (+    46us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:10.983501 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.983557 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:10.983672 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.015961  9941 rpcz_store.cc:276] 0709 12:26:09.591406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.974168 (+1382762us) service_pool.cc:224] Handling call
0709 12:26:10.974475 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.015208 (+ 40733us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.015237 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.015284 (+    47us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:11.015329 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.015390 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.015500 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.025426  9941 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:26:11.026257  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1305) took 1400 ms. Trace:
I20260709 12:26:11.026749  9941 rpcz_store.cc:276] 0709 12:26:09.625570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.024553 (+1398983us) service_pool.cc:224] Handling call
0709 12:26:11.024966 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.026010 (+  1044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.026200 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.039111 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:11.041898 15872 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:11.046537 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.048518  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.050945  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1304) took 1435 ms. Trace:
I20260709 12:26:11.051515  9942 rpcz_store.cc:276] 0709 12:26:09.615608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.024051 (+1408443us) service_pool.cc:224] Handling call
0709 12:26:11.050182 (+ 26131us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.050430 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.050471 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.050533 (+    62us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:11.050622 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.050706 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.050866 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.053539 10099 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:11.056841 10017 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:11.059008 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.061966  9949 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:10.940850 16221 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:11.067628 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.068625 10090 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:10.886859 16009 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.010215 16017 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.078890 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:11.093559 10097 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:11.074115 16358 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.074569 16342 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:11.075248 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.075625 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:11.096982 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:11.072451 10017 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
W20260709 12:26:11.093693  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.097968 10017 pending_rounds.cc:85] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 10
I20260709 12:26:11.098281 10017 pending_rounds.cc:107] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 12:26:11.098538 10017 raft_consensus.cc:2889] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:11.100337 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.102242  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.103430  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.104423  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.109428 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:11.109517 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:11.111778 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:11.112717 10668 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:26:11.115953 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:11.144644  9991 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 2.140s (SENT) [suppressed 7 similar messages]
I20260709 12:26:11.242686 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:11.243706 10672 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:11.242565  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2484) took 1610 ms. Trace:
I20260709 12:26:11.244360  9941 rpcz_store.cc:276] 0709 12:26:09.631862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.241719 (+1609857us) service_pool.cc:224] Handling call
0709 12:26:11.242041 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.242231 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.242259 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.242300 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:11.242330 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.242385 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.242510 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.260004 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.286271 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:11.100260 16170 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:11.291220  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.292344  9942 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:26:11.292382 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:11.102078 16315 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:11.295367 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:11.099495 16286 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.173122 16319 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:11.242141 15992 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:11.242528 16099 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:11.317430 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 74044.9us
I20260709 12:26:11.319645 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:11.256505 16179 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:11.291893 16341 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:11.325640 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:11.296689 16315 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:11.327309 10024 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
I20260709 12:26:11.296401 16170 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.100896 16358 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.290058 15758 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:11.291018 16295 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:11.297817 16332 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:11.333012 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:11.334091 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:11.347879  9942 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:26:11.352666  9942 pending_rounds.cc:85] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 11
I20260709 12:26:11.354911 10022 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:11.357004 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:11.357760 10024 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:11.353052  9942 pending_rounds.cc:107] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:26:11.322933 16184 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:11.325939 16362 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:11.362891 10090 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:11.334350 16170 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:11.334743 16315 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:11.333405 16343 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:11.359017  9942 raft_consensus.cc:2889] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:11.385492  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1306) took 1706 ms. Trace:
W20260709 12:26:11.388638  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 244053us
W20260709 12:26:11.367717 16200 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.768s	user 0.012s	sys 0.000s
I20260709 12:26:11.325275 15992 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:11.381816 16351 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
I20260709 12:26:11.329661 16099 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:11.383095 16353 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.469s	user 0.001s	sys 0.000s
I20260709 12:26:11.387842  9942 rpcz_store.cc:276] 0709 12:26:09.679360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.347318 (+1667958us) service_pool.cc:224] Handling call
0709 12:26:11.347596 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.385240 (+ 37644us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.385436 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.366647 16363 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:11.398813  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:11.389639 16352 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.370s	user 0.000s	sys 0.002s
W20260709 12:26:11.399746  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.336691 16341 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.354702 15758 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.346837 16179 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.404942  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2270) took 1694 ms. Trace:
I20260709 12:26:11.391932 16171 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:11.463443  9942 rpcz_store.cc:276] 0709 12:26:09.710680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.404182 (+1693502us) service_pool.cc:224] Handling call
0709 12:26:11.404474 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.404635 (+   161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.404671 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.404710 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:11.404749 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.404800 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.404891 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.459192 10670 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:11.369619 16185 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
W20260709 12:26:11.469990 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47051.3us
I20260709 12:26:11.369131 16184 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.356303 16332 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.354573 16295 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:11.381721 16362 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.403640 16082 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.119s
I20260709 12:26:11.394171 16200 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.389285 16343 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.403983 16363 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:11.630507  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.660058 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.556880 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:11.647307 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.490793  9942 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:11.748521 10093 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 12: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:11.747715 10021 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 12.
W20260709 12:26:11.459038 16361 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.303s	user 0.001s	sys 0.000s
I20260709 12:26:11.645051 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:11.754285  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2487) took 2034 ms. Trace:
I20260709 12:26:11.511408  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2267) took 1931 ms. Trace:
I20260709 12:26:11.469959 10024 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:11.460490 16009 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.705530 16190 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:11.746201 16160 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:11.754904  9942 rpcz_store.cc:276] 0709 12:26:09.719811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.490097 (+1770286us) service_pool.cc:224] Handling call
0709 12:26:11.490443 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.748281 (+257838us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.754219 (+  5938us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:11.761102  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.459625 16169 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:11.711918 16372 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.241s	user 0.000s	sys 0.000s
I20260709 12:26:11.674005 16381 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:11.729985 16293 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.123s	user 0.011s	sys 0.000s
I20260709 12:26:11.759588 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:11.645252 16369 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 12:26:11.776348 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1075) took 1219 ms. Trace:
I20260709 12:26:11.624966 16286 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:11.466749 16221 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:11.618913 16166 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.653s	user 0.010s	sys 0.000s
I20260709 12:26:11.499711 16171 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:11.711259 16378 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:26:11.768769 10098 raft_consensus.cc:2368] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:11.795646  9944 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:11.645515 16367 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.731455 15758 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.728981 16321 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.460752 16255 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.297s
I20260709 12:26:11.644605 16358 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.503259 16017 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.468541 16233 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.776793 10097 rpcz_store.cc:276] 0709 12:26:10.557342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.558000 (+   658us) service_pool.cc:224] Handling call
0709 12:26:10.577789 (+ 19789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.094230 (+516441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.776085 (+681855us) spinlock_profiling.cc:243] Waited 515 ms on lock 0x7b5400163854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:11.776286 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185140096}
I20260709 12:26:11.755182  9946 rpcz_store.cc:276] 0709 12:26:09.580076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.431801 (+851725us) service_pool.cc:224] Handling call
0709 12:26:10.432087 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.432396 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.432424 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.432869 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.729183 (+296314us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:10.729245 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.729368 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.511064 (+781696us) raft_consensus.cc:1685] finished
0709 12:26:11.511098 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.511307 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1386,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":10934,"thread_start_us":295969,"threads_started":1,"wal-append.queue_time_us":437615}
I20260709 12:26:11.781304  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.785638 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:11.760970 16380 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.760830 16099 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:11.789144  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 171144us
I20260709 12:26:11.808270  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.789645 16341 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.811028  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:11.815021 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:11.807672 16233 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.817504 16171 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:11.818049 16233 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:11.818624 16356 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.815s	user 0.004s	sys 0.000s
I20260709 12:26:11.819748  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.797713 15859 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.820394  9949 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:11.764839 16190 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.821314  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.767156 16160 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.790755 16009 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:26:11.840433 16170 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:11.822403  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.823294  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.828425 16160 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.844935  9949 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:11.823346 16341 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:11.845615 16341 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.845847 16200 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.846055 16341 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:11.827906 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2238) took 1379 ms. Trace:
I20260709 12:26:11.846530  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1308) took 1393 ms. Trace:
I20260709 12:26:11.847733  9949 rpcz_store.cc:276] 0709 12:26:10.453258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.845680 (+1392422us) service_pool.cc:224] Handling call
0709 12:26:11.845950 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.846226 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.846252 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.846308 (+    56us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:11.846335 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.846387 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.846482 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1131,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":78}
I20260709 12:26:11.849051 16315 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:11.832250 16364 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:11.850322 16364 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.804723 16367 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:11.852209 10018 rpcz_store.cc:276] 0709 12:26:10.448235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.448435 (+   200us) service_pool.cc:224] Handling call
0709 12:26:10.448896 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.449350 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.449391 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.449954 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:26:10.802732 (+352778us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:10.802778 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:10.802863 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.818023 (+1015160us) spinlock_profiling.cc:243] Waited 9.48 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:11.824591 (+  6568us) raft_consensus.cc:1685] finished
0709 12:26:11.824613 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.827834 (+  3221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1754,"raft.queue_time_us":856,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":3312,"spinlock_wait_cycles":21814528,"thread_start_us":227525,"threads_started":1,"wal-append.queue_time_us":496118}
I20260709 12:26:11.854566 16367 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:11.855271 16364 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:11.856158 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.860119  9946 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 20 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:26:11.787436 16293 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:11.863278  9941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.062s	sys 0.085s
W20260709 12:26:11.865293  9941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.062s	sys 0.085s
I20260709 12:26:11.865196  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2268) took 2284 ms. Trace:
I20260709 12:26:11.866395  9940 rpcz_store.cc:276] 0709 12:26:09.580362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.953606 (+1373244us) service_pool.cc:224] Handling call
0709 12:26:10.953972 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:10.954372 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:10.954399 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.003017 (+ 48618us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.256991 (+253974us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:11.257038 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.257116 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:11.864967 (+607851us) raft_consensus.cc:1685] finished
0709 12:26:11.864987 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.865139 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":138,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":253330,"threads_started":1,"wal-append.queue_time_us":378103}
I20260709 12:26:11.794919 16286 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.866554  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2485) took 2234 ms. Trace:
I20260709 12:26:11.870203  9941 rpcz_store.cc:276] 0709 12:26:09.632196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.263317 (+1631121us) service_pool.cc:224] Handling call
0709 12:26:11.263786 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.264109 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:11.311798 (+ 47689us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.866098 (+554300us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.866147 (+    49us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:11.866202 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.866373 (+   171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:11.866494 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":47490,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":82,"thread_start_us":553904,"threads_started":1}
I20260709 12:26:11.863768 10016 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:11.775549 16381 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:11.825035 15758 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:11.886168 10091 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:11.887517  9947 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:11.864553 15992 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.803664 16321 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.824884 15859 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:11.889415 15758 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:11.811157 16380 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:11.924834 16190 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:11.926527 16184 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.927196 16295 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:11.931955 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:11.875805 16341 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:11.953572  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:11.959872  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.965373 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:11.934520 15758 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:11.978401  9942 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:11.983625  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.986052  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:11.978215 16171 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:11.950424 16315 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:11.967000 16184 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:11.951035 15992 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.007606  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1313) took 1044 ms. Trace:
I20260709 12:26:12.018401 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.021595 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.021886 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:11.992472 16184 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.051146  9949 rpcz_store.cc:276] 0709 12:26:10.963063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.848892 (+885829us) service_pool.cc:224] Handling call
0709 12:26:11.849174 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.849353 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:11.849620 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 12:26:11.994957 (+145337us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.995009 (+    52us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:11.995050 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.995112 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.007509 (+ 12397us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":646,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":86,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"thread_start_us":145014,"threads_started":1,"wal-append.queue_time_us":182147}
I20260709 12:26:11.995937 16362 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.030921 16170 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:12.037367 15758 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.063652 16383 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:12.064257 16383 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.039059  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.066061  9946 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
W20260709 12:26:12.067376 16374 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:26:12.067595 16384 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.068193 16384 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.069723 16380 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.071027 16384 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.071239 16341 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:11.994271 10675 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:11.992380 16171 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.051537 10674 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:12.052685 16382 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:12.027495 16293 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.038245 15859 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.056411  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:12.059157 16286 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.054023 16332 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.072387  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:12.086642  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46442.2us
I20260709 12:26:12.052014  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2275) took 1037 ms. Trace:
I20260709 12:26:12.040814 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.064579 15758 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:12.365625 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1081) took 1114 ms. Trace:
I20260709 12:26:12.022905 16343 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.345831 15872 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.366330 10092 rpcz_store.cc:276] 0709 12:26:11.251437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.040481 (+789044us) service_pool.cc:224] Handling call
0709 12:26:12.365247 (+324766us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.367440 15872 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:12.099753  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 120986us
I20260709 12:26:12.377408  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:12.377748  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2276) took 1362 ms. Trace:
I20260709 12:26:12.378504  9947 rpcz_store.cc:276] 0709 12:26:11.015153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.021385 (+  6232us) service_pool.cc:224] Handling call
0709 12:26:12.377001 (+1355616us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b540042d954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:12.377436 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1988795648}
I20260709 12:26:12.056432 15992 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.025332 16363 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.066409 16383 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.073012 16184 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:12.053669 16221 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:12.402945 16386 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:12.093180 16321 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:12.403724 16386 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.403714 16221 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:12.404574 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3633) took 1311 ms. Trace:
I20260709 12:26:12.405067 10096 rpcz_store.cc:276] 0709 12:26:11.093296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.402513 (+1309217us) service_pool.cc:224] Handling call
0709 12:26:12.403433 (+   920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.403932 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.403956 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.404045 (+    89us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:12.404133 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.404203 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.404352 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.194419  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.364022  9940 rpcz_store.cc:276] 0709 12:26:11.014903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.867910 (+853007us) service_pool.cc:224] Handling call
0709 12:26:11.868257 (+   347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:11.868603 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.868630 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:11.879558 (+ 10928us) log.cc:844] Serialized 52 byte log entry
0709 12:26:11.983346 (+103788us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:26:11.983385 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.983450 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.049442 (+ 65992us) raft_consensus.cc:1685] finished
0709 12:26:12.049464 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.051958 (+  2494us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":446,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6717,"thread_start_us":70329,"threads_started":1,"wal-append.queue_time_us":119379}
I20260709 12:26:12.361182  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:12.387722 16394 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.417207 16394 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.417824  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.389351 16017 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:12.419282 16017 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.420909 16394 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.422528 16017 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.422843 15872 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:12.404309 16386 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:12.387981 15859 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.429970 15859 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.431607 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.432896 15859 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.433797 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.410844  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:12.406092 16221 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.441169 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.448371 16394 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.449406 16394 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:12.443485  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.446390  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.446949 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.452004  9947 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:12.457919 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.459702 16394 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.460187  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1309) took 1727 ms. Trace:
I20260709 12:26:12.460820  9943 rpcz_store.cc:276] 0709 12:26:10.732480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.847438 (+1114958us) service_pool.cc:224] Handling call
0709 12:26:11.847747 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.904646 (+ 56899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.904704 (+    58us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:11.904765 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:11.904814 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.459595 (+554781us) spinlock_profiling.cc:243] Waited 56.6 ms on lock 0x7b540009c4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:12.459674 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.459919 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":130234240}
W20260709 12:26:12.463438  9944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2486) took 2744 ms (client timeout 1623 ms). Trace:
I20260709 12:26:12.464479  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2278) took 1432 ms. Trace:
W20260709 12:26:12.464701  9948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1298) took 3459 ms (client timeout 2151 ms). Trace:
I20260709 12:26:12.464933 16171 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.465533  9945 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:12.468442  9946 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:12.466399  9948 rpcz_store.cc:269] 0709 12:26:09.004952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.283546 (+278594us) service_pool.cc:224] Handling call
0709 12:26:12.464159 (+3180613us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540018b0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:12.464459 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3816574208}
I20260709 12:26:12.466024  9941 rpcz_store.cc:276] 0709 12:26:11.032061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.873927 (+841866us) service_pool.cc:224] Handling call
0709 12:26:11.874308 (+   381us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:11.874767 (+   459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:11.874809 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:11.879926 (+  5117us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.990622 (+110696us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:11.991289 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.991487 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.463994 (+472507us) raft_consensus.cc:1685] finished
0709 12:26:12.464031 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.464232 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2869,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":9666,"spinlock_wait_cycles":1764224,"thread_start_us":109465,"threads_started":1,"wal-append.queue_time_us":218987}
I20260709 12:26:12.471015  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2279) took 1433 ms. Trace:
I20260709 12:26:12.471419  9948 rpcz_store.cc:276] 0709 12:26:11.037188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.469662 (+1432474us) service_pool.cc:224] Handling call
0709 12:26:12.470197 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.470571 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.470620 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.470709 (+    89us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:12.470746 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.470803 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.470942 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:12.465956  9944 rpcz_store.cc:269] 0709 12:26:09.719382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.760162 (+ 40780us) service_pool.cc:224] Handling call
0709 12:26:12.462912 (+2702750us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b540003fa54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:12.463193 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4003233152}
I20260709 12:26:12.473290 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.474172  9948 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:26:12.474718  9948 pending_rounds.cc:85] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 10
I20260709 12:26:12.474511 10099 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
W20260709 12:26:12.475224 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1080) took 1361 ms (client timeout 1703 ms). Trace:
W20260709 12:26:12.475693 10099 rpcz_store.cc:269] 0709 12:26:11.113694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.472736 (+1359042us) service_pool.cc:224] Handling call
0709 12:26:12.475139 (+  2403us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.476817  9948 pending_rounds.cc:107] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:26:12.477963 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.477275  9948 raft_consensus.cc:2889] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:12.480077  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2283) took 1420 ms. Trace:
I20260709 12:26:12.480496  9948 rpcz_store.cc:276] 0709 12:26:11.059994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.472920 (+1412926us) service_pool.cc:224] Handling call
0709 12:26:12.473468 (+   548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.479834 (+  6366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.480008 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.484632 10096 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:26:12.480681  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.471874  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2281) took 1434 ms. Trace:
I20260709 12:26:12.487702  9941 rpcz_store.cc:276] 0709 12:26:11.037840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.470829 (+1432989us) service_pool.cc:224] Handling call
0709 12:26:12.471205 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.471497 (+   292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.471515 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.471597 (+    82us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:12.471634 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.471691 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.471786 (+    95us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.492952 16381 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:12.493721  9943 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:26:12.495505 15992 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.496112  9943 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:26:12.496382  9940 raft_consensus.cc:2410] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 15 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
W20260709 12:26:12.499593  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.500917 15859 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:12.501852 15859 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.506063 16201 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.506548 16179 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:12.511489 16368 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.067s	user 0.005s	sys 0.000s
I20260709 12:26:12.511762 16230 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.512722 16230 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.513605 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1329) took 3496 ms. Trace:
I20260709 12:26:12.514117 10677 rpcz_store.cc:276] 0709 12:26:09.017509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.018388 (+   879us) service_pool.cc:224] Handling call
0709 12:26:09.042677 (+ 24289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:09.495085 (+452408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:09.495138 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:10.984394 (+1489256us) spinlock_profiling.cc:243] Waited 4.82 ms on lock 0x7b2c000d07e8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:26:10.984830 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 12:26:11.245515 (+260685us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:11.245550 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:11.245620 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.510199 (+1264579us) spinlock_profiling.cc:243] Waited 3.32 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:12.513191 (+  2992us) raft_consensus.cc:1685] finished
0709 12:26:12.513220 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.513394 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13292,"raft.queue_time_us":70889,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":64,"spinlock_wait_cycles":18723584,"thread_start_us":688191,"threads_started":2,"wal-append.queue_time_us":376100}
I20260709 12:26:12.515889 16230 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.513640 16063 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:12.517413 16063 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.518553 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1082) took 1204 ms. Trace:
I20260709 12:26:12.519057 10098 rpcz_store.cc:276] 0709 12:26:11.314293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.314449 (+   156us) service_pool.cc:224] Handling call
0709 12:26:12.518044 (+1203595us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b5400340b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:12.518333 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":959848832}
I20260709 12:26:12.523910 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:12.524791  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:12.525539  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.531844 16394 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:12.532195 16329 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.530224  9942 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:12.532824 16394 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.540647 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.547119  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1316) took 1144 ms. Trace:
I20260709 12:26:12.549448  9941 rpcz_store.cc:276] 0709 12:26:11.389205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.488758 (+1099553us) service_pool.cc:224] Handling call
0709 12:26:12.496115 (+  7357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.522054 (+ 25939us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.522086 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.522141 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:12.533378 (+ 11237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.533456 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.533580 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.555907 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:12.553090 16329 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:12.508344  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63291.1us
I20260709 12:26:12.566288 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.570533 16329 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.580240 16063 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.584216 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.585427 10092 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:12.566955  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.570576  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1317) took 1181 ms. Trace:
I20260709 12:26:12.592259  9946 rpcz_store.cc:276] 0709 12:26:11.389595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.490145 (+1100550us) service_pool.cc:224] Handling call
0709 12:26:12.548900 (+ 58755us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.569871 (+ 20971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.569921 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:12.570018 (+    97us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:12.570094 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.570210 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.570447 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:12.574347 16329 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.605345 16395 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.606240 16395 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.609417 16395 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:12.612886  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.614902 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.616058 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:12.616255  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.617308  9941 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:12.617750 10677 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:12.618974 16009 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:12.621948 16349 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.715s	user 0.001s	sys 0.002s
I20260709 12:26:12.627306 10096 raft_consensus.cc:2368] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20.
I20260709 12:26:12.606361 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1348) took 1306 ms. Trace:
I20260709 12:26:12.630571 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.631311 16296 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:12.631412 10020 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:12.632105 16296 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.621994 16009 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:12.628120 10670 rpcz_store.cc:276] 0709 12:26:11.300100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:11.303604 (+  3504us) service_pool.cc:224] Handling call
0709 12:26:12.605906 (+1302302us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54004d35d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:12.606133 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295929088}
I20260709 12:26:12.629302  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.622833 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1290) took 2393 ms. Trace:
I20260709 12:26:12.633417 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.598498 16063 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:12.642740 16394 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.643254 16063 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.643702 16323 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:12.645251 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.645748 16063 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.648543 10020 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:12.650287 10024 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:12.635325 10673 rpcz_store.cc:276] 0709 12:26:10.229099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:10.232860 (+  3761us) service_pool.cc:224] Handling call
0709 12:26:10.233315 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:11.007281 (+773966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.622282 (+1615001us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b5400615b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:12.622652 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778389248}
I20260709 12:26:12.649257 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.652527 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.653271 10098 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:12.653534 16296 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:12.649914 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:12.655426 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.656605 10017 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:12.634759 16009 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:12.660583 16009 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:12.665964  9946 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:12.666580  9940 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:12.672729  9946 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:12.598554 16386 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.697958  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.700157  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:12.704239  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74178.3us
I20260709 12:26:12.709944  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:12.712369  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:12.713431 16394 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:12.714267 16394 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:12.710166 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.718861 16386 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:12.722965  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:12.724262 16255 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:26:12.726369 16255 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:12.726924 16255 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.727463 16255 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:12.025393  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:12.732791  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:12.734885 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.735396 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.736243 10677 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:12.736851 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.736756 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.739637 10670 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:12.741065 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.741858 10674 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:12.705824 16063 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:12.747543 16063 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.749697 16063 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.749923 16009 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.760515 16009 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:12.764889  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:12.707971 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.772471 16179 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:12.773519  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:12.758400  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4317022370000814s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1678s for a response
W20260709 12:26:12.775921  9912 connection.cc:256] RPC call timeout handler was delayed by -1.3138022370003455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
I20260709 12:26:12.776279  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2286) took 1441 ms. Trace:
I20260709 12:26:12.782804 15893 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:12.777982  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.783557  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2285) took 1459 ms. Trace:
I20260709 12:26:12.784174  9941 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 18 because replica has last-logged OpId of term: 15 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:26:12.784760 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.785128  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.785519 10668 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:12.782801 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.779224  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2475) took 3797 ms. Trace:
I20260709 12:26:12.779523 16355 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.771162  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.779270 16264 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:12.781952 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.782107 16190 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:12.793313 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:12.793975 10018 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:12.793732 16190 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.797794 16190 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.773746 16179 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:26:12.780015 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65257.8us
W20260709 12:26:12.732852  9949 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.644s	user 0.013s	sys 0.009s
I20260709 12:26:12.786305  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.786811  9945 rpcz_store.cc:276] 0709 12:26:08.982059 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:09.402195 (+420136us) service_pool.cc:224] Handling call
0709 12:26:12.465027 (+3062832us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.472054 (+  7027us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.778940 (+306886us) spinlock_profiling.cc:243] Waited 3.06 s on lock 0x7b540018b0d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:12.779114 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7042754304}
I20260709 12:26:12.782562  9948 rpcz_store.cc:276] 0709 12:26:11.334685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.481618 (+1146933us) service_pool.cc:224] Handling call
0709 12:26:12.482234 (+   616us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.483545 (+  1311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:12.483590 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.484456 (+   866us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.591528 (+107072us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:12.591591 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.591669 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.776000 (+184331us) raft_consensus.cc:1685] finished
0709 12:26:12.776027 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.776181 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":165,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":269,"thread_start_us":106572,"threads_started":1,"wal-append.queue_time_us":223823}
I20260709 12:26:12.783962  9944 rpcz_store.cc:276] 0709 12:26:11.324476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.473383 (+1148907us) service_pool.cc:224] Handling call
0709 12:26:12.507426 (+ 34043us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:12.509304 (+  1878us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:12.509346 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:12.509965 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 12:26:12.556558 (+ 46593us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:12.558844 (+  2286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:12.558920 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:12.724427 (+165507us) raft_consensus.cc:1685] finished
0709 12:26:12.724447 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:12.783290 (+ 58843us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b5400206f50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:12.783502 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77092608,"thread_start_us":45200,"threads_started":1,"wal-append.queue_time_us":194386}
I20260709 12:26:12.790478 16355 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:12.790925 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 733302us
I20260709 12:26:12.791492 16264 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.789410  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:12.798501 16221 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.803614  9947 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:12.806782 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:12.801461  9912 connection.cc:256] RPC call timeout handler was delayed by -1.2310022370002116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19010000000000002s for a response
W20260709 12:26:12.814288  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0447022370003651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
W20260709 12:26:12.814889  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0437086450001516s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1957s for a response
W20260709 12:26:12.815289  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0051022370003011s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
I20260709 12:26:12.813501  9949 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:12.818500  9838 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 14, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:12.819890 15859 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.801537 16179 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:12.821120  9838 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 9 to 16, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 16 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:12.823539 16179 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.823802  9838 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 12, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 12 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:12.825032 16179 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:12.827026  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.828233  9940 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:12.830864 16230 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:12.833348  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.835143  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.826087  9838 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 9 to 12, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 12 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:12.838079 10018 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
W20260709 12:26:12.840355  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:12.841889 16230 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.842283 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.842948 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.843626 10020 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
W20260709 12:26:12.844213  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56548.6us
I20260709 12:26:12.847138 10674 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 11.
I20260709 12:26:12.851387 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:12.852139  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 93101.7us
I20260709 12:26:12.855166 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:12.852445  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51365.7us
I20260709 12:26:12.843734 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.858274 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.856164  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:12.842357 16230 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.860052  9945 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:12.860242  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:12.863838 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.868113 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.844393 16329 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:12.862890 10017 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:12.867419 10099 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:12.873657 10098 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:26:12.867336 10016 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:12.874327 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.879057 15796 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 12:26:12.879410 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:12.880846 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:12.881614 10024 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:12.882740 16406 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:12.883155 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53014.6us
I20260709 12:26:12.879468 10025 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:12.886754 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:12.887508 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:12.888142 16190 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:12.889592 16190 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:12.890003 16190 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:12.889592 16080 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:12.891074 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.892150 10024 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:12.892671 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:12.885989 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.893822 10677 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:12.895381 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.907473 16329 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:12.908275 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:12.909600 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:12.911734 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.887535 10671 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:12.940639  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 85055.8us
W20260709 12:26:12.942214 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52569us
I20260709 12:26:12.948539 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.961987 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.964219 16056 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:26:12.964550 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.965562 10021 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:12.967046 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.917516 16406 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:12.981405 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.982101 15869 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:12.982791 15869 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:12.984772  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 127471us
W20260709 12:26:12.985775  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 116581us
I20260709 12:26:12.991349 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:12.988289  9838 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 14, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 14 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:12.992172 10021 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:12.992448 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:12.993453 10095 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:12.994738 16255 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:12.985571 16406 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:12.999491 16329 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.984611 16063 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:26:13.003762 16264 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:12.967077 16200 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.006772 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:13.006999 16200 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:13.007589 15869 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:13.007769 16017 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:13.008432 16017 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:13.009295 16200 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:13.009402 16017 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:13.010470 16184 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.006s
W20260709 12:26:13.010754 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71688.4us
I20260709 12:26:13.011008 16184 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.013290 16184 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:13.014318  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.014945 10092 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:13.015327 16355 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:13.011339 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:13.020145  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.020845 10090 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:12.994510 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.021399 16113 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:12.988204 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:13.027223 10670 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:26:13.031036 10020 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:13.031723 10018 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:13.036105 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.036864 10097 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:13.045842 16255 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:13.046347 16063 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:13.047049 16063 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.047869 16383 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:13.048431 16383 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.035940 16113 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:13.050025 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79623.5us
I20260709 12:26:13.058981 10675 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:13.035436 10671 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:13.068138  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.069090  9948 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:13.070952  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.071897  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.072710 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:13.072935 16382 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.073387 10021 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:13.073550 16382 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.076720 16382 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:13.077294 16382 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.077755 16382 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.078749 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:13.079438 10674 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:13.080472 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:13.081642 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:13.071026 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:13.070299  9945 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:13.082775 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:13.086124 10021 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:13.082667 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.086980 10025 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:13.081851 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:13.089957  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.049938 16394 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.093840 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:13.094455 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:13.094475 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:13.098892 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.097268 10669 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:13.104115 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:13.062536 16063 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:13.105213 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.070946 16190 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.107520 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:13.107718 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 147878us
I20260709 12:26:13.108245 10020 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:13.108706 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:13.109120 16264 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:13.114379 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52229.5us
I20260709 12:26:13.107964 16190 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:13.123831 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:13.114802 10091 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:13.114656  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.124132 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.123335 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:13.144289 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:13.147044 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52872.9us
I20260709 12:26:13.147974 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:13.149279 16201 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:13.151921 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:13.154068 16383 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:13.114097 16264 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:13.155503 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46906.9us
I20260709 12:26:13.155922 16264 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:13.157271 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:13.158057 16201 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:13.158491 16201 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:13.158895  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:13.109933 16394 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:13.158972 16201 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:13.161865 16296 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:13.163697 16382 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:13.164462 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.160706 16184 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:13.166157 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:13.122309 16179 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:13.166874 10025 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:13.168140 10642 leader_election.cc:400] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:13.168473 10642 leader_election.cc:403] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:13.169127 10096 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:26:13.158304 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:13.184832 15904 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:13.185504 15904 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:13.188764 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.199949  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:13.186450 16406 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:13.203481 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:13.203510 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 93372.8us
I20260709 12:26:13.207125 10022 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:13.186442 16200 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:13.209393 10090 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:13.186743 16113 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:13.216436 10094 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 12:26:13.210058 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51825.2us
I20260709 12:26:13.218637 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.196954 16386 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:13.224222 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:13.223466 16355 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:13.228165 16363 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:13.229254 16185 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:13.233237  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:13.233976  9941 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:13.186430 15904 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:13.238873  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2298) took 1158 ms. Trace:
I20260709 12:26:13.265784  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.258296  9946 rpcz_store.cc:276] 0709 12:26:12.080933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.673673 (+592740us) service_pool.cc:224] Handling call
0709 12:26:12.674040 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:12.674389 (+   349us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:12.805162 (+130773us) log.cc:844] Serialized 14 byte log entry
0709 12:26:13.217092 (+411930us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:13.217168 (+    76us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:13.223148 (+  5980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.238559 (+ 15411us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:13.238784 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5272,"prepare.queue_time_us":199206,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":75480,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":2476,"spinlock_wait_cycles":15701248,"thread_start_us":461074,"threads_started":3,"wal-append.queue_time_us":278584}
I20260709 12:26:13.223590 16383 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:13.272501  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84419.5us
I20260709 12:26:13.258080 16200 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:13.262506 16185 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:13.283789  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 211351us
I20260709 12:26:13.282918  9940 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
W20260709 12:26:13.281845 16414 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:26:13.288594 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.276492 16230 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:13.269219 16383 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:13.290995 16383 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:13.291523  9948 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 12:26:13.292403 16113 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:13.292609  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:13.292917 16230 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:13.293248  9948 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:13.293021 16113 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.294667 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.295446 10091 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:26:13.295964 16113 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:13.296413 16394 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:13.306170 16413 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.198s	user 0.000s	sys 0.002s
I20260709 12:26:13.298471 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:13.309803 16394 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:13.318197 16200 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:13.319324 16394 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.321653 16394 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:13.310961  9839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 312) took 38197 ms (client timeout 15000 ms). Trace:
I20260709 12:26:13.311591  9838 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 14, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:26:13.324182 16418 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:26:13.325136  9838 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 15, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 15 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:13.326272  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.315953 16113 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:26:13.313232  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:13.328380 16113 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:13.328797 16113 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:13.328989  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.329768 16113 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:13.331049 16113 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:13.327450  9838 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:13.331288 16264 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.271649 16386 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.330360  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.327487  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.332775  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:13.358321 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 134201us
I20260709 12:26:13.359472 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.359861 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.284686 16406 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:13.367368  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.363173 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:13.364446 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.368894 10097 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:13.364938 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.371471 10024 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:13.371136 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.374296  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.375986 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.377131  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.377466 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.379024  9947 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:13.379689  9941 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:13.380903 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:13.382100 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:13.384070 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 223788us
I20260709 12:26:13.385757 10091 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:13.386710  9942 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:13.389721  9945 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:13.393253 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.381632 10090 raft_consensus.cc:2376] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 18. Re-sending same reply.
W20260709 12:26:13.396672  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.475911 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 86987.8us
W20260709 12:26:13.472569 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79790.6us
I20260709 12:26:13.480351  9838 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 14, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:26:13.484544 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 125518us
I20260709 12:26:13.489631 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.501196 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.388273 16056 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:13.515534 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:13.516527 10093 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:13.533128 16389 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:13.534420  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 529727us
I20260709 12:26:13.543423  9838 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 12, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 12 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:13.364382 16190 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.137s
I20260709 12:26:13.541419 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:13.556337 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:13.563299  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 278286us
I20260709 12:26:13.564965 16113 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:13.565317  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:13.567157 16113 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:13.391515 16184 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.106s
I20260709 12:26:13.613226  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:13.618973 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:13.620198 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:13.619442  9838 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 8 to 11, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 11 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:13.641778 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:13.648994 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 92726.2us
I20260709 12:26:13.651649 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:13.648077  9838 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:13.665959  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52813.8us
I20260709 12:26:13.667373  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:13.653512 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:13.673813  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116399us
I20260709 12:26:13.619542 16056 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:13.676244 10091 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:13.663414  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:13.679719 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 199501us
I20260709 12:26:13.683981  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:13.700886  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:13.704121 10024 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:26:13.642958 16394 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:13.726958  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63624.3us
W20260709 12:26:13.322758  9839 rpcz_store.cc:269] 0709 12:25:35.117444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:35.127238 (+  9794us) service_pool.cc:224] Handling call
0709 12:26:13.310749 (+38183511us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:25:59.267522 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:25:59.268327 (+   805us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:25:59.268376 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:25:59.268402 (+    26us) tablet.cc:662] Decoding operations
0709 12:25:59.347693 (+ 79291us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:25:59.367363 (+ 19670us) write_op.cc:641] Partition lock acquired for write op
0709 12:25:59.368733 (+  1370us) tablet.cc:685] Acquiring locks for 23 operations
0709 12:25:59.621917 (+253184us) tablet.cc:701] Row locks acquired
0709 12:25:59.621949 (+    32us) write_op.cc:260] PREPARE: finished
0709 12:25:59.622190 (+   241us) write_op.cc:270] Start()
0709 12:25:59.622371 (+   181us) write_op.cc:276] Timestamp: P: 1783599959622096 usec, L: 0
0709 12:25:59.622405 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 12:25:59.866763 (+244358us) log.cc:844] Serialized 9193 byte log entry
0709 12:26:00.947363 (+1080600us) op_driver.cc:464] REPLICATION: finished
0709 12:26:01.021512 (+ 74149us) spinlock_profiling.cc:243] Waited 94.3 ms on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09551e28 00007f4e0cf8c581 00007f4e0cf8bd58 00007f4e0cf909f1 00007f4e0cf909a9 00007f4e0cf90939 00007f4e0cf90901 00007f4e0cf8fb2d 0000000000388634 0000000000388479
0709 12:26:01.852866 (+831354us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b180001a8c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:01.859842 (+  6976us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b180001a8c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:01.860050 (+   208us) write_op.cc:301] APPLY: starting
0709 12:26:01.860180 (+   130us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:02.741008 (+880828us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:02.741065 (+    57us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:03.354577 (+613512us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:03.596109 (+241532us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 12:26:03.627696 (+ 31587us) write_op.cc:312] APPLY: finished
0709 12:26:03.733691 (+105995us) log.cc:844] Serialized 204 byte log entry
0709 12:26:03.947554 (+213863us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:03.967062 (+ 19508us) write_op.cc:454] Released schema lock
0709 12:26:03.967156 (+    94us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:03.967823 (+   667us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:03.967860 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":62826,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":59852,"apply.run_cpu_time_us":60211,"apply.run_wall_time_us":2107680,"cfile_cache_hit":94,"cfile_cache_hit_bytes":100313,"num_ops":23,"prepare.queue_time_us":419481,"prepare.run_cpu_time_us":74532,"prepare.run_wall_time_us":1754216,"raft.queue_time_us":236944,"raft.run_cpu_time_us":17154,"raft.run_wall_time_us":102492,"replication_time_us":1293662,"spinlock_wait_cycles":367947392,"thread_start_us":916124,"threads_started":5,"wal-append.queue_time_us":1077960,"wal-append.run_cpu_time_us":26426,"wal-append.run_wall_time_us":2506863}]]}
W20260709 12:26:13.707947 16422 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:26:13.598670 16201 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:13.735206  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57204.8us
W20260709 12:26:13.783550  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 301096us
I20260709 12:26:13.685911 16171 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:13.788990 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 119930us
W20260709 12:26:13.796339  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:13.789722 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 136315us
I20260709 12:26:13.713605  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:13.715569 16221 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:13.785748  9838 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:13.742749 16395 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:13.857990 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:13.858799 10671 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:13.733976 16080 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:13.849712 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:13.852939 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:13.855298 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:13.869303 10670 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:26:13.860999  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73729.7us
I20260709 12:26:13.877249 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:13.925973 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:13.926308  9838 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:26:13.930078  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 216544us
W20260709 12:26:13.932626 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 390501us
I20260709 12:26:13.742290 16230 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 12:26:13.934752 16230 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:13.934844  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:13.935206 16230 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:13.935633  9940 raft_consensus.cc:2368] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20.
I20260709 12:26:13.936617  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.936209 16230 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:13.937914 10675 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:13.937637 16230 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:13.943028 16386 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:13.943581 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 145387us
I20260709 12:26:13.937590 10643 leader_election.cc:400] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20.
I20260709 12:26:13.948067 10643 leader_election.cc:403] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:26:13.949024 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 251677us
I20260709 12:26:13.937808  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.952597 10670 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:26:13.879099  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:13.851289 16394 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:13.962625 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:13.963364  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 215565us
I20260709 12:26:13.963426  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:13.961356 10673 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
W20260709 12:26:13.951443  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78229.2us
I20260709 12:26:13.869822 16395 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:13.937731 16367 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:13.984236  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:13.871748 16264 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:13.987927 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:13.991945 10669 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:26:14.005792  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:14.002642 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 58034us
W20260709 12:26:13.944923 15992 consensus_queue.cc:1175] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" responder_term: 14 status { last_received { term: 14 index: 9 } last_committed_idx: 8 last_received_current_leader { term: 14 index: 9 } }
I20260709 12:26:13.996047 16395 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:14.032716  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.033581  9838 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 13, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 13 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:26:14.041523 16436 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:26:13.993686 16367 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:14.050892 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:14.003005 16221 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:14.056074  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:14.072810 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54408.4us
I20260709 12:26:14.024981 16382 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:14.035105 16383 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.744s	user 0.012s	sys 0.000s
W20260709 12:26:14.095093  9948 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.801s	user 0.009s	sys 0.002s
I20260709 12:26:14.095645  9948 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:14.081408  9838 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 13, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 13 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:14.075295  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:14.075119 16367 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:14.062330 16221 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:14.130412 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 104713us
I20260709 12:26:14.129055 10671 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
W20260709 12:26:14.134730 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.091292 16201 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.100986 16329 raft_consensus.cc:2764] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:14.156038 16230 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:14.156140 15771 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:14.108896 16386 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:14.157346 16430 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.300s	user 0.001s	sys 0.000s
I20260709 12:26:14.157137 16386 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.158947  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1319) took 1431 ms. Trace:
W20260709 12:26:14.159024 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 207523us
I20260709 12:26:14.159853 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:14.160746 16447 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:14.159394  9944 rpcz_store.cc:276] 0709 12:26:12.727785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.958322 (+230537us) service_pool.cc:224] Handling call
0709 12:26:13.002837 (+ 44515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.048197 (+ 45360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.048229 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.048785 (+   556us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.270517 (+221732us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:13.270616 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.270687 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.158691 (+888004us) raft_consensus.cc:1685] finished
0709 12:26:14.158713 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.158857 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28310,"raft.queue_time_us":60060,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":221202,"threads_started":1,"wal-append.queue_time_us":625377}
I20260709 12:26:14.161229 16447 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.161394  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2308) took 1471 ms. Trace:
W20260709 12:26:14.161541 16401 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:26:14.161830 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:14.161948  9943 rpcz_store.cc:276] 0709 12:26:12.689831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.952671 (+262840us) service_pool.cc:224] Handling call
0709 12:26:12.966278 (+ 13607us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.107533 (+141255us) spinlock_profiling.cc:243] Waited 899 us on lock 0x7b18003abb80. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:13.277427 (+169894us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.277482 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.330809 (+ 53327us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.673429 (+342620us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:13.673473 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.677104 (+  3631us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.161069 (+483965us) raft_consensus.cc:1685] finished
0709 12:26:14.161093 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.161325 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":526,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":111,"spinlock_wait_cycles":2067328,"thread_start_us":645918,"threads_started":2,"wal-append.queue_time_us":828643}
I20260709 12:26:14.164671 16179 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:26:14.165069 16179 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:14.165436 16179 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:26:14.167845 16440 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:14.170274 16190 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:14.175218 16426 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:26:14.159277 16386 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:14.108223 16383 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.191112  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1322) took 1462 ms. Trace:
I20260709 12:26:14.191766  9949 rpcz_store.cc:276] 0709 12:26:12.728857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.965335 (+236478us) service_pool.cc:224] Handling call
0709 12:26:12.965707 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.110325 (+144618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.110371 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.122321 (+ 11950us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.384721 (+262400us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:13.384758 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.384825 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.190717 (+805892us) raft_consensus.cc:1685] finished
0709 12:26:14.190754 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.191015 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":832,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":382260,"threads_started":2,"wal-append.queue_time_us":896211}
I20260709 12:26:14.193116 16435 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:14.193640 16179 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.139927 16080 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20.
I20260709 12:26:14.199587 16179 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:14.151306 16264 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:14.200765 16264 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:14.201436 16264 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:14.202045  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61269.8us
I20260709 12:26:14.203555 16456 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:14.204394 16456 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.200850 15771 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:14.207948 15771 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.212296  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:14.214855 16329 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:14.215607 16329 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:14.216789 16329 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:14.219894 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60130.8us
I20260709 12:26:14.219288  9838 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:14.220741 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:14.222842 16201 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:14.223474 16201 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:14.223968 16201 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:14.225442 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:14.227353 16184 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.223489 16009 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:14.228570 16009 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:14.231392  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 176011us
I20260709 12:26:14.233115  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:14.234200 15869 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:14.235118  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:14.139487 16221 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:14.236042 16230 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:14.236465 16230 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:14.236867 16230 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:14.237793  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.161660 16447 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:14.246197  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:14.246910 16230 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.249114 16383 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:14.254662  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:14.262326 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:14.310402 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48225.8us
I20260709 12:26:14.262198 16230 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:14.321455 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:14.323028  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 311488us
I20260709 12:26:14.229805 16009 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:14.329957  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.294566 15869 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:14.345778  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:14.325263 16200 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:14.354290 16450 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:14.448652 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:14.434952 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:14.462255 10016 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:14.387622  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:14.356635 16367 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:14.410898 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 89572.9us
W20260709 12:26:14.396870  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.464288 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:14.389529 16444 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.310s	user 0.001s	sys 0.001s
I20260709 12:26:14.339252 15869 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:14.372814  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:14.364079 16230 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:14.338690 16423 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:14.353116 16200 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:14.491279 10021 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:26:14.486939 16431 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:14.492144 16431 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:14.492631 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:14.493371 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.493121 16431 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.494124 10095 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:14.479959 16449 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:14.494917 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:14.495060 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:14.495237 16449 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.495664 10096 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:14.496011 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:14.496345 16438 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.415s	user 0.003s	sys 0.000s
I20260709 12:26:14.497393 16449 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:14.483958  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 111252us
I20260709 12:26:14.410663 16264 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.389711 16427 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:14.488245 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:14.481633 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.483701 16230 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:14.444522 15992 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:14.488246 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.548919 15992 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.493155 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.551596 15992 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.551770 10096 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:14.505059 16447 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.490392 16429 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:14.562891 16429 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:14.505393 16427 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.566061 16427 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.563448 16429 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:14.569422 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.569790 10677 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:14.570170 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.570820 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.572032 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.572743 10673 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:14.504803 16264 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:14.519356 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:14.490309 16446 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
I20260709 12:26:14.479277 15869 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.372071 16395 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:14.521221  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 133728us
I20260709 12:26:14.521675  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2311) took 1366 ms. Trace:
I20260709 12:26:14.562294 16447 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:14.576428 10017 raft_consensus.cc:2368] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13.
I20260709 12:26:14.575510  9947 rpcz_store.cc:276] 0709 12:26:13.155023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.545331 (+390308us) service_pool.cc:224] Handling call
0709 12:26:13.570886 (+ 25555us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.571316 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.571345 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.571826 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.716044 (+144218us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:13.716112 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.716246 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.521475 (+805229us) raft_consensus.cc:1685] finished
0709 12:26:14.521494 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.521611 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37983,"raft.queue_time_us":26840,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":8719,"thread_start_us":143765,"threads_started":1,"wal-append.queue_time_us":419235}
I20260709 12:26:14.491020 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:14.508591 16200 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:14.509178 16230 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:14.527536 16187 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.518584 16367 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.533402 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:14.534126 16190 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:14.535532 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:14.537688 10022 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:26:14.540954 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:14.593798 10016 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 18 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
I20260709 12:26:14.593593 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.542115 10021 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:14.542053 16221 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.543035 16113 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.546098 16455 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.501662 16184 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:14.546556 16386 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:14.547677 16425 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
W20260709 12:26:14.548520 16462 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:26:14.364589 16456 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.556162 16201 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.579538 10023 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:14.582507 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.587195  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:14.587579  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:14.587860  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:14.595381 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:14.613210 10092 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:14.603391  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2515) took 1499 ms. Trace:
I20260709 12:26:14.604116 10098 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:14.603475 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:14.612505 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79196.6us
I20260709 12:26:14.615257  9945 rpcz_store.cc:276] 0709 12:26:13.104111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.479683 (+375572us) service_pool.cc:224] Handling call
0709 12:26:13.515099 (+ 35416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:13.546132 (+ 31033us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:13.546178 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:13.576537 (+ 30359us) log.cc:844] Serialized 26 byte log entry
0709 12:26:13.851625 (+275088us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:13.851694 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:13.851795 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:14.603060 (+751265us) raft_consensus.cc:1685] finished
0709 12:26:14.603092 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:14.603313 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":92254,"raft.queue_time_us":35910,"raft.run_cpu_time_us":238,"raft.run_wall_time_us":47085,"thread_start_us":195008,"threads_started":1,"wal-append.queue_time_us":697705}
I20260709 12:26:14.605486 16113 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.604795 16190 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.614511  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:14.605770 16447 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:14.624434 16190 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:14.635061 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 236222us
I20260709 12:26:14.635291 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:14.636610  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:14.637876  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:14.638554 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.001s	sys 0.019s
W20260709 12:26:14.653239 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.002s	sys 0.020s
W20260709 12:26:14.653647 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 562959us
I20260709 12:26:14.592044 16187 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.616966 16427 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:14.626611 16367 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.632006 16383 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:14.656507 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 178637us
I20260709 12:26:14.636555 15992 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:14.610776 16455 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:14.663456 10642 leader_election.cc:336] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.516s (SENT)
I20260709 12:26:14.613046 16386 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.673038 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:14.675583 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:14.662425 16447 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:14.660707 16423 raft_consensus.cc:2764] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 12:26:14.684324 10018 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:14.682327  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:14.683629  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:14.685168  9945 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:14.686417 16395 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.689787 16113 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:14.665385 15992 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:14.690850 15992 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:14.693010 16455 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:14.686477 16221 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:14.698539 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:14.698935 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.702188 16201 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:14.698625 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.703053 10096 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:14.703336  9940 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:14.706323  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:14.707020  9948 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:14.707995  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:14.709012 16230 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.710156  9948 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:26:14.710533  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:14.713359  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:14.717103 10092 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:14.713953  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:14.720129 10018 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
W20260709 12:26:14.728399 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63458.1us
I20260709 12:26:14.728863  9945 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:26:14.729923  9943 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
W20260709 12:26:14.737795 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.727243  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:14.739969  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:14.747627  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:14.705531 15992 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.748958 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.712231 16447 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:14.763199 10096 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:14.764010  9946 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:14.751554 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:14.766938 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.767807 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:14.771013 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.772249 15992 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.773500 16429 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:14.775810 16429 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:14.776170  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:14.776769  9941 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:14.748227 16221 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:14.676638 16367 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.771013  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:14.750510 16113 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:14.780855 16009 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:14.824069 16009 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:14.824546 16009 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:14.780995 16230 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.783174  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:14.783314 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.826071  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:14.769522 16447 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:14.827553 16394 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:14.827649 16395 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:14.828133  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:14.784827  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:14.807610 10090 pending_rounds.cc:85] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 11
I20260709 12:26:14.829912 10090 pending_rounds.cc:107] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:26:14.810317 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:14.830118  9948 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:14.811275 10092 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:14.830188 10090 raft_consensus.cc:2889] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:14.831609 10642 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.703s (SENT)
I20260709 12:26:14.832113 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:14.812041 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:14.812381 16464 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:14.813074 10094 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:14.777582  9941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:26:14.835408  9941 raft_consensus.cc:606] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:14.817260 16386 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:14.818127 10097 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:14.810583 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:14.840790  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:14.820297 16113 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:14.753687 16329 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.083s
I20260709 12:26:14.842195  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:14.843020  9942 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:14.758335 16457 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 12:26:14.842617 16113 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.843641 16457 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:14.802520 16221 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:14.822691  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:14.810578 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:14.820026 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.769954 16435 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:14.963537 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:14.877346 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:14.963063 10023 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:26:14.876289  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53711.7us
I20260709 12:26:14.944263 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:14.939589 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.982263 10097 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:14.945391 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.935596 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.924118  9941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:26:14.984041  9941 raft_consensus.cc:606] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:14.984684  9941 raft_consensus.cc:614] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:14.906786  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:14.922494 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 90915.3us
I20260709 12:26:14.988032 10673 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:14.987895 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.988564 10677 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:14.877213 16200 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:14.922235 16179 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:14.851150  9945 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:14.849257 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:14.831971 16389 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:14.994462 10098 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:14.924041 16363 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:14.996464 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:14.997068 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:14.999619 10024 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:14.843999 16457 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:14.997556 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:14.932538 16431 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:14.829066 15904 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.187s
I20260709 12:26:15.009617 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:14.923861 16355 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:14.897706 16190 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:14.845528 16221 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:14.845278 16367 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:15.021533 10677 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:14.886658 16463 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:14.843875 16187 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.029994 10019 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:14.923010 16329 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 12:26:14.994659 16435 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:15.069080 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:15.032609  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 113908us
I20260709 12:26:15.069365 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.070448  9991 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:14.893940 15869 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.081585  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:15.086997 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48320us
I20260709 12:26:14.999815 16179 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.094345 16187 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.097554 16187 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.028892 16431 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.099974 16431 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.100251 16187 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.032756 15796 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:15.103258  9947 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:15.112141 10024 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:15.068114 16427 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.134552  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.017107 16363 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.029757 16264 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 12:26:15.101168 16435 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.120764 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.144181 10675 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:15.113936 16187 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.121531 16329 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:15.122244 15869 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:15.122695  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:26:15.095280 16457 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:15.123443  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 89702.4us
I20260709 12:26:15.154687  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.156426  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:15.033185 16423 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 12:26:15.126436  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.128717  9945 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:15.130782  9941 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:15.159914 16427 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.068058 16056 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:26:15.161439 16056 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:15.161870 16056 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:15.162094 16427 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:15.162317  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:15.163434  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:15.164666  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:15.166006 16480 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.166682 16480 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.167116 16480 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.033036 16355 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.171491 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.130260 16460 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:15.138399 16200 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:15.140548 16363 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.142180 16264 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:15.182049 16264 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:15.183276 16264 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.184561 16264 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:15.145754  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 75432.8us
I20260709 12:26:15.187623  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:15.188319 16429 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:15.188725 16429 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:15.189497 16429 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.192484 16179 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.193060  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.193917 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.194828 16485 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:15.194959 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.195393 16485 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.195983 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.196467 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:15.195348 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:15.197194 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.197685 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.197923 10668 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:15.198053 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.090437 16383 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:15.146306 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.200248 10675 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 13: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:15.095767 16487 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.202364 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:15.145038 16187 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.159704 16423 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.160753 16463 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:15.212416 16463 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:15.213248 16187 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.213855 16427 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:15.213426 16463 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.214257 16427 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.196352 16367 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:15.214689 16461 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:15.215268 16367 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:15.215665 16367 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:15.216503 16367 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.217922 16367 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:15.200686 10022 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:15.151863 16389 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:15.146111 10671 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:15.208518 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.219930 10668 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:15.209329 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:15.214696 16427 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:15.227133 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.227839 10673 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:15.228377  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:15.209183 10024 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.195824 16485 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.214872 16487 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.229221 16389 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:15.230882 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:15.229990  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:15.234184 10677 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
W20260709 12:26:15.234766  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 70176.2us
I20260709 12:26:15.177718 16460 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:15.240619  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:15.264057 10024 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:15.267227  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:15.133756 15796 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.275000 10019 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:15.280390  9945 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:15.278043  9941 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:15.239526 16429 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:15.277463 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:15.264943 16460 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.351459 10023 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:15.256722 16487 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:15.273989 16113 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:15.368816 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 299804us
W20260709 12:26:15.361248 16447 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.534s	user 0.008s	sys 0.004s
I20260709 12:26:15.374050 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:15.376308 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.374207 16431 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:15.392238 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:15.394856  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.404207 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:15.401571 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:15.390517 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 270720us
I20260709 12:26:15.409726  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.383208 16431 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.415521 10093 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:15.276964 16056 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:15.423444 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72317.2us
I20260709 12:26:15.424304 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:15.424501 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.425609 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:15.391403 16187 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:15.391640 16363 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.380087 16447 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:15.452122  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 99041us
W20260709 12:26:15.418088 16009 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.589s	user 0.010s	sys 0.002s
I20260709 12:26:15.383460 16200 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:26:15.419912 16431 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:15.476231 15796 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:15.477144 16355 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:15.477705 10095 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:15.478931 16447 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:15.454674 16363 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.482482 16363 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.451691 16482 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:26:15.480129 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:15.515731 16500 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:26:15.471855 16435 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
W20260709 12:26:15.455626 16481 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
W20260709 12:26:15.535926  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:15.420354 16456 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:26:15.549455 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 153746us
W20260709 12:26:15.563695  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 90522.6us
I20260709 12:26:15.456527 16179 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:15.579555 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 153961us
I20260709 12:26:15.585512  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.471601 16009 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.574312 16187 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:15.474063 16449 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.111s
W20260709 12:26:15.596681  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 329552us
I20260709 12:26:15.597616  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:15.590155 16492 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
W20260709 12:26:15.598426 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 131525us
I20260709 12:26:15.582458 16494 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.594035 16463 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:15.589818 16355 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:15.609700 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 184148us
I20260709 12:26:15.615799 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.609541  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:15.617410 10096 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:15.619794  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.629415  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:15.624408  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:15.613575 16394 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.622831 16009 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:15.614066 16363 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:26:15.657655 16465 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.009s
W20260709 12:26:15.659857 16498 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:26:15.637629 16367 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:15.660722 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59817.1us
I20260709 12:26:15.674320 10669 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:15.676072 16506 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:15.626272 16355 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.645915 16187 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:26:15.613559 16494 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.634130 16427 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.692854  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.694123  9948 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:15.695030 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:15.678660  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:15.705080  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:15.696987 15869 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.717072 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:15.678560 16221 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.608s	user 0.007s	sys 0.002s
I20260709 12:26:15.717319 16461 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.717883 16221 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.717078 10094 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:15.680308 16363 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:15.680204 16465 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:15.642844 16507 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:26:15.684412 16187 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:15.688552 16494 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.694936 16355 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.729979  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:15.717433 16423 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:26:15.724733 16363 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:15.723691 16508 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.727806 16465 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:15.737283 16480 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.570s	user 0.009s	sys 0.004s
I20260709 12:26:15.717343 16485 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:15.745123 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 69715.8us
I20260709 12:26:15.732620 16187 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:15.748065 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:15.745270 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 129443us
I20260709 12:26:15.750504 10640 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:15.753786 16427 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.756328 16485 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.747984 16329 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:15.760078 16465 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.761375 16465 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:15.762207 16363 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.771396  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:15.739518 16367 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:26:15.773531 10640 leader_election.cc:400] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13.
I20260709 12:26:15.775331 10640 leader_election.cc:403] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:15.775452  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:15.776955 16395 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:26:15.777263 16496 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 12. Current term is 13.
I20260709 12:26:15.778059 16395 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:15.778254  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:15.778479 16395 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:15.779642  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.779457 16395 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.780673 16395 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.780457 16463 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:15.780874  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:15.782109 16463 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:15.782073  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:15.781771 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:15.770048 16493 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:15.769905 16355 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:15.781396 16429 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.773833 16363 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.776399 16329 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:15.766819 16517 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.749558 16480 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.787247 16080 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:15.791458 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:15.791458  9912 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:15.784360 16423 raft_consensus.cc:2764] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:26:15.796446  9941 raft_consensus.cc:2368] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:26:15.798267 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:15.800745 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.790685 16463 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.791028 16429 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.808964 16221 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.808038 16080 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:26:15.791049 16513 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:26:15.811571  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:15.812233  9946 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:15.810426 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:15.813304  9943 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:26:15.796461 16187 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.815447 16187 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.797865 16427 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.817448 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:15.792698 16431 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.808328 16516 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.820295 16508 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:15.808544 16429 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.816166 16080 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:15.838480 16080 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.839934 16383 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:15.840597 16383 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.827610 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.844007 16431 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:15.847179  9940 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:15.847465  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:15.850302  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.850688  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.852499  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.821708 16264 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:15.822333 16463 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.858114 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:15.798844 16517 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.827405 16507 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:15.816339 16329 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:26:15.863497 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:15.855832 16423 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:15.855441 16427 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:15.823227 16467 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:26:15.865513 16201 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:15.859098 16264 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.875015 16201 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:15.886514 16201 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:15.890941 16386 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:15.891530 16386 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.893906 16386 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.889773 16429 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.907287 15796 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:26:15.907743 15796 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:15.908339 16480 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:15.909020 16480 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:15.908681 15796 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.910166 15796 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:26:15.911440 16507 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:15.875336 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47822.8us
I20260709 12:26:15.865080 16383 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.824113 16460 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:15.919116 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 97563.2us
I20260709 12:26:15.862167 16463 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.913578  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:15.868891 16423 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:15.912122 16264 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:15.924376  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:15.918579 16515 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.166s	user 0.007s	sys 0.001s
W20260709 12:26:15.920462  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 68069.1us
I20260709 12:26:15.926018 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:15.929374 10093 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:15.931310 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.923346 16463 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.933338 10669 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:15.924713 16465 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:15.933488 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:15.934525 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.935320 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:15.924921 16423 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:15.937804 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:15.937170 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:15.937587 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.937668 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:15.937041 10677 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:15.939038  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.917609 16190 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:26:15.940369 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:15.938516 10025 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:15.940394 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:15.930383 16521 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:15.943089 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:15.944226 10016 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:15.946983 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:15.947214 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:15.947795 10019 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:15.948290 16521 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.950480 16056 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:15.951112 16514 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:26:15.942670 16456 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:15.956292 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:15.946180 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:15.932449 16507 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:15.956986 10676 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:15.939930 16465 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:15.966189 10020 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:15.961066  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:15.961663  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.962016 16456 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:15.971635 15904 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:15.971391 16456 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:15.972054 15904 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:15.972316  9947 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:15.972894 15904 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:15.974125 15904 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:15.967960 10669 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
W20260709 12:26:15.968189 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44487.8us
I20260709 12:26:15.979439 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:15.982520 10020 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:15.984368 10025 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:26:15.984010 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:15.985236  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:15.988502  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:15.988582  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:15.990710 10098 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:15.992290 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:15.992239 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:15.960628 16521 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:15.969457 16465 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:15.992973 10098 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
W20260709 12:26:15.999524 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.007344  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:16.009299 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.011621  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:16.013031 10018 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:16.015110  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:16.017019 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:16.018105 10025 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:15.996505 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:16.020439 10094 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:15.975445 16507 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.033605 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:16.019819 16435 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:16.023871  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:16.024448  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:16.014035 16386 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:16.026712 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47356.8us
I20260709 12:26:16.038420 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:16.023061 16427 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:16.016511 16524 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:16.040706  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:16.041759 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:16.041909 10020 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.024533 16221 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.008183 16455 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:26:16.055035 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:16.059332  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.040843 16435 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:16.043807 16427 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:16.044003 16461 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:16.051925 16524 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:16.063319 16383 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:16.085637 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:16.082528  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67524.1us
I20260709 12:26:16.083182 16531 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:26:16.064147 16460 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.087141  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:16.068744 16427 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:16.067001 16517 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:16.067754 16435 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:16.093897 16511 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:16.094355 16531 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:16.095326 16355 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:16.078414 16524 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.104940 10094 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:16.084856 16383 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:16.097222  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:16.113314 10092 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:16.114990 16447 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:16.115367 16329 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.114181  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:16.118333  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:16.128127 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:16.124526 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.130036 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:16.155588  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.157572 10669 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.159829 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:16.111531 16531 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 14 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:16.116909 16355 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:16.119249 16383 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:16.167639 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:16.190627 10676 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.168982 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:16.190316 10023 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
W20260709 12:26:16.170717 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.171677 10670 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:26:16.195499 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.177202 10669 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
W20260709 12:26:16.181449 16527 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:26:16.182648 16532 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:16.183110 16522 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:26:16.185690 16465 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.186631 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:16.187858 10095 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:26:16.157078 16221 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:16.123092 16447 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:16.198173 16490 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:26:16.199937 16491 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:16.205796 10020 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:16.207096 16531 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.208194 16201 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.116444 16511 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.212294 16516 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.190253 16457 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:16.190546 16495 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.033s
I20260709 12:26:16.167135 16460 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:16.233999 16530 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:16.216315 16480 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:16.218992 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:16.221720 10670 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 12. Current term is 13. Ops: []
I20260709 12:26:16.179697 16190 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:26:16.246484 10675 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:16.239176 16447 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:16.265879  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:16.281996 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 418516us
I20260709 12:26:16.247800 16221 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:16.296233  9945 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:16.297083 10670 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:16.297075 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:16.297875 10675 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:16.298296 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:16.298990 10670 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:26:16.261260 16329 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:16.244498 16423 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.356351 10677 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:16.285213 16530 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:16.295321 16080 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.405570 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:16.259321 16535 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.150s
I20260709 12:26:16.432595 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:16.439049 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:16.444391 10020 raft_consensus.cc:2368] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:26:16.444696 10023 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:16.419901 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:16.446004 10676 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:16.446930 10669 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:16.425097 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:16.470206 10020 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
W20260709 12:26:16.413748 16536 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 12:26:16.450058 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:16.502233 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52273.9us
I20260709 12:26:16.428807 15869 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:26:16.443919 16463 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.000s
I20260709 12:26:16.434500 16383 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.405926 16080 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.510162 16383 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:16.510238 16080 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:16.424310 16201 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.511063 10016 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:16.513360 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.413461 16465 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:16.414716 16296 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.092s
I20260709 12:26:16.514696 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.514827 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.516021 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.473989 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:16.514401 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.413146 16539 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.522552 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.524258 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.418143 16495 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:16.403021 16530 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.519654 10020 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.520081 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.529943 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:16.420562 16515 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:16.519150 10099 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:16.408372 16533 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:16.527051 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:16.532598 10090 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:16.532375 16201 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:16.530037 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:16.534488 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:16.521196 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.535326 16460 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:16.535640 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:16.536831 16080 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:16.537482 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:16.539441 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.430219 16355 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.552634 16429 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.504834 16455 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:16.406395 16264 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.541129 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:16.542732 16497 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:16.527873 16539 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.547111 16547 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:16.548179 16530 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:16.547808 16495 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:16.556495  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 438243us
I20260709 12:26:16.559378 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:16.504731 15869 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:16.434954 16221 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.472417 16524 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.544248 16506 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:26:16.698585  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 86865.5us
I20260709 12:26:16.700510 10671 raft_consensus.cc:2368] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20.
I20260709 12:26:16.546301 16080 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:16.703841 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 144589us
I20260709 12:26:16.729384 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:16.525422 16463 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.557561 16383 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.733641  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:16.741173  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:16.707808 16505 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.711392 16423 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.749702  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:16.715164 16547 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:16.564424 16455 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.712682 16506 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:16.735944 16519 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:26:16.731565 16526 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:16.785633 16447 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:16.794386  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.752439 16532 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:16.718439 16530 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:26:16.772187 16329 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:26:16.779495 16435 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.662s	user 0.009s	sys 0.001s
I20260709 12:26:16.761842 16080 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:16.728374 16507 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.017s	sys 0.015s
I20260709 12:26:16.780349 16480 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:16.793785 16447 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.746064 16383 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.812001 10020 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
W20260709 12:26:16.823163 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 93868.7us
W20260709 12:26:16.815562 16507 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.018s	sys 0.015s
I20260709 12:26:16.804883 16549 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 12:26:16.826761  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:16.806005 16329 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:16.832976  9942 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:16.835559 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.836570 10090 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:16.783120 15869 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.827399 16530 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:16.855008  9946 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:16.856773  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2548) took 1032 ms. Trace:
I20260709 12:26:16.815045 16435 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.838625 16329 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:16.773772 16505 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.857157  9946 rpcz_store.cc:276] 0709 12:26:15.824149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.824879 (+   730us) service_pool.cc:224] Handling call
0709 12:26:16.848726 (+1023847us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:26:16.848926 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.855400 (+  6474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.856714 (+  1314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7027968}
I20260709 12:26:16.823519 16547 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:16.829130 16506 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.860311 16539 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:16.864792 16495 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.865936 15869 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:16.866983 16463 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:16.867367 16542 consensus_queue.cc:1175] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aa345bcd43b44088114b7b6f2830b17" responder_term: 19 status { last_received { term: 17 index: 11 } last_committed_idx: 11 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:16.857475 16429 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:16.871924 16383 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:16.874660 16455 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:16.851037 16427 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.734s	user 0.003s	sys 0.008s
I20260709 12:26:16.883302 16531 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:16.890188  9942 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:16.897059  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:16.855988 16530 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:16.857785 16507 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:16.842173 16511 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.887586 16264 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:16.914040  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 151021us
I20260709 12:26:16.924952  9944 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:16.936551  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:16.927050 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:16.927413 10670 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:16.932857 10022 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:16.917182 16480 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:16.937546 16507 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.938748 16480 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.939893 16507 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:16.940706 16530 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.917191 16524 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.941951 16530 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:16.942773  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:16.917308 16429 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:16.946817  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:16.947454  9944 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:16.936368 16465 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:16.956699 16435 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.949930  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:16.945569 16056 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:16.961359 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:16.962112 10020 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:16.962452  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:16.962761 10643 leader_election.cc:400] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:26:16.963058 10643 leader_election.cc:403] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:16.961813 16056 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:16.963656 16429 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.965085 16429 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:16.964830 16056 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.966360 16556 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
I20260709 12:26:16.933939 16427 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.949162 16524 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:16.968063 16524 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:16.968751 16427 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.968963 16524 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.966985 16463 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:16.973412 16463 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:16.974447 16463 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.979447 16505 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:16.983870 16535 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:16.984508 16535 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:16.985227 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:16.985378 16535 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:16.986178 16521 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:16.993433 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3461) took 1002 ms. Trace:
I20260709 12:26:16.995111  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:16.995954  9946 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:16.998311  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:16.999297  9942 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:16.993901 10016 rpcz_store.cc:276] 0709 12:26:15.991052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.062350 (+ 71298us) service_pool.cc:224] Handling call
0709 12:26:16.072742 (+ 10392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.511557 (+438815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:16.993204 (+481647us) spinlock_profiling.cc:243] Waited 438 ms on lock 0x7b54001f6154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:16.993370 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1007596288}
I20260709 12:26:17.006132 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:17.007061 10677 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.025619  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:16.935550 16532 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:17.009832 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:17.045560 10677 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:17.047523  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.027190 16511 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:17.061583 10642 leader_election.cc:336] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.762s (SENT)
I20260709 12:26:17.062173 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:17.062211 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:17.063439 10675 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:17.029294 16221 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.085891 16221 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.088155 16221 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.059149 16395 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 12:26:17.096230 16511 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.067924 16562 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.098049 16562 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:17.059964 16457 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.797s	user 0.011s	sys 0.000s
I20260709 12:26:17.100658 16533 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.101114 16533 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.105176 16355 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.069365 16532 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.070258 16556 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.042908 16383 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.050053 16329 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.080873 16530 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:17.083853 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3462) took 1089 ms. Trace:
I20260709 12:26:17.063128 16463 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.136008 16463 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.137207 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.138003 10672 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:17.138878 10025 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: [14.11-14.11]
I20260709 12:26:17.139648  9943 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:17.096195  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.115147 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1362) took 1180 ms. Trace:
I20260709 12:26:17.061232  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:17.130314 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.131594  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.132243  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.133374  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:17.148691  9941 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:17.133805 10090 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:17.134639  9987 leader_election.cc:336] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.664s (SENT)
I20260709 12:26:17.135524 10016 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:17.127514 10022 rpcz_store.cc:276] 0709 12:26:15.994832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:16.163008 (+168176us) service_pool.cc:224] Handling call
0709 12:26:16.187262 (+ 24254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:16.937946 (+750684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:17.083632 (+145686us) spinlock_profiling.cc:243] Waited 745 ms on lock 0x7b54001379d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:17.083793 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1714070912}
I20260709 12:26:17.152398  9946 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:17.141011 10671 rpcz_store.cc:276] 0709 12:26:15.934846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:15.935205 (+   359us) service_pool.cc:224] Handling call
0709 12:26:17.114938 (+1179733us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b54005267d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:17.115085 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1751360768}
I20260709 12:26:17.153975 10090 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:17.150305  9987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 2.200s (SENT) [suppressed 33 similar messages]
I20260709 12:26:17.156751 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.159016 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:17.131217 10640 leader_election.cc:400] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16.
W20260709 12:26:17.163892  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 102739us
I20260709 12:26:17.165307  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.165791 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.165946  9949 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:26:17.167577  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:17.169023 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:17.166352 10091 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:17.176275 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.177464 10090 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:26:17.178565 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.181536 16429 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:17.182435 16429 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:17.182777 16429 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:17.183525 16429 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.184733 16429 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:17.186882 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:17.187384 16329 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.187949 16517 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:17.188409 16517 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.188831 16517 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.190982 16560 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.126670 16383 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.192337  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:17.180065 10093 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
W20260709 12:26:17.194779  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.192200 16463 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.194651  9940 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:17.165328 16080 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:17.178203 16480 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:17.203177 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:17.208518  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 58241us
I20260709 12:26:17.153324 16511 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:17.193114 16544 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:26:17.229763  9987 leader_election.cc:336] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.770s (SENT)
I20260709 12:26:17.229756 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.230926 10092 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:26:17.231379 16480 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:17.232002 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:17.232450 16480 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.235306 16533 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:17.236145 16496 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:17.236586 16496 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.236907 16496 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:17.240767 16562 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.241460 16521 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:17.243861  9912 leader_election.cc:400] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20.
I20260709 12:26:17.244217  9912 leader_election.cc:403] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:17.201530 15869 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.230521  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:17.243991 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:17.246582 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:17.249099 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.255046 10675 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:17.258102  9941 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:17.258500 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.259061  9945 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:17.259184 10090 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:17.245191 16080 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:17.260198 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:17.260809 10671 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.261154 16080 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.218443 16463 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:17.266959  9948 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:17.244081 16329 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:26:17.268801 16221 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:17.269620 16221 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:17.269990 16221 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:17.201496 16493 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
W20260709 12:26:17.285173 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:17.288120 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44221.2us
I20260709 12:26:17.285355 10092 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:17.285951  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56218.4us
I20260709 12:26:17.289925 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:17.294823 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.298879 10671 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:17.300614  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:17.303471 10093 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:17.321139 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:17.258700 16511 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.327406 16511 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.282166 16530 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20.
I20260709 12:26:17.330291 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:17.332120 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:17.245594 16505 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.332479 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:17.303133 16184 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.287922 16463 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:17.356251 16542 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:17.356725 16542 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:17.356935 16463 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:17.335366 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46552.2us
I20260709 12:26:17.304550 16485 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.336292 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:17.339073 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:17.374197 10022 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:17.342948 16495 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:17.343744 16383 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.283685 15869 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.271112 16221 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.329470 16562 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:17.349370 16505 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.350725 16511 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.351402 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.330744 16507 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:17.425753 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.331265 16526 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
I20260709 12:26:17.310829 16457 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:17.304488 16113 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.324769 16535 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:17.361802 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:17.375648 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:17.376721 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:17.378365  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:17.324885 16367 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.389534 16542 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.386341 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:17.456594 10019 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:17.348910 10020 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:17.392400 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:17.426462 10091 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:17.391115 16565 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:17.379828 16467 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:26:17.452064 10095 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:17.454795 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:17.453969 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 117491us
I20260709 12:26:17.455166  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:17.463238 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.469967 10024 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:17.472010 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.474010  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:17.477905 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 145644us
I20260709 12:26:17.472687 10093 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:17.480715 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:17.475472 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.487805 10669 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:17.385924 16517 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.495633 10671 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:17.496918  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:17.486799 16531 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.443377 16363 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:26:17.498411 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.484019 16427 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.463225 16535 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:17.491115 16560 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.501717 16531 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.425290 16562 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.454221 16113 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.444655 16457 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.512784 16505 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:17.515139 16579 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
I20260709 12:26:17.499991 16524 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:17.515848 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:17.499825 16480 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.483518 16565 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:17.489329 16563 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:17.462409 16367 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.500046 16056 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:17.530226  9946 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:17.536816 10021 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:17.532044 16542 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:17.531947 16524 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.535118 16563 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.521307 16560 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.531752 16184 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
W20260709 12:26:17.562309 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 122019us
I20260709 12:26:17.564152 16584 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.582552 16584 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.583408 16531 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.584673 16584 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:17.585431 16506 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.538329 16201 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:26:17.586099 16113 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.586723 16201 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:17.587117 16201 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:17.587487 16562 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.587988 16201 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.588732 15869 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.589388 16201 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.589918 16517 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.590853 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.591403 16457 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:17.591511 10093 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:26:17.555773 16524 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.591744 16565 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.536010 16480 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.596549 16480 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.597720 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.598740 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.599385 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.600467 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.601114 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:17.599980 10675 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:17.604619 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.607055  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:17.610258  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:17.615104 16463 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:17.615921 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.560432 16563 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.615272 16480 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:17.575201 16580 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:17.619588 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:17.621584 10670 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:17.580295 16184 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:17.622746 16435 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:26:17.624428 16184 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:17.625222 16367 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.625496 16546 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:17.623117 16480 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:17.626185 16480 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:17.627540 16546 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:17.628679 16435 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:17.629024 16435 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:17.629798 16435 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.581911 16560 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.631937 16435 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.632797 16496 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.640045  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.641531  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.641793  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.642462  9944 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:17.646664 15869 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:17.647251 15869 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.649569 15869 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.621579 16080 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.654392 16080 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.655268 16555 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.655810 16555 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.656327 10671 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:17.656484 16080 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:17.658077 10675 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:17.658574  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.663045 16429 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:17.569278 16535 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.664458 16539 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:17.665056 16539 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:17.667635 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.668429 10090 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:17.675401  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:17.681394  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 138533us
I20260709 12:26:17.656270 16480 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.696460 16490 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:17.697093 16490 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.666039 16539 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.683671 16429 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:17.738668 16595 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:26:17.764621  9941 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 16 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:17.765036 16595 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:17.765542 16595 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:17.766068  9940 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 12. (term mismatch)
I20260709 12:26:17.766460  9940 pending_rounds.cc:85] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 11
I20260709 12:26:17.767042  9940 pending_rounds.cc:107] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:26:17.767385  9940 raft_consensus.cc:2889] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:17.754638 10677 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:17.739045  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:17.706822 16594 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:17.771498  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.776327  9941 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:17.776789  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:17.775350  9942 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 12:26:17.777362  9942 pending_rounds.cc:85] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 10
I20260709 12:26:17.777598  9942 pending_rounds.cc:107] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 12:26:17.777693  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:17.777839  9942 raft_consensus.cc:2889] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:17.781850  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:17.737311 16485 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:17.787914  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.741813 16539 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:17.746768 16496 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.795969 16447 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:17.796514 16447 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:17.796620 16485 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.797318 16555 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.797384 16447 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.798866 16485 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.792582 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:17.803614 16184 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.804972 16184 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.753937 16490 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.810745 16604 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.740001 16367 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:26:17.740219 16429 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.812207 16480 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:17.821405 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.775120 16531 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
W20260709 12:26:17.822762 16597 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:26:17.823832 16521 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:17.824374 16521 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.824806 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.815830  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:17.816359  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:17.828502 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.818075 10677 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:17.784942 16455 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:17.831811  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:17.832073 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 119857us
I20260709 12:26:17.837368 10672 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
W20260709 12:26:17.837731  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 459470us
I20260709 12:26:17.837898  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:17.839912 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:17.827801 16447 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.840015  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:17.841436 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.840612 10099 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:17.819468 16506 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:17.822530 16429 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.756971 16563 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.835292 16505 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:17.836498 16521 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.839366 16485 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:17.836524 16455 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:17.851764  9990 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:17.840683 16531 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:17.842136 16367 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:17.848120 16533 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:17.843245 16480 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:17.847007 16113 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:17.778318 16595 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.852864  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:17.865242 16606 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:17.875975 16606 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:17.876432 16606 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:17.854864 16395 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:17.853546 16485 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:17.902967 16485 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.869783 16594 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 12:26:17.875149 16595 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.909489 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.866365 16533 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:17.910742 10098 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:17.911405 16533 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.912184 16600 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:17.912822 16533 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.913781 16600 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:17.941735 16600 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:17.942130 16600 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:17.881029 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.881625 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.943271 10097 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:17.882022 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:17.843874 16524 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.944780  9940 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:17.945776 16524 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.945567 16600 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.885797 16429 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.882525 16480 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:17.946384 16524 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:26:17.892946 16563 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.847321 16506 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.949414 16429 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:17.951212 16506 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:17.895524 16541 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.412s	user 0.003s	sys 0.000s
I20260709 12:26:17.951643 16429 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:17.859805 16367 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:17.953291 16367 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.955371 16513 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:17.955909 16480 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.955902 16513 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.960877 16513 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:17.963358 10672 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:17.859081 16455 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:17.877249  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:17.923846  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.963376 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:17.967406  9944 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:17.967727 10670 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.924677  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:17.928040 16545 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:17.969477 16545 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.932670 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.972627 16545 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:17.935568 16493 consensus_queue.cc:1175] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd983415389444e292b2330ef630850a" responder_term: 16 status { last_received { term: 16 index: 14 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:17.946291 16221 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:17.975445 16539 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.975664 10096 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:17.976082 16539 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.978775 16539 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.982820 16367 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:17.989344 16603 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:17.990012 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:17.990003 16603 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:17.992635 16603 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:17.993474 16113 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:17.993906 16113 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:17.994316 16113 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:17.997715 10091 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:17.998962 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:18.000613 16594 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:18.001071 16594 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:18.003304 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:18.004076 10096 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 14: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:18.001971 16594 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.005112 16594 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:18.005795 10091 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:18.007377 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:18.008000 10094 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:18.002266 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:18.011375 10092 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:17.990844 10672 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:18.016919  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:17.936878 10669 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:17.937965 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:18.020269 10098 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:17.940724 16531 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:18.020736 10016 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
W20260709 12:26:18.020892  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 168332us
I20260709 12:26:18.044273  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:18.019325  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:18.030005 10670 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:18.031877  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:18.058104  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:18.062971  9944 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:18.066968 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.089864  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:18.092486 10094 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:18.096730  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:18.102350 10099 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:18.061519 16563 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:18.107030 16602 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:18.110499  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52501.7us
I20260709 12:26:18.113473 10098 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:18.117393  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:18.083349 16329 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:18.121923  9987 leader_election.cc:400] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:26:18.122272  9987 leader_election.cc:403] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:18.125012 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:18.044418 16546 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:26:18.102237 16221 raft_consensus.cc:2764] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:26:18.061033 16539 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 12:26:18.105211 16526 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.082129 16556 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:26:18.090678 16429 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:26:18.113116 16563 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.160923 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:18.099619 15869 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:18.144529  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:18.114703 16463 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.094334 16493 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:26:18.129079 16604 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:18.188463 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.158717 16506 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:18.196098  9945 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:18.125538 16511 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:26:18.159662 16383 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:26:18.163825 16546 consensus_queue.cc:1243] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:26:18.182464 16596 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
W20260709 12:26:18.211906  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:18.225973 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.245105 10669 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:18.185324 16595 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:18.248796 10099 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:18.192664 16604 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.260040 10090 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:18.185170 16555 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:18.272579  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:18.195111 16495 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:18.285310 10099 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:26:18.287801  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 101335us
I20260709 12:26:18.297664 10670 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:18.202112 16563 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:18.120467 16329 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.302966  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:18.317198 16535 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:18.321796 16555 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:18.323014 16539 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:18.263098 16604 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:18.175624 16526 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.190389 15869 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:18.190037 16463 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.334431 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.299481 16495 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:18.335100 16602 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.338852 16455 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:18.360013 16608 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
I20260709 12:26:18.360273 16584 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.360555 16455 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:18.348541 16532 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:18.360877 16584 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.363271 16584 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:18.352859  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.004s	sys 0.021s
I20260709 12:26:18.349468 15869 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:18.365056 16521 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:26:18.365984  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.005s	sys 0.022s
W20260709 12:26:18.371220  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 589450us
I20260709 12:26:18.373910 10671 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:18.375021 10099 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:26:18.375419 10099 pending_rounds.cc:85] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 9
I20260709 12:26:18.375636 10099 pending_rounds.cc:107] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:26:18.375888 10099 raft_consensus.cc:2889] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:18.362417 16612 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.051s	user 0.002s	sys 0.000s
I20260709 12:26:18.188827 16606 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.381345 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:18.333649 16329 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.382129  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:18.388985 10094 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:18.391904 16485 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:18.391880  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:18.396526 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.397467 10022 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:18.399277 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.400179 10021 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:18.402525 10024 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:18.402566 16606 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:18.403674 16485 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:18.393971 16221 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:18.402575 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.408105 10025 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:26:18.410794 16221 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:18.330796 16423 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:26:18.412447  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:18.413096 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:18.414134 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.418773 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.420665  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:18.421505  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:18.340796 16467 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:26:18.422950 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.431371 10021 raft_consensus.cc:2368] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
I20260709 12:26:18.423228  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:18.434567 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.439589  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:18.440989 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.466284 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.471800  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:18.472743  9944 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.473475 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.382963 16532 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:26:18.384440 16113 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.474113 10021 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:18.430190 16521 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:18.484774 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.495687 10023 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:18.423393 16606 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:18.487658 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.494863 10024 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:18.502777 10018 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:18.510779 16606 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.511756 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.534093 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:18.515472 16600 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:18.517701 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.518239 10023 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:18.439149 16511 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.525274 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:18.527642  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:18.540536  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:18.542182  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:18.542572 16485 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:18.531808 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:18.543068 16542 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:18.543511 16542 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:18.546030 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.546587  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:18.547214 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.547824 10024 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:18.542757 16617 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.543437 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.538727 10021 raft_consensus.cc:2368] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
I20260709 12:26:18.552654 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.553551 10021 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:18.529462 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:18.554404  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:18.548645 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.555717 10024 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:18.555894 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:18.556473 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.432855 16367 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.063s
I20260709 12:26:18.557145 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.557792 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.555583  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:18.552675 16363 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:18.564774 16594 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.559s	user 0.008s	sys 0.003s
W20260709 12:26:18.565351 16611 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:26:18.565958 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.557433 16595 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:18.566833 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.567260 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.556778 16485 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.495404 16521 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:18.573325 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:18.606853 16521 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:18.606979 10092 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.607786 10025 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.608081 16485 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:18.574316 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:18.614078 16521 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.615058 16520 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:18.615621 16520 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.616053 10025 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:18.616604 16521 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:18.617278 16521 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:18.617764 16521 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.618309 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.619926 16521 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:18.626806 16493 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:26:18.628636 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.629377 10023 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:18.626174 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:18.631138 10021 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:18.632691 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:18.549763 16609 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:26:18.642041 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.642902 10023 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:18.581429 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.582971 10022 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:18.583163 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:18.538610 16606 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:18.549619 16542 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:18.552822 16524 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:18.439736 16555 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.628249 16221 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:18.639516 10025 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:18.661041 10092 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:18.602048 10024 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:18.559857 16617 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:18.601503 16363 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.627643 16493 consensus_queue.cc:1243] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:26:18.636252 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.550088 16511 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:18.649201 10020 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:18.571695 16563 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:18.578192  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:18.689321  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 111237us
I20260709 12:26:18.655506 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:18.693418 10023 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:18.656101 16467 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.664830 10091 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
W20260709 12:26:18.606609  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 51083.1us
I20260709 12:26:18.664626 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:18.701308 10024 raft_consensus.cc:2376] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17. Re-sending same reply.
I20260709 12:26:18.702493 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.703285 10024 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:18.675410 10025 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:18.687361 10021 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:18.693845 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:18.657382 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.711264 10022 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:18.712915 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
W20260709 12:26:18.713434 16610 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 12:26:18.713084 16329 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.713617 10024 raft_consensus.cc:606] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:18.714231 10024 raft_consensus.cc:614] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:18.715425 16329 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:18.687428 16520 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:18.693964 16463 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:18.665541 16524 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:18.698072 16506 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:18.708148 16493 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:18.695784 15869 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:18.694427 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:18.635891 16435 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:26:18.762863 10021 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:18.789631 10021 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:18.771610  9941 raft_consensus.cc:2368] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20.
I20260709 12:26:18.780115 10018 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:18.781929 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 88197.7us
I20260709 12:26:18.793414 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:18.794015 10096 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:18.794234 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:18.787392  9990 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:18.796239 10096 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
W20260709 12:26:18.788071 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:18.769652  9944 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:18.800480 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:18.796051  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:18.804291 10062 leader_election.cc:400] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
I20260709 12:26:18.804715 10062 leader_election.cc:403] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 12:26:18.806170 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:18.801977  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:18.807396 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:18.809633  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:18.811084 10097 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:18.746008 16447 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.900s	user 0.010s	sys 0.000s
W20260709 12:26:18.752954 16613 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.417s	user 0.002s	sys 0.000s
W20260709 12:26:18.813138 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 417621us
I20260709 12:26:18.772434 16563 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:18.771456 16490 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:18.823861 10093 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:18.767882 16363 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:18.791580 16594 raft_consensus.cc:2764] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:18.845422  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:18.776671 16532 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:18.827255 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:18.799680 16467 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:18.772989 16493 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:18.773422 15869 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:18.771100 16511 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:18.809499 16602 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:18.803086 16367 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:26:18.810791 16533 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:18.839435 16539 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:18.825569 16480 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:18.814174 16505 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:18.859972 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:18.845798 16591 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:18.914057 10669 raft_consensus.cc:2368] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18.
I20260709 12:26:18.861845 10023 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:18.856321 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:18.886065 10021 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:26:18.864981  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 318449us
I20260709 12:26:18.900848 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:18.840492 16447 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16.
W20260709 12:26:18.929399 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 73179us
I20260709 12:26:18.851408 16535 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:18.794910 16542 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:18.831101 16490 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:18.908552 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 338645us
I20260709 12:26:18.742520 16604 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.844579 16485 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:18.760247 16524 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.932514 16634 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:18.987466 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:18.988476 10669 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:18.990011 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:18.989717 15869 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.906716 16467 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:18.937970 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:18.942940 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:18.943154 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:19.002565 10668 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:18.944489 16591 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:18.944756 16600 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:18.928500 16435 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:18.851716 16455 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:26:18.949880 16546 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:18.860255 16532 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.015182 16532 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:18.851536 16595 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
W20260709 12:26:19.019136 16618 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:26:18.952890 10098 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 12:26:18.975504 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:19.019877 10098 pending_rounds.cc:85] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 14
I20260709 12:26:19.020102 10677 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.020639 16546 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:19.021570 16546 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:18.851847 16363 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:19.023294 16363 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.023888 16604 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.024627 16632 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.025105 16632 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.025575 16363 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:18.984035 16535 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.027297 16632 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:18.980384 16505 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:19.028177 16467 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.028460 16505 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:18.974073 16506 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:19.029709 16493 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 14.14, Last appended by leader: 14, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.030273 16506 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.031177 16591 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:18.978300 16485 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.033609 16485 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:19.010567 16435 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.036643 16606 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.037686 16435 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:19.037837 10025 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:18.940420  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:19.038887  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 99072.1us
I20260709 12:26:19.038816 16490 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.039812  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:19.008899 16565 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:19.040594 16565 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:19.041006 16565 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:18.928210 16480 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.043283 16563 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.043602 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:19.044152 16480 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:19.044842 16480 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:26:19.004637 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67145.2us
I20260709 12:26:19.020197 10098 pending_rounds.cc:107] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:26:19.027254 10671 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:19.045699 10098 raft_consensus.cc:2889] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:19.045917 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:19.047076 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:19.008541 10016 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:18.982422 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:19.048993 10096 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:19.044196 10023 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:19.042409  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:19.053645  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:19.053890  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:19.054499  9944 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:19.058260  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:19.061188 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:19.065387  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:19.066304  9944 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:19.058907  9942 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:19.071036 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:19.072721 10677 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:19.047242 16542 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:19.103629 10023 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:19.116060 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:19.059456 16513 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:19.137519 10017 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:19.138576 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:19.059715 16329 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:19.070243 16423 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:19.141357  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:19.070884 16363 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:19.094566 16600 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.149906  9944 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:19.094187 16634 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:19.152307 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:19.144266  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:19.153685 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 337923us
I20260709 12:26:19.106921 16184 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:19.137207 16555 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:19.169028 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:19.170120 16363 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:19.137271 16435 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.174654 10023 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:19.159296 16532 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:26:19.217653 16435 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.173811 16526 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:19.103009 16520 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:26:19.103868 16626 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 12:26:19.244037  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:19.264878  9949 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:19.231662 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:19.308354 10023 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.308528 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2513) took 1428 ms. Trace:
W20260709 12:26:19.173631 16622 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:26:19.331125 10023 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
W20260709 12:26:19.340927 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 200457us
I20260709 12:26:19.342700  9940 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:19.342264 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:19.346738 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:19.352347 10023 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:19.309311 10093 rpcz_store.cc:276] 0709 12:26:17.880513 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.880664 (+   151us) service_pool.cc:224] Handling call
0709 12:26:19.308251 (+1427587us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b5400164254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:19.308438 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025709312}
I20260709 12:26:19.220106 16435 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.175499 16363 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:19.227653 16480 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:19.381472 10023 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:19.168480 16600 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.345943 16329 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:19.362824 16608 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
W20260709 12:26:19.405256 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48844us
I20260709 12:26:19.379866 16363 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.409336  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:19.409948 10019 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.005s	sys 0.013s
I20260709 12:26:19.410354 10023 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:19.412062 10019 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.006s	sys 0.013s
W20260709 12:26:19.408702 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 337761us
I20260709 12:26:19.425827 10023 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:19.428673 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2292) took 1026 ms. Trace:
I20260709 12:26:19.429435 10019 rpcz_store.cc:276] 0709 12:26:18.402540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.405687 (+  3147us) service_pool.cc:224] Handling call
0709 12:26:18.406006 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:18.406199 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:18.791486 (+385287us) log.cc:844] Serialized 14 byte log entry
0709 12:26:19.428189 (+636703us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:19.428269 (+    80us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:19.428335 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.428426 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.428593 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15370,"prepare.queue_time_us":597721,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"thread_start_us":1001185,"threads_started":2}
I20260709 12:26:19.410838 16602 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:19.433374 16602 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:19.483330 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 76456.2us
I20260709 12:26:19.408526 16329 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.393317 16480 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.531273 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:19.551963 10023 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:19.551623  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:19.556562 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:19.555068  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:19.559588 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:19.557264 10019 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:19.541162 16526 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:19.553217 16640 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
I20260709 12:26:19.549443 16604 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.502930 16657 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:19.449949 16495 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.558813 16542 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:19.605373 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:19.563125 16429 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:19.558637 16435 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:19.606669 16542 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:19.562155 16656 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:26:19.601708 16657 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:19.608209 16644 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.640898 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:19.577797 16637 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:26:19.648326 10019 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:19.602041 16526 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:19.630617 16642 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 12:26:19.625938 16495 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:19.556811 16535 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:26:19.603624 16604 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.638095 16542 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.622162 16435 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:19.723106 16651 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:26:19.697053 16600 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:19.723754 16435 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.723855 10019 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:19.724552 16663 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:19.728674 16480 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.732254 16604 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.627589 16520 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.039s
W20260709 12:26:19.684793 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 260446us
I20260709 12:26:19.734014 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:19.695166 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1359) took 1151 ms. Trace:
I20260709 12:26:19.645169 16657 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:19.735343 16531 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.735991 16531 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.738904 16447 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:19.739324 16447 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:19.739711 16447 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:19.735014 10676 rpcz_store.cc:276] 0709 12:26:18.534840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.535023 (+   183us) service_pool.cc:224] Handling call
0709 12:26:18.554246 (+ 19223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.707766 (+153520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.707795 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.713995 (+  6200us) log.cc:844] Serialized 26 byte log entry
0709 12:26:18.971228 (+257233us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:18.991528 (+ 20300us) log.cc:844] Serialized 14 byte log entry
0709 12:26:18.991723 (+   195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:18.991784 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:19.685906 (+694122us) raft_consensus.cc:1685] finished
0709 12:26:19.685925 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:19.686054 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":25823,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":47048,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":25590,"spinlock_wait_cycles":17950592,"thread_start_us":409190,"threads_started":2,"wal-append.queue_time_us":459385}
I20260709 12:26:19.738375 16531 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.742955 16435 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:19.744086 16629 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:19.667303 16526 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:19.744496 16629 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:19.745398 16629 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.746410 16629 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:19.746760 16634 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.573s	user 0.007s	sys 0.003s
I20260709 12:26:19.748780 16634 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:26:19.753824 16555 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.577s	user 0.011s	sys 0.000s
I20260709 12:26:19.754887 16555 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:19.716742 16513 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.000s
W20260709 12:26:19.720330 16485 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.686s	user 0.006s	sys 0.005s
W20260709 12:26:19.700184 16647 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
W20260709 12:26:19.722275 16658 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:26:19.656445 16644 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.778535  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:19.768735 16652 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.278s	user 0.001s	sys 0.001s
I20260709 12:26:19.770659 16513 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:19.783404 16447 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.776713 16485 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.801878 16429 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:19.817510 16429 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:19.819561  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 259773us
I20260709 12:26:19.822726 16656 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:26:19.824903  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:19.827687 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 405220us
I20260709 12:26:19.830045 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:19.831213 10640 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:19.833355  9987 leader_election.cc:400] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18.
I20260709 12:26:19.833798  9987 leader_election.cc:403] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:19.839059 10020 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:19.840066 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:19.825646 16535 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:26:19.841174 10669 raft_consensus.cc:2410] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
W20260709 12:26:19.846086 10018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2302) took 1330 ms (client timeout 1590 ms). Trace:
I20260709 12:26:19.834401 16531 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:19.846660 10018 rpcz_store.cc:269] 0709 12:26:18.508362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.544527 (+ 36165us) service_pool.cc:224] Handling call
0709 12:26:19.838628 (+1294101us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b54001f8954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:19.838777 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":305778432}
I20260709 12:26:19.849856 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:19.851953 10676 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:19.835270 16635 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:19.833053 16513 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:19.837010 16648 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18.
I20260709 12:26:19.867033  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2583) took 1444 ms. Trace:
I20260709 12:26:19.867445  9941 rpcz_store.cc:276] 0709 12:26:18.422804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.423008 (+   204us) service_pool.cc:224] Handling call
0709 12:26:19.866827 (+1443819us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b5400222ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:19.866971 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":780341504}
I20260709 12:26:19.868508 16617 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:19.851070 16531 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:19.866459 16563 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.823s	user 0.013s	sys 0.000s
I20260709 12:26:19.838864 16657 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.875838 16563 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:19.876674 16655 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:19.874426 16531 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:19.888290 16655 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.845337 16480 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:26:19.862699 16673 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:19.889595 16673 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:19.879197 16650 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:26:19.883039 16644 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.892833 16485 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:19.882273  9843 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 10 to 14, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 14 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:19.883610 16656 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:19.857854 16542 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.894502 16113 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.895093 16542 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.898937 16673 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:19.857067 16676 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:19.899868 16676 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:19.902660 16676 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:19.904275 16480 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:19.904654 16480 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:19.905455 16480 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.861442 16635 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:19.907492 16635 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:19.915607 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.892969 16655 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:19.922456  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 86454.7us
I20260709 12:26:19.918542  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:19.925597 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 642810us
W20260709 12:26:19.911580 16565 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.870s	user 0.013s	sys 0.000s
I20260709 12:26:19.887633 16526 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.925580 16480 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:19.940151 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2304) took 1431 ms. Trace:
I20260709 12:26:19.940650 16655 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:19.941771 16655 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.942417  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:19.943252  9943 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:19.947913 10676 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:19.948114 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:19.949841 10669 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:26:19.893395 16506 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:19.940573 10025 rpcz_store.cc:276] 0709 12:26:18.508950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.788081 (+279131us) service_pool.cc:224] Handling call
0709 12:26:18.788492 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.038205 (+249713us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.939809 (+901604us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b540032aad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:19.939991 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":572978432}
I20260709 12:26:19.952157 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:19.954708 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.938378 16526 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:19.921542 16644 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:19.961725 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:19.963109  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:19.906469 16531 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.958006 16490 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:19.956213 16363 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.002s
I20260709 12:26:19.969610 16645 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.970227 16645 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.971144 16676 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:19.971730 16676 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.973217 16645 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.969113 16363 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.973980 16676 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.966682 16505 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:19.975507 16363 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:19.975360 16505 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.974581 16676 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:19.978524  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:19.958950 16533 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.983769 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:19.986909 10092 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:19.964474 16594 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:19.980346 16676 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:19.984951 16683 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:26:19.917038 16656 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:19.961652 16604 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:19.989400  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:19.991207 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:19.983884 16505 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:19.950443 16565 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.004755 10092 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:19.967219 16644 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.028532  9940 raft_consensus.cc:2368] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:20.036660 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3711) took 1260 ms. Trace:
I20260709 12:26:20.039400 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.040318 10670 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:20.024039  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:20.037070 10092 rpcz_store.cc:276] 0709 12:26:18.775941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.777946 (+  2005us) service_pool.cc:224] Handling call
0709 12:26:19.983656 (+1205710us) spinlock_profiling.cc:243] Waited 1.88 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431eee 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:26:20.036598 (+ 52942us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4334208}
I20260709 12:26:19.987564 16685 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:20.045517  9940 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:19.994428 16657 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:20.012454  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.024114 16187 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.033268 16656 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.047518 16467 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:26:20.047132 16505 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:20.049665 16187 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.050303 16505 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.051107 10643 leader_election.cc:400] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:20.051479 10643 leader_election.cc:403] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:26:20.052558  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.053629  9940 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 13.
I20260709 12:26:20.056084 10669 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:20.056363 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:20.030419 16591 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:20.027518 16447 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:20.055936 16677 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 12:26:20.065897 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 80494us
I20260709 12:26:20.023613 16684 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:20.043025 16655 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.071075 16531 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.048120 16626 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:26:20.002647 16594 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.076426 16505 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.076846 16598 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:20.083935 10022 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:20.065409  9948 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:20.047627 16645 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:19.993763 16533 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:20.087503 10016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1167) took 1668 ms (client timeout 1522 ms). Trace:
I20260709 12:26:20.062085 16635 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:20.030027 16604 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.151755 16329 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.152709 16604 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:20.109692  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 186318us
W20260709 12:26:20.112161 10016 rpcz_store.cc:269] 0709 12:26:18.415290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.415882 (+   592us) service_pool.cc:224] Handling call
0709 12:26:20.083511 (+1667629us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b54001f6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:20.083695 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1231341440}
I20260709 12:26:20.044441 16685 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.122166 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1150) took 1931 ms. Trace:
I20260709 12:26:20.111181 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:20.063269 16546 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:26:20.144506 16644 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.106698 16684 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.145684  9943 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:20.106492 16598 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.159444 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:20.189929 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 127041us
I20260709 12:26:20.164361 10098 rpcz_store.cc:276] 0709 12:26:18.180851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.181022 (+   171us) service_pool.cc:224] Handling call
0709 12:26:18.181361 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.046145 (+864784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.111664 (+1065519us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b5400163854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:20.111852 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1773473024}
I20260709 12:26:20.170773 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.194119 10024 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:20.191418 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.195084 10017 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
W20260709 12:26:20.194696 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83619.9us
I20260709 12:26:20.180527 10025 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:20.183945 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.184890 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.186082 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.177273 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:20.190223 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.197711 10019 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:20.202188 10093 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:20.204165 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.204362 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.204926 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:20.205601 10096 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:20.206789 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:20.201376 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.210863 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.212983 10025 raft_consensus.cc:2368] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:20.215302 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:20.225945 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.226683 10092 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:20.227197 10024 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:20.229300 10670 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:20.230161 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:20.235836 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.230607 10025 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:20.187181 16668 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:20.237031 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.154337 16645 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.189213 16633 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:20.250078 10024 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:20.250344  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:20.239298  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:20.250579 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 194350us
I20260709 12:26:20.206422 16363 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:20.239464 10062 leader_election.cc:400] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:20.206575 16505 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.244565 16187 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:20.244869 16676 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.246768 16679 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:20.247210 16687 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:20.191872 16520 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:20.199390 16598 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:20.248749 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:20.278524 10018 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:20.252764 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.240473 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.200240 16626 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:20.257952 10062 leader_election.cc:403] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:20.239087 16447 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:20.270444 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:20.228207 16644 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:26:20.277081 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.291529 10025 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:20.292057 10024 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
W20260709 12:26:20.294148 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57891.7us
I20260709 12:26:20.301064 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:20.299906 10672 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:20.303697  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:20.305146 10024 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 15. (index mismatch)
I20260709 12:26:20.305994  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:20.307332 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:20.310098 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:20.274816 16594 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.302263 16629 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:20.316123 16686 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
W20260709 12:26:20.316151 16680 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:26:20.248519 16668 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.256230 16633 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.268997 16505 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.289188 16363 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:20.338604 16600 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:20.338984  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:20.343345 16691 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:20.344101 16691 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.348380 16629 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:20.349505 16447 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:20.349587 16668 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:20.351907 16676 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:20.352450 16676 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:20.352767 16606 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.353403 16600 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:20.353309 16676 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.354359 16600 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.356839 16633 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.312407 16546 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:20.354745 16684 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.372116 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.373091 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.329695 16617 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:20.329970 16594 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:20.281117 16520 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:20.377136 16495 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.695s	user 0.013s	sys 0.000s
I20260709 12:26:20.345870 16187 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:26:20.383905 16533 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.299672 16626 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.281291 16687 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.382711 16617 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:20.402745 16617 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:20.403662 16505 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.383548 16594 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:20.403880 16694 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.414160 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:20.416234 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 82323.4us
I20260709 12:26:20.418392 10090 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:20.416309 10096 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:20.417320 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:20.417670 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.421456 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.422079 10094 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:20.420459 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.410144 16495 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:20.424423 16495 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.424787 16520 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:20.409543 16594 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:20.400523 16684 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.427163 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.428366 10090 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
W20260709 12:26:20.427289 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 68247.6us
W20260709 12:26:20.427341  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:20.430437 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 129461us
I20260709 12:26:20.406517 16533 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.437705 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.409433 16685 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.446452 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.447633 10672 raft_consensus.cc:2410] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:26:20.273604 16679 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.449932 16691 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:20.412945 16644 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.451141 16679 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.450961 16644 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.452028 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:20.452261 10019 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:20.452602 10016 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:20.453126 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.453718 16644 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.444936 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.458799 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:20.460471 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.423146 16467 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:20.461128 10090 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:20.466945 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:20.466945 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.468458 10090 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:20.472615  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "840d8356270745f492c7d2c7099794c1"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:26:20.473222  9944 raft_consensus.cc:606] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:20.473326  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:20.473582  9944 raft_consensus.cc:614] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:20.456614 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:20.454232 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.445585 16645 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.480389  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.472010 16674 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:20.500458  9943 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:26:20.428498 16655 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:26:20.461601 16631 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:20.446274 16533 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:20.477661 16695 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:26:20.432457 16363 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.482513 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:20.479363 16632 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.520773 16632 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.483611 16526 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:20.471282 16467 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.523643 10098 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:20.524753 10092 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:20.525422 10093 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
W20260709 12:26:20.525938 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43489.7us
I20260709 12:26:20.526664 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:20.527736 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:20.528697 16668 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:20.529335 16668 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:20.529491 16635 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:20.530702 10673 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:20.489060  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.526448 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:20.482498 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61415.4us
W20260709 12:26:20.498131  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 239210us
I20260709 12:26:20.498833 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.501749 10016 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:20.426342 16694 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.433991 16684 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.509086  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.470077 16520 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:20.542438 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:20.510447  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:20.543215  9991 leader_election.cc:400] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20.
I20260709 12:26:20.515599  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:20.488134 16662 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:20.554824 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.555114 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.532343 10093 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:20.536427 10019 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:20.537634  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.556176 10016 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:20.539065  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.552378  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:20.533437 16635 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:20.529735 16668 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:20.539710 16495 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.541923 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.567164 10672 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:20.568280 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:20.544374  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:20.522974 16632 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.557312 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.573035 10094 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:26:20.542470 16520 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:20.545948 16633 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:20.540690  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.535712  9949 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:20.632344  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.633658  9949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:26:20.563608 16635 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:20.564080 16661 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 8, Time since last communication: 0.133s
I20260709 12:26:20.565665 10024 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:20.557219  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.547269 16662 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:20.656206 16662 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:20.656105 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.657552 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:20.658244 10016 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:20.490617 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:20.613464 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:20.634327  9949 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:20.659783  9949 raft_consensus.cc:614] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:20.631259  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.637406  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.543134 16657 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:26:20.565842 16668 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:20.650485 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.545730 16631 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.628363 16701 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:20.565527 16645 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:20.628592 16520 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:20.669795 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 237813us
I20260709 12:26:20.629136 15869 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:20.689915 15869 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:20.690862 15869 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.679785 16702 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:20.676620 16626 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:20.628422 16429 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.051s
W20260709 12:26:20.679294 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 151619us
I20260709 12:26:20.634807 16329 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:20.703059 16329 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:20.685122  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 132873us
I20260709 12:26:20.639207 16676 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:20.720134 10021 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:20.638685 16633 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.683259 16435 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:20.730674  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71883.7us
I20260709 12:26:20.734987  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.735664  9940 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:20.701963 16645 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.707952 16699 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:20.721761 16661 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:20.676908 10094 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:20.758386 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.726266 16429 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:20.734365 16546 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.766767 16699 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.766326 16717 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.765805 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.723714 16495 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:26:20.730022 16626 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:26:20.729892 16635 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.730675 16435 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.783155 16633 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.725699 16676 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:20.767187 16705 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:20.695267 16701 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.784682 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.761102 16329 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:20.815745 16706 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:26:20.817934 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3709) took 2042 ms. Trace:
I20260709 12:26:20.818310 10095 rpcz_store.cc:276] 0709 12:26:18.775371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.781189 (+  5818us) service_pool.cc:224] Handling call
0709 12:26:18.803930 (+ 22741us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:18.804299 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:18.804358 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:18.804838 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:26:19.163966 (+359128us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:19.164008 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:19.164116 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.817113 (+1652997us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b2400223868. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a7fb0 00007f4e0911e488 00007f4e0911ebe8 00007f4e090f6398 00007f4e090f176a 00007f4e090f0bb6
0709 12:26:20.817658 (+   545us) raft_consensus.cc:1685] finished
0709 12:26:20.817686 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.817874 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":111,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6423,"spinlock_wait_cycles":4456192,"thread_start_us":358641,"threads_started":1,"wal-append.queue_time_us":556251}
I20260709 12:26:20.820173 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.821285 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.824852  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:20.824873 16533 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:20.815249 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:20.825296 16600 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:20.830214 16600 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.831912 16604 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:20.832957 16600 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.822293 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.783510 16699 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.826413  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:20.841557 16682 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.785s	user 0.003s	sys 0.000s
I20260709 12:26:20.846170 16665 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:20.784127 16717 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.857529 16665 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.859822 16717 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:20.851590 16641 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.449s	user 0.001s	sys 0.004s
I20260709 12:26:20.850145 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.863489 16631 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.863907 10090 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:20.863999 16694 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:20.856653 10097 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1156) took 1875 ms (client timeout 2198 ms). Trace:
W20260709 12:26:20.864733 10097 rpcz_store.cc:269] 0709 12:26:18.981508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.981640 (+   132us) service_pool.cc:224] Handling call
0709 12:26:20.856470 (+1874830us) spinlock_profiling.cc:243] Waited 356 ms on lock 0x7b5400340654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:20.856597 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":819234688}
I20260709 12:26:20.865995 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.867300 16676 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.778168 16546 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.868808 10095 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:20.868688  9843 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:20.870859 16692 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.599s	user 0.003s	sys 0.000s
I20260709 12:26:20.872130 16435 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.872159 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.873191 10090 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:20.875595 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.876879 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.877990 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.879895 16506 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:20.836913 16684 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.856889 16699 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:20.859894 16665 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.864130 16631 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:20.903631 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.881079 10018 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:20.859859 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.905002 10669 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:20.855782 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:20.880303 16506 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:20.910329 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.775676 16429 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.898854 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:20.929416 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.930485  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:20.930897 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.900574  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.896464 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.914911  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.915241 16701 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.810554 15869 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:20.809727 16687 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:20.919142 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2299) took 2444 ms. Trace:
I20260709 12:26:20.954499 10022 rpcz_store.cc:276] 0709 12:26:18.474575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:18.712136 (+237561us) service_pool.cc:224] Handling call
0709 12:26:20.083741 (+1371605us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.109835 (+ 26094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.918934 (+809099us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b54001f6b50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:20.919057 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3153711872}
I20260709 12:26:20.956326 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.956517 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.957927 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.919230 16506 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:20.911945 16598 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.619s	user 0.006s	sys 0.005s
I20260709 12:26:20.764662 16661 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.920766 16520 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:20.922859 16717 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.923887  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1398) took 1007 ms. Trace:
I20260709 12:26:20.970467  9945 rpcz_store.cc:276] 0709 12:26:19.916260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.916431 (+   171us) service_pool.cc:224] Handling call
0709 12:26:19.946648 (+ 30217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.033582 (+ 86934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.033618 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.033673 (+    55us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:20.033711 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.923550 (+889839us) spinlock_profiling.cc:243] Waited 86.7 ms on lock 0x7b540009c4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:20.923617 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:20.923807 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199385856}
I20260709 12:26:20.925334 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:20.952641  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 225903us
I20260709 12:26:20.870290 16546 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:20.939491 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:20.917186 16617 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.003s
I20260709 12:26:20.953665 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.896257 16447 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:20.961583 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.981016 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.967056 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.981393 10668 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:20.967218 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.983002 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:20.983206 10022 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:20.984190 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:20.892953 16604 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:20.970499  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.986495  9940 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:20.973376 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.976929 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:20.990418 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.997651 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:21.004657 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.047303 10022 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:20.926761 16672 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.105s
I20260709 12:26:20.948997 16363 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:21.021025 10673 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:20.953847 16480 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:20.953096 16184 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:21.044550  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:20.902900 16594 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:21.044898  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 134779us
I20260709 12:26:21.025081 10668 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:21.049187 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.016767 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:21.008018 16604 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:21.084085 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:21.080972 10022 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 14: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:21.080715  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:21.085340 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.021509 16546 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:21.086853  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:21.022048 16521 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.048602 16506 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:21.096870 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.048558 16665 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.098093  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:21.098116 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.090787 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:20.959141 15869 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.104146 10025 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
W20260709 12:26:21.104908 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 88241.4us
I20260709 12:26:21.106959  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.107571  9940 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:21.099560 10022 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:21.108866 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.099640 10091 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 13.
I20260709 12:26:21.103152 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:21.110562 10675 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:21.111929 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:21.113605  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.004696 16717 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:20.979465 16598 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.131920 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.080096 16184 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:21.074631 16480 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:21.087126 16604 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.018044 16617 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.022961 16447 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:21.103077  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 130882us
I20260709 12:26:21.103555 16521 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.183805 16521 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:21.171862 16447 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.172264  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.185595 16598 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.185817 10096 raft_consensus.cc:2368] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15.
I20260709 12:26:21.172886 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:21.186962 10099 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:21.174105 16645 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.175148 16661 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.183837 16717 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.191332 10017 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:21.071027 16363 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:21.192040 16363 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.115513 16716 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:21.087487 16672 consensus_queue.cc:1243] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:21.087376 16684 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:21.162138  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64109.1us
I20260709 12:26:21.211567  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:21.087193 16687 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:21.131714 16703 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:21.177304 16520 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:21.179883 16594 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.190117 16632 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:21.192137 16617 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:21.194763  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 14 committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:26:21.285190  9827 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 90475.2us
I20260709 12:26:21.284929 10019 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:26:21.130769 16665 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:21.290228 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61454.7us
W20260709 12:26:21.159372  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66725.9us
I20260709 12:26:21.291453  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:21.200711  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:21.207218 10675 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:21.296448 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:21.166429 16735 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:21.161273 16480 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:21.170256 16723 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:21.088418 16546 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.307188 16546 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.289893  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.297752  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:21.299746  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 88306.7us
I20260709 12:26:21.312141 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:21.312325  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:21.300705  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.313220  9940 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:21.312788 10675 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:21.283129 16703 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:21.320511  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 119939us
I20260709 12:26:21.320749  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:21.320864  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:21.324298  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:21.327703  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:21.306299 16661 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:21.314963 16717 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:21.340040  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.342033  9944 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:21.314626 16657 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:21.342658  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:21.344313  9991 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:21.350806  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.351665  9948 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 13.
I20260709 12:26:21.362141  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:21.363121  9949 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:21.366550  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.370376  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:21.284498 16687 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:21.305544 16690 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:26:21.328459 16726 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:21.380386 16700 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:21.381624 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3497) took 1010 ms. Trace:
I20260709 12:26:21.338150 16617 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:21.402254 10025 rpcz_store.cc:276] 0709 12:26:20.371125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.372867 (+  1742us) service_pool.cc:224] Handling call
0709 12:26:21.381412 (+1008545us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54001f7f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:21.381542 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679812480}
I20260709 12:26:21.403920 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:21.383132 16743 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:21.383497 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3503) took 1008 ms. Trace:
I20260709 12:26:21.383029 16745 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.405251 10023 rpcz_store.cc:276] 0709 12:26:20.374916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.893512 (+518596us) service_pool.cc:224] Handling call
0709 12:26:20.893849 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.925802 (+ 31953us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.925840 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.926373 (+   533us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.299545 (+373172us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:21.299606 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.299715 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.383291 (+ 83576us) raft_consensus.cc:1685] finished
0709 12:26:21.383311 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.383443 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32901,"raft.queue_time_us":32594,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":372573,"threads_started":1,"wal-append.queue_time_us":454630}
I20260709 12:26:21.415984 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.416843 10023 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:21.383642 16737 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:21.327402 16524 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:21.326325 16703 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:21.384220 16739 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:26:21.388612 16429 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:21.390375 16668 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.000s
I20260709 12:26:21.390671 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.392439 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:21.440591 10673 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:21.394035 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:21.394886  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.328531 16716 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:21.399535  9944 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:21.421494 10023 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:21.376579 16735 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:21.376058 16505 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:21.427712 10675 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
W20260709 12:26:21.421481  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 97267.9us
I20260709 12:26:21.429020 16524 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.521847 16524 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:21.521957  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:21.523488 16665 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:21.385805 16657 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.525717 16657 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.490995 10025 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:21.377664 16661 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.528168 16661 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.414403 16726 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:21.494450  9940 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:21.534283 16524 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.493481 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:21.536887  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:21.538585  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.495173 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3486) took 2942 ms. Trace:
I20260709 12:26:21.539238  9943 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:21.414779 16743 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:21.402143 16700 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:21.434217 16662 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.777s	user 0.010s	sys 0.001s
I20260709 12:26:21.491292  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.554775  9949 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
W20260709 12:26:21.496630 16732 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 12:26:21.498759  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 167064us
I20260709 12:26:21.556973  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:21.541376 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:21.539275 10020 rpcz_store.cc:276] 0709 12:26:18.552950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:19.389714 (+836764us) service_pool.cc:224] Handling call
0709 12:26:19.838887 (+449173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:19.839458 (+   571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.438127 (+1598669us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54001f8950. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:21.495083 (+ 56956us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032268160}
I20260709 12:26:21.456249 16668 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.547219 10098 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:21.547777 16617 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.548070 16716 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:21.549947 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2319) took 1348 ms. Trace:
I20260709 12:26:21.558018 10675 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:21.575398 10018 rpcz_store.cc:276] 0709 12:26:20.201384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.279486 (+ 78102us) service_pool.cc:224] Handling call
0709 12:26:20.300660 (+ 21174us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.904530 (+603870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.549670 (+645140us) spinlock_profiling.cc:243] Waited 580 ms on lock 0x7b54001f6654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:21.549879 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1334458752}
I20260709 12:26:21.554255 16700 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.544546 16743 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:21.537701 16744 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:26:21.576478 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
I20260709 12:26:21.577783  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.579176 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:21.589632 10675 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:21.591915 10676 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:21.588577 16655 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:21.593309  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:21.593736 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3498) took 1222 ms. Trace:
I20260709 12:26:21.594251 16750 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:21.594825 16750 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.595314 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:21.594130 10016 rpcz_store.cc:276] 0709 12:26:20.371457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.687371 (+315914us) service_pool.cc:224] Handling call
0709 12:26:20.719215 (+ 31844us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.960272 (+241057us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.960318 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.973474 (+ 13156us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.198963 (+225489us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:21.304218 (+105255us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.311065 (+  6847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.311156 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.593544 (+282388us) raft_consensus.cc:1685] finished
0709 12:26:21.593562 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.593682 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22152,"prepare.queue_time_us":105261,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":75525,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":7993,"spinlock_wait_cycles":23257344,"thread_start_us":363785,"threads_started":2,"wal-append.queue_time_us":320231}
I20260709 12:26:21.596944 10098 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:26:21.562992 16661 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:21.566560 15869 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.545562 10673 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:21.617481 16661 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.616048 16716 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.623570 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1412) took 1437 ms. Trace:
I20260709 12:26:21.625198 10675 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:26:21.635983 10673 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:21.642463 16749 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:21.643126 16749 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:21.644156 16749 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:21.584223 16752 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:21.576608 16617 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.649187 16617 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.650346 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:21.650957 16687 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.651037 10675 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:21.651541 16737 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:21.651892 16524 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.654958 16493 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:21.655329 16668 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:21.561426 16662 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.659232 10670 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:21.659515 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3712) took 1245 ms. Trace:
I20260709 12:26:21.659946 10092 rpcz_store.cc:276] 0709 12:26:20.413701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.413853 (+   152us) service_pool.cc:224] Handling call
0709 12:26:21.659347 (+1245494us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b5400163354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:21.659455 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":252394368}
I20260709 12:26:21.660349 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1418) took 1106 ms. Trace:
I20260709 12:26:21.660805 10670 rpcz_store.cc:276] 0709 12:26:20.554448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.554565 (+   117us) service_pool.cc:224] Handling call
0709 12:26:21.660187 (+1105622us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54005b95d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:21.660297 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2538306944}
I20260709 12:26:21.661226 16184 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:21.662269 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3496) took 1291 ms (client timeout 1701 ms). Trace:
W20260709 12:26:21.662662 10024 rpcz_store.cc:269] 0709 12:26:20.370776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.371835 (+  1059us) service_pool.cc:224] Handling call
0709 12:26:21.662090 (+1290255us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54001c5f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:21.662217 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":425478912}
I20260709 12:26:21.665040 16661 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.655350 16493 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:21.667826 16493 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:21.660360 16662 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.668794 16493 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.640201 15869 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:21.669374 15869 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:21.669776 15869 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:21.640228 10674 rpcz_store.cc:276] 0709 12:26:20.186420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.186538 (+   118us) service_pool.cc:224] Handling call
0709 12:26:20.186828 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.350667 (+163839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.350718 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:20.350782 (+    64us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:20.350841 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.617839 (+1266998us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54007396d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:21.623312 (+  5473us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.623492 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376019712}
I20260709 12:26:21.595329 16750 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.654453 16546 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.685344 16546 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.686918 10091 raft_consensus.cc:2368] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20.
W20260709 12:26:21.692698  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 135793us
I20260709 12:26:21.694504  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:21.695909  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.696966  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:21.697603  9943 raft_consensus.cc:2368] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17.
I20260709 12:26:21.698472  9942 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:21.702237  9942 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:21.694646  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:21.706789  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.710415  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:21.704975 16676 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:21.713901  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:21.678092 16604 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.677070 16700 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:21.729876  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 170190us
I20260709 12:26:21.741237 10643 leader_election.cc:400] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17.
I20260709 12:26:21.699944 16480 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.741689 10643 leader_election.cc:403] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
W20260709 12:26:21.742446  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 113550us
I20260709 12:26:21.745215 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:26:21.745903 10024 raft_consensus.cc:606] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:21.746223 10024 raft_consensus.cc:614] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:21.755811  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.719120 16714 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 12:26:21.758575 16716 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.753258  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:21.764386  9944 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:21.753839 16668 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:21.768833  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.777304 16765 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:21.777980 16765 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:21.778942 16708 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.123s	user 0.004s	sys 0.000s
I20260709 12:26:21.780983  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.782055 16764 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:21.782958 16563 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:21.786231 16563 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:21.787850 16184 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:21.791208 16480 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:21.798079 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.804692 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1424) took 1140 ms. Trace:
W20260709 12:26:21.812476  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59388.8us
I20260709 12:26:21.814908 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:21.814963  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:21.815814  9943 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:26:21.799286 16113 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:21.816962 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:21.800339  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.754130 16661 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:26:21.801051  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:21.800386  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:21.801685 16563 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.802032  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:21.802492 16363 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.005s
I20260709 12:26:21.782071 16757 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:21.804198 16745 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:21.804133 16429 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:21.831453  9944 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:26:21.831956 16757 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.836109 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1204) took 1634 ms. Trace:
I20260709 12:26:21.805534 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3727) took 1004 ms. Trace:
I20260709 12:26:21.808207 10097 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:21.837105 16363 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.839001 16467 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:21.839582 16363 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.840324 10671 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:21.841995 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1423) took 1182 ms. Trace:
I20260709 12:26:21.841733 16750 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:21.765901 16716 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.807443 15869 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:21.809073 10668 rpcz_store.cc:276] 0709 12:26:20.664691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.018105 (+353414us) service_pool.cc:224] Handling call
0709 12:26:21.018488 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.080436 (+ 61948us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.804457 (+724021us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b5400501ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:21.804636 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14726912}
W20260709 12:26:21.823441 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.836984 10094 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:21.828032 16113 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.874612 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:21.883260 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:21.849342  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:21.868974 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.869987 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1425) took 1107 ms. Trace:
I20260709 12:26:21.782950 16764 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:26:21.870111 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.836542 10677 rpcz_store.cc:276] 0709 12:26:20.201222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.201711 (+   489us) service_pool.cc:224] Handling call
0709 12:26:20.201958 (+   247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:20.202277 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:20.202305 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:20.226677 (+ 24372us) log.cc:844] Serialized 26 byte log entry
0709 12:26:20.333730 (+107053us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:20.333769 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:20.333832 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:20.357173 (+ 23341us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b480002ea58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:26:21.833912 (+1476739us) spinlock_profiling.cc:243] Waited 2.72 ms on lock 0x7b180043df80. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:21.835884 (+  1972us) raft_consensus.cc:1685] finished
0709 12:26:21.835907 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.836047 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":66138,"raft.queue_time_us":459,"raft.run_cpu_time_us":1954,"raft.run_wall_time_us":149539,"spinlock_wait_cycles":33628800,"thread_start_us":73713,"threads_started":1,"wal-append.queue_time_us":175619}
I20260709 12:26:21.818639 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:21.872427 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.836644 10095 rpcz_store.cc:276] 0709 12:26:20.801630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.822044 (+ 20414us) service_pool.cc:224] Handling call
0709 12:26:21.805377 (+983333us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:21.805481 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76109056}
I20260709 12:26:21.870852 10018 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:21.847940 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:21.869783 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:21.895444 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.842378 10671 rpcz_store.cc:276] 0709 12:26:20.659473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.018679 (+359206us) service_pool.cc:224] Handling call
0709 12:26:21.019026 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.840729 (+821703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.841775 (+  1046us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b54007b0cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:21.841934 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1888336768}
I20260709 12:26:21.866573 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:21.917428 10091 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:21.917608  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:21.872370 10025 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:21.919152 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:21.919623 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.920212 10023 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:21.856869 16565 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:21.881098  9990 leader_election.cc:400] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20.
I20260709 12:26:21.922672  9990 leader_election.cc:403] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:26:21.922700 10024 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:21.877961 10094 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:21.885202 10676 rpcz_store.cc:276] 0709 12:26:20.762903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.991623 (+228720us) service_pool.cc:224] Handling call
0709 12:26:20.992015 (+   392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.592373 (+600358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.869751 (+277378us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b5400545bd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:21.869935 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1379121536}
W20260709 12:26:21.885756  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 153673us
I20260709 12:26:21.834291 16757 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.888830  9943 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:21.920162 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:21.879719 16726 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.900339 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:21.802300 16743 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:21.936067  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:21.938980 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:21.936408 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:21.844017 16563 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.863936 16668 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17.
I20260709 12:26:21.925343 10094 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.940970 10094 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:21.844712 16760 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:21.938210 16184 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:21.950824 16760 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.951311 16760 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:21.951325 16184 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:21.898497 16546 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:21.843971 16750 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:21.937657 16772 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:21.955264 10018 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:26:21.957058 10094 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 12:26:21.957470 10094 pending_rounds.cc:85] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 14
I20260709 12:26:21.957691 10094 pending_rounds.cc:107] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:26:21.957923 10094 raft_consensus.cc:2889] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:21.958755 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1174) took 1104 ms. Trace:
I20260709 12:26:21.959085 10094 rpcz_store.cc:276] 0709 12:26:20.854047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.179898 (+325851us) service_pool.cc:224] Handling call
0709 12:26:21.180183 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.836881 (+656698us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b54001db8a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:26:21.958472 (+121591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.958703 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63501056}
W20260709 12:26:21.960757 16435 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:26:21.918111 16691 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:21.946165 16113 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.804198 16703 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:21.932212 16752 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:21.949790 16716 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.954804 16546 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:21.932454 16531 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:21.969286 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1419) took 1410 ms. Trace:
I20260709 12:26:21.971127 16187 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:21.973905 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:21.976480 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:22.056077 10677 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:21.980589 16748 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.592s	user 0.002s	sys 0.000s
I20260709 12:26:21.993297 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:21.996512 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3505) took 1621 ms. Trace:
W20260709 12:26:22.011714 10096 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3717) took 1591 ms (client timeout 1701 ms). Trace:
W20260709 12:26:22.058688 10096 rpcz_store.cc:269] 0709 12:26:20.420084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.420186 (+   102us) service_pool.cc:224] Handling call
0709 12:26:22.011446 (+1591260us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400094054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:22.011629 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752528128}
I20260709 12:26:22.060420 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:22.021904 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2532) took 1479 ms. Trace:
I20260709 12:26:22.061829 10093 rpcz_store.cc:276] 0709 12:26:20.542835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.667230 (+124395us) service_pool.cc:224] Handling call
0709 12:26:21.394891 (+727661us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.395077 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.395099 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:21.395138 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:21.395165 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.395214 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.021644 (+626430us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b540002dd50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:22.021817 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1672421120}
I20260709 12:26:22.063541 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:22.064913 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:22.025878 16700 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:21.940995  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:22.051378 16703 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:22.066608  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 125657us
I20260709 12:26:22.066947 10091 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
W20260709 12:26:22.067384 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74237us
I20260709 12:26:22.049592 16770 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:22.052611 10099 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:22.067862 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:22.053830 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2540) took 1364 ms. Trace:
I20260709 12:26:22.055649 10672 rpcz_store.cc:276] 0709 12:26:20.559389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.567989 (+  8600us) service_pool.cc:224] Handling call
0709 12:26:21.969027 (+1401038us) spinlock_profiling.cc:243] Waited 593 ms on lock 0x7b5400528fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:21.969206 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1363025792}
I20260709 12:26:22.068097 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:22.050110 16759 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.114s
I20260709 12:26:22.068933 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:22.068528 10097 rpcz_store.cc:276] 0709 12:26:20.689074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.872659 (+183585us) service_pool.cc:224] Handling call
0709 12:26:21.807895 (+935236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.837117 (+ 29222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.053529 (+216412us) spinlock_profiling.cc:243] Waited 935 ms on lock 0x7b5400162e50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:22.053735 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2150013184}
I20260709 12:26:22.058526 10022 rpcz_store.cc:276] 0709 12:26:20.375413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.165147 (+789734us) service_pool.cc:224] Handling call
0709 12:26:21.165525 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.431046 (+265521us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.431077 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.434559 (+  3482us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.674004 (+239445us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:21.674061 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.995931 (+321870us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b54001c6954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:21.996004 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.996079 (+    75us) raft_consensus.cc:1685] finished
0709 12:26:21.996101 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:21.996428 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610134784,"thread_start_us":154926,"threads_started":1,"wal-append.queue_time_us":258642}
I20260709 12:26:22.069697 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:22.070926 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:22.072396 10093 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 17: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:21.963037 16676 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:21.961259 16762 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.116s
I20260709 12:26:22.107692 10091 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:22.108378 10096 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:22.108580 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:22.110042 16770 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.076990 16531 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:22.110504 16676 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:22.133944 10677 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:22.045735 15869 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:22.171365 16770 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.171651 16467 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.076197 16668 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:22.055649 16546 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:22.179347  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 63095us
I20260709 12:26:22.230978  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:22.114758 16662 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:22.067770 16629 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.104940 16750 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:22.076632 16363 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:22.109499 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:22.248278 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:22.249305 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:22.104908 16714 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.185420 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:22.178340 16703 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.189379 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:22.050899 16752 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:22.240406 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:22.240509 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:22.113391 16565 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.140s
I20260709 12:26:22.243243 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:22.114620 16524 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.132s
W20260709 12:26:22.255698 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66427.1us
I20260709 12:26:22.255707 10093 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:22.257447  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:22.241425 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:22.254478 10091 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:22.172066 16739 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:26:22.323724 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:22.112382 16429 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 12:26:22.326973  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2415) took 1281 ms. Trace:
I20260709 12:26:22.249876 16657 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:22.336666 10091 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:22.329737  9945 rpcz_store.cc:276] 0709 12:26:21.045941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.046081 (+   140us) service_pool.cc:224] Handling call
0709 12:26:21.084115 (+ 38034us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.084475 (+   360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.084514 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.093552 (+  9038us) log.cc:844] Serialized 26 byte log entry
0709 12:26:21.325175 (+231623us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:21.325222 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:21.325295 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:21.979846 (+654551us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:22.323321 (+343475us) raft_consensus.cc:1685] finished
0709 12:26:22.323344 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.326892 (+  3548us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":77272,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":25964,"spinlock_wait_cycles":27488000,"thread_start_us":118384,"threads_started":1,"wal-append.queue_time_us":256093}
W20260709 12:26:22.332486  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 75127.9us
I20260709 12:26:22.341667 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:22.340720 16113 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:22.344359 16690 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:22.344426 16113 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:22.346665 16617 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:22.346973 16626 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:22.344988 16690 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:22.347396 16626 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.347613 16690 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.347783 16626 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:22.349241 10672 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:22.351157  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:22.352025 16699 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:22.353587 16735 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:22.354029 16735 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:22.353818 16699 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:22.359964 16699 consensus_queue.cc:1243] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 20.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:26:22.360524  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:22.360680 16699 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:22.354470 16735 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:22.362382 16113 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:22.364179  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:22.357781  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:22.358866  9941 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:22.358916  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:22.361616 16699 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:22.375612  9946 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:22.359325 10096 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:22.388124 10091 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:22.309759 16691 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:22.403704  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:22.247635 15869 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.133s
I20260709 12:26:22.247114 16629 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:22.406260 16749 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.050s	sys 0.091s
I20260709 12:26:22.413286 16694 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:26:22.414000 16749 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.563s	user 0.050s	sys 0.092s
I20260709 12:26:22.415167 10022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8453bea911c4756add721edebd5d124"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:26:22.415900 10022 raft_consensus.cc:606] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:22.416251 10022 raft_consensus.cc:614] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:22.415076 16716 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:22.420949  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:22.255820 16714 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:22.189139 16467 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:22.298053 16703 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:22.373077 16699 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:22.430830 16687 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:22.448537 16759 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:22.558614  9948 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.008s	sys 0.085s
W20260709 12:26:22.582878  9948 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.838s	user 0.009s	sys 0.086s
I20260709 12:26:22.584571  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2609) took 1226 ms. Trace:
I20260709 12:26:22.182906 16632 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.366s
I20260709 12:26:22.185125 16745 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.360s
W20260709 12:26:22.591711 10020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.019s	sys 0.022s
W20260709 12:26:22.615535 10020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.019s	sys 0.022s
I20260709 12:26:22.585139  9948 rpcz_store.cc:276] 0709 12:26:21.358165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.358350 (+   185us) service_pool.cc:224] Handling call
0709 12:26:21.358763 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.359002 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:21.573504 (+214502us) log.cc:844] Serialized 14 byte log entry
0709 12:26:22.584064 (+1010560us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:22.584147 (+    83us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:22.584206 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.584293 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:22.584490 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13512,"prepare.queue_time_us":315200,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":698,"raft.queue_time_us":87911,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":9759,"thread_start_us":1202540,"threads_started":3,"wal-append.queue_time_us":223303}
W20260709 12:26:22.561964 16495 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.019s	sys 0.041s
I20260709 12:26:22.620015 10669 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:22.558670 16505 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:22.561967 16648 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:22.583570 16700 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:22.646869  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:22.625998 16495 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.832s	user 0.020s	sys 0.041s
I20260709 12:26:22.639791 16467 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:22.642367 10091 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:22.643108 16743 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:22.643185 16714 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:22.644487 16629 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:22.587090 16759 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:22.661638 10676 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.687s	user 0.021s	sys 0.004s
W20260709 12:26:22.662319 10676 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.021s	sys 0.004s
I20260709 12:26:22.649714  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:22.650383 16716 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:22.670142  9990 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:22.637007 16707 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.894s	user 0.013s	sys 0.006s
I20260709 12:26:22.590454 16687 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:22.680481 16694 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:22.647279 16505 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:22.643234 16765 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:22.653875 10671 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:26:22.658533 16685 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
W20260709 12:26:22.668463 10095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.009s	sys 0.016s
I20260709 12:26:22.675863  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:22.679003 16774 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:22.649050 16648 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:22.651427 16700 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:22.657166 16752 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:22.667361 16746 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:22.696779 10095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.010s	sys 0.016s
W20260709 12:26:22.664090 10023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.009s	sys 0.020s
W20260709 12:26:22.700203 10023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.009s	sys 0.020s
W20260709 12:26:22.700415 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.006s	sys 0.014s
W20260709 12:26:22.701712 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.006s	sys 0.014s
W20260709 12:26:22.703905 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 595415us
I20260709 12:26:22.704854 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:22.697487 16716 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:22.681785 16759 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:22.701236 16737 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.003s	sys 0.011s
I20260709 12:26:22.693903 16735 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:22.691879 16690 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:22.699366 16617 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:26:22.699414 16746 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:22.722544 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.009s	sys 0.013s
W20260709 12:26:23.129508 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.010s	sys 0.013s
W20260709 12:26:23.090217  9914 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.106s	user 0.007s	sys 0.008s
W20260709 12:26:23.131278  9914 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.147s	user 0.008s	sys 0.008s
W20260709 12:26:23.131829 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 426894us
I20260709 12:26:23.107674 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1204) took 1228 ms. Trace:
I20260709 12:26:23.133517 10023 rpcz_store.cc:276] 0709 12:26:21.879064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.920956 (+ 41892us) service_pool.cc:224] Handling call
0709 12:26:21.921272 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.937418 (+ 16146us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.937461 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.938723 (+  1262us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.700993 (+762270us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:22.701038 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.701102 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.106906 (+405804us) raft_consensus.cc:1685] finished
0709 12:26:23.106967 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.107389 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15587,"raft.queue_time_us":15752,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":4581,"thread_start_us":761958,"threads_started":1,"wal-append.queue_time_us":1000661}
W20260709 12:26:23.115288 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.174s	user 0.013s	sys 0.018s
W20260709 12:26:23.137883 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.197s	user 0.013s	sys 0.018s
W20260709 12:26:23.132007  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.1482e+06us
W20260709 12:26:23.140756 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 1.20432e+06us
W20260709 12:26:23.116283 10094 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.155s	user 0.002s	sys 0.017s
W20260709 12:26:23.143316 10094 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.002s	sys 0.017s
I20260709 12:26:23.143714 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:23.145233 10091 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:23.146013 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3737) took 1326 ms. Trace:
I20260709 12:26:23.118221 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3734) took 1303 ms. Trace:
W20260709 12:26:23.273670 10676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.008s	sys 0.034s
I20260709 12:26:22.735078 16759 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:22.735355 16772 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.780s	user 0.012s	sys 0.000s
I20260709 12:26:23.445959 16772 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20.
W20260709 12:26:23.011986 16626 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.664s	user 0.013s	sys 0.001s
I20260709 12:26:23.448135 16626 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:23.002290 16563 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.058s	user 0.009s	sys 0.002s
I20260709 12:26:22.991271 16690 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:22.822952 16735 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:22.992058 16746 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:23.011626 16760 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.060s	user 0.011s	sys 0.002s
W20260709 12:26:23.029434 16770 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.840s	user 0.011s	sys 0.002s
I20260709 12:26:23.055994 16774 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:23.018416 16546 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.766s	user 0.012s	sys 0.000s
I20260709 12:26:22.725453 16716 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:23.318851 16495 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.016s	sys 0.052s
W20260709 12:26:23.472602 16495 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.017s	sys 0.052s
I20260709 12:26:23.146507 10094 rpcz_store.cc:276] 0709 12:26:21.819649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.960056 (+140407us) service_pool.cc:224] Handling call
0709 12:26:21.960402 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.960744 (+   342us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.960773 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.961267 (+   494us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.144082 (+1182815us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:23.144894 (+   812us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.145321 (+   427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.145532 (+   211us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.145608 (+    76us) raft_consensus.cc:1685] finished
0709 12:26:23.145619 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.145925 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":485,"raft.run_cpu_time_us":310,"raft.run_wall_time_us":81756,"spinlock_wait_cycles":1948416,"thread_start_us":1182100,"threads_started":1,"wal-append.queue_time_us":787488}
I20260709 12:26:22.757963 16604 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:23.130142 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 947451us
I20260709 12:26:22.723219 16703 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:22.850980 16505 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:22.857297 16648 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:23.116017 10090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.133s	user 0.001s	sys 0.009s
W20260709 12:26:23.479954 10090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.497s	user 0.002s	sys 0.010s
I20260709 12:26:22.850867 16687 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:22.861290 16700 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:22.723985 16737 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.003s	sys 0.011s
I20260709 12:26:23.147190 10095 rpcz_store.cc:276] 0709 12:26:21.814571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.935496 (+120925us) service_pool.cc:224] Handling call
0709 12:26:21.938144 (+  2648us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.938505 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.938539 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.941811 (+  3272us) log.cc:844] Serialized 26 byte log entry
0709 12:26:22.703073 (+761262us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:22.703121 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:22.703206 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:23.117434 (+414228us) raft_consensus.cc:1685] finished
0709 12:26:23.117488 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.117931 (+   443us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":21637,"thread_start_us":753183,"threads_started":1,"wal-append.queue_time_us":903305}
I20260709 12:26:23.356392  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:26:23.274257 10672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.003s	sys 0.015s
W20260709 12:26:23.483505 10672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.003s	sys 0.025s
I20260709 12:26:23.133937 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:23.486246 16746 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:23.486606 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.691s (SENT) [suppressed 10 similar messages]
W20260709 12:26:23.486567 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 353014us
I20260709 12:26:23.142464 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:23.115502 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.015s	sys 0.040s
I20260709 12:26:23.419754 16776 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:23.423657 16764 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.424656 16676 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.425480 16752 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.497210 16752 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.118341 10097 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:23.463418 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:23.464578 16546 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.465016 16602 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.465986 16617 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.465898 16770 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.467092 16760 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.492496 16776 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:23.494344 16764 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.495833 16676 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.523149 16700 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.523305 16776 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.524511 16690 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:23.525243 16690 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:23.526754 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 384648us
I20260709 12:26:23.505873 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1170) took 2662 ms. Trace:
I20260709 12:26:23.530470 16764 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.531246 16546 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.531405 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.530388 10090 rpcz_store.cc:276] 0709 12:26:20.828488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.873799 (+ 45311us) service_pool.cc:224] Handling call
0709 12:26:20.874085 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:20.874295 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:20.881501 (+  7206us) log.cc:844] Serialized 14 byte log entry
0709 12:26:21.983076 (+1101575us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b18002e6120. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:26:23.487868 (+1504792us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.487972 (+   104us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:23.488101 (+   129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.488220 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.490760 (+  2540us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6980,"prepare.queue_time_us":6404,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":15391,"raft.queue_time_us":7783,"spinlock_wait_cycles":41121024,"thread_start_us":1497214,"threads_started":1,"wal-append.queue_time_us":2102689}
I20260709 12:26:23.532356 10098 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:23.527731 16467 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:23.532827 16467 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.533303 16467 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:23.533236 16602 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.534377 16676 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.524221 16700 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.535105 16787 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.536160 16602 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.535799 16787 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.536623 16700 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.540439 16788 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.541281 16788 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.544438  9949 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:26:23.545596 16668 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.547483  9941 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:23.486268 16626 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.548614 16770 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.550451 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.550908 16760 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.529263 16687 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.552435 16687 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.551515 10090 raft_consensus.cc:2368] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15.
I20260709 12:26:23.554394 10095 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:23.554999 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.528959 16617 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:23.513545 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.020s	sys 0.041s
W20260709 12:26:23.490110 10676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.011s	sys 0.035s
I20260709 12:26:23.527875 16690 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.567036 16717 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:23.567562 16717 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:23.568450 16798 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.568873 16717 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.569270 16798 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:23.572861 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 1.22665e+06us
I20260709 12:26:23.573540 16531 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.574100 16798 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.575584 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:23.557281 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.577656 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.579308 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:23.546396 16668 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.583254 16687 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.583793 16668 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.560511 16617 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.503144 16752 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.585897 16787 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.561347 16565 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:23.563081 16467 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.470031 16716 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.586607 16565 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:23.587113 16565 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:23.593025 16467 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:23.594699 16795 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.618s	user 0.012s	sys 0.014s
I20260709 12:26:23.594897 16661 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:23.595361 16661 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:23.596658 16661 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.598048 16661 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:23.598680  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1430) took 1746 ms. Trace:
I20260709 12:26:23.599035 16429 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:23.599476 16429 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:23.599133  9949 rpcz_store.cc:276] 0709 12:26:21.851879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.932688 (+ 80809us) service_pool.cc:224] Handling call
0709 12:26:21.942098 (+  9410us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.545298 (+1603200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.598290 (+ 52992us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b540042e854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:23.598552 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3683940736}
I20260709 12:26:23.600979 16429 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.517802 10091 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:23.603061 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1177) took 2748 ms. Trace:
I20260709 12:26:23.603475 10099 rpcz_store.cc:276] 0709 12:26:20.854848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.188620 (+333772us) service_pool.cc:224] Handling call
0709 12:26:21.188925 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:22.068204 (+879279us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.602441 (+1534237us) spinlock_profiling.cc:243] Waited 863 ms on lock 0x7b54002fdd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:23.602882 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1985663360}
I20260709 12:26:23.604423 10091 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 12:26:23.604907  9942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2614) took 1849 ms (client timeout 1816 ms). Trace:
W20260709 12:26:23.605341  9942 rpcz_store.cc:269] 0709 12:26:21.755385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.755544 (+   159us) service_pool.cc:224] Handling call
0709 12:26:23.604484 (+1848940us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400016dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:23.604659 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3079305088}
W20260709 12:26:23.606813  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.607074 10093 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:23.607618 16429 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:23.612494 16363 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.359s	user 0.005s	sys 0.018s
I20260709 12:26:23.608901 16685 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.609066 16714 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:23.544301 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:23.614851 16788 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:23.536784 16602 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:23.615898 16752 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:23.619371 15869 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:23.620934  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:23.608587 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2563) took 1615 ms. Trace:
I20260709 12:26:23.623234 16329 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.623942 16673 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:23.607568 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3736) took 1788 ms (client timeout 2239 ms). Trace:
W20260709 12:26:23.668079 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 181670us
I20260709 12:26:23.570358 16717 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:23.484582  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:23.625540  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:23.667371 10093 rpcz_store.cc:276] 0709 12:26:21.992829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.322968 (+330139us) service_pool.cc:224] Handling call
0709 12:26:23.601742 (+1278774us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:26:23.606753 (+  5011us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.607488 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.608204 (+   716us) spinlock_profiling.cc:243] Waited 4.68 ms on lock 0x7b54002fce50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:23.608515 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13773056}
I20260709 12:26:23.628057 16739 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:23.627562  9843 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 6 to 14, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 14 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:23.631052 16690 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:23.631528 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56130.7us
I20260709 12:26:23.634696 16710 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:23.635113 10096 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:23.639112 16699 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.640096 16776 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:23.642549 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2566) took 1001 ms. Trace:
I20260709 12:26:23.642735 16531 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:23.643760  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:23.644197 16749 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:26:23.645856  9987 connection.cc:256] RPC call timeout handler was delayed by -1.6550049590000526s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
I20260709 12:26:23.649085 16694 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:23.651161  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2625) took 1266 ms. Trace:
I20260709 12:26:23.651811 16662 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:23.751497  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 108067us
I20260709 12:26:23.751801 16662 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.752868  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:23.752753 16565 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.753847 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:23.754873 16788 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:23.755091 16772 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.755620 16531 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.755998 16772 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.756737 16565 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:23.758159 16531 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.652808 16633 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:23.656136 10090 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:23.651247 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1220) took 1678 ms. Trace:
W20260709 12:26:23.687491 10098 rpcz_store.cc:269] 0709 12:26:21.819283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.827198 (+  7915us) service_pool.cc:224] Handling call
0709 12:26:23.607277 (+1780079us) spinlock_profiling.cc:243] Waited 1.61 s on lock 0x7b54003aded4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:23.607464 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3702390144}
I20260709 12:26:23.761616 16113 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:23.609216 16657 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.746768 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.747516 16329 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.748685 16673 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.742724 16749 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:23.750850  9987 connection.cc:256] RPC call timeout handler was delayed by -1.08795086300006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18130000000000002s for a response
I20260709 12:26:23.742957 16694 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:23.758745 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.759199 16801 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:23.759362 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.759682 16685 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.759882 16524 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:23.669844 16787 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:26:23.760972 16728 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.669691 15869 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:23.755299  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.755203 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 91694.3us
I20260709 12:26:23.743122  9941 rpcz_store.cc:276] 0709 12:26:22.384472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.384599 (+   127us) service_pool.cc:224] Handling call
0709 12:26:22.416598 (+ 31999us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:23.547942 (+1131344us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.650761 (+102819us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001a40d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:23.651083 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2599783168}
I20260709 12:26:23.744815 16602 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.738647 10097 rpcz_store.cc:276] 0709 12:26:22.640883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:22.641430 (+   547us) service_pool.cc:224] Handling call
0709 12:26:22.641730 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:23.515833 (+874103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.642249 (+126416us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b54003ae3d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:23.642469 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094463744}
I20260709 12:26:23.726361 16739 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:23.758375 16772 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.737363 16363 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:23.780550  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:23.781504 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 237492us
I20260709 12:26:23.754936 16759 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:23.782829 16687 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:23.784472 16765 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:23.737402 16668 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:26:23.754936 16629 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:23.766952 16690 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:23.793023 16657 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.794375 10091 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:26:23.794931 16184 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:23.777607  9843 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 10 to 13, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 13 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:23.798761 10093 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:26:23.775563 16752 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.808526 16801 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:23.764910 10676 rpcz_store.cc:276] 0709 12:26:21.972684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.973645 (+   961us) service_pool.cc:224] Handling call
0709 12:26:21.973966 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:21.974148 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:22.668237 (+694089us) log.cc:844] Serialized 14 byte log entry
0709 12:26:23.650677 (+982440us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:23.650766 (+    89us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:23.650917 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.651030 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:23.651171 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44089,"prepare.queue_time_us":968250,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":48,"raft.queue_time_us":44257,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":198,"spinlock_wait_cycles":85632,"thread_start_us":1582113,"threads_started":2,"wal-append.queue_time_us":893170}
W20260709 12:26:23.846860 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.850219 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:23.769767 16714 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.833431 16685 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.852312 16493 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:23.853252 16493 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.856005 16762 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:23.858798  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78506.7us
I20260709 12:26:23.858732 16184 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.859747 16329 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.861438 16184 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.861963 16772 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:23.863575 16772 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:23.864086 16772 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:23.867492 16690 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.869282 16493 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.847270 16429 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.884454 16429 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.885340 16673 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.870748 16728 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.872963 16662 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.804167 16710 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.875638 16661 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:23.859220 16765 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:23.805892 16113 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.877664 16801 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.878032 16749 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.879668 16694 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.881778  9843 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 10 to 14, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:23.883277 15869 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.894017 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.875933 16602 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.904644 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.906220 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:23.907147 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 58886.4us
I20260709 12:26:23.865576 16772 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.897149 16759 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:23.902982 16661 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:23.904771 16765 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:23.925846 10091 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:23.906741 16493 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:23.936932 16800 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.937916  9948 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:23.937700 16800 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.944120  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:23.944327  9838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 323) took 31406 ms (client timeout 15000 ms). Trace:
I20260709 12:26:23.945072 16714 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.945676 16800 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:23.947958  9945 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:23.947234  9843 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 13 to 15, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 15 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:23.944797  9838 rpcz_store.cc:269] 0709 12:25:52.537758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:52.538038 (+   280us) service_pool.cc:224] Handling call
0709 12:26:23.944031 (+31405993us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:14.928908 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:14.929862 (+   954us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:14.929931 (+    69us) write_op.cc:435] Acquired schema lock
0709 12:26:14.929957 (+    26us) tablet.cc:662] Decoding operations
0709 12:26:14.932428 (+  2471us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:14.932529 (+   101us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:14.932555 (+    26us) tablet.cc:685] Acquiring locks for 25 operations
0709 12:26:15.124645 (+192090us) tablet.cc:701] Row locks acquired
0709 12:26:15.124711 (+    66us) write_op.cc:260] PREPARE: finished
0709 12:26:15.124933 (+   222us) write_op.cc:270] Start()
0709 12:26:15.125113 (+   180us) write_op.cc:276] Timestamp: P: 1783599975124851 usec, L: 0
0709 12:26:15.125162 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 12:26:15.126113 (+   951us) log.cc:844] Serialized 9829 byte log entry
0709 12:26:15.791979 (+665866us) op_driver.cc:464] REPLICATION: finished
0709 12:26:15.962302 (+170323us) write_op.cc:301] APPLY: starting
0709 12:26:15.962403 (+   101us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:16.205817 (+243414us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:16.206022 (+   205us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:16.982931 (+776909us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:16.983358 (+   427us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=50,key_file_lookups=50,delta_file_lookups=50,mrs_lookups=0
0709 12:26:16.983419 (+    61us) write_op.cc:312] APPLY: finished
0709 12:26:17.134745 (+151326us) log.cc:844] Serialized 220 byte log entry
0709 12:26:17.300699 (+165954us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:17.335022 (+ 34323us) write_op.cc:454] Released schema lock
0709 12:26:17.338809 (+  3787us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:17.339389 (+   580us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:17.339429 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7262085,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":154297,"apply.run_cpu_time_us":68041,"apply.run_wall_time_us":1377031,"cfile_cache_hit":102,"cfile_cache_hit_bytes":108677,"num_ops":25,"prepare.queue_time_us":313149,"prepare.run_cpu_time_us":63548,"prepare.run_wall_time_us":527808,"raft.queue_time_us":128388,"raft.run_cpu_time_us":63380,"raft.run_wall_time_us":139831,"replication_time_us":660696,"thread_start_us":641681,"threads_started":5,"wal-append.queue_time_us":542782,"wal-append.run_cpu_time_us":21013,"wal-append.run_wall_time_us":2198989}]]}
I20260709 12:26:23.945078 16685 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:23.951572 10099 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:23.920805 16673 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.931833 16765 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.958858 10020 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:23.960616  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:23.961450  9944 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:23.933606 16493 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:23.955932 10023 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:23.956682 16800 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:23.962807 16493 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:23.963310 16800 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.964483 16694 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:23.966221 16800 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.967291 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.967418 16673 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:23.970052 16673 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.971920 16710 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:23.976017 16752 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.977646  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:23.978668  9946 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:23.979847  9948 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:23.979965 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:23.980707 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:23.982303  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:23.982568 16657 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.983392  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:23.983773 10677 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:26:23.987001  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:23.940045 16776 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:23.988369 16776 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:23.989951  9948 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
W20260709 12:26:23.990912  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 155011us
I20260709 12:26:23.992738  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:23.994894  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:23.992765  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:23.995996  9944 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:23.989693 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:23.956885 15869 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:23.949748  9843 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 5 to 11, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 11 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:23.997622  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.001871 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61525.3us
I20260709 12:26:24.005486  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.006634  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.007380  9946 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:24.008574 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.008407  9843 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 7 to 11, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 11 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:24.006518  9945 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:23.958854 16429 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:23.996778 16685 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.028580 16685 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:24.002804 16759 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.023291  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1428) took 2203 ms. Trace:
I20260709 12:26:24.024236 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.025265 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.025179 16661 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:24.026695 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 180558us
I20260709 12:26:23.956718 16739 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.027837 16749 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:24.034385 10022 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:26:24.010609 16752 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.031833  9943 rpcz_store.cc:276] 0709 12:26:21.820087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:21.929415 (+109328us) service_pool.cc:224] Handling call
0709 12:26:21.929875 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:21.933259 (+  3384us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:21.933305 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:21.933903 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 12:26:23.575707 (+1641804us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b1800497620. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:26:23.850362 (+274655us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:23.850414 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:23.852310 (+  1896us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:24.022863 (+170553us) raft_consensus.cc:1685] finished
0709 12:26:24.022883 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:24.023102 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2722,"raft.queue_time_us":2844,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":5386,"spinlock_wait_cycles":3135360,"thread_start_us":232218,"threads_started":1,"wal-append.queue_time_us":2075389}
I20260709 12:26:24.060299 16760 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.060927 16760 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.063524 16760 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:24.067247 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.072364 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.074654  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:24.078917 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:24.079422 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:24.079936 10673 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:24.011693 16710 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:24.068625 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:24.040419 16429 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:24.062268 16493 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.087538 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:24.111126  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.105973  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.113973  9949 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:24.115190 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.115916 10671 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:24.117261 10671 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:24.117372 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.118029 10676 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:24.118821  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.118937 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.119499 10674 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:24.119712 10675 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:24.118284 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:24.123122 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.124756 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:24.125519 10668 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.127189  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.128095  9940 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:26:24.105635  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:24.129030  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.131243  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.131990  9947 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:24.133867 16799 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:24.134344 16717 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:24.134872 16799 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:24.135375 16799 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:24.135854  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:24.137060  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:24.106902 10669 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:24.138723 16687 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:24.139107 16687 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:24.139461 16687 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:24.140306 16728 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.144498 16759 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.145807 16661 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:24.149407  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 43894.7us
I20260709 12:26:24.150457  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:24.122772 16710 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.153069 16809 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:24.153278 16803 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 12:26:24.103111 10093 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:26:24.130203  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.157953 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.153640 16759 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:24.163548  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.158869  9940 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:24.165556  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:24.173296 16759 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:24.174455 16759 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.176331 16798 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:24.176435  9946 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:26:24.194212 10017 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:24.152040 16739 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.195531 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.228981 10674 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:24.136467 16799 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.230149 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.202445 15869 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:26:24.203946  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.276022 10017 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:24.193657 16776 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.211040 16772 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.156512 16429 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.219590 10025 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.12-21.12]
W20260709 12:26:24.302248 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 273388us
I20260709 12:26:24.219663  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:24.201984  9914 leader_election.cc:400] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15.
I20260709 12:26:24.304164  9914 leader_election.cc:403] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:24.238453 16690 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.321590 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.322364  9944 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:24.230461 16799 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.226018 16685 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:24.231427 16749 consensus_queue.cc:1175] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" responder_term: 16 status { last_received { term: 16 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 16 index: 10 } }
I20260709 12:26:24.233402 16687 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.177265 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.151666 16752 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.246296 10021 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 12:26:24.262058  9943 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:24.332624 10095 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:24.248507 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:24.332396 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 133764us
I20260709 12:26:24.237804 16710 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.251178 16113 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.231519 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:24.177461 16657 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.281553 10668 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:24.301708 10019 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:24.220881 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.178540 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:24.180124 16493 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:24.329254 16776 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 14. Current term is 15.
I20260709 12:26:24.238533 16765 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.262639  9949 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:24.333323 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.353802 10017 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:24.322855 16184 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:24.359445 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.360646 10019 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:24.332469 10021 pending_rounds.cc:85] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 9
W20260709 12:26:24.342288 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.344699 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.363184 16363 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:24.365379 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:24.366966 10025 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:24.368191 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.368311 16687 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:24.368983 16662 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:24.369470 16662 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:24.369849 16662 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.370500 16685 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:24.371832  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 170107us
I20260709 12:26:24.373031  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:24.373934  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:24.374454  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.375283 15869 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:24.375834 15869 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:24.376807 15869 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.380192 16739 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:24.380795 16739 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:24.381733 16739 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.383221 16662 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.385644 16662 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:24.389786 10024 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:26:24.393103  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 222937us
I20260709 12:26:24.362031 10021 pending_rounds.cc:107] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 12:26:24.399505 10021 raft_consensus.cc:2889] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:24.341454 16699 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.354702 16113 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.409718 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.357540 16710 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:24.412668 10019 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:24.413336 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:24.415704  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 496408us
I20260709 12:26:24.414254  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:24.351442 16657 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.418920 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.440456 16113 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.441807 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.412307 16184 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.443114 10018 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:24.444311 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 212891us
I20260709 12:26:24.444950 16184 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.445422 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:24.389758 16661 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:26:24.450706 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:24.451478  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57528.1us
I20260709 12:26:24.455036  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:24.453501 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:24.464462 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.465651 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:24.417609 16602 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.472796 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:24.463655  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:24.469538 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.470393 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.475931 10019 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:24.415413 16710 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:24.478916 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.420437 16662 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:24.489418 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:24.486552 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 308109us
I20260709 12:26:24.494182 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.485514  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:24.400976 16690 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.438972 16493 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:24.496802 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:24.500025 10091 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:24.464462 16687 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:24.501832 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.505796 16800 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:24.507018 16657 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:24.468724 16798 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:24.493538 16694 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:24.521247 10024 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:24.560488 10094 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:24.528807 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.468070 16184 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:24.510540  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.574404  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.574867  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.482131 16710 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:24.570845  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.527806 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:24.573249  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67883.2us
I20260709 12:26:24.581477  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:24.590330 10093 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:24.522501 16687 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:24.522344 16661 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:24.592931 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.598868 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.567052 16798 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:24.571281 16694 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:24.605273 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.495580 16602 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.610615 10095 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:24.618564 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.601176 16687 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.625250 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.612422 16694 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.572235 16493 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.607445 16657 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:24.608208 16798 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:24.631130 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 126985us
I20260709 12:26:24.633919 10024 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:24.615177 16800 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:24.581199 16662 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.640718 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.621258 16429 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:24.636229 16813 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:26:24.606123 16661 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:24.646271  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 141096us
I20260709 12:26:24.653321 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.648952 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:24.654930  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:24.635221 16113 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.634394 16657 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:24.668753 16657 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.658932 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.658128 16717 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:24.618059 16184 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:24.667214 16429 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:26:24.678272 10677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.031s
I20260709 12:26:24.647342 16800 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:24.642835 16765 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:24.680323 10677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.010s	sys 0.031s
I20260709 12:26:24.685875 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.670512 16657 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:24.699481 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.705724 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.676314 16113 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:24.713255 10091 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
W20260709 12:26:24.679606 16714 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.029s	sys 0.073s
W20260709 12:26:24.728715 16817 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:26:24.729257 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.741317 16807 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:24.725522 16714 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.029s	sys 0.074s
I20260709 12:26:24.710772 16113 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:24.729856 16814 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 12:26:24.749166 10019 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:24.745055 10099 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:24.749387 16113 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:24.752421 16800 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:24.757818 10019 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:24.759799 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 385680us
I20260709 12:26:24.759260  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:24.766103  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:24.819998  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.841801 10099 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:24.842644 10019 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:26:24.848856 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.854656  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.855681  9940 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:24.858157  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.769554 16690 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.865510  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.867091  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:24.866681 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.873559  9944 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:24.876513 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.877873 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:24.866150 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:24.887018 10019 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:24.768378 16429 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.099s
I20260709 12:26:24.892858 16184 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:24.894023 10670 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.029s	sys 0.072s
W20260709 12:26:24.894430 10670 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.029s	sys 0.072s
I20260709 12:26:24.894658 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.895346 10022 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:24.896241 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.896783  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:24.897509 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.847652 16759 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:24.845428 16493 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.905759  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:24.906615  9940 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:24.908277 16633 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:24.908851 16633 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:24.909274 16633 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:24.911190 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.912655 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.914350 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.915982 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.916862 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.904403  9949 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:24.918345 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.919272 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:24.920720 10669 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:24.924014 16752 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:24.927032 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:24.932888 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:24.952831 10017 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:24.952899 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:24.934988 16809 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:24.942994 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 497731us
I20260709 12:26:24.954538 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:24.914168 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.960053 10675 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:24.890772 16690 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:24.950467 10676 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 14.
I20260709 12:26:24.967242 10090 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:24.960650 10016 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:24.983291 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:24.987758 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:24.946341 16602 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:24.904623 16749 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.001603 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:25.003726  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:24.987794 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:25.004623 10671 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
W20260709 12:26:25.002878  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 348005us
I20260709 12:26:25.005887  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:25.010020  9944 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
W20260709 12:26:25.011878  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 260868us
I20260709 12:26:25.012897  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:25.014662 16830 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.015553 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.015432 16830 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:25.016261 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.020356 16807 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.021370 16807 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.033838 16832 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:25.034554 16832 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.035890 16832 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:24.994730 10674 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:24.904412 16739 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.156s
W20260709 12:26:25.035211 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 80793.5us
I20260709 12:26:25.042270 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:25.052637 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:25.056885 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81838.6us
I20260709 12:26:24.955269 16759 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:25.057826  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 161177us
I20260709 12:26:24.956669 16809 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:25.117332 16759 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:24.975569 16661 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:24.976034 16690 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:25.126796 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:25.235896 10022 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.128515 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:25.128644  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 115858us
W20260709 12:26:25.139411 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 498122us
I20260709 12:26:25.139475 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:25.117460  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 256876us
I20260709 12:26:25.118655 16662 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:25.223961  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 218176us
W20260709 12:26:25.225615 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 396799us
I20260709 12:26:25.241357 10675 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:25.242043 10024 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.001s
I20260709 12:26:25.242811 10024 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:25.225968 10668 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:24.978402 16363 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:25.245658 10023 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:25.239868 10017 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:26:25.239694  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:25.002481 16717 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:25.234879 10018 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:25.237001  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:25.231901 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:25.236020 16699 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.823s	user 0.013s	sys 0.002s
I20260709 12:26:25.288098 10019 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:25.268332 10016 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:25.253767 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:25.241032 16690 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:25.294649 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:25.292392  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51473.5us
I20260709 12:26:25.296761  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:25.297405 10672 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:25.299350  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59774.8us
W20260709 12:26:25.299584  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.249980 16760 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:26:25.302611  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.299755 15869 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.303443 15869 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.303565 10099 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 21 because replica has last-logged OpId of term: 20 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 9.
I20260709 12:26:25.303848 15869 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:25.300097  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 63159.6us
W20260709 12:26:25.295549 16824 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.320s	user 0.003s	sys 0.006s
I20260709 12:26:25.252453 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:25.312844  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.305006  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:25.322053 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 269495us
I20260709 12:26:25.245749 16363 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:25.233394 16716 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:25.296087 16429 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:25.251348 16662 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:25.326156 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:25.298821 16699 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:25.335752 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87718.8us
I20260709 12:26:25.336230  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:25.295598 16717 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.335363 10019 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:25.342099  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.302966 16739 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:25.300652 16714 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:25.356189 16749 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:25.356931  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:25.359023  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:25.359854  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:25.361756 16687 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.727s	user 0.016s	sys 0.011s
I20260709 12:26:25.361567  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:25.361923 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:25.362564 16661 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:25.336890 16363 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:25.242933 16113 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.297653 16800 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.050s
I20260709 12:26:25.362697  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:25.371116  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66447.6us
I20260709 12:26:25.366647 15869 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.407938  9943 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:25.408916 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 186632us
I20260709 12:26:25.412086 16830 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:25.412321 10024 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.367524 16714 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:25.427632 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 128227us
W20260709 12:26:25.371119  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66191.7us
I20260709 12:26:25.345155 16762 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:25.418387 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3539) took 1199 ms. Trace:
W20260709 12:26:25.417131 16633 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.003s
I20260709 12:26:25.432847 10020 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:25.338295 16429 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:25.437820  9940 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:26:25.457782 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:25.458009 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1450) took 1336 ms. Trace:
I20260709 12:26:25.334707 16716 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:25.414254 16798 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.761s	user 0.008s	sys 0.008s
W20260709 12:26:25.421464 16799 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.087s	user 0.011s	sys 0.010s
W20260709 12:26:25.415680 16657 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.009s
I20260709 12:26:25.458765 10023 rpcz_store.cc:276] 0709 12:26:24.301148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.757501 (+456353us) service_pool.cc:224] Handling call
0709 12:26:25.243586 (+486085us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.249849 (+  6263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.418155 (+168306us) spinlock_profiling.cc:243] Waited 415 ms on lock 0x7b54001c6950. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:25.418323 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955533056}
I20260709 12:26:25.655279 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1221) took 1419 ms. Trace:
I20260709 12:26:25.419212 16687 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.465075  9945 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:25.579689  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:25.578617  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:25.535091  9991 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:25.723346  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 257232us
W20260709 12:26:25.725075  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.542362 16633 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.719273 10016 rpcz_store.cc:276] 0709 12:26:24.235768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.990637 (+754869us) service_pool.cc:224] Handling call
0709 12:26:25.268071 (+277434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.302311 (+ 34240us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.655067 (+352756us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54001c7350. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:25.655214 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339539712}
I20260709 12:26:25.432952 16809 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.546382 16807 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:25.589089 10091 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:25.631565 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:25.590838 16836 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:25.420092 16832 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:25.536787  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:25.580116 16844 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.144s	user 0.000s	sys 0.000s
W20260709 12:26:25.746488  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.746721 16602 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.542641 16694 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.913s	user 0.015s	sys 0.004s
I20260709 12:26:25.747416 16776 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:25.543607 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 181803us
I20260709 12:26:25.600411 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:25.718423 16716 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:25.748617 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 148288us
I20260709 12:26:25.531934  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:25.639456 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:25.660202 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 225393us
W20260709 12:26:25.530655 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 69406us
I20260709 12:26:25.637758 10022 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:25.413136 16661 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:25.482045 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:25.528538 16762 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.460736 16714 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:25.543939 16834 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
W20260709 12:26:25.542865  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 320283us
I20260709 12:26:25.605511 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:25.364286 16739 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:25.412912 16752 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.363988 16493 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:25.762279 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2346) took 1327 ms. Trace:
I20260709 12:26:25.765362 10017 rpcz_store.cc:276] 0709 12:26:24.434745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.294648 (+859903us) service_pool.cc:224] Handling call
0709 12:26:25.295273 (+   625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:25.343205 (+ 47932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:25.343331 (+   126us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:25.422560 (+ 79229us) log.cc:844] Serialized 26 byte log entry
0709 12:26:25.728548 (+305988us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:26:25.728650 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.728718 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.761929 (+ 33211us) raft_consensus.cc:1685] finished
0709 12:26:25.762004 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.762197 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22109,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":10729,"thread_start_us":159559,"threads_started":1,"wal-append.queue_time_us":298416}
I20260709 12:26:25.713200  9943 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:25.715245 15869 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:25.715399 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 257721us
I20260709 12:26:25.720261 10025 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:25.715243 10677 rpcz_store.cc:276] 0709 12:26:24.122043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.135869 (+ 13826us) service_pool.cc:224] Handling call
0709 12:26:24.136288 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:24.136787 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.136816 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:24.137342 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:26:24.696419 (+559077us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:24.696503 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:24.696594 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.457789 (+761195us) raft_consensus.cc:1685] finished
0709 12:26:25.457810 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.457948 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1231,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"thread_start_us":558604,"threads_started":1,"wal-append.queue_time_us":692671}
I20260709 12:26:25.724782  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:25.718348 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3535) took 1501 ms. Trace:
I20260709 12:26:25.725859  9949 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
W20260709 12:26:25.717285 16710 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.116s	user 0.007s	sys 0.007s
I20260709 12:26:25.728245  9941 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:25.738436 16429 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.740706 16363 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.718029 16841 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:25.773636 16848 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:26:25.749593 10671 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:25.747345 16602 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.776927 16363 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:25.777277 16602 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:25.750645  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 218723us
I20260709 12:26:25.778141  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:25.752377 16799 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.778455 16363 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 12:26:25.752772 10022 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:25.775048 16841 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:25.755519 16661 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:25.718184 16798 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.768150 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:25.747488 16694 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.748106 16853 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:25.771687  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47031.5us
I20260709 12:26:25.805583  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:25.768956 10020 rpcz_store.cc:276] 0709 12:26:24.217394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.757582 (+540188us) service_pool.cc:224] Handling call
0709 12:26:24.825616 (+ 68034us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.687166 (+861550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.716958 (+ 29792us) spinlock_profiling.cc:243] Waited 607 ms on lock 0x7b54002666d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:25.718280 (+  1322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1395608448}
I20260709 12:26:25.751456 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:25.755144 16807 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.797933 16836 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:26:25.779042 16429 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:25.814782 16807 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:25.815280 16807 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.815054 16853 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.753065 16760 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.752844 16776 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.781706 16841 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.781874 16845 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:25.782009 16840 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 12:26:25.782130 16801 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:26:25.829185 10099 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:25.782786 16662 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 12:26:25.784142 15869 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.830785 16830 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:25.830982 15869 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.831441 16830 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:25.783869 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:25.766855 16787 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:25.788187 16762 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:25.832998 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:25.784230 16363 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:25.833316 16839 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:25.833726 16363 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:25.833344 16787 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.788767 16773 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.834712 16363 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.835470 16773 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.804188 16835 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:25.836251 16787 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:25.836364 16835 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:25.823760 16699 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:25.836864 16835 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:25.823817 16838 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:25.837631 16773 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:25.837926 16838 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:25.838263 10090 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:26:25.782971 16842 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:26:25.839470 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:25.787751 16633 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:25.812794 16855 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:26:25.817804 16760 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.841631 16760 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.842316 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1217) took 1607 ms. Trace:
I20260709 12:26:25.835958 16363 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.842753 10021 rpcz_store.cc:276] 0709 12:26:24.234577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.754847 (+520270us) service_pool.cc:224] Handling call
0709 12:26:24.755342 (+   495us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:24.755864 (+   522us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:24.755899 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:24.756855 (+   956us) log.cc:844] Serialized 52 byte log entry
0709 12:26:25.226373 (+469518us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:25.452773 (+226400us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.453056 (+   283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.453126 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:25.842075 (+388949us) raft_consensus.cc:1685] finished
0709 12:26:25.842096 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.842256 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":343310,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":2672,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":33538,"thread_start_us":516381,"threads_started":2,"wal-append.queue_time_us":539054}
I20260709 12:26:25.844096 16853 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:25.790536 16687 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:25.814090  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:25.844900 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:25.845516 10021 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.845775  9987 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:25.813901 16850 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 12:26:25.847052 16837 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:25.814348 10024 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:25.847321  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:25.812846 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:25.848203 16772 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:25.813750 16832 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:25.848544 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:25.848732 16772 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:25.849792 16772 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.850351 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2353) took 1089 ms. Trace:
I20260709 12:26:25.850728 10024 rpcz_store.cc:276] 0709 12:26:24.761378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.452968 (+691590us) service_pool.cc:224] Handling call
0709 12:26:25.557269 (+104301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.847790 (+290521us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.850112 (+  2322us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b5400216bd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:25.850294 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":590841216}
I20260709 12:26:25.851922 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:25.853080 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:25.853506 16809 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.850750 16762 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:25.854022  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:25.812310 16798 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:25.855175 16798 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:25.855566 16798 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:25.855916 16798 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.856557 15869 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:25.818030 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68813us
I20260709 12:26:25.783529 16752 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.833070 10022 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:25.791172 16800 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:25.806257 16759 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.542s	user 0.007s	sys 0.004s
I20260709 12:26:25.824576 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:25.859278 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:25.827131 10025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3538) took 1608 ms (client timeout 1789 ms). Trace:
I20260709 12:26:25.827474 16840 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:25.823727 16845 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:25.828843 16493 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:25.791658 16710 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.767489 16739 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:25.833844 10093 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:25.837126 16749 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:25.812105 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:25.817868 16113 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:25.847518 16837 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.814311 10018 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:25.879410 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:25.857800 16716 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.886701  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:25.855862  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:25.812299 16694 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:25.778576 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:25.821065  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:25.863365 10020 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
W20260709 12:26:25.864327 10025 rpcz_store.cc:269] 0709 12:26:24.218388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:24.367874 (+149486us) service_pool.cc:224] Handling call
0709 12:26:25.822431 (+1454557us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:25.827044 (+  4613us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3107383168}
I20260709 12:26:25.866650 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:25.871502  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:25.830686 16657 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:25.858932 16752 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.878160 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:25.877974  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:25.897284  9949 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:25.880565 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:25.881392 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:25.883530 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2357) took 1026 ms. Trace:
I20260709 12:26:25.884474 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:25.899185 10676 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:25.854343 16762 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.898088 10095 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:25.882378 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:25.888530 10671 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:25.887879 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2356) took 1030 ms. Trace:
I20260709 12:26:25.913793 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:25.888978 10096 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:25.888629 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:25.890539  9941 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.865713 16184 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:25.886894 10098 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 14.
I20260709 12:26:25.832998 16776 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:25.873281 16855 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:25.865098 16840 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:25.917927 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2347) took 1445 ms. Trace:
W20260709 12:26:25.913954 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 101950us
I20260709 12:26:25.914538 10096 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:25.865129 16845 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:25.914746 10023 rpcz_store.cc:276] 0709 12:26:24.857001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.731231 (+874230us) service_pool.cc:224] Handling call
0709 12:26:25.756497 (+ 25266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.797712 (+ 41215us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:25.803973 (+  6261us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.887550 (+ 83577us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.887605 (+    55us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:25.887658 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.887723 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.887824 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26730,"prepare.queue_time_us":16656,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":50261,"thread_start_us":56291,"threads_started":1}
I20260709 12:26:25.876199 16699 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:25.885048 16837 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:25.832348 16830 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.921350 10019 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:25.918314 10018 rpcz_store.cc:276] 0709 12:26:24.483185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.295462 (+812277us) service_pool.cc:224] Handling call
0709 12:26:25.295968 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.879698 (+583730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.917710 (+ 38012us) spinlock_profiling.cc:243] Waited 518 ms on lock 0x7b54002657d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:25.917873 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1191665024}
I20260709 12:26:25.887238 16836 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:25.884634 16602 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:25.932570 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:25.933840 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:25.870456 16759 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.935972 10099 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:25.886965 16801 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.046s
I20260709 12:26:25.936616 10093 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
W20260709 12:26:25.943322 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 164780us
I20260709 12:26:25.947264 10097 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:25.947938 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:25.917120 16184 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:25.926904 16493 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:25.954934 10091 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:25.921183 16699 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:25.930687 16799 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:25.930907 16113 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.949648 10096 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.935357 16836 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:25.994410 16836 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:25.900007 10016 rpcz_store.cc:276] 0709 12:26:24.857359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.743492 (+886133us) service_pool.cc:224] Handling call
0709 12:26:25.754736 (+ 11244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.759228 (+  4492us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:25.779270 (+ 20042us) log.cc:844] Serialized 14 byte log entry
0709 12:26:25.883067 (+103797us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:25.883142 (+    75us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:25.883210 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:25.883299 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:25.883454 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12072,"prepare.queue_time_us":8302,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":35,"raft.queue_time_us":12209,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":11527,"thread_start_us":91197,"threads_started":1}
I20260709 12:26:25.976583 16839 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:25.996598 16759 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:25.996940 16839 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:25.997917 16839 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:25.954998 16184 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:25.934770 16633 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:26.001960 16699 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.010571 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2354) took 1154 ms. Trace:
I20260709 12:26:26.010842 16851 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:26.011198 10019 rpcz_store.cc:276] 0709 12:26:24.856090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.569063 (+712973us) service_pool.cc:224] Handling call
0709 12:26:25.591055 (+ 21992us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:25.936434 (+345379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:26.010221 (+ 73787us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54002161d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:26.010484 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":755517696}
I20260709 12:26:26.011565 16851 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.929989 16855 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:25.993858 10096 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.019016 16855 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.956037 16662 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:25.926462 16657 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:25.938213 16838 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:26.027284  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48484.2us
I20260709 12:26:25.985966  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:26.031782  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45880.1us
W20260709 12:26:26.137403  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 109263us
W20260709 12:26:26.151177  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.029012 16866 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:26.155460 16866 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:26.155874 16866 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.162540 16840 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.164410 16840 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.151273 16799 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:26.171856 16799 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:26.176777 16864 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.166s	user 0.002s	sys 0.000s
I20260709 12:26:26.151724 16687 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:26.179483 16687 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.179939 16687 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.181957 16867 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:25.997372 16759 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.156s
W20260709 12:26:26.201803 16863 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
W20260709 12:26:26.209307 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 246232us
W20260709 12:26:26.215131  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 359339us
I20260709 12:26:26.218920  9941 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
W20260709 12:26:26.220654  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 308371us
W20260709 12:26:26.206528 16862 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
W20260709 12:26:26.234316 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 344092us
W20260709 12:26:26.239081  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.247383  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.210376 16830 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:26.248291  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:26.254048 16762 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.255062  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:26.256531 16690 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.915s	user 0.009s	sys 0.011s
I20260709 12:26:26.257048 16690 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.257614  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:26.258909 16657 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.224303 16752 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.261711 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.262264 16800 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:26.263016 16800 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:26.215179 16839 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:26.252807  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.268779  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.269178  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.247929 16493 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:26.249568 10096 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
W20260709 12:26:26.263881 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 397312us
I20260709 12:26:26.263955 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:26.269748 10096 pending_rounds.cc:85] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 8
I20260709 12:26:26.270012 10674 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.270165 10096 pending_rounds.cc:107] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:26:26.271364 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.272198 10672 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:26.254534 10677 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:26.266155  9949 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:26.267424 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.276863 10096 raft_consensus.cc:2889] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:26.277802  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 456825us
W20260709 12:26:26.278369 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 364681us
I20260709 12:26:26.279785 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.242683 16699 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.283208 16876 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:26.284039 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 349774us
I20260709 12:26:26.283917 16876 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.286223  9949 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:26.282379 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:26.288355 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.291751  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.294698  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.295589  9946 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:26.287483  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.301520  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.286305 16876 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.302314  9949 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:26.301426 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.310245 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.310431 10017 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:26.269551 16710 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:26.311058 10671 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:26.330384  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.310629  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.273455 16749 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:26.267622 16657 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.320608  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.326670  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.344677  9940 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.348335 10674 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:26.327870  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.334640 10021 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:26.273464 16716 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.337024 15869 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:26.338119  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.355525  9944 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:26.297756 16717 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.936s	user 0.019s	sys 0.004s
W20260709 12:26:26.351786 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 69545.5us
I20260709 12:26:26.345574 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.303612 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:26.366315  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.363299 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:26.325913 16773 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:26.374197  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 161231us
I20260709 12:26:26.380106  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:26.353415 16835 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.516s	user 0.013s	sys 0.006s
I20260709 12:26:26.343873 16749 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:26.389099 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.346850 16861 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:26.411746 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.412281 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:26.414341  9944 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:26.378153 16710 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:26.415231  9949 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
W20260709 12:26:26.400719 16860 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.397s	user 0.004s	sys 0.000s
I20260709 12:26:26.407366 16749 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:26.357371 16762 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:26:26.378319 16773 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.425410 16184 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.426092  9940 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:26:26.426265 16760 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.584s	user 0.011s	sys 0.021s
W20260709 12:26:26.427629 16429 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.612s	user 0.009s	sys 0.022s
I20260709 12:26:26.428378  9945 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
W20260709 12:26:26.429438 16363 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.586s	user 0.013s	sys 0.002s
W20260709 12:26:26.435017  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 177484us
I20260709 12:26:26.399701 16876 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:26.437947  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.468262  9949 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
W20260709 12:26:26.438261 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.471180  9943 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:26.357245 16657 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:26.479719 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48380.3us
W20260709 12:26:26.439270 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50288.6us
W20260709 12:26:26.436214  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 95841.1us
I20260709 12:26:26.454438  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.461187  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.486501 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:26.488420 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:26.370628 16717 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.414906 16861 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:26.514946  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.517709  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.435804 16710 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:26.521209 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:26.522152  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:26.518781  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57720.7us
I20260709 12:26:26.523907  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:26.525254  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:26.434700 16690 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.440590 16867 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:26.532240 10098 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:26.532825 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:26.533499 10099 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:26.526372  9940 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:26.476729 16773 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.522854 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:26.514760 16842 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:26:26.534973  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.460409 16851 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.404436 16835 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.506347 16870 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.543716 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:26.529331 16714 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.757s	user 0.013s	sys 0.008s
I20260709 12:26:26.479075 16694 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:26.546959 16807 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.731s	user 0.009s	sys 0.003s
W20260709 12:26:26.532771 16661 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.722s	user 0.009s	sys 0.004s
I20260709 12:26:26.551471 10668 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:26.515830 16861 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.527818 16710 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.477541 16760 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:26.527654 16363 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:26.542063 16798 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.681s	user 0.016s	sys 0.004s
I20260709 12:26:26.530773 16113 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:26.531242 16716 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:26.550366  9942 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:26.568707  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.543159 16838 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:26.569087  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 170080us
I20260709 12:26:26.569887  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:26.574548  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 114333us
I20260709 12:26:26.581270  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.430570 16493 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.591641 16868 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:26.557109 16690 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:26.592466 16184 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.527544 16602 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:26:26.592501  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.551697 16830 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:26.553031 16867 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:26.599479 10093 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:26.603314  9945 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.563862 16875 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:26.617167 16875 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:26.601622 16739 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.721s	user 0.019s	sys 0.004s
I20260709 12:26:26.625196  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.571237 16694 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:26.691412 16633 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.692s	user 0.013s	sys 0.023s
I20260709 12:26:26.691708  9945 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:26.604758 16800 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:26.692693 16429 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:26.693179 16800 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:26.694310 16800 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.578382 16807 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.698491 16807 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.627352 16716 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:26.703575 16716 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:26.706139 16836 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.711s	user 0.018s	sys 0.013s
I20260709 12:26:26.706981 10671 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:26.710909  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.712409  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.713760  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.623613 16830 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.714970 16830 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.715699 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:26.716487 10099 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:26.717504 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:26.718106 10099 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.719134 16890 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:26.719781  9940 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:26.720546 16890 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:26.720911 16890 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:26.724664  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.725260 16837 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.778s	user 0.011s	sys 0.006s
I20260709 12:26:26.726804 16739 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.641721  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:26.643451 16882 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:26:26.728142  9942 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:26.605819 16798 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.729028  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.729585  9942 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:26.730224 16798 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.591445 16661 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:26.732547 16840 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.568s	user 0.012s	sys 0.000s
I20260709 12:26:26.732975 16661 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.733511  9944 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:26.733853 10673 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:26.562688 16870 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.626668 16869 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.117s
I20260709 12:26:26.663076 16717 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.664216 16760 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:26.667155  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.610926 16113 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:26.667263 16773 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:26.667892  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61582.4us
I20260709 12:26:26.670326 16835 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.672302 16657 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:26.673935 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.800288 10019 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
W20260709 12:26:26.674556 16809 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.821s	user 0.009s	sys 0.017s
I20260709 12:26:26.643821 16888 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:26.677340 16866 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.007s
I20260709 12:26:26.602692 16363 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.679520 16867 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:26.680474 16841 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.852s	user 0.010s	sys 0.002s
I20260709 12:26:26.809552  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.682565 16875 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.813764  9948 raft_consensus.cc:2368] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17.
I20260709 12:26:26.692135 16694 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.567662 16714 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.621256 16690 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.737679  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:26.817764  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 80165.5us
W20260709 12:26:26.807148 16845 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.874s	user 0.015s	sys 0.006s
I20260709 12:26:26.688514  9946 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:26.819624 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:26.822355 10019 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:26.766997 16836 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.742980 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.824532  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.737695  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.747241 16739 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:26.767928 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 104136us
W20260709 12:26:26.776129  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 241230us
I20260709 12:26:26.725929 16837 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.721846 16890 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:26.742022 16749 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.802399  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:26.810334  9943 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:26.691983 16633 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:26.812258 16888 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:26.742642 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.844357 16892 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:26:26.829659 16429 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:26:26.824643 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 81790.3us
I20260709 12:26:26.844115 16716 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:26.845988 16429 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:26.746025  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.846236 16716 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:26.847021  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 101072us
I20260709 12:26:26.846968 16429 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.825361  9944 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:26.825515  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 87919.5us
I20260709 12:26:26.843115 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:26.825541  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.803678 16184 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:26:26.836620  9946 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:26.850274  9946 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:26.815872 16839 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.535s	user 0.013s	sys 0.000s
I20260709 12:26:26.823943 16694 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:26.854287 16694 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.854765 16694 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:26.854838 16363 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.856225 16773 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.829418 16657 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:26.858847 16657 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.858959  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.859234 16657 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:26.860052 16749 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.864718 16837 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.865515 10024 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:26.866178 16690 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.850870  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:26.866816 16760 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:26.866988 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1226) took 1073 ms. Trace:
I20260709 12:26:26.867293 16760 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.867350 10024 rpcz_store.cc:276] 0709 12:26:25.793346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:25.852848 (+ 59502us) service_pool.cc:224] Handling call
0709 12:26:26.866813 (+1013965us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001c5f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:26.866933 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2326994688}
I20260709 12:26:26.867759 16760 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:26.867846 16895 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:26.868577 16895 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:26.869544 16888 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.869513 16895 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:26.870921  9940 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
W20260709 12:26:26.871001  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 68715.5us
I20260709 12:26:26.871706  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:26.849630 16687 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.006s
I20260709 12:26:26.875053 16687 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:26.877264 16687 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:26.877561 16710 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.878652 10097 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:26.851857 10676 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:26.852599  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.858438 10099 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:26.866460 10019 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
W20260709 12:26:26.880282 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45188.5us
I20260709 12:26:26.808900 16866 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:26.892930  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.881417  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:26.887276  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.880870  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:26.881386  9946 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:26.896726 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:26.887744  9944 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:26.902374  9943 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:26.910079 16363 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:26:26.914800 16870 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:26.915548 16896 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:26.916157 16896 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:26.916618 16896 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:26.879480 16113 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:26.918687 16661 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:26.919953  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:26.913872 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:26.942967  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61630.4us
W20260709 12:26:26.944636 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:26.945330 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48689.6us
I20260709 12:26:26.947620  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.947526 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.946524 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:26.948309 10019 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:26.948271  9944 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:26.948309 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.949972 10021 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:26.950172 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.951002 10019 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:26.816390 16841 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.953069  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:26.953516 10095 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:26.951004  9940 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:26.953069 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.955464 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.956123 10019 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:26.957422 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:26.959930 10674 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:26.960597 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:26.962680 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.962754 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:26.958185 10025 raft_consensus.cc:2376] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 19. Re-sending same reply.
I20260709 12:26:26.974341 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:26.975148 10675 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:26.974912 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:26.976356 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.977028 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:26.889176 16837 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:26.974726 10016 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:26.896075 16699 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:26.823985 16845 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:26.974427 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.975225 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.913898 16890 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:26.968405  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:26.980744 10025 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:26:26.981321 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:26.916402 16836 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:26.986405 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:26.986516  9944 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:26.987870 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:26.963617 16184 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:26.987143 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.012943 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.012872 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:27.019243 10093 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 17
W20260709 12:26:27.011472  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56419.5us
I20260709 12:26:27.017405 10020 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:26:27.020792 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.021521 10022 raft_consensus.cc:2368] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17.
I20260709 12:26:26.949960 16888 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:27.018635 10096 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:27.025967  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:27.027163 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:27.027930 10065 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:27.032341 10023 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 12. (term mismatch)
I20260709 12:26:27.032768 10023 pending_rounds.cc:85] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 11
I20260709 12:26:27.032717  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:27.033219 10023 pending_rounds.cc:107] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:26:27.033528  9941 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:27.033756 10023 raft_consensus.cc:2889] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:26.931754 16830 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.033602  9940 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:26.981194 16690 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:26.981362 16493 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:26.986536 16800 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.044409  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:27.046216  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:27.017820 16809 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:27.017634 16876 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.051126  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.054229  9944 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:27.055154 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.020366 16875 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:27.060818 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:26.988024 16869 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:26:27.069213 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.061280  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:27.073870 10091 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:27.015578 16837 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.101295 16885 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.102881 16871 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:26:27.104480 16871 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:27.104831 16871 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:27.106271 16894 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:27.077100  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:27.109193  9940 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:27.033150 16762 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:26:27.043474 16184 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.102144 16885 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.098474 16809 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:27.048053 16690 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:27.099012 16839 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:27.063474 16699 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:27.242964 16716 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.116396 16894 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:27.219372 16363 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:27.071033 16876 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.218956 16898 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.210s
I20260709 12:26:27.246716 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:27.214504 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 186592us
I20260709 12:26:27.105759 16871 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.064942 16800 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.267271 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.275866 16716 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:27.276517 16716 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.245344 16839 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.278797 16839 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:27.269474 16363 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:27.270038 16903 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:26:27.270469 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.275897 10065 leader_election.cc:400] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17.
I20260709 12:26:27.285171 10065 leader_election.cc:403] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:27.270906 16906 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:26:27.216179 16714 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:27.289669 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.292256 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.293238 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.266364 16710 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.263437 16699 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:27.286093 16861 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.689s	user 0.013s	sys 0.001s
W20260709 12:26:27.290019 16897 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.313s	user 0.000s	sys 0.001s
I20260709 12:26:27.291937 16716 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.313428 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.314260 10021 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:27.315062 10019 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:27.308255 16113 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.306788 16866 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.325088 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.327816 10674 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.326298 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.332028 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.312415 16841 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.307238 16759 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.020s
I20260709 12:26:27.375627 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.376681 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:27.380185  9948 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.508s	user 0.015s	sys 0.001s
I20260709 12:26:27.381008 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.402151 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.415302 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.419371 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.402639 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.420037 10024 raft_consensus.cc:2376] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20. Re-sending same reply.
I20260709 12:26:27.416916 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:27.421384 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.422061 10675 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:27.422487 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.423383 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.424625 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.423079 10017 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:27.425340 10675 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:27.427359 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.427901 10018 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:27.428829 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:27.421972 10016 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:27.432768 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:27.433526 10025 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:27.408540  9948 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:27.375613 16830 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:27.431210  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 147822us
W20260709 12:26:27.427083 16909 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:26:27.423967 16787 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.440481 16661 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.446164 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.426342 16714 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.443640 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:27.330976 16113 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.444708 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.445115 10674 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:27.448283 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.461218 10016 raft_consensus.cc:2376] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19. Re-sending same reply.
I20260709 12:26:27.434049 16836 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:27.453329 10093 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
W20260709 12:26:27.453958 16760 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.586s	user 0.010s	sys 0.000s
I20260709 12:26:27.334652 16809 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.426644 16807 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:27.470139 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.444118 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.464408 10018 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:27.454273 10021 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:27.498871 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:27.445940 16832 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.456110 10094 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:26:27.459858 16907 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.389s	user 0.001s	sys 0.000s
I20260709 12:26:27.499500 10093 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:27.500174  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56066.6us
I20260709 12:26:27.502036 10019 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:26:27.469688 16184 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17.
I20260709 12:26:27.507926 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.455264 16787 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.510015 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.511926 16787 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.502264 16807 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.512545 16113 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.512705 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:27.515475 16694 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.000s
I20260709 12:26:27.516007 16694 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.518924 10022 raft_consensus.cc:2368] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:27.508809 10670 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
W20260709 12:26:27.469460 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63060.5us
I20260709 12:26:27.523388 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:27.431538 16699 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.526469 16807 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.526907 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:27.308884 16861 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.537129 10670 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:27.538825 16661 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.539069 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.542809 10093 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
W20260709 12:26:27.528513 16896 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.611s	user 0.006s	sys 0.006s
I20260709 12:26:27.547200 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.470430 16760 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.551340 10090 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:27.552474 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.465226 16714 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.519440 16363 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.332019 16894 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.523696 16837 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.550280 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.515511 16885 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.583168 10671 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:27.583948 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2606) took 1098 ms. Trace:
I20260709 12:26:27.542388 16832 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.591096 10096 rpcz_store.cc:276] 0709 12:26:26.485651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.486908 (+  1257us) service_pool.cc:224] Handling call
0709 12:26:26.522656 (+ 35748us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.026252 (+503596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.583718 (+557466us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b54002fc954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:27.583890 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140247296}
I20260709 12:26:27.594687 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.580901 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:27.555258  9947 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:27.542941 16876 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:27.595288 10016 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:27.597383 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.598060 10021 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:27.559939 16896 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.600667 16896 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:27.605703  9912 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.600s (SENT)
I20260709 12:26:27.608206  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:27.609622 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.611773 16714 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.553442 16800 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.544363 16739 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:26:27.580457 16760 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:27.615262 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 795331us
W20260709 12:26:27.598223 16913 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:26:27.619915 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.616202 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:27.621124 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.621757 10021 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:27.622717 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.623514 10671 raft_consensus.cc:2410] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:26:27.624416 10090 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:27.626091 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.629760 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.630694 10671 raft_consensus.cc:2410] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:26:27.630853 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.631423 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.631879 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.632474 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.633240 10021 raft_consensus.cc:2376] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17. Re-sending same reply.
I20260709 12:26:27.633718 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.634384 10671 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:27.635277 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.594161 16874 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 12:26:27.635841 10671 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:27.636771 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.601912 16832 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.638157 10671 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:27.638762 16832 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.639426 10671 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:26:27.641124 16657 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.781s	user 0.009s	sys 0.000s
I20260709 12:26:27.641207 10016 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:27.641628 16657 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.642930 10096 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:27.643317 10096 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:27.644168 10096 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.645182 10096 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.645802  9943 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:27.647267 16845 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.647817 10670 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:27.648882 16776 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:27.649451 16776 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.646030 16920 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.651736 16920 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.652508 16888 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.652992 16888 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.655164 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.655642 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1461) took 1384 ms. Trace:
I20260709 12:26:27.655759 10670 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.656001 10668 rpcz_store.cc:276] 0709 12:26:26.271247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.272216 (+   969us) service_pool.cc:224] Handling call
0709 12:26:27.655450 (+1383234us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b5400545bd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:27.655560 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":602438528}
W20260709 12:26:27.658437 10676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1453) took 1413 ms (client timeout 1863 ms). Trace:
W20260709 12:26:27.658897 10676 rpcz_store.cc:269] 0709 12:26:26.244568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.244705 (+   137us) service_pool.cc:224] Handling call
0709 12:26:27.658232 (+1413527us) spinlock_profiling.cc:243] Waited 579 ms on lock 0x7b54007f4754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:27.658380 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1332270336}
I20260709 12:26:27.586122 16839 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.663308 10094 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:27.666052 10670 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:27.664995 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.669135 10016 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:27.618886 16916 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:27.672235 10098 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:27.672114 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.672941 10671 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.673492 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.673946 10094 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:27.676394 10094 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:27.677752 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:27.691056 16839 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.699657 16694 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:27.701363  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 95680.7us
I20260709 12:26:27.702641 10093 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:27.703073 10093 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:27.699775 16657 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:27.214761  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:26:27.705780 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77411.2us
I20260709 12:26:27.707543 10096 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:27.708421 10671 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:27.714066 10676 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:27.715792 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.716593 10016 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:27.717159 10670 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:27.718235 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.719098 10671 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:27.719173 16699 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.720569 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.721261 10668 raft_consensus.cc:2368] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19.
I20260709 12:26:27.721520 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.735203 16919 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:27.736125 16890 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:27.737759 16855 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:27.737851 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:27.738426 16855 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:27.738862 10668 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:27.729327 16888 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.739846 16855 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.741602 16855 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:27.729893 10093 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.742429 10640 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:27.743300 10093 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:27.743544 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.744290 10677 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:26:27.744484 16919 consensus_queue.cc:1175] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "9aa345bcd43b44088114b7b6f2830b17" responder_term: 21 status { last_received { term: 20 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:27.745661 16888 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:27.746102  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.746366 16888 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.748584 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:27.745203 16657 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:27.749531 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:27.750727 16888 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.752067 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:27.752384 16876 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:27.752864 16876 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.753249 16876 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.754798 16885 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:27.755349 16885 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:27.755537 16841 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:27.756300 16885 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.756778 16896 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.749610 16657 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:27.762131  9945 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:27.762319 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:27.763726  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:27.771031  9948 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:27.764284  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:27.771538  9945 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:27.765161 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:27.765600  9946 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:27.769807  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:27.778810  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 76760.3us
I20260709 12:26:27.779124 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.779965  9949 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:27.781682  9949 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:27.784332  9945 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:27.719895 16699 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:27.795017 10093 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:27.797644  9945 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:27.798275 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.798930 10668 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:27.800231 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:27.800869 16739 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:27.801743 16739 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:27.802079 16739 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:27.802316 10018 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:27.802919 16739 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:27.803373 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:27.804097 16739 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:27.804181 16919 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:27.805477 16919 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:27.806013  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:27.804816 16896 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:27.799348  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:27.806416 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:27.829274  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:27.799073  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:27.822532  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.817229 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3784) took 1268 ms. Trace:
I20260709 12:26:27.851395  9946 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
W20260709 12:26:27.851466  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52475.3us
I20260709 12:26:27.852612  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:27.853169  9946 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:27.828609  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:27.780573 16657 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:27.854468  9948 raft_consensus.cc:2410] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:26:27.825057  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:27.860121  9942 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:27.825215  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2452) took 1232 ms. Trace:
I20260709 12:26:27.856092  9945 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:27.829941  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:27.813347 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:27.862288  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56359.8us
I20260709 12:26:27.862479  9944 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:27.862545 10025 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:27.863392  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:27.863929  9944 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:27.851761 10097 rpcz_store.cc:276] 0709 12:26:26.548329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.565929 (+ 17600us) service_pool.cc:224] Handling call
0709 12:26:26.595815 (+ 29886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.879012 (+283197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.816848 (+937836us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b54002fc454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:27.817119 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":650015616}
I20260709 12:26:27.812951 16873 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:26:27.812813 16699 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:27.859673  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:27.868794 16773 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:27.869258 16773 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:27.861166  9947 rpcz_store.cc:276] 0709 12:26:26.592419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.624937 (+ 32518us) service_pool.cc:224] Handling call
0709 12:26:27.824966 (+1200029us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b540042de54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:27.825152 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1989162880}
I20260709 12:26:27.871423 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:27.872117 10671 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
W20260709 12:26:27.872351 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66016.5us
I20260709 12:26:27.873107 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:27.873278  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:27.875766  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:27.876623 10671 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:26:27.872363  9912 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.517s (SENT)
I20260709 12:26:27.877278  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:27.879190 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:27.879431  9912 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.613s (SENT)
I20260709 12:26:27.880535  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:27.882764 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:27.811654 16633 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:27.886732  9949 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:27.866448 16739 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:27.912110 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:27.879179 16868 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.238s	user 0.014s	sys 0.023s
W20260709 12:26:27.914088  9912 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.707s (SENT)
I20260709 12:26:27.870555 16853 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:27.915510 16875 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:26:27.918891 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2608) took 1391 ms. Trace:
I20260709 12:26:27.878615 16772 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:27.932652  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1464) took 1681 ms. Trace:
I20260709 12:26:27.935271  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1468) took 1330 ms. Trace:
I20260709 12:26:27.938138 16879 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:27.950160  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:27.959558 16889 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.302s	user 0.004s	sys 0.006s
I20260709 12:26:27.961750 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:27.962308 16871 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:27.964484 16798 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:26:27.965418 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.965979 16776 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:27.940410 16837 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:27.967005 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:28.106731 10096 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:27.968856 16717 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:27.979167 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:27.980976 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.115234 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:28.116044 10099 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.982333 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:27.924932 16908 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.012s	user 0.010s	sys 0.000s
I20260709 12:26:27.885008 16699 raft_consensus.cc:2764] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:26:27.886466 16493 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.216s
I20260709 12:26:27.878684 16873 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:27.885804 16830 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.226s
I20260709 12:26:27.912155 16881 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:27.878865 16874 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:28.081224 10095 rpcz_store.cc:276] 0709 12:26:26.527149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.527266 (+   117us) service_pool.cc:224] Handling call
0709 12:26:26.551338 (+ 24072us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:26.953869 (+402531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.918518 (+964649us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400304154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:27.918802 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924503296}
W20260709 12:26:28.089584  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 139510us
I20260709 12:26:28.088878  9941 rpcz_store.cc:276] 0709 12:26:26.251683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.253058 (+  1375us) service_pool.cc:224] Handling call
0709 12:26:26.253434 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.033965 (+780531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.932297 (+898332us) spinlock_profiling.cc:243] Waited 780 ms on lock 0x7b540009c4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:27.932566 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1793685888}
I20260709 12:26:28.090869  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.077725 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:28.173549 10673 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.089193  9942 rpcz_store.cc:276] 0709 12:26:26.604963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.732136 (+127173us) service_pool.cc:224] Handling call
0709 12:26:27.832437 (+1100301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:27.860655 (+ 28218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:27.934991 (+ 74336us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540042de50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:27.935185 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2529744128}
I20260709 12:26:27.991292  9946 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:28.102664 10097 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:27.984604 16868 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.343s	user 0.015s	sys 0.023s
I20260709 12:26:27.886560 16687 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:28.104452 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 130415us
W20260709 12:26:28.105029 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 232005us
I20260709 12:26:27.869757 16866 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.210s
I20260709 12:26:27.912001 16710 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:27.910779 16633 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:28.147799  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:27.878944 16890 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.223s
I20260709 12:26:27.875727 16113 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:27.806363 16919 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.080080 16836 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:28.082828 16853 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:28.215595  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:28.216133  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:27.885624 16773 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.216799  9944 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:28.083009 16739 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:27.886482 16918 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.274s
I20260709 12:26:28.184924 10674 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:28.186676  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 294533us
I20260709 12:26:28.215663  9945 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:28.191090 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.198736 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:28.203617  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 248999us
I20260709 12:26:28.208042 16891 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:28.212123  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:28.220299  9949 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:28.212927 16931 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.213743  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:28.224865  9948 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.214689  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:28.225636  9941 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:28.215492 10099 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:28.104387 16429 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:28.163657 16699 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:28.186062 16687 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.222690  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:28.213899 16772 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.225419  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:28.241518 10675 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:28.242249  9949 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:26:28.186582 16916 raft_consensus.cc:2764] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:28.185838 16868 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:28.151065 16717 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:28.248600  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63491.8us
I20260709 12:26:28.244122  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:28.191958 16710 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:28.223901 16853 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:28.221318 16891 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:28.226867 16749 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:28.300534 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:28.249045 16773 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:28.264043 16113 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:28.261595  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:28.275622 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84649.9us
I20260709 12:26:28.292927  9940 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 12:26:28.296334  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 382247us
I20260709 12:26:28.319990 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:28.320402  9912 leader_election.cc:336] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.717s (SENT)
I20260709 12:26:28.280344 16714 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:28.248785 16687 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:28.311980 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.323129 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.324504 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.270290 16710 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:28.278451  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.305502 16906 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:26:28.327469  9912 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.743s (SENT)
I20260709 12:26:28.336170  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.342142  9944 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:28.318166 16890 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:28.261761 16699 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.342917  9948 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:28.269541 16760 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:26:28.319478 16113 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:28.339979 16885 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:28.361533 16955 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:28.390287 16841 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:28.390805 16841 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:28.391642 16841 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.269913 16772 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.243562 16931 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.120405 16879 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.343394 16836 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:28.354820 16759 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:28.366679 16835 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:28.368233 16717 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:28.328867 16749 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.363157 16760 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:28.333877 16837 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:26:28.370220 10025 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:26:28.372669 16896 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.501s	user 0.008s	sys 0.002s
W20260709 12:26:28.353904  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 109892us
I20260709 12:26:28.393002 16113 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:28.343457  9912 leader_election.cc:336] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.789s (SENT)
I20260709 12:26:28.421641  9948 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:28.421782  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.343528 16917 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:26:28.386619 16930 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:26:28.363003 16936 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:28.427474 16717 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:28.427932 16717 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:28.428743 16717 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.417985 16879 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:28.431349 16879 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:28.433027 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.375695 16870 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 12:26:28.395604 10673 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:28.395972 16661 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:28.435007 16661 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.436190 16931 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:28.400848 16836 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.436772 16931 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:28.401474 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1455) took 2154 ms. Trace:
I20260709 12:26:28.402971 16910 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:28.403236 16954 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:28.355175 16184 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:26:28.352986 16941 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:28.408998 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.437172 16931 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:28.405808 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 85797.2us
I20260709 12:26:28.415190  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:28.374719 16891 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.408289 16760 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:28.406859 16835 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:28.416891 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.333197 16739 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.417186 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.417960 16749 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:28.418438 16896 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:28.418974 16920 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:28.424459  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:28.339370 16866 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:28.375137 16868 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:28.440634  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.487674  9948 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
W20260709 12:26:28.441838  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 98413.3us
W20260709 12:26:28.488497  9912 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.804s (SENT)
I20260709 12:26:28.488907  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.489076 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:28.449399  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 204586us
I20260709 12:26:28.349162 16918 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:28.341666 16873 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:28.464918  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49815.9us
I20260709 12:26:28.421348 16429 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:28.468845 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:28.440007 10669 rpcz_store.cc:276] 0709 12:26:26.247177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:26.264584 (+ 17407us) service_pool.cc:224] Handling call
0709 12:26:26.280542 (+ 15958us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:26.341953 (+ 61411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:26.341994 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:26.342698 (+   704us) log.cc:844] Serialized 52 byte log entry
0709 12:26:27.936405 (+1593707us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b18000ed400. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1
0709 12:26:27.936626 (+   221us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:28.192865 (+256239us) log.cc:844] Serialized 14 byte log entry
0709 12:26:28.395660 (+202795us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.395767 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:28.395843 (+    76us) raft_consensus.cc:1685] finished
0709 12:26:28.401140 (+  5297us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:28.401361 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47672,"prepare.queue_time_us":393959,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"raft.queue_time_us":117157,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":17130,"spinlock_wait_cycles":13682816,"thread_start_us":476359,"threads_started":4,"wal-append.queue_time_us":399574,"wal-append.run_cpu_time_us":2027,"wal-append.run_wall_time_us":1057250}
I20260709 12:26:28.470986 16694 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:28.471799 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
W20260709 12:26:28.475548  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 255670us
I20260709 12:26:28.405846 16841 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:28.442751  9946 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:28.503177  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.917s (SENT) [suppressed 37 similar messages]
W20260709 12:26:28.456562 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 112999us
I20260709 12:26:28.503283 10092 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:28.504814 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:28.471453 16894 consensus_queue.cc:1175] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 16 status { last_received { term: 15 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:28.455931 16853 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.486866 16941 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:28.505802 16787 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.512110 16429 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.496465 16879 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:28.501456 16876 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.748s	user 0.006s	sys 0.006s
I20260709 12:26:28.491406 16918 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:28.433044 16690 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.500941 16866 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:28.524085  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.506184 16920 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:28.509723 16714 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:26:28.520493 16749 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:26:28.542927 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.542182 16861 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:28.546681 16956 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:28.537503 16760 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.550823 16835 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.509053 16931 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.527251 16184 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:26:28.527493 16687 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.531427 16876 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.522782 16879 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:28.556777 16694 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:28.551141 16960 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:28.519052 16916 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:26:28.564081 16941 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.520305 16952 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:26:28.628378  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.631227 10668 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:28.628361 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.628840 10093 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:28.555608 16837 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:28.546301 16944 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:26:28.529592 16918 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:28.632448  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79016.8us
I20260709 12:26:28.514701 16917 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.129s
W20260709 12:26:28.649711 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.657121 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 112954us
W20260709 12:26:28.658257  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 114539us
I20260709 12:26:28.670095  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.661605 16937 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:28.662647 16960 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:28.655218 16931 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:28.664754 16876 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:28.650539 16879 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:28.519585  9946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:26:28.678427  9946 raft_consensus.cc:606] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:28.678946  9946 raft_consensus.cc:614] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:28.648793 16657 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.765s	user 0.004s	sys 0.011s
I20260709 12:26:28.653409 16699 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:28.663791 16959 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:28.698189 16956 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:28.705399 16871 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.562s	user 0.009s	sys 0.005s
I20260709 12:26:28.675796  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:28.645470 16896 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:26:28.710218 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.666728 16861 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:28.712774  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.714159 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.715633 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.716645 10099 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:28.680936 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.681316  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:28.690734 16866 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.710218 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:28.693070 16717 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:28.693567 16113 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.696058 16853 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:28.646581 16710 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.671128 16917 consensus_queue.cc:1243] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:26:28.699595 16881 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.526s	user 0.005s	sys 0.008s
I20260709 12:26:28.750785 16881 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:28.752744 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3559) took 1548 ms (client timeout 1804 ms). Trace:
I20260709 12:26:28.753504 16113 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:28.754884 10096 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:28.717209 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:28.711045  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 208303us
I20260709 12:26:28.731813 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:28.732039 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:28.753177 10023 rpcz_store.cc:269] 0709 12:26:27.204706 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.204870 (+   164us) service_pool.cc:224] Handling call
0709 12:26:28.752532 (+1547662us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54002161d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:28.752682 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3186604928}
I20260709 12:26:28.739042  9943 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:28.740262  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.676774 16960 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:28.744067 16965 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:26:28.761282  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:28.761458  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:28.748379 16710 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:28.762866  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 255002us
I20260709 12:26:28.756524 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:28.763576 10094 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.758098  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:28.764094  9987 leader_election.cc:400] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19.
I20260709 12:26:28.765107  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:28.765678  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:28.766109 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:28.756359 10097 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:28.711051 16955 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:26:28.727980 16956 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:28.769753  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:28.759615  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 259433us
I20260709 12:26:28.767172 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:28.770689  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:28.767647 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:28.755779 16861 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:28.732640 16959 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:28.772205  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:28.758208 16888 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.711535 16657 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.800405  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:28.795420 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:28.785780  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:28.804198 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:28.777820 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:28.804979 10094 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:28.805727  9940 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:28.852479 10094 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:28.774941 16910 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:28.768308 16845 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:28.823381  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:28.889918  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 89598.4us
W20260709 12:26:28.818343  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53294.8us
I20260709 12:26:28.775892 16868 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:28.854349 10025 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:28.890749 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:28.893599  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:28.850382 16888 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:28.773512 16773 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:28.904827 10025 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:28.907521  9941 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:28.777833 16800 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:28.912222  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:28.912744 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:28.913054 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:28.913719 10020 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:28.915488 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:28.773715 16809 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.916548 10025 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:28.918535 10674 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:28.919800 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:28.920679 10021 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:28.780857 16749 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:28.795143 16961 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.793910 16687 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:28.780647 16895 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.143s
I20260709 12:26:28.771538 16870 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.118s
I20260709 12:26:28.773896 16920 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.120s
I20260709 12:26:28.925675  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:28.849786 16776 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:28.850216 16830 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:28.928534  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 105271us
I20260709 12:26:28.958459  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:28.777597 16772 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:28.925755 10091 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
W20260709 12:26:28.948740  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.950799  9940 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:28.959873 16870 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:28.951126 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:28.768414 16113 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.087s
I20260709 12:26:28.912698 16861 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:28.789320 16879 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:28.922884 16945 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:28.923394 16919 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:28.912009 16944 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:26:28.964870  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:28.965523  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:28.852990 16916 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.100s
W20260709 12:26:29.049325 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.925487 16868 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:29.051021 16772 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.051285 16868 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:29.051573 16772 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:29.053694 16772 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:29.054883 16970 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:26:28.853276 16918 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.059165 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:28.960268 16870 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:28.968418 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:28.971004  9940 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:28.972775 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:28.975106 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:28.975659  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:28.976718  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64585.6us
I20260709 12:26:28.977053  9946 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:28.854760 16699 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 12:26:28.892264 16710 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
W20260709 12:26:28.981254 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.975952 16973 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:29.019059  9941 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
W20260709 12:26:29.021526  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:28.960616 16687 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:29.089650 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.091986  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:29.092312 10676 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:29.092708 10021 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:28.952796 16800 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:28.951851 16845 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:28.926678 16809 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.096460 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:28.983225 16749 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:29.097301 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:29.116982 10022 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:26:29.097538 10021 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:29.099545 10017 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:29.101562 10025 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:29.113741 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:29.104782 10024 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:28.960994 16776 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.107342  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:29.109866  9912 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.237s (SENT)
I20260709 12:26:28.795517 16960 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.121315  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:29.122028  9940 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.120098 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.054750 16910 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:29.122438 10021 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:29.130522 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:29.108739  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 183363us
I20260709 12:26:29.124311 10016 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:28.978644 16830 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.126552 16809 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:29.126986 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:28.982362 16876 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.137s
I20260709 12:26:29.143760 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:29.144927 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.145663 10669 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:29.145979 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.147261 16657 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:29.019304 16959 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.154191 16956 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:29.154760 16956 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.023062 16919 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:29.160466 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3795) took 1195 ms. Trace:
I20260709 12:26:29.160879 10093 rpcz_store.cc:276] 0709 12:26:27.965376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.965526 (+   150us) service_pool.cc:224] Handling call
0709 12:26:27.965820 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.656326 (+690506us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.160243 (+503917us) spinlock_profiling.cc:243] Waited 578 ms on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:29.160410 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1329011584}
I20260709 12:26:29.161410 16919 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:29.162215 10094 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:29.163775 16918 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:29.122138 16687 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:29.165098 16687 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.168370 16888 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:29.170431 16944 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:29.171089 16944 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:29.171979 16944 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.172956 16661 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:29.173785 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60143.2us
I20260709 12:26:29.131636 10018 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:29.132540 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.135413 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.127305  9947 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:29.149880 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:29.138170 10670 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:29.140228 16870 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.141163 10673 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:29.141871 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:29.142760 10022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:26:29.136583 16910 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:29.108201 16800 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:29.179770 10019 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 15: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:29.179962 10022 raft_consensus.cc:606] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:29.182065 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:29.182413 10022 raft_consensus.cc:614] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:29.171345 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:29.183195 10668 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.184826 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:29.185365  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64131.2us
I20260709 12:26:29.185451 10099 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
W20260709 12:26:29.206854 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.219434 10670 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 12:26:29.222092 10093 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:29.133998 16776 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:29.135588 16890 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:29.250669  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63975.5us
I20260709 12:26:29.182067 16866 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:29.186837 16800 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:29.255756 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.215684 16941 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.256265 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:29.256538 10016 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:29.226863 16837 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:29.257118 10093 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:29.214583 16787 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.261080 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:29.225448 16919 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:29.269563 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.270406 10016 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:29.268111  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:29.262128  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:29.262481 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.261437 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:29.267042 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:29.272389 10018 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:29.272508 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 98906.7us
I20260709 12:26:29.272724 10093 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:29.274261 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:29.274446  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:29.250109 16945 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:29.253790 16870 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.284044 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.284783 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:29.224721 16881 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:26:29.251910 16493 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:26:29.287134 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:29.294363 10016 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:29.256006 16920 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:26:29.268028 16837 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.284420 16918 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.268000 16845 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:26:29.304459 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:29.259471 16866 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.298861 16714 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:26:29.295558 16657 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:29.284065 16699 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:26:29.308985 16971 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:26:29.268226 16941 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.319047 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:29.294356 16873 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:29.259483 16890 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:29.293677 16861 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:29.319725 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:29.340507 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.321237  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:29.312682  9940 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:29.341368 10016 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:29.312618 10668 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:29.318439 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:29.286212 16868 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:29.273356 16787 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.333123 16699 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:29.342514 10093 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.346768  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:29.311425 16955 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.317811 16983 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:29.317692 16845 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:29.319051 16972 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.134s
I20260709 12:26:29.333567 16832 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:29.351183  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2459) took 1588 ms (client timeout 2043 ms). Trace:
W20260709 12:26:29.963152  9943 rpcz_store.cc:269] 0709 12:26:27.762925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.763077 (+   152us) service_pool.cc:224] Handling call
0709 12:26:29.351021 (+1587944us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b5400207954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:29.351125 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2187710720}
I20260709 12:26:29.352245 16890 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:29.313588 16977 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
W20260709 12:26:29.322036 16982 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:29.358522 10093 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:29.331547 16876 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:29.331724 16657 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.077306 10018 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:29.985932 16931 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.280s	user 0.010s	sys 0.002s
I20260709 12:26:29.972110 16986 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.085700 16986 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.088812 16986 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:29.997627 16657 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:29.998190 16876 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.104684 10023 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:26:30.106889 10018 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
W20260709 12:26:30.161545 16776 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.910s	user 0.011s	sys 0.001s
I20260709 12:26:30.162290 16776 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.333813 16809 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:29.482227  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:29.599413 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.227406  9940 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.340296 16879 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:29.356113 16699 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:29.607306  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.231612  9942 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:29.613931 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:29.318946 16945 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.643980 10677 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:26:29.690840 16949 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:29.331882 16918 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:29.332544 16956 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:29.365176 16845 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:29.342744 16873 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.600039 16714 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:29.554734 16983 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:29.757709 16989 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.411s	user 0.000s	sys 0.000s
W20260709 12:26:29.760809 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.041s	sys 0.043s
W20260709 12:26:30.258553 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.041s	sys 0.043s
I20260709 12:26:29.760644 16941 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:29.779443  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2474) took 1338 ms. Trace:
I20260709 12:26:29.643514 16910 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:29.366025 16955 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:29.778290 16883 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:29.959785  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:29.974180 10093 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:30.274420 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2637) took 1009 ms. Trace:
I20260709 12:26:30.274963 10093 rpcz_store.cc:276] 0709 12:26:29.264911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.973608 (+708697us) service_pool.cc:224] Handling call
0709 12:26:29.973950 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.274008 (+300058us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.274337 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:29.982944  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:30.277210  9943 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:30.280146  9940 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
W20260709 12:26:30.208742 16995 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.231s	user 0.000s	sys 0.000s
W20260709 12:26:30.209998 16870 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.920s	user 0.005s	sys 0.005s
I20260709 12:26:30.209944 16837 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:29.479146 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 141999us
W20260709 12:26:30.158476 16800 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.890s	user 0.011s	sys 0.000s
I20260709 12:26:30.237092 16853 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:30.251955 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.952s	user 0.111s	sys 0.166s
W20260709 12:26:30.300448 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.000s	user 0.111s	sys 0.166s
I20260709 12:26:30.257088  9945 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:30.258883 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 1.03485e+06us
I20260709 12:26:30.261845  9944 rpcz_store.cc:276] 0709 12:26:28.441163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.441647 (+   484us) service_pool.cc:224] Handling call
0709 12:26:28.441987 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:28.451986 (+  9999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:28.452020 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:28.476852 (+ 24832us) log.cc:844] Serialized 26 byte log entry
0709 12:26:28.715518 (+238666us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:28.715559 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:28.715628 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:29.779156 (+1063528us) raft_consensus.cc:1685] finished
0709 12:26:29.779182 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:29.779365 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9643,"raft.queue_time_us":10164,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11124,"thread_start_us":200985,"threads_started":1,"wal-append.queue_time_us":420710}
I20260709 12:26:30.247265 16949 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:30.281901  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.281697 16883 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.302610 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:30.307578  9944 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:30.311398  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1494) took 1198 ms. Trace:
I20260709 12:26:30.311709  9944 rpcz_store.cc:276] 0709 12:26:29.113258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.302616 (+1189358us) service_pool.cc:224] Handling call
0709 12:26:30.307405 (+  4789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.311119 (+  3714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.311340 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.312861  9944 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:30.313409  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1495) took 1199 ms. Trace:
I20260709 12:26:30.313761  9944 rpcz_store.cc:276] 0709 12:26:29.113602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.312436 (+1198834us) service_pool.cc:224] Handling call
0709 12:26:30.312704 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.313189 (+   485us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.313361 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.310564  9942 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 21 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:30.317057 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:30.318141 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:30.299609 16879 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:30.329972 16879 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.331339 16883 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:30.332139  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1498) took 1156 ms. Trace:
I20260709 12:26:30.332284 16879 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.331966 16657 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.332551  9942 rpcz_store.cc:276] 0709 12:26:29.175726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.260550 (+1084824us) service_pool.cc:224] Handling call
0709 12:26:30.332069 (+ 71519us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.333803  9942 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:30.333884 16879 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:30.334260 16657 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.318986 16845 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.335954 16955 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.334384  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2482) took 1159 ms. Trace:
I20260709 12:26:30.346335  9942 rpcz_store.cc:276] 0709 12:26:29.175457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.333309 (+1157852us) service_pool.cc:224] Handling call
0709 12:26:30.333642 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.334155 (+   513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.334337 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.320348 16861 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.226018 16832 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.283106 16988 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:30.344689 10090 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.099s	sys 0.161s
I20260709 12:26:30.290906 16991 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:30.347500  9943 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 12:26:30.396718 10090 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.103s	sys 0.165s
I20260709 12:26:30.331965 16883 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:30.396884  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1496) took 1283 ms. Trace:
I20260709 12:26:30.263295 16714 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.407141  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2686) took 1442 ms. Trace:
I20260709 12:26:30.397605  9940 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:30.405597  9943 rpcz_store.cc:276] 0709 12:26:29.113918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.973631 (+859713us) service_pool.cc:224] Handling call
0709 12:26:30.396819 (+423188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.407696  9948 rpcz_store.cc:276] 0709 12:26:28.965081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.110768 (+145687us) service_pool.cc:224] Handling call
0709 12:26:29.136982 (+ 26214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:29.293822 (+156840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.293854 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:29.301010 (+  7156us) log.cc:844] Serialized 26 byte log entry
0709 12:26:29.332663 (+ 31653us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:29.332699 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:29.332775 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.406859 (+1074084us) raft_consensus.cc:1685] finished
0709 12:26:30.406884 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.407070 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":52001,"raft.queue_time_us":113140,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":7229,"thread_start_us":134281,"threads_started":2,"wal-append.queue_time_us":295826}
I20260709 12:26:30.341120 16987 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:30.355679 16687 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.012s	sys 0.029s
I20260709 12:26:30.345300 16773 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.345809 16944 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.309762 16800 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.313105 16853 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.355427 16835 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:30.424219 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 106138us
I20260709 12:26:30.404793 16845 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:30.429427 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 126983us
W20260709 12:26:30.431910 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.432190 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:30.425873 16687 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.102s	user 0.012s	sys 0.029s
W20260709 12:26:30.435953 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.439527 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.424021 16987 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:30.398762 16861 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.441980 10018 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:30.446278 10675 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:30.440889 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:30.427142 16883 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:30.407927 16991 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.340320 16949 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.434192 16879 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:30.458045 16955 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:30.467084 16955 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.468014 16832 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.436410 16657 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:30.433578 16944 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.405251 16988 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.432885 16873 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.427279 16699 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.341954 16983 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.442975 16945 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.445266 16987 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:30.433183 16773 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.462970 16883 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.469365 16955 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.467026 16879 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.521488  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1487) took 1739 ms. Trace:
I20260709 12:26:30.519420  9949 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:30.528657  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2684) took 1564 ms. Trace:
I20260709 12:26:30.529065  9949 rpcz_store.cc:276] 0709 12:26:28.964528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.964641 (+   113us) service_pool.cc:224] Handling call
0709 12:26:30.519248 (+1554607us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b2c001a2ad8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07806291 00007f4e078210ca 00007f4e0954f4ad 00007f4e0955b4db 00007f4e0955a2af 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2
0709 12:26:30.528596 (+  9348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28423936}
I20260709 12:26:30.483737 16949 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:30.530854  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65473.6us
I20260709 12:26:30.527360  9947 rpcz_store.cc:276] 0709 12:26:28.782620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.803374 (+ 20754us) service_pool.cc:224] Handling call
0709 12:26:28.803793 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.176780 (+372987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.521106 (+1344326us) spinlock_profiling.cc:243] Waited 323 ms on lock 0x7b540018b0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:30.521406 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":743179008}
I20260709 12:26:30.483429 16956 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.483031 16918 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:30.539263  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:30.540001 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3590) took 1394 ms. Trace:
I20260709 12:26:30.488703 16910 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.485541 17010 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:30.544265 10016 rpcz_store.cc:276] 0709 12:26:29.145232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.246874 (+1101642us) service_pool.cc:224] Handling call
0709 12:26:30.258909 (+ 12035us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.259284 (+   375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.259329 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.259935 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.441432 (+181497us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:30.537683 (+ 96251us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.539691 (+  2008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.539768 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.539812 (+    44us) raft_consensus.cc:1685] finished
0709 12:26:30.539823 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.539946 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":96222,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1963,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":8628,"thread_start_us":232543,"threads_started":2,"wal-append.queue_time_us":240601}
I20260709 12:26:30.504091 16988 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.502261 17006 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:30.568521  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.563179  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:30.557883 16837 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.488188 16945 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.573628  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.575222  9949 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:26:30.581390  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:30.582073  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2481) took 1486 ms. Trace:
I20260709 12:26:30.582574  9940 rpcz_store.cc:276] 0709 12:26:29.095686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.297443 (+1201757us) service_pool.cc:224] Handling call
0709 12:26:30.397361 (+ 99918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.408578 (+ 11217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.581823 (+173245us) spinlock_profiling.cc:243] Waited 99.4 ms on lock 0x7b54000b5c50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:30.581999 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228714880}
I20260709 12:26:30.583618 16835 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.580049 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:30.584860 10095 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:26:30.558909 16918 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:30.588243  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.590116 10016 pending_rounds.cc:85] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 7
I20260709 12:26:30.593926 10016 pending_rounds.cc:107] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:26:30.594358 10016 raft_consensus.cc:2889] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.573957 16931 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:30.573829 16956 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.572151 16776 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.562562 16870 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:26:30.570041 16687 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:26:30.595271 17004 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.615269 16800 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.614238 17018 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.592212 16941 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:30.562235 16910 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.615676 16991 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.602185 17006 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:30.652624 16931 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:26:30.568135 17010 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:30.659189 17000 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:26:30.659646  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1499) took 1483 ms. Trace:
I20260709 12:26:30.659796 16714 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:30.647079 17004 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.667519 17006 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.669044  9942 rpcz_store.cc:276] 0709 12:26:29.175994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.347822 (+1171828us) service_pool.cc:224] Handling call
0709 12:26:30.348136 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.443146 (+ 95010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.443172 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.455723 (+ 12551us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.526182 (+ 70459us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:30.526259 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.526367 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:30.659380 (+133013us) raft_consensus.cc:1685] finished
0709 12:26:30.659400 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.659579 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":108005,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":18009,"thread_start_us":129058,"threads_started":2,"wal-append.queue_time_us":99247}
W20260709 12:26:30.659904 17002 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:26:30.672945 17006 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:30.661172 16879 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:30.661585 16699 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.597793 16835 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.643831 16853 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:30.667886 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3792) took 2708 ms. Trace:
I20260709 12:26:30.601801 16918 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:30.680774  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 95208.8us
W20260709 12:26:30.653025 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 212211us
W20260709 12:26:30.681428 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.642370 17016 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:30.652638 16941 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:30.682835 10092 rpcz_store.cc:276] 0709 12:26:27.960027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:27.960200 (+   173us) service_pool.cc:224] Handling call
0709 12:26:27.960632 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.504829 (+544197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.667608 (+2162779us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b5400341f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:30.667801 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1242991872}
I20260709 12:26:30.664798 17018 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.698175 17015 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:30.698212 16879 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:30.696969 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2629) took 1514 ms. Trace:
I20260709 12:26:30.699626 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:30.699571 10090 rpcz_store.cc:276] 0709 12:26:29.182972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.355336 (+172364us) service_pool.cc:224] Handling call
0709 12:26:29.551734 (+196398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.552023 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:26:30.406794 (+854771us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.696506 (+289712us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.696583 (+    77us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:30.696645 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.696738 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.696900 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4045,"prepare.queue_time_us":880129,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":2431,"raft.queue_time_us":5088,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":1092655,"threads_started":2,"wal-append.queue_time_us":286068}
W20260709 12:26:30.669906 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89950.1us
I20260709 12:26:30.706394 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:30.708395 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:30.709057  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1497) took 1534 ms. Trace:
I20260709 12:26:30.613227 16956 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.706202 16861 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.695638 16835 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.696350 16941 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:30.709462  9944 rpcz_store.cc:276] 0709 12:26:29.175098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.314553 (+1139455us) service_pool.cc:224] Handling call
0709 12:26:30.315183 (+   630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.315380 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:30.497372 (+181992us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.708702 (+211330us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.708754 (+    52us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:30.708796 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.708860 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.708977 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16664,"prepare.queue_time_us":220992,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":72584,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":13720,"thread_start_us":347530,"threads_started":3,"wal-append.queue_time_us":100577}
W20260709 12:26:30.715225  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.708391 16944 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:30.722250 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.723784 17021 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:30.724076 17025 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:30.725329 16910 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.707803 16853 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:26:30.715651 16773 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.718371 16870 consensus_queue.cc:1243] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:26:30.728190 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1239) took 1961 ms (client timeout 1688 ms). Trace:
W20260709 12:26:30.736302 10023 rpcz_store.cc:269] 0709 12:26:28.766538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.767785 (+  1247us) service_pool.cc:224] Handling call
0709 12:26:30.726050 (+1958265us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002652d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:30.728083 (+  2033us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2751767296}
I20260709 12:26:30.703558 17016 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.720111 16835 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 12:26:30.720785 16879 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:30.758039 10023 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:30.758895 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2401) took 1609 ms. Trace:
I20260709 12:26:30.759279 10023 rpcz_store.cc:276] 0709 12:26:29.149584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.741760 (+1592176us) service_pool.cc:224] Handling call
0709 12:26:30.753586 (+ 11826us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.758637 (+  5051us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.758837 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.760779 10023 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:26:30.766749 10023 pending_rounds.cc:85] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 15
I20260709 12:26:30.767513 10023 pending_rounds.cc:107] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:26:30.767853 10023 raft_consensus.cc:2889] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:30.767731 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3807) took 1960 ms. Trace:
I20260709 12:26:30.768504 10094 rpcz_store.cc:276] 0709 12:26:28.806860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.922344 (+115484us) service_pool.cc:224] Handling call
0709 12:26:28.922670 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.162570 (+239900us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.767385 (+1604815us) spinlock_profiling.cc:243] Waited 239 ms on lock 0x7b54002fbf54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:30.767641 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":550125952}
I20260709 12:26:30.750387 16944 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.769942 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1252) took 1603 ms. Trace:
I20260709 12:26:30.732672 17021 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:30.771507 17027 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:26:30.775175 10023 rpcz_store.cc:276] 0709 12:26:29.166910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.760155 (+1593245us) service_pool.cc:224] Handling call
0709 12:26:30.760453 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.769694 (+  9241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.769887 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.781908 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2625) took 1611 ms. Trace:
I20260709 12:26:30.782352 10095 rpcz_store.cc:276] 0709 12:26:29.170318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.184257 (+ 13939us) service_pool.cc:224] Handling call
0709 12:26:29.249179 (+ 64922us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.589735 (+1340556us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.781641 (+191906us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400093154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:30.781843 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3070791424}
I20260709 12:26:30.785145 10023 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:30.788220 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1253) took 1621 ms. Trace:
I20260709 12:26:30.789031 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1254) took 1621 ms. Trace:
I20260709 12:26:30.790138 10019 rpcz_store.cc:276] 0709 12:26:29.167526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.788222 (+1620696us) service_pool.cc:224] Handling call
0709 12:26:30.788522 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.788720 (+   198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.788748 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:30.788787 (+    39us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:30.788818 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.788873 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.788976 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.791509 10023 rpcz_store.cc:276] 0709 12:26:29.167256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.784485 (+1617229us) service_pool.cc:224] Handling call
0709 12:26:30.784834 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.787909 (+  3075us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.788143 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:30.750793 16945 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:30.833977 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1221) took 2118 ms (client timeout 1571 ms). Trace:
I20260709 12:26:30.834746 16918 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:30.834452 10098 rpcz_store.cc:269] 0709 12:26:28.715744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.715890 (+   146us) service_pool.cc:224] Handling call
0709 12:26:30.790544 (+2074654us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b5400304154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:30.833868 (+ 43324us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1227921280}
I20260709 12:26:30.744387 17016 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.788455 16941 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.773310 16956 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.846194 16800 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:30.846783 16800 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:30.849520 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3803) took 2436 ms. Trace:
I20260709 12:26:30.849938 10096 rpcz_store.cc:276] 0709 12:26:28.413057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.413227 (+   170us) service_pool.cc:224] Handling call
0709 12:26:28.417269 (+  4042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:28.755343 (+338074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.849253 (+2093910us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b5400192dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:30.849450 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":776018816}
I20260709 12:26:30.847903 16800 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.851686 16800 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:30.855314 16881 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:30.855896 16881 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:30.857206  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.857861  9940 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:30.835588 16944 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:30.845415 16984 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.498s	user 0.003s	sys 0.010s
W20260709 12:26:30.844825 17026 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:26:30.858306  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:30.859146 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.859083  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:30.864732 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 156438us
I20260709 12:26:30.873798  9940 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:30.874214 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:30.875378  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.876969  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.868784  9942 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:30.879878  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.880599  9942 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:30.868577  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.881098 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:30.881811  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.776067 16837 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:30.878325  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.849520 16891 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:30.886013  9940 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:30.887648  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:30.888365 16918 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:30.896083 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2397) took 1789 ms. Trace:
W20260709 12:26:30.902311 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 193581us
I20260709 12:26:30.901274  9944 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:26:30.902535 10024 rpcz_store.cc:276] 0709 12:26:29.106515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.121601 (+ 15086us) service_pool.cc:224] Handling call
0709 12:26:29.123736 (+  2135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.140995 (+ 17259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:29.141030 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:29.141069 (+    39us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:29.141100 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.895754 (+1754654us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b54002d4e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:30.895838 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:30.896015 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39279872}
I20260709 12:26:30.863972 16941 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.880477 16988 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.857008 16881 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.886085 16873 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:30.898965 16657 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:30.900323 16837 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.903244 16867 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
W20260709 12:26:30.932418  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 1.66183e+06us
I20260709 12:26:30.922797 17006 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.934531 17006 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.921561 17010 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.936209 10669 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:30.907137 16946 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:26:30.923694 16918 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:30.919077 16184 consensus_queue.cc:1175] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 21 status { last_received { term: 18 index: 13 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:30.947208 16867 consensus_queue.cc:1243] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:30.937798 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:30.940434 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:30.941721 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:30.948755 10674 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:30.948925 17036 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.949504 16944 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.928966 16879 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.942571 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:30.943321 17006 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.944307 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:30.915089 16988 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.945591 16837 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:30.945708 10095 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:30.921922 16873 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.960057 16873 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.942128 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:30.961189 10671 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:30.915230 16853 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:26:30.880944 16987 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.962472 10674 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:30.953140 10090 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:30.951855 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:30.945225 10094 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 12:26:30.955157  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.915509 17021 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:30.974228  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.974242 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:30.974812  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.949469 17036 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.970944 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:30.986948 16699 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.979884 16809 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.987645 16699 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:30.987768 16809 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:30.989765 16699 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:30.990995 17022 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:30.990070 16657 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:30.998149  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:30.956245 16961 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:30.990283 16931 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:31.000486  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:30.991513 17022 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.001255 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.001163 16661 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:31.002655 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.024600 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.003070 16800 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.024745  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:31.003249 17029 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:26:31.003799 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.028088 10096 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:31.029047 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.029786 10096 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:31.004921 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.005985 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.031418 10093 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:31.008399 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1262) took 2195 ms. Trace:
I20260709 12:26:31.032342 10677 rpcz_store.cc:276] 0709 12:26:28.812732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.813379 (+   647us) service_pool.cc:224] Handling call
0709 12:26:29.643740 (+830361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.237023 (+593283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.008070 (+771047us) spinlock_profiling.cc:243] Waited 830 ms on lock 0x7b54007d3cd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:31.008304 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1908775168}
I20260709 12:26:30.996476 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:31.018136 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:30.952303 16918 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.022897 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:30.980463 16836 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:30.955283 16879 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.044381 16836 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.049733 16809 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.026865 16874 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:31.027004  9912 connection.cc:256] RPC call timeout handler was delayed by -1.6703323849997105s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
I20260709 12:26:31.054370 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.027392 10094 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:31.038318 16866 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.041153 10098 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:31.041676 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45288.3us
I20260709 12:26:31.046914 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.058835 10096 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:31.048473  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:31.026721 10095 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:31.054421  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:31.054780  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.061044  9912 connection.cc:256] RPC call timeout handler was delayed by -1.3157323850000466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2048s for a response
I20260709 12:26:31.057816 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:31.061463  9912 connection.cc:256] RPC call timeout handler was delayed by -1.1690323849998094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 12:26:31.036311 16956 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:31.061796  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.057783 10677 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:26:31.064396 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.060177  9942 raft_consensus.cc:2376] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17. Re-sending same reply.
I20260709 12:26:31.066184  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.067068  9942 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:31.050138 16714 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:31.049975 16836 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.044653 16657 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:31.062924 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:31.057417 16661 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:26:31.069608  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.072067 10092 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:31.053131 16987 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.072024  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.074350 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:31.080533 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.071605  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:31.077730 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.082580 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.082832 10676 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:31.025657 16931 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.083925  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.086964  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:31.081014 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:31.085449  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.087353 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:31.090546  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.090670 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:31.091336 10676 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.070426 16917 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.068259 16714 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.094703  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.044929 16800 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.096045 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:31.100188  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:31.097020 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.098174  9942 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:31.075073 16773 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.099802 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.115597 17030 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.116282 10023 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:31.087074 17021 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:31.101082 10024 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:31.103055 17028 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.095986 17006 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:31.103358 17038 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:31.095062 16883 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:31.104938 17036 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.105052 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.091588 17020 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:31.110795 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.110942 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.103291 16876 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.112134 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.112459 17023 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:31.117102 16871 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:31.078483 16809 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.119764 16855 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.108700 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:31.165514 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56880.3us
I20260709 12:26:31.166345 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:31.100991  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.094230 16874 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:26:31.128890 17021 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:31.136330 16879 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:31.128664 16883 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.130858 17038 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:31.130890 16800 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.131228 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:31.114605 16714 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:31.134688 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.111128 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1413) took 2140 ms. Trace:
I20260709 12:26:31.135488 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:31.113688 10674 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:31.153003 16988 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.020525 16961 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.114375 16917 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.128773 16866 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:26:31.196292 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.122072 16969 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.108677 16760 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:31.170524 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:31.171314 16876 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:31.167652 16809 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.118005 16873 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:26:31.122560 17006 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.118384 16773 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.153860 16699 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.159497 17019 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:26:31.185942 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.153003 17022 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.117429 17030 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.192287 10023 raft_consensus.cc:2410] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:31.169550 16855 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.198298 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.184330 16883 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:31.208163 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.195369 16661 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:31.211593 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:31.214656 10098 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:26:31.222671 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:31.129040 17028 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.250259 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.250834 16866 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:31.227936 10676 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:31.193279 16874 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.169003 16776 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:26:31.228273 17006 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:31.270643 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.258348 16687 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:31.186766 17038 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:31.233384 10020 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:31.237277 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.190137 10673 rpcz_store.cc:276] 0709 12:26:28.970708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.970845 (+   137us) service_pool.cc:224] Handling call
0709 12:26:28.971111 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:29.178948 (+207837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.110879 (+1931931us) spinlock_profiling.cc:243] Waited 170 ms on lock 0x7b54004c9ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:31.111047 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":390660864}
I20260709 12:26:31.251012 10098 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
W20260709 12:26:31.227461 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.279907 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.225149 16988 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:31.282536  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 100589us
I20260709 12:26:31.283003 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.283942 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.284562 10673 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:31.285712 10673 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:31.232352 16657 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:26:31.159534 17020 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.244632 16832 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 12:26:31.256763 16773 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.203961 17021 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.259482 16873 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:31.261909 17030 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.262701 16931 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.263494 16874 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.265502 10670 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:31.176528 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:31.303838 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 127381us
I20260709 12:26:31.304072 10099 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:31.304582 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:31.301396 16773 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.004s
I20260709 12:26:31.305485 16966 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:31.282500 16988 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:31.306142 16966 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:31.306470 16988 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.276664 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.309433 10676 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:31.280246 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.153055 17023 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.310483 10023 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:31.286757 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.237259 16699 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.294349 10673 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:31.296648 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:31.262962 16690 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:31.324251 16855 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.277163 16687 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:31.218487 16969 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.306730 16966 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.306699 16957 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:31.315863 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.338099 10020 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:31.327620 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:31.335038  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:31.333905 16855 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:31.344626 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:31.308682 16972 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.312266 17036 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:31.344108  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:31.344158 16809 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.352823 10024 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:31.337133 16739 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:31.364687 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:31.364991 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 153493us
I20260709 12:26:31.337450 16957 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:31.348636 16910 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:31.351308 16941 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:31.349404 16891 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 12:26:31.350704 17036 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.379936 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:31.382791  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.358582 17030 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:31.359069 16919 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:31.362183  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:31.364158 16739 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:31.370229 16917 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.400630 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:31.400630  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:31.380352 16941 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.404209 16800 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:31.411285 16800 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:31.411780 16800 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:31.410171 16941 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.415185 16657 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:31.380153 16910 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:31.418184 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.426406 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.417752 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.397418 16944 raft_consensus.cc:2764] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:31.367635 16931 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:26:31.398824 17036 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.431751 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:31.376543 16832 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:26:31.440199 17020 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.429807 17041 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:31.367595 16839 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:26:31.399766 16919 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:31.463696 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.350996 16972 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:31.481591  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.481755  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.473439 16874 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.373087 16885 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 12:26:31.392779 16687 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.406426 17030 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:31.478996 10023 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:31.430892 16910 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:31.484194 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52522.5us
I20260709 12:26:31.492244 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.493726 10023 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:31.424576 17021 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:31.489563  9987 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:36302 dropped due to backpressure: service queue is full with 50 items
W20260709 12:26:31.523109  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.480971 16961 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.480003 16776 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:31.416059 16836 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:31.537832  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55223.5us
I20260709 12:26:31.497406 16690 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:31.542629 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.543429 10676 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:31.551288  9912 leader_election.cc:336] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.232s (SENT)
I20260709 12:26:31.495373 17021 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:31.542176 16633 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.578361 16919 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.579439 16919 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.586704 10677 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:31.590984 16966 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:31.591912 16950 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.592629 16950 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.594631 17030 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.604821 16952 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:31.605458 16952 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:31.539654 16687 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:31.606631 16952 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.613971 16772 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.615644 17021 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.630741 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.665076 10676 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:31.632335 16839 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.636830 16830 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:31.639140 17048 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.273s	user 0.003s	sys 0.000s
I20260709 12:26:31.640967 17015 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:31.670552 17015 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.645128  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.671591  9948 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:31.542230 16690 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.475163 16957 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.567251 16931 consensus_queue.cc:1243] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:26:31.533886 17022 consensus_queue.cc:1175] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 20 status { last_received { term: 17 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:31.667476 17018 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.545835 16776 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.611088 17023 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.675413 16690 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:31.527415  9987 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:36302 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2415)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1271)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3598)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3613)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1270)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1264)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2412)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3618)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1268)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3611)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3606)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59744 (request call id 38)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1258)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2419)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1275)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3594)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3607)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3608)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3610)
I20260709 12:26:31.687847 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.687158 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:31.690527 16776 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.691684 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:31.704438 10670 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
W20260709 12:26:31.693419 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2628) took 2522 ms (client timeout 1688 ms). Trace:
W20260709 12:26:31.705439 10099 rpcz_store.cc:269] 0709 12:26:29.171217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.215173 (+ 43956us) service_pool.cc:224] Handling call
0709 12:26:31.693216 (+2478043us) spinlock_profiling.cc:243] Waited 2.03 s on lock 0x7b5400305554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:31.693344 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4672424704}
I20260709 12:26:31.694525  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.706700  9948 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:31.711386  9949 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:31.712154 16920 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.713922 16836 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.713790 16920 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.715142 16690 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.721352 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:31.688452 17021 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.707780 16959 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:31.674340 16839 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.730955  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.733776 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:31.748495 16739 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.749277 16699 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.749679 16739 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:31.672416 16772 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.726969 16957 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.706004 16883 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.725780 16890 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:31.750996 16841 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:31.771044 16690 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:31.775741 16890 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:31.776676 16941 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.777050 16890 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.762830 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3586) took 2620 ms. Trace:
I20260709 12:26:31.778788 10018 rpcz_store.cc:276] 0709 12:26:29.144011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.108269 (+964258us) service_pool.cc:224] Handling call
0709 12:26:30.108654 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:30.448598 (+339944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.762584 (+1313986us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540012c854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:31.762772 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765867392}
I20260709 12:26:31.780117 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:31.780617 16714 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.583s	user 0.010s	sys 0.000s
I20260709 12:26:31.780711 10018 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:31.762871  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.781132 16714 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.782002  9948 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:31.782106 17038 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.782737 10674 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:31.782965  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.728289 16891 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:26:31.742581 16959 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:31.787575  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:31.788739 10671 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:31.764492 17052 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:26:31.790256 10669 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:31.785156  9987 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45930 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3613)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3609)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1272)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1270)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3592)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3605)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1264)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3614)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3615)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1266)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3597)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2412)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3599)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2414)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3616)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3601)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3617)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3618)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1268)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3611)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3602)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3593)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3619)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3604)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1269)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3603)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3612)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3600)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3606)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3620)
Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59744 (request call id 38)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1258)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2410)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2411)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1273)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2413)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2417)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2418)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2419)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1275)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3594)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3595)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3596)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3607)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3608)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3610)
W20260709 12:26:31.793129  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.796037  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.735888 16800 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.801009 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:31.801611 10021 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:31.803444 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:31.786880 16959 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.807349 10018 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:26:31.806767  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.810680 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.811942  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.759742 16988 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.816196 10021 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:26:31.782523 17006 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.800056 16874 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:26:31.821712 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:31.816180  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:31.829298 16841 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:31.829769 16839 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.853183 10675 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:31.831148 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.854977 10024 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:31.834777  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:31.835640 10017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1244) took 3018 ms (client timeout 2896 ms). Trace:
W20260709 12:26:31.857008 10017 rpcz_store.cc:269] 0709 12:26:28.817091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.817352 (+   261us) service_pool.cc:224] Handling call
0709 12:26:31.835422 (+3018070us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b5400137ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:31.835544 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":427468160}
I20260709 12:26:31.836737  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.858821  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.859400  9944 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:31.837272 16760 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:31.837906 16776 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:31.838207 16772 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.838994 10019 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:31.840144 16941 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.840616  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.866384  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:31.842787 16690 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.798031 17041 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:31.846688 10021 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:31.869472  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.869763 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.848616  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:31.871466 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.849613 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.871971 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.870350 10021 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:31.873695 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:31.874166 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.875288 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.862643 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.864003 10097 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:31.861275  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:31.867729  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.847438 16918 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:31.879132 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.877938 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.881052 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.882105 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:31.883157  9944 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:31.881229 10097 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:31.885028  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:31.885890  9948 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:31.886759 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:31.851897 16874 consensus_queue.cc:1243] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:31.892161 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.893025 10099 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:31.890298 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:31.897058  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.863112 16891 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.898684 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.899852 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.866395 17020 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.901957 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.906423 10090 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:31.912525  9948 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:31.887933 16839 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:31.915773 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.920006 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.864403 17015 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.903945 16941 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:31.922936 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.926426 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.880362 16690 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:31.931443 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.913756 17038 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.933334  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.933965 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.935741  9944 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:26:31.936349 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:31.937047 10099 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:31.924283 16687 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:31.939476 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.941296 10024 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:31.924511 16874 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:31.906770 16891 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.927839 16941 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:31.945683 10094 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:31.933113 17004 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:31.913580 17020 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.948573 16874 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.951342 16760 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:31.961699  9945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2685) took 2996 ms (client timeout 1500 ms). Trace:
I20260709 12:26:31.961829 16760 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.968618 16760 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.946305 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:31.970757 16787 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.972595 17020 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.973238  9946 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:31.934206 17041 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.976122 16874 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.976208  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:31.954885  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:31.977444  9944 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:31.951152 17006 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.957217 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.958956 17015 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:31.960639 16891 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.960914 16714 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.921401 17059 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 12:26:31.962133  9945 rpcz_store.cc:269] 0709 12:26:28.964790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:28.965304 (+   514us) service_pool.cc:224] Handling call
0709 12:26:31.961466 (+2996162us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54000163d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:31.961599 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2840343936}
I20260709 12:26:31.964222 16661 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:31.965354 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:31.953368 16941 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:31.945181 16883 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:32.002247 10676 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:31.978461  9946 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:26:32.016191 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:31.999320 17062 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:32.033710  9945 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:32.033707 10097 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:26:32.020265 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:32.020654 10094 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:31.998247 16972 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.037010  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:32.037413 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:31.981760 10092 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:31.982568 16861 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:32.030274 10096 raft_consensus.cc:2376] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17. Re-sending same reply.
I20260709 12:26:31.945446 16687 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.039639 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.996851 16969 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:31.978384 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:32.032133 16661 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.044593 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:31.987658 16690 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:31.984577 16988 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:31.961854 17036 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.046056  9945 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:32.033708 17037 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.087s	user 0.009s	sys 0.000s
I20260709 12:26:32.044327 10024 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:31.988390 16910 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:31.985520  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:32.059499  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74076.6us
I20260709 12:26:32.043032 17020 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:32.051363 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.040491 16861 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:32.061259  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:32.059994 17037 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.113s	user 0.010s	sys 0.000s
I20260709 12:26:32.048630 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:32.072152 10096 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:32.061787 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.045640 16798 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:32.067639 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.069633 10093 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:32.070009  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:32.029630 16800 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.074954 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.077184  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:32.063140 17041 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:32.078042 16798 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.081632 16798 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.081877 16687 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:32.074776 16717 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
W20260709 12:26:32.089776  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.071321 17020 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:32.074057  9945 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:26:32.082720 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.084530 10024 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:32.086656 16881 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:32.072803 16969 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:32.087479  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:32.074414 10018 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:32.097414 10092 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:32.098555  9942 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:26:32.088254 17036 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:32.099431 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:32.100052 10095 raft_consensus.cc:2376] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 18. Re-sending same reply.
I20260709 12:26:32.077792 16800 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:32.104179 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:32.104934 10020 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:32.103273  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:32.109740 17020 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.101154 16687 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:32.118921 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.121083 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.121887 10092 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 12:26:32.123761  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.115077 16739 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.129144 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.131770 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.132408 10095 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 15.
I20260709 12:26:32.115406 16687 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:32.123087 16888 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:32.135337 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.139219 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.139592 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.139927 10095 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:32.144902 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 1268) took 1007 ms. Trace:
I20260709 12:26:32.139230  9946 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:26:32.148159 10672 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:32.145465 10675 rpcz_store.cc:276] 0709 12:26:31.137227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.270364 (+133137us) service_pool.cc:224] Handling call
0709 12:26:32.144618 (+874254us) spinlock_profiling.cc:243] Waited 582 ms on lock 0x7b54005b95d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:32.144812 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1337848064}
I20260709 12:26:32.149080  9946 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:26:32.136513 16687 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.151676 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.151690 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.153288 10095 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:32.136219 16888 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:32.155113  9946 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:32.102394 16861 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.156032 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:32.156926 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.156987 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2636) took 2896 ms. Trace:
I20260709 12:26:32.159705 10091 rpcz_store.cc:276] 0709 12:26:29.261042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.552885 (+291843us) service_pool.cc:224] Handling call
0709 12:26:29.553290 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.264262 (+710972us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.264293 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.287158 (+ 22865us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.437466 (+150308us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:30.437505 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.150003 (+1712498us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b540014f0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:32.150076 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.150159 (+    83us) raft_consensus.cc:1685] finished
0709 12:26:32.150174 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.156926 (+  6752us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":84,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":16710,"spinlock_wait_cycles":1634054912,"thread_start_us":94594,"threads_started":1,"wal-append.queue_time_us":142272,"wal-append.run_cpu_time_us":2320,"wal-append.run_wall_time_us":1055554}
I20260709 12:26:32.164215  9946 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:32.165515 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.166855 10018 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:32.131209 16739 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.175143 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:32.178426  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.179337 10091 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:32.181147 10092 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:26:32.163817 17065 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:26:32.180429 16910 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:32.190954  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.182713 16861 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:32.194922 10092 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:32.195905 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.196190 10092 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:32.196579 10020 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
W20260709 12:26:32.197391 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.287569  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.232494 16888 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.197376 10092 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:32.284309 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.286727 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.287123 16837 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:32.295849 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.298660 10095 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:32.290977 17020 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:32.300063 17020 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.302886 16739 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.303330 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2409) took 1201 ms. Trace:
I20260709 12:26:32.303664 10022 rpcz_store.cc:276] 0709 12:26:31.119158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.130577 (+ 11419us) service_pool.cc:224] Handling call
0709 12:26:31.130898 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.340493 (+209595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:31.340547 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:31.340614 (+    67us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:31.340667 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.303090 (+962423us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540011b054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:32.303135 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.303276 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481328000}
I20260709 12:26:32.305032 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.306102 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.306569 17020 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:32.226814 16772 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:26:32.307410 16772 consensus_queue.cc:1243] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:32.313575  9948 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:32.290527 17023 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.319975 17023 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.322191 17023 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.290688 17041 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:32.324527 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.324994 17041 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.327235 17041 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.328254  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:32.328994 16837 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.331264 16837 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:32.331926  9949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1504) took 1476 ms (client timeout 1677 ms). Trace:
W20260709 12:26:32.332548  9949 rpcz_store.cc:269] 0709 12:26:30.855962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.858078 (+  2116us) service_pool.cc:224] Handling call
0709 12:26:32.331669 (+1473591us) spinlock_profiling.cc:243] Waited 838 ms on lock 0x7b540018a6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:32.331837 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1928307968}
W20260709 12:26:32.291821 17071 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.090s	user 0.004s	sys 0.000s
I20260709 12:26:32.338032 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.290261 10091 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:26:32.340466 10673 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
W20260709 12:26:32.293418 17070 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.149s	user 0.001s	sys 0.002s
I20260709 12:26:32.336591  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:32.341980 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.337641 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.342581 10673 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:32.340646  9948 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:26:32.335691 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:32.343628 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.343669 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.344255 10092 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:32.344455 10673 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:32.310073 16941 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.347540 10669 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:32.346911 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:32.347930 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.348414 10022 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:32.350154 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.350987 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:32.351537 10670 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.333709 16888 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.341734 17041 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:32.357931 10095 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:32.359807 10021 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:26:32.361680 16959 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.519s	user 0.005s	sys 0.008s
I20260709 12:26:32.366307 16661 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.347515 16837 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:32.373066 10022 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:32.383771 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.376355 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.372835 10670 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:32.395802  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1519) took 1120 ms. Trace:
I20260709 12:26:32.396409  9944 rpcz_store.cc:276] 0709 12:26:31.275681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.038186 (+762505us) service_pool.cc:224] Handling call
0709 12:26:32.050125 (+ 11939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.050456 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.050498 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.050981 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.109046 (+ 58065us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:32.109084 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.109146 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.395511 (+286365us) raft_consensus.cc:1685] finished
0709 12:26:32.395541 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.395721 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3918,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":3811,"thread_start_us":57757,"threads_started":1,"wal-append.queue_time_us":79585}
I20260709 12:26:32.367918 16874 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:32.400306  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:32.377254 16837 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.401226  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2492) took 1355 ms. Trace:
I20260709 12:26:32.378695 17041 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:32.403262  9941 rpcz_store.cc:276] 0709 12:26:31.046263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.047632 (+  1369us) service_pool.cc:224] Handling call
0709 12:26:32.400147 (+1352515us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:26:32.401169 (+  1022us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3080960}
I20260709 12:26:32.405103 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.404201 16837 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.402777 16800 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.375015 16959 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.415520 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.420028 10675 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:32.420485 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.421133 10020 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:32.421541 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1269) took 1239 ms. Trace:
I20260709 12:26:32.421800 10020 rpcz_store.cc:276] 0709 12:26:31.182514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.409306 (+1226792us) service_pool.cc:224] Handling call
0709 12:26:32.421498 (+ 12192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.417577 10673 raft_consensus.cc:1217] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Deduplicated request from leader. Original: 16.12->[17.13-18.14]   Dedup: 17.13->[18.14-18.14]
I20260709 12:26:32.422948 10673 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: [17.13-18.14]
I20260709 12:26:32.423558  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:32.423902 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.424533 10673 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:32.425181 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.425693 10673 raft_consensus.cc:2410] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:26:32.426546 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.426784  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:32.426616 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.428901 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:32.429663 10675 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:32.430186 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.435494 10099 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:32.436681 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.438283 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:32.440227 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.440241 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:32.408420 16874 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.444550 16807 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:32.445045 16807 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:32.445453 16807 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.447607 16941 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.448328 16941 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:32.448701 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.449631 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3603) took 1011 ms. Trace:
I20260709 12:26:32.449962 10021 rpcz_store.cc:276] 0709 12:26:31.437875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.448433 (+1010558us) service_pool.cc:224] Handling call
0709 12:26:32.449571 (+  1138us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.443315 16969 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:32.451261 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.452584 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.453447 10020 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:26:32.444567 17022 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.010s
I20260709 12:26:32.455686 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3591) took 3310 ms. Trace:
I20260709 12:26:32.456187 16874 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.435236 16699 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:26:32.454308 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:32.458228 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.456072 10016 rpcz_store.cc:276] 0709 12:26:29.145498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.568671 (+1423173us) service_pool.cc:224] Handling call
0709 12:26:30.589853 (+ 21182us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:30.670643 (+ 80790us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.701885 (+ 31242us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:30.707256 (+  5371us) log.cc:844] Serialized 52 byte log entry
0709 12:26:30.899221 (+191965us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:30.911006 (+ 11785us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.932620 (+ 21614us) log.cc:844] Serialized 14 byte log entry
0709 12:26:30.933056 (+   436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:30.933154 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:31.315249 (+382095us) spinlock_profiling.cc:243] Waited 1.49 ms on lock 0x7b180038dee0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:31.983076 (+667827us) spinlock_profiling.cc:243] Waited 7.79 ms on lock 0x7b180038dee0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:32.358469 (+375393us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b180038dee0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:32.455481 (+ 97012us) raft_consensus.cc:1685] finished
0709 12:26:32.455501 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.455630 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14751,"prepare.queue_time_us":67236,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":1879,"raft.queue_time_us":47956,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":299,"spinlock_wait_cycles":31374976,"thread_start_us":245359,"threads_started":2,"wal-append.queue_time_us":215109}
I20260709 12:26:32.460208 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3600) took 1023 ms. Trace:
I20260709 12:26:32.460697 10021 rpcz_store.cc:276] 0709 12:26:31.437176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.457983 (+1020807us) service_pool.cc:224] Handling call
0709 12:26:32.460157 (+  2174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.461879 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:32.463037 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.464210 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.456511 17041 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.465620 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.466341  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:32.467103  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2488) took 1423 ms. Trace:
I20260709 12:26:32.467414  9947 rpcz_store.cc:276] 0709 12:26:31.043376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.044105 (+   729us) service_pool.cc:224] Handling call
0709 12:26:32.466177 (+1422072us) spinlock_profiling.cc:243] Waited 3.91 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:26:32.467045 (+   868us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8998784}
I20260709 12:26:32.468101 16920 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.469239  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:32.469861  9940 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
W20260709 12:26:32.470309  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2490) took 1424 ms (client timeout 1645 ms). Trace:
W20260709 12:26:32.470674  9940 rpcz_store.cc:269] 0709 12:26:31.045671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.047223 (+  1552us) service_pool.cc:224] Handling call
0709 12:26:32.469092 (+1421869us) spinlock_profiling.cc:243] Waited 1.67 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:26:32.470248 (+  1156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3851136}
W20260709 12:26:32.471638  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.421166 16800 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.476295 10020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:26:32.477157 10020 raft_consensus.cc:606] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:32.477538 10020 raft_consensus.cc:614] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:32.427990 17056 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:26:32.478477 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.481099 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3606) took 1042 ms. Trace:
I20260709 12:26:32.481561 10016 rpcz_store.cc:276] 0709 12:26:31.438550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.460671 (+1022121us) service_pool.cc:224] Handling call
0709 12:26:32.481020 (+ 20349us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.460119 16910 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:32.459293 16710 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.486193 10021 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:26:32.488375 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1258) took 1337 ms. Trace:
I20260709 12:26:32.488870 10021 rpcz_store.cc:276] 0709 12:26:31.150895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.478574 (+1327679us) service_pool.cc:224] Handling call
0709 12:26:32.479019 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.488038 (+  9019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.488319 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.489670 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59744 (request call id 38) took 1346 ms. Trace:
I20260709 12:26:32.490195 10020 rpcz_store.cc:276] 0709 12:26:31.132214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.465090 (+1332876us) service_pool.cc:224] Handling call
0709 12:26:32.478060 (+ 12970us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.490984 10021 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: [18.13-20.14]
I20260709 12:26:32.491873 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2411) took 1322 ms. Trace:
I20260709 12:26:32.492266 10021 rpcz_store.cc:276] 0709 12:26:31.169421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.490142 (+1320721us) service_pool.cc:224] Handling call
0709 12:26:32.490651 (+   509us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:32.491519 (+   868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.491805 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.495903 16710 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.477710 16661 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:32.499907 10090 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:32.505442 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2404) took 1421 ms. Trace:
I20260709 12:26:32.505893 10019 rpcz_store.cc:276] 0709 12:26:31.083728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.084265 (+   537us) service_pool.cc:224] Handling call
0709 12:26:31.838684 (+754419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:31.863482 (+ 24798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.505233 (+641751us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b5400137ed0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:32.505379 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1734317824}
I20260709 12:26:32.506824 10016 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.507386 10019 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:32.507414 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2410) took 1355 ms. Trace:
I20260709 12:26:32.508103 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2413) took 1276 ms. Trace:
I20260709 12:26:32.508219 10016 rpcz_store.cc:276] 0709 12:26:31.152502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.482427 (+1329925us) service_pool.cc:224] Handling call
0709 12:26:32.506661 (+ 24234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.507191 (+   530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.507366 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.508489 10019 rpcz_store.cc:276] 0709 12:26:31.232000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.506856 (+1274856us) service_pool.cc:224] Handling call
0709 12:26:32.507105 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.507895 (+   790us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.508059 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.509967 10019 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:32.511667 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:32.512313 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1497) took 1227 ms. Trace:
I20260709 12:26:32.514606 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3579) took 3363 ms. Trace:
I20260709 12:26:32.512703 10672 rpcz_store.cc:276] 0709 12:26:31.284882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.307462 (+ 22580us) service_pool.cc:224] Handling call
0709 12:26:32.147078 (+839616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.150475 (+  3397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.512129 (+361654us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54005b95d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:32.512255 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1930239360}
I20260709 12:26:32.517750 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2419) took 1183 ms. Trace:
I20260709 12:26:32.518209 10019 rpcz_store.cc:276] 0709 12:26:31.334072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.509362 (+1175290us) service_pool.cc:224] Handling call
0709 12:26:32.509725 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.510420 (+   695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.517678 (+  7258us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.512552 10095 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:32.515174 10025 rpcz_store.cc:276] 0709 12:26:29.140745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:29.142687 (+  1942us) service_pool.cc:224] Handling call
0709 12:26:29.143066 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.735055 (+1591989us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.735103 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:30.735703 (+   600us) log.cc:844] Serialized 26 byte log entry
0709 12:26:30.904928 (+169225us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:31.166737 (+261809us) log.cc:844] Serialized 14 byte log entry
0709 12:26:31.167001 (+   264us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:31.999068 (+832067us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b18002925a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:32.324503 (+325435us) spinlock_profiling.cc:243] Waited 3.93 ms on lock 0x7b18002925a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419
0709 12:26:32.503242 (+178739us) spinlock_profiling.cc:243] Waited 1.59 s on lock 0x7b54002666d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:32.503310 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.503391 (+    81us) raft_consensus.cc:1685] finished
0709 12:26:32.503411 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.503605 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5805,"prepare.queue_time_us":28859,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":3204,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":7173,"spinlock_wait_cycles":3688457728,"thread_start_us":369770,"threads_started":2,"wal-append.queue_time_us":211769}
I20260709 12:26:32.521448 16961 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:32.495198 16910 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.527401 16959 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:32.530110 16837 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.531105 10093 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:32.535815 10025 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:32.536780 10016 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:26:32.543247 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1275) took 1176 ms. Trace:
W20260709 12:26:32.542665 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 100295us
I20260709 12:26:32.544312 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2418) took 1210 ms. Trace:
I20260709 12:26:32.544683 10025 rpcz_store.cc:276] 0709 12:26:31.366946 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.535059 (+1168113us) service_pool.cc:224] Handling call
0709 12:26:32.535411 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.543060 (+  7649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.543206 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.544859 10016 rpcz_store.cc:276] 0709 12:26:31.333718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.509143 (+1175425us) service_pool.cc:224] Handling call
0709 12:26:32.536561 (+ 27418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.544072 (+  7511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.544253 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.546079 10025 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:32.546667 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3595) took 1110 ms. Trace:
I20260709 12:26:32.547047 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:32.538920 16961 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:32.546972 10025 rpcz_store.cc:276] 0709 12:26:31.436043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.545601 (+1109558us) service_pool.cc:224] Handling call
0709 12:26:32.545917 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.546429 (+   512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.546619 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.506075 16687 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.555997 10022 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:32.537364 10019 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:26:32.556226 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:32.560568 10640 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:32.561898 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:32.559525 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3594) took 1123 ms. Trace:
I20260709 12:26:32.564009 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:32.565635 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1486) took 1501 ms. Trace:
I20260709 12:26:32.566190 10677 rpcz_store.cc:276] 0709 12:26:31.063980 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.064151 (+   171us) service_pool.cc:224] Handling call
0709 12:26:32.565348 (+1501197us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b540073a354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:32.565544 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1177971968}
I20260709 12:26:32.549664 16873 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:32.578915 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.579243 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.580633 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.584718 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:32.580861 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.585321 10675 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:32.586022 10018 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:32.586395 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.587301 10675 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:32.589195 10671 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:32.570631 16661 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:32.593537 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.589780  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:32.594461 10023 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
W20260709 12:26:32.595273 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2415) took 1312 ms (client timeout 1628 ms). Trace:
W20260709 12:26:32.595813 10023 rpcz_store.cc:269] 0709 12:26:31.282737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.565377 (+1282640us) service_pool.cc:224] Handling call
0709 12:26:32.595183 (+ 29806us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.597456 10023 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:32.558130 10025 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [19.12-21.13]
I20260709 12:26:32.598682 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3607) took 1151 ms. Trace:
I20260709 12:26:32.599040 10025 rpcz_store.cc:276] 0709 12:26:31.446948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.556712 (+1109764us) service_pool.cc:224] Handling call
0709 12:26:32.557892 (+  1180us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:32.598375 (+ 40483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.598628 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.599411 10670 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:26:32.599985 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.601089 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.602802 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3608) took 1155 ms. Trace:
I20260709 12:26:32.603561  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:32.606356 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.603338 10023 rpcz_store.cc:276] 0709 12:26:31.447217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.596694 (+1149477us) service_pool.cc:224] Handling call
0709 12:26:32.597098 (+   404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.597906 (+   808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.602728 (+  4822us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.563503 10019 rpcz_store.cc:276] 0709 12:26:31.435827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.536795 (+1100968us) service_pool.cc:224] Handling call
0709 12:26:32.537090 (+   295us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.558969 (+ 21879us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.559460 (+   491us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:26:32.608193  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.608238 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1427) took 1177 ms. Trace:
I20260709 12:26:32.609135 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3610) took 1137 ms. Trace:
I20260709 12:26:32.609357 10676 rpcz_store.cc:276] 0709 12:26:31.431269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.708137 (+276868us) service_pool.cc:224] Handling call
0709 12:26:32.001987 (+293850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.031719 (+ 29732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.608059 (+576340us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54006ff750. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:32.608185 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627218176}
I20260709 12:26:32.619874 10019 rpcz_store.cc:276] 0709 12:26:31.471814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.608354 (+1136540us) service_pool.cc:224] Handling call
0709 12:26:32.608630 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.608818 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.608859 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:32.608904 (+    45us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:32.608941 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.608995 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.609085 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:32.607334  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:32.625113  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:32.585170 16944 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:32.675745  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:32.628688 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.614585 10672 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:32.588968 16993 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:32.617085 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.680509 10671 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:32.611405  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:32.681993  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 70685.4us
I20260709 12:26:32.683265  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:32.627255  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:32.611196 10023 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.615252 16946 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:32.659302 16739 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:32.659894 17020 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:32.662272  9941 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:32.612666 16874 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:32.663278 16910 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.588932 16836 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:32.591559 16800 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:32.607304 16798 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:32.665783 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.679942 16993 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.666622 17028 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.713793 10094 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 12:26:32.668143 16837 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.697302 16760 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:32.669119 16855 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.669919 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:32.672837 17040 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:32.673496 16952 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.584434 16873 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:32.677338  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:32.724606  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47371us
I20260709 12:26:32.724925 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:32.725306  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:32.678088  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.683741  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:32.685073 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:32.726017 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 141394us
W20260709 12:26:32.726308  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.690797 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:32.727036  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:32.694095 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.612180 17033 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:32.697613 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.728334 10676 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:32.615664 16861 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:32.697177 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.676110 16944 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:32.703346  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.601835 16920 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.102s
I20260709 12:26:32.705118 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:32.687374 17056 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.666064 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:32.733305 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67323.6us
I20260709 12:26:32.687912 17041 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:32.714387 10094 pending_rounds.cc:85] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 13
I20260709 12:26:32.740010 10094 pending_rounds.cc:107] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:26:32.740298 10094 raft_consensus.cc:2889] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:32.726809 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:32.714416 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:32.743095 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:32.707664 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:32.744998 10091 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:32.702432 17037 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:32.668226 16950 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:32.721629 10092 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:32.722154 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:32.722574 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:32.700766 16874 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:32.703663 17022 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:32.693308  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66172.7us
I20260709 12:26:32.730010 10671 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:32.716418 16993 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.781407 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:32.690356 16739 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:32.738997 16944 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:32.748487  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 304423us
I20260709 12:26:32.687816 16961 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.727080 16873 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.763798  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:32.796277 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.798961 10676 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:32.715435 16798 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.803287 16798 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:32.729236 17033 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:32.719429 16910 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:26:32.806744 16910 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:32.807264 16910 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:32.808482 16910 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.729962 16917 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:32.810904 16961 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:32.731278 16837 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:32.731238 16800 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:32.752240 16956 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:32.794800 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:32.747799 16898 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:32.814623 16898 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.814906 16687 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.815181 16898 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:32.815994 17040 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:32.760918 16992 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.725378 16855 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.825202 16855 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:32.772833 17022 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:32.826110 17022 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:32.804217 10019 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:26:32.826843 10019 pending_rounds.cc:85] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 9
I20260709 12:26:32.811014 16917 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:32.827010  9843 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 331) took 42968 ms (client timeout 15000 ms). Trace:
I20260709 12:26:32.827560 16917 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:32.761126 16969 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.080s
I20260709 12:26:32.753379 16986 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:32.832880  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2483) took 3656 ms. Trace:
I20260709 12:26:32.833484 16986 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.833428  9943 rpcz_store.cc:276] 0709 12:26:29.176561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:30.409119 (+1232558us) service_pool.cc:224] Handling call
0709 12:26:30.422585 (+ 13466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:30.570346 (+147761us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:30.570380 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.543754 (+1973374us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b2c00012928. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:26:32.548372 (+  4618us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.744611 (+196239us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:32.744644 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.744723 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.832490 (+ 87767us) raft_consensus.cc:1685] finished
0709 12:26:32.832512 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.832787 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":244,"raft.queue_time_us":78642,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":13442,"spinlock_wait_cycles":8384384,"thread_start_us":331718,"threads_started":2,"wal-append.queue_time_us":234974}
I20260709 12:26:32.836416 16986 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:32.744220 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:26:32.843262 17086 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
W20260709 12:26:32.843897 17085 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
W20260709 12:26:32.843240 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 99099.5us
I20260709 12:26:32.827149 10019 pending_rounds.cc:107] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:26:32.847208 10019 raft_consensus.cc:2889] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:32.848517 16881 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:32.849123 16881 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.849547 16881 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:32.778108 10671 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:32.810253 16910 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.772141 16874 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:32.780769 16861 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:32.782562 17025 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:32.790450 16739 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:32.827632  9843 rpcz_store.cc:269] 0709 12:25:49.824114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:49.824322 (+   208us) service_pool.cc:224] Handling call
0709 12:26:32.792818 (+42968496us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:24.688764 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:24.741009 (+ 52245us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:24.741073 (+    64us) write_op.cc:435] Acquired schema lock
0709 12:26:24.741093 (+    20us) tablet.cc:662] Decoding operations
0709 12:26:24.742151 (+  1058us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:24.742258 (+   107us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:24.742300 (+    42us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:26:24.743088 (+   788us) tablet.cc:701] Row locks acquired
0709 12:26:24.743153 (+    65us) write_op.cc:260] PREPARE: finished
0709 12:26:24.743326 (+   173us) write_op.cc:270] Start()
0709 12:26:24.743469 (+   143us) write_op.cc:276] Timestamp: P: 1783599984743258 usec, L: 0
0709 12:26:24.743502 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:26:24.846316 (+102814us) log.cc:844] Serialized 3223 byte log entry
0709 12:26:25.746352 (+900036us) op_driver.cc:464] REPLICATION: finished
0709 12:26:26.004615 (+258263us) write_op.cc:301] APPLY: starting
0709 12:26:26.004781 (+   166us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:26.255446 (+250665us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:26.255580 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:26.642444 (+386864us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:26.657669 (+ 15225us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=16,mrs_lookups=0
0709 12:26:26.657739 (+    70us) write_op.cc:312] APPLY: finished
0709 12:26:26.658136 (+   397us) log.cc:844] Serialized 81 byte log entry
0709 12:26:26.658347 (+   211us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:26.658482 (+   135us) write_op.cc:454] Released schema lock
0709 12:26:26.658552 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:26.659052 (+   500us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:26.659083 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5413214,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":189661,"apply.run_cpu_time_us":17415,"apply.run_wall_time_us":706882,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37583,"num_ops":8,"prepare.queue_time_us":619089,"prepare.run_cpu_time_us":46438,"prepare.run_wall_time_us":619664,"raft.queue_time_us":171107,"raft.run_cpu_time_us":29251,"raft.run_wall_time_us":132502,"replication_time_us":1002693,"thread_start_us":925351,"threads_started":4,"wal-append.queue_time_us":662925,"wal-append.run_cpu_time_us":20838,"wal-append.run_wall_time_us":2112055}]]}
I20260709 12:26:32.902738 16861 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:32.903539 16969 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:32.904148 16969 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:32.903854 16861 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:32.905179 16969 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:32.907114  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 143477us
I20260709 12:26:32.907275 17020 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:32.908195 17020 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:32.909855  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:32.910923 17084 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:32.911523 17084 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.911973 17084 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:32.858685 16855 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:32.915004 16855 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.916680 16969 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:32.917128 16855 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:32.890046 17025 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:32.925196 17025 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:32.925595  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:32.920733 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2417) took 1587 ms. Trace:
I20260709 12:26:32.927717 16855 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:32.928452 16855 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.928005 10020 rpcz_store.cc:276] 0709 12:26:31.333389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.508048 (+1174659us) service_pool.cc:224] Handling call
0709 12:26:32.531842 (+ 23794us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:32.532259 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:32.532301 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:32.540060 (+  7759us) log.cc:844] Serialized 26 byte log entry
0709 12:26:32.721843 (+181783us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:32.721882 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:32.721949 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:32.920500 (+198551us) raft_consensus.cc:1685] finished
0709 12:26:32.920524 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:32.920668 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":563,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":4449,"thread_start_us":166991,"threads_started":1,"wal-append.queue_time_us":314209}
I20260709 12:26:32.922986 17041 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:32.779508 16959 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 12:26:32.931648 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:32.933293 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:32.937861 10020 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:32.939544  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:32.963893 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.935761 17041 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:32.885571 16661 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:32.973366 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 269057us
I20260709 12:26:32.982489  9940 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:32.998152  9943 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:26:32.918045 16873 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:32.985107  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:32.995683  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.001215  9947 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:33.001659  9949 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:32.991319  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 51905.6us
I20260709 12:26:33.003404  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:32.978104 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:33.007030  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:32.970366 16807 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.001s
I20260709 12:26:33.006541  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:32.974138 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 239524us
W20260709 12:26:33.010326  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 223267us
W20260709 12:26:32.999348 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:32.987697  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:32.990265  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:33.011858  9948 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:33.013213 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:33.013768  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.013816 10674 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.014477  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:33.012015  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 267283us
I20260709 12:26:33.016407  9944 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:32.893723 16798 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.111s
W20260709 12:26:33.019295  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 228667us
W20260709 12:26:33.019758  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 294524us
I20260709 12:26:33.020500  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.025918  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:33.028688  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:33.023191  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:33.030539  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.032618 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.032447  9942 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 15.
I20260709 12:26:33.031939  9949 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:33.022332  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:33.041841  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.047132  9949 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:33.051656  9944 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:33.051911 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.056133  9949 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:33.039970 17056 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:33.068939 10674 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:33.071158  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.076017  9949 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:33.079954  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.083691  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.020227 16807 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.091092 10017 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:33.094300 16873 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:33.094753 16710 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.095315 16710 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:33.096958 17084 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.099071 17028 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:33.099722 16633 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.102892  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.105405 16986 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:33.105806 16986 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:33.106988 16699 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:33.107648 16699 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:33.108232 16699 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:33.109247 16986 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.109128 17033 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.112205 16760 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:33.115540 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2433) took 1045 ms. Trace:
I20260709 12:26:33.115962 10018 rpcz_store.cc:276] 0709 12:26:32.070255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.292428 (+222173us) service_pool.cc:224] Handling call
0709 12:26:33.115355 (+822927us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54001388d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:33.115474 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":558892800}
I20260709 12:26:33.117508 10018 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:33.119135 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3625) took 1044 ms. Trace:
I20260709 12:26:33.119503 17022 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.119935 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2646) took 1266 ms. Trace:
I20260709 12:26:33.120422 10097 rpcz_store.cc:276] 0709 12:26:31.853804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.891914 (+ 38110us) service_pool.cc:224] Handling call
0709 12:26:33.119720 (+1227806us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54003ae3d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:33.119874 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":323464960}
I20260709 12:26:33.119536 10022 rpcz_store.cc:276] 0709 12:26:32.074246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.383418 (+309172us) service_pool.cc:224] Handling call
0709 12:26:33.118930 (+735512us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54002170d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:33.119069 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375909120}
W20260709 12:26:33.122582 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3601) took 1685 ms (client timeout 2004 ms). Trace:
W20260709 12:26:33.123159 10024 rpcz_store.cc:269] 0709 12:26:31.437398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.056193 (+618795us) service_pool.cc:224] Handling call
0709 12:26:33.122417 (+1066224us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b54002d5354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:33.122524 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25059584}
I20260709 12:26:33.124313 10022 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:26:33.121677 17022 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.127231 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1273) took 1917 ms. Trace:
I20260709 12:26:33.127780 10021 rpcz_store.cc:276] 0709 12:26:31.209640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.496334 (+1286694us) service_pool.cc:224] Handling call
0709 12:26:33.125468 (+629134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.125720 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.125760 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.125818 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:33.125863 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.125935 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.127004 (+  1069us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b54002d5350. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:33.127153 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1445943680}
I20260709 12:26:33.129695 17099 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:33.130134 17100 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:33.130785 16986 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:33.131078 17100 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:33.131299 10016 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 20. Ops: []
I20260709 12:26:33.131595 17100 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:33.132611 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3596) took 1696 ms. Trace:
I20260709 12:26:33.132512 17100 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.132994 10016 rpcz_store.cc:276] 0709 12:26:31.436264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.545715 (+1109451us) service_pool.cc:224] Handling call
0709 12:26:33.131137 (+585422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.131808 (+   671us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.132422 (+   614us) spinlock_profiling.cc:243] Waited 572 ms on lock 0x7b54002170d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:33.132554 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1316256640}
I20260709 12:26:33.133956 17056 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:33.124962 10024 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:33.081573 16920 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:33.140858  9947 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:33.092747 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:33.084568 16855 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.146697 10673 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:33.149154 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56645.7us
I20260709 12:26:33.147048  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:33.151772 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.090294 17018 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:33.152606 17018 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:33.092065 16717 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:33.144084 16920 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:33.164518 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.177110  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:33.147315 17005 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.175879  9942 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:33.154162 16992 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:33.080051 16661 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.180888 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 3818) took 1210 ms. Trace:
I20260709 12:26:33.155390 17033 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.181720 10099 rpcz_store.cc:276] 0709 12:26:31.970904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.037587 (+ 66683us) service_pool.cc:224] Handling call
0709 12:26:33.180677 (+1143090us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b540028fd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:33.180819 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893536128}
I20260709 12:26:33.154944 16950 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:33.185568  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 161222us
I20260709 12:26:33.202571  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:33.205734  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:33.181380  9947 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
W20260709 12:26:33.184291  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 158457us
I20260709 12:26:33.207211  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:33.141549 17093 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.041s
W20260709 12:26:33.184861  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 160105us
I20260709 12:26:33.169266  9949 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
W20260709 12:26:33.186347  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 157775us
I20260709 12:26:33.211948  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:33.212950  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:33.186930 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 329745us
I20260709 12:26:33.218370  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:33.220294 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:33.221803  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.179075 16855 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.228193 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:33.228145  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:33.182713 17005 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.155725 16898 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.238235  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.242420  9944 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: [16.10-18.11]
I20260709 12:26:33.244539  9945 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:33.245855 10674 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:33.232152 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:33.230916 16633 raft_consensus.cc:2764] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:26:33.251214  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.223760 17020 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:33.263312  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 241115us
I20260709 12:26:33.239457 17084 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.155102 17018 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:33.217356 16760 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:33.250473 10024 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:33.253080 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:33.269963  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:33.224035 16739 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.261289 16710 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:26:33.252031  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.250484 17102 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.281023  9949 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:33.265709 10093 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:33.259039  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:33.265080 10668 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:33.273602 16807 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:33.239041 16994 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 12:26:33.268415 16760 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.217935 16874 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.278920 16739 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.274487 10020 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:26:33.305393 10020 pending_rounds.cc:85] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 13
I20260709 12:26:33.305802 10020 pending_rounds.cc:107] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:26:33.247469 16873 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:26:33.306099 10020 raft_consensus.cc:2889] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:33.274824 16898 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.257328 16950 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:33.262768 17020 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.299680  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:33.337450  9947 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:33.276714 17109 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:33.239876 16855 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:33.322165  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52275.1us
I20260709 12:26:33.339296  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:33.253978 17040 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:26:33.343892  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:33.263792 16881 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.253196 17091 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 12:26:33.250054 17103 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.083s
I20260709 12:26:33.299005 16760 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:33.344780 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:33.356838 10016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:26:33.357560 10016 raft_consensus.cc:606] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:33.358045 10016 raft_consensus.cc:614] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:33.246933 16920 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:33.279803 16969 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 12:26:33.345342 17018 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:33.341604 17020 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:33.341203 17108 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:33.389650  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:33.398748  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.299943 17102 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.382575 17056 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:33.334908 17107 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:33.345191 16944 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.346535 17093 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:33.425940 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 193424us
I20260709 12:26:33.386554 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:33.395869 16917 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.568s	user 0.008s	sys 0.003s
W20260709 12:26:33.421689  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 209800us
I20260709 12:26:33.390508 17092 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:26:33.385746 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 135490us
I20260709 12:26:33.430559  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:33.353381 17025 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.400388 16898 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:26:33.431465 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:33.513984  9949 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:33.525810 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.439812 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:33.458236 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1278) took 1166 ms. Trace:
I20260709 12:26:33.530897 10018 rpcz_store.cc:276] 0709 12:26:32.291433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.118275 (+826842us) service_pool.cc:224] Handling call
0709 12:26:33.134992 (+ 16717us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.135386 (+   394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.135419 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.136272 (+   853us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.351098 (+214826us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:33.377039 (+ 25941us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.421081 (+ 44042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.421186 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:33.457958 (+ 36772us) raft_consensus.cc:1685] finished
0709 12:26:33.457988 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.458161 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43425,"prepare.queue_time_us":25681,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":3817,"raft.queue_time_us":50889,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":6016,"spinlock_wait_cycles":581888,"thread_start_us":214529,"threads_started":1,"wal-append.queue_time_us":315541}
I20260709 12:26:33.441166  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:33.556226  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 115130us
W20260709 12:26:33.442853 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62329.9us
W20260709 12:26:33.518463 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 87090.2us
I20260709 12:26:33.594698 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:33.596510 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:33.611132  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.562968 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:33.628921 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 66027.6us
W20260709 12:26:33.632436 16986 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
W20260709 12:26:33.637036 16993 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.844s	user 0.009s	sys 0.000s
I20260709 12:26:33.637533 16993 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.638767 10671 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:33.446029 17022 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.195s
I20260709 12:26:33.430231 17108 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.425485 17107 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:33.647131  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 216642us
I20260709 12:26:33.648375  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:33.455250 17100 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.456344 16881 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.442344 17092 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:33.595096 17103 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:33.614204 16952 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:33.447486 17091 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:33.432181 17097 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:33.433250 17056 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:33.446710 17109 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.243s
I20260709 12:26:33.684129 17101 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:33.686832 17093 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:33.435431 16917 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.684494 16363 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:33.691186 16874 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.617105 16955 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 12:26:33.685132 17107 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:33.686776 17108 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:33.699940 16952 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:33.767522 16952 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:33.706497 16881 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:33.714515 17109 consensus_queue.cc:1243] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:33.715687 16855 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:33.721974 17093 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:33.776479 17093 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:33.697999 10023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.045s	sys 0.038s
W20260709 12:26:33.779407 10023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.045s	sys 0.039s
I20260709 12:26:33.724042 17101 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.780735 17104 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:33.781934 17101 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.739866  9946 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:33.738308 17106 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:33.733088 16944 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:33.741279 17092 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.746021 17056 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.748553 16993 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:33.751890 17094 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:33.752161 17102 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:33.753413 17108 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:33.725710 17103 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:33.808892 17103 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:33.809369 17103 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:33.811509 17104 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:33.763092  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:33.762306  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:33.803769 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2426) took 1776 ms. Trace:
W20260709 12:26:33.756166 16910 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.897s	user 0.007s	sys 0.004s
I20260709 12:26:33.693555 16760 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:33.819267 10023 rpcz_store.cc:276] 0709 12:26:32.027648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.726807 (+699159us) service_pool.cc:224] Handling call
0709 12:26:32.727085 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.727292 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:33.054120 (+326828us) log.cc:844] Serialized 14 byte log entry
0709 12:26:33.780374 (+726254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:33.780451 (+    77us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:33.780522 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.803494 (+ 22972us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:33.803692 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":429293,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":181,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":102,"thread_start_us":1040050,"threads_started":2,"wal-append.queue_time_us":683369}
W20260709 12:26:33.851693  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 203416us
I20260709 12:26:33.630236 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:33.857803 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 227662us
W20260709 12:26:33.861430  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:33.864878 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 268447us
W20260709 12:26:33.904788 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:33.867543 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:33.749228 17025 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:33.965858 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.045s	sys 0.049s
I20260709 12:26:33.966450 16855 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:33.846371 17018 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:33.983776 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.046s	sys 0.050s
I20260709 12:26:33.726279 16363 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:34.040086 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 731603us
I20260709 12:26:34.044325 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:34.046446 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:33.948359 16955 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:33.747026 16950 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:34.060900 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:33.958680 17121 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.067827 17025 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:34.045979 16881 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:33.958495 16807 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.110s
I20260709 12:26:33.854189 16952 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.046914 16363 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:34.085924 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:34.114259 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.115039 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.117460 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.109588 17033 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.899s	user 0.012s	sys 0.000s
I20260709 12:26:34.107007 16910 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:34.117520 16993 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:34.060607 17018 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.134207 16993 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:34.161424 16993 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.137595 16873 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:26:34.106376 17121 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.163753 16873 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:34.164199 16873 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:34.165055 17121 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.133124 16910 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.165171 16873 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.166019 16910 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:34.166986 16873 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:34.173292 17018 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.176002 16993 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:34.162918 17132 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:26:34.179975 16917 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:34.181236 17130 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:26:34.195570 16760 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:34.199492 17133 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:34.118891 17025 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:34.216893 16952 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.229008 16950 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:34.233669 17020 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.834s	user 0.008s	sys 0.005s
I20260709 12:26:34.234912 17020 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.186432 17103 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:34.276811 17107 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.537s	user 0.009s	sys 0.001s
I20260709 12:26:34.277776 17107 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.248878 17025 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:34.139098 16955 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:34.341668 17100 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.648s	user 0.009s	sys 0.003s
W20260709 12:26:34.360726 17127 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:34.242398 17093 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.365317 17025 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.369961 16955 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.378286 17103 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.383205 17020 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.385008 16910 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.388512 16910 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:33.382153  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 163882us
W20260709 12:26:34.573138  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:34.573733  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.573478  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:34.469882  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.579437 16760 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.580490 16760 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.470069  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2729) took 1227 ms. Trace:
I20260709 12:26:34.582460  9940 rpcz_store.cc:276] 0709 12:26:33.242888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.245518 (+  2630us) service_pool.cc:224] Handling call
0709 12:26:33.245920 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.427410 (+181490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:33.427443 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.437101 (+  9658us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.750976 (+313875us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:33.751030 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.751129 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.469299 (+718170us) raft_consensus.cc:1685] finished
0709 12:26:34.469388 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.469827 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10182,"raft.queue_time_us":97068,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5713,"thread_start_us":452172,"threads_started":2,"wal-append.queue_time_us":403775}
I20260709 12:26:34.584205 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1492) took 3478 ms. Trace:
I20260709 12:26:34.584692 10668 rpcz_store.cc:276] 0709 12:26:31.105963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.109225 (+  3262us) service_pool.cc:224] Handling call
0709 12:26:31.109523 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.282756 (+2173233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.583593 (+1300837us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b540058e654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:34.583977 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4952388480}
I20260709 12:26:34.585691  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:34.587227 17005 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.588153 17005 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.589120 17005 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.589522 16363 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.578261 17100 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:34.471356  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2723) took 2044 ms (client timeout 1696 ms). Trace:
W20260709 12:26:34.590564  9941 rpcz_store.cc:269] 0709 12:26:32.426317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.426462 (+   145us) service_pool.cc:224] Handling call
0709 12:26:34.470843 (+2044381us) spinlock_profiling.cc:243] Waited 235 ms on lock 0x7b5400016dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:34.471093 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":540283776}
I20260709 12:26:34.592674 17100 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.592734  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:34.475242  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:34.593845 17100 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.478029  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:34.478785  9949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.009s	sys 0.009s
I20260709 12:26:34.479529  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:34.481292  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1523) took 1429 ms. Trace:
I20260709 12:26:34.482577  9947 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:33.967792 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 100319us
W20260709 12:26:34.488420  9991 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.696s (SENT) [suppressed 33 similar messages]
W20260709 12:26:34.488874 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.495788 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2424) took 2516 ms. Trace:
I20260709 12:26:34.496800 16772 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.498447 10025 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:34.500293 17020 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:34.501511  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:34.501760 17025 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.390915 17093 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.394577 17107 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.510128 17056 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:34.521414 17109 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:34.467942  9842 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 18, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 18 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.607095 17109 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.608207  9842 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 15, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 15 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:34.522607 16961 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.522877 16699 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:34.524955  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46345.7us
I20260709 12:26:34.527804 16836 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:34.534910 16798 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:34.505041 16956 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:34.536010 10017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1270) took 3298 ms (client timeout 1920 ms). Trace:
W20260709 12:26:34.537720 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.537652 16661 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:34.545836 16993 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.567899 17150 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.470075  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2726) took 1227 ms. Trace:
I20260709 12:26:34.601668  9940 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:34.600412  9945 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:34.604120  9943 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:34.606271 17056 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:34.610767  9949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.009s	sys 0.009s
W20260709 12:26:34.644652 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.601073 10019 rpcz_store.cc:276] 0709 12:26:31.978999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.686182 (+707183us) service_pool.cc:224] Handling call
0709 12:26:32.686513 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:32.847850 (+161337us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.495124 (+1647274us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540038d1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:34.495559 (+   435us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269567872}
I20260709 12:26:34.610340 17109 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:34.645802  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 157594us
I20260709 12:26:34.599048  9946 rpcz_store.cc:276] 0709 12:26:33.051921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.052061 (+   140us) service_pool.cc:224] Handling call
0709 12:26:33.052420 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.783069 (+730649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.480731 (+697662us) spinlock_profiling.cc:243] Waited 687 ms on lock 0x7b540042e854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:34.481088 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1579116800}
W20260709 12:26:34.637323 10017 rpcz_store.cc:269] 0709 12:26:31.195853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:31.857934 (+662081us) service_pool.cc:224] Handling call
0709 12:26:34.494045 (+2636111us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b5400021054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:34.494325 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2780927872}
I20260709 12:26:34.637552 17100 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.663556 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:34.666730 10672 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.671100 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:34.671484  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 98346.9us
I20260709 12:26:34.671859 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1504) took 2090 ms. Trace:
I20260709 12:26:34.672435 10668 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:34.673208 10671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1449) took 1694 ms (client timeout 2213 ms). Trace:
W20260709 12:26:34.673774 10671 rpcz_store.cc:269] 0709 12:26:32.975681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.977755 (+  2074us) service_pool.cc:224] Handling call
0709 12:26:34.669990 (+1692235us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54007e4fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:34.670365 (+   375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1452918016}
I20260709 12:26:34.646476 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:34.672457 10673 rpcz_store.cc:276] 0709 12:26:32.580869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.581011 (+   142us) service_pool.cc:224] Handling call
0709 12:26:32.581368 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.148698 (+567330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.671205 (+1522507us) spinlock_profiling.cc:243] Waited 565 ms on lock 0x7b54007396d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:34.671609 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1299670912}
I20260709 12:26:34.677395 16760 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.645251 16956 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:34.678800 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.636818 16836 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:34.679828 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79391.5us
I20260709 12:26:34.680725 17005 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:34.681203 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.683892 10674 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:34.684564 17094 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:34.684839 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.686094 17109 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.688971 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:34.689863 17022 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:34.693853 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:34.694419 16959 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.696609 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:34.695951 10672 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:34.636129 16798 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:34.697223 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:34.705255 10023 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:34.705093 10668 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:34.708547 17109 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.710371  9942 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:34.677256 16699 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.611176 17025 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.799501 16710 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:34.800312 16710 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:34.800756 17025 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:34.801081 16699 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.614851 16772 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:34.801759 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.801591 16710 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.803532 16772 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.645928  9941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db738483c1444059b752648d302dc0d9"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:26:34.702962 17094 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:34.610443  9842 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 18, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 18 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.636488  9948 rpcz_store.cc:276] 0709 12:26:33.242012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.243318 (+  1306us) service_pool.cc:224] Handling call
0709 12:26:33.243598 (+   280us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:33.248346 (+  4748us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:33.248395 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:33.249082 (+   687us) log.cc:844] Serialized 26 byte log entry
0709 12:26:33.737347 (+488265us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:33.737389 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:33.737495 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:34.469348 (+731853us) raft_consensus.cc:1685] finished
0709 12:26:34.469399 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:34.469824 (+   425us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4301,"raft.queue_time_us":90020,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":6991,"spinlock_wait_cycles":15232,"thread_start_us":445399,"threads_started":1,"wal-append.queue_time_us":680814}
I20260709 12:26:34.841840 17099 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:34.845165  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:34.845317  9946 pending_rounds.cc:85] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 14
W20260709 12:26:34.845954  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 172270us
I20260709 12:26:34.851559 16363 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.858043 17099 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.861929 17104 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:34.851744 17059 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.862718  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:34.862845 17104 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.863744 16836 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.865146  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:34.865610 16836 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:34.829013 17005 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:34.848630  9946 pending_rounds.cc:107] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:26:34.866935  9946 raft_consensus.cc:2889] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:34.866993 17005 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.867914 17100 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:34.804409 16661 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.870250 17059 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.871812 17056 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.832463 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:34.872632 16661 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.821728 17150 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.872779 17059 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.901197  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:34.839445 16993 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.870311 17005 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.867231 17099 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.906277 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:34.913748 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82667.6us
I20260709 12:26:34.916138 10017 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:34.855508 17022 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.917092 16870 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:34.917303 16993 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:34.917569 16870 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:34.918660 17022 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.832239 16961 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.919359  9943 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:34.918848 16870 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:34.920135 16870 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:34.921031 16961 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:34.930380  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100837us
I20260709 12:26:34.934715  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:34.955955  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48292.5us
I20260709 12:26:34.960505 16993 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:34.961097 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 113708us
I20260709 12:26:34.938122 16961 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:34.962056 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:34.962312 16961 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:34.963905 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:34.964828 16961 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:34.933431  9842 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:34.960556 16798 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:34.973253 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 275795us
I20260709 12:26:34.973855 16798 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:34.978523  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:34.979449  9942 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
W20260709 12:26:34.982337 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44134.2us
I20260709 12:26:34.982685 17025 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:34.983146 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:34.983867 10018 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:34.984025 10065 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.628s (SENT)
W20260709 12:26:34.984843 10065 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Timed out: RequestConsensusVote RPC to 127.8.137.193:34909 timed out after 1.645s (SENT)
I20260709 12:26:34.985256 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:34.969776  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:34.986887  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:34.987304 17025 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:34.987546 16861 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:34.988087 16861 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:34.988525 16861 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:34.989154 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 185354us
I20260709 12:26:34.989851 17020 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:34.990284 17020 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:34.990334 16888 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:34.990815 17020 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:34.991528 16888 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:34.991784 17018 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:34.992655 17018 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:34.993000 17018 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:34.995368  9842 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 15, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 15 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:34.997538  9842 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 16, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 16 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:34.999114 17109 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.002097 16898 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:34.989226  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:35.005601  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:35.010176 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46479.9us
I20260709 12:26:34.892294 16959 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.012406 16993 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:35.012539 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:35.014550 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.017907  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2734) took 1621 ms. Trace:
I20260709 12:26:35.017948 16874 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:35.018823 10019 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:26:35.018492  9949 rpcz_store.cc:276] 0709 12:26:33.396768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.515220 (+118452us) service_pool.cc:224] Handling call
0709 12:26:33.515528 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.515755 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:34.805028 (+1289273us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.003536 (+198508us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:35.008407 (+  4871us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:35.013387 (+  4980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.017637 (+  4250us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.017838 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7480,"prepare.queue_time_us":624244,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"raft.queue_time_us":10655,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":41157,"thread_start_us":1119977,"threads_started":2}
I20260709 12:26:35.020253 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:35.020927 17153 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:26:35.021003 10019 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 13.
I20260709 12:26:34.994166 17018 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.023747 17104 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.029451 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1276) took 2831 ms. Trace:
I20260709 12:26:35.034878  9842 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 14, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 14 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.040714 17150 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.041389 17018 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:35.043138 10093 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1253) took 2337 ms (client timeout 2086 ms). Trace:
I20260709 12:26:35.043375 17037 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.043624 10093 rpcz_store.cc:269] 0709 12:26:32.705607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.713544 (+  7937us) service_pool.cc:224] Handling call
0709 12:26:35.042609 (+2329065us) spinlock_profiling.cc:243] Waited 510 ms on lock 0x7b5400164754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:35.042848 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1172934016}
I20260709 12:26:35.047570 16959 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.050963 17109 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.051821 17109 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.053539 10022 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:35.044201 17037 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.057550 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.059161 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.060096 10097 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:35.060935 16959 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:35.061199 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.061806 10095 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:35.062896 17104 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:35.063504 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2427) took 3027 ms. Trace:
I20260709 12:26:35.063897 10025 rpcz_store.cc:276] 0709 12:26:32.036224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.729505 (+693281us) service_pool.cc:224] Handling call
0709 12:26:34.497315 (+1767810us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.601924 (+104609us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.063276 (+461352us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b5400021050. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:35.063429 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4013410560}
I20260709 12:26:35.064812 17150 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:35.065254 17150 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:35.066519 17147 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.067130 17147 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.067669 16874 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.069499  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.070755 17108 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:35.071066  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.071292 17108 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:35.071851 17108 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:35.029917 10020 rpcz_store.cc:276] 0709 12:26:32.197857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:32.950911 (+753054us) service_pool.cc:224] Handling call
0709 12:26:33.011095 (+ 60184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:33.306813 (+295718us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.028822 (+1722009us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b5400217ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:35.029244 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":604863872}
I20260709 12:26:35.037671  9842 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.080089  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.046469  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2731) took 1803 ms. Trace:
I20260709 12:26:35.091485  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2736) took 1229 ms. Trace:
I20260709 12:26:35.094964 10017 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:35.091493  9947 rpcz_store.cc:276] 0709 12:26:33.243390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:33.343605 (+100215us) service_pool.cc:224] Handling call
0709 12:26:35.046229 (+1702624us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54001b9254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:35.046402 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2614604032}
W20260709 12:26:35.095580  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 106544us
I20260709 12:26:35.090977 17100 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:35.106552 17147 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.107015 17005 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.107455 16946 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:35.107210  9842 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.107903 16946 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:35.109022 16946 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.110082 17094 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.110801 16946 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:35.112476 17094 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.112650 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:35.113466 10099 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:35.114428 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:35.114845 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54239.3us
I20260709 12:26:35.116179 17109 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.116370 10097 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:35.013849 16956 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.117293 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:35.117791  9941 raft_consensus.cc:606] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:35.118247  9941 raft_consensus.cc:614] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:35.119414 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:35.122231 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:35.107645 17005 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.123374  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2741) took 1261 ms. Trace:
I20260709 12:26:35.108971  9842 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 15, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 15 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.093793  9945 rpcz_store.cc:276] 0709 12:26:33.860080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.644268 (+784188us) service_pool.cc:224] Handling call
0709 12:26:34.802775 (+158507us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:34.803192 (+   417us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:34.803514 (+   322us) log.cc:844] Serialized 14 byte log entry
0709 12:26:35.089180 (+285666us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:35.089236 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:35.089278 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:35.089351 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.089444 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1890,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":9256,"raft.queue_time_us":109,"thread_start_us":285196,"threads_started":1}
W20260709 12:26:35.106401  9912 connection.cc:256] RPC call timeout handler was delayed by -1.3174631470001259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.188s for a response
I20260709 12:26:35.125000  9946 rpcz_store.cc:276] 0709 12:26:33.861910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.664586 (+802676us) service_pool.cc:224] Handling call
0709 12:26:34.806790 (+142204us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:34.868429 (+ 61639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:34.868464 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:34.869167 (+   703us) log.cc:844] Serialized 52 byte log entry
0709 12:26:34.936559 (+ 67392us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:34.936586 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.936633 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.123127 (+186494us) raft_consensus.cc:1685] finished
0709 12:26:35.123153 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.123312 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":608,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":181,"thread_start_us":66959,"threads_started":1,"wal-append.queue_time_us":146523}
I20260709 12:26:35.128849 16898 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 12:26:35.141201 17111 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.141978 17111 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.143904 16898 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.144483 17152 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.091s	user 0.002s	sys 0.005s
I20260709 12:26:35.146072  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2525) took 1382 ms. Trace:
W20260709 12:26:35.147089 17156 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.082s	user 0.003s	sys 0.000s
I20260709 12:26:35.147471 16959 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:35.147951 16959 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:35.148360 16959 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:35.148787 17158 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.076s	user 0.007s	sys 0.000s
I20260709 12:26:35.149706 16861 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.150285  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2737) took 1289 ms. Trace:
I20260709 12:26:35.146939 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:35.144335 17111 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.154103  9940 rpcz_store.cc:276] 0709 12:26:33.860488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.660611 (+800123us) service_pool.cc:224] Handling call
0709 12:26:34.816133 (+155522us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:34.836528 (+ 20395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:34.836574 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:34.841919 (+  5345us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.934018 (+ 92099us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:34.934079 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.934298 (+   219us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.150017 (+215719us) raft_consensus.cc:1685] finished
0709 12:26:35.150039 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.150217 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":786,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":16368,"thread_start_us":91575,"threads_started":1,"wal-append.queue_time_us":209961}
I20260709 12:26:35.157245 10093 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: [15.13-15.13]
I20260709 12:26:35.159976 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.144513 16898 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.165983  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:35.168821 10668 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:35.146561  9944 rpcz_store.cc:276] 0709 12:26:33.763185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:34.595531 (+832346us) service_pool.cc:224] Handling call
0709 12:26:34.596219 (+   688us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:34.596926 (+   707us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:34.596979 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:34.597896 (+   917us) log.cc:844] Serialized 26 byte log entry
0709 12:26:34.777626 (+179730us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:34.777669 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:34.777735 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:35.068291 (+290556us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:35.145708 (+ 77417us) raft_consensus.cc:1685] finished
0709 12:26:35.145735 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:35.145980 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2219,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":40820,"spinlock_wait_cycles":5835008,"thread_start_us":179170,"threads_started":1,"wal-append.queue_time_us":258176}
I20260709 12:26:35.180944 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:35.181363 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64163.3us
I20260709 12:26:35.191541 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.192265 10668 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:35.192744 16861 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.195214 17100 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:35.196875 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:35.197654 17037 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.144337 17020 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.199559 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.200518 10090 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 24 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:26:35.201444 17020 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.184484 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:35.169827 16956 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.189988  9842 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 16, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 16 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:35.209388 16898 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.210033 10091 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:35.210930 10673 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:26:35.168272 17018 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.212813 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.223009 10018 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
W20260709 12:26:35.228506  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94969.1us
I20260709 12:26:35.229408 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:35.235411  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:35.236718  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 58448.2us
I20260709 12:26:35.235677 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.218832 16861 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:35.239513 10669 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:35.243016 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:35.246475 10017 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:35.242758 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.249742  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.222518 16920 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:35.252339 10672 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:35.252732 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.249858  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.255249  9944 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:35.256386  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.261408  9842 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:35.267185  9948 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:35.274916 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:35.278677 17105 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:35.279578 17025 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:35.282446  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:35.284637 10098 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:26:35.286119  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:35.286921 10674 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:35.235528 17018 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.293035 17018 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.294021 10673 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:35.254344 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:35.300657  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.247270 17150 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.303895 17150 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.268783  9842 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.305460  9842 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 14, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 14 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.307279  9842 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 16, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 16 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:26:35.300012 17164 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:26:35.301092 10670 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:35.313750  9943 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: [16.10-18.11]
I20260709 12:26:35.301692  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.293409 10093 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:26:35.299611  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 60231.8us
I20260709 12:26:35.315991 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.316840 10097 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 15.
I20260709 12:26:35.317023  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:35.317852 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.324667  9940 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:35.318540 10097 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:35.334647  9948 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:35.349474 10017 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:35.338315 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:35.357717 10672 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.331483  9842 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 17, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 17 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.318953 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.331981  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.319358 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:35.358836 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.322826  9945 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:35.363832 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.364426 10674 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:35.365236 10670 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:35.363505 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:35.368950 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.370064  9947 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:35.372077 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.371300 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.330688 16898 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.387418  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:35.387986  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 51575.4us
I20260709 12:26:35.376027  9842 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 15, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 15 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.377818  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.390298  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:35.393328  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.394332  9949 raft_consensus.cc:2368] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:35.395629  9940 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:35.388139  9945 raft_consensus.cc:2376] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16. Re-sending same reply.
I20260709 12:26:35.399390  9945 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:35.400780 10097 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
W20260709 12:26:35.402468 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 153729us
I20260709 12:26:35.403312 10017 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:26:35.401569  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:35.413873  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 173700us
I20260709 12:26:35.418298  9944 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:35.323935 17108 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.419113  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:35.330821 17018 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.073s
I20260709 12:26:35.421692 10095 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:26:35.422093  9948 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:35.423947  9942 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:26:35.425115 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.399869 17005 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.426110  9945 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:35.426318 10024 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:35.428267 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.419739  9842 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:35.430137 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.430819  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:35.438441  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71590.1us
I20260709 12:26:35.438828 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:35.441725 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.435750  9944 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:35.434319 10090 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:35.445601 10097 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:26:35.446382 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:35.451407 10676 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:35.452170  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.453229  9945 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
W20260709 12:26:35.454674  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.456185 17108 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.456346 10674 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
W20260709 12:26:35.457080 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.457799 17121 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.458388 17121 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.458936  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.459595  9948 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:35.466143 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:35.466837 10676 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:35.471192  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.472232  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.473136  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.475675 16987 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.476305 16987 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.477541 10672 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:35.473002 16956 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:35.482537  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:35.483249 16836 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:35.484732 16836 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:35.485260 16836 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:35.454700  9944 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:26:35.488408  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.489181  9944 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:35.490145  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:35.491334  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.491941 17099 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.491995 16861 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:35.493175 16861 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:35.492805 17099 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.494167 16861 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.492168 16920 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:35.494837 16888 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:35.495373 16888 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:35.495847 16888 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:35.496181 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:35.497984  9912 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.491187  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.447079 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:35.499143  9940 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:35.499212  9949 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:35.500670  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:35.498576  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.507278  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:35.499500 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 211541us
W20260709 12:26:35.502682 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55718.8us
I20260709 12:26:35.544288  9948 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:35.505808  9842 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 15, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 15 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.530241 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.546468 10672 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:35.547246 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:35.549062  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 182548us
I20260709 12:26:35.520988  9944 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:35.567205 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:35.499656 17020 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:35.522378  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:35.576336  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 77831.2us
I20260709 12:26:35.579738 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:35.430935 16898 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.513324 17005 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:26:35.501543 17099 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:35.614450 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.585412  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.495378 16920 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:35.593502  9842 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.618029 10676 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:35.597533  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:35.594091 10017 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:35.627013 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:35.502882 16987 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:35.591531  9944 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
W20260709 12:26:35.585429  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 63214.1us
I20260709 12:26:35.613339  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:35.664690  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51457.2us
W20260709 12:26:35.622741 17166 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
I20260709 12:26:35.623204 10640 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:35.654835  9949 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:35.660931 10671 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:35.577803 17020 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.598856 16760 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:35.593727 17108 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.584896 16772 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:35.614319 17040 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:35.618228 16739 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.679893 10024 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:35.613893 17094 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:35.648444 17005 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:35.676827  9842 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 16, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 16 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.604506 16870 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 12:26:35.675472 17020 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.724596 16987 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:35.725170 16987 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:35.725447  9940 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:35.617025 17109 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 12:26:35.726286 16987 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:35.731148  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:35.731369 17090 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:26:35.608723 17025 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.085s
W20260709 12:26:35.718581 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64251.1us
I20260709 12:26:35.739444 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:35.722162 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 110476us
I20260709 12:26:35.725219 16710 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:35.733181 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 110445us
W20260709 12:26:35.733958 16959 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.000s
I20260709 12:26:35.736063 17121 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.761412  9842 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 15, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 15 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:26:35.758504  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 88951.2us
I20260709 12:26:35.699470 16772 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:35.718066 17005 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:35.672219 17150 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:26:35.698866 16760 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:35.703346 17040 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:35.663628 16836 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.771039  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.773448 16941 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:35.783751 16941 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.783798  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.784389 16941 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:35.773598 16772 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.787078 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.765565 16959 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.775297 17109 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:35.767571 16739 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.790958 17109 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:35.792047 17109 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.793213 17109 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:35.793892 17025 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.794721 17025 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.774021 16710 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.796787 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:35.800993 16920 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.802376 10098 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:35.804451 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:35.821553 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 163129us
W20260709 12:26:35.822173 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48234.9us
I20260709 12:26:35.824802 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:35.823768 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:35.831965 10017 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:35.829388  9842 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 14, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 14 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.840513 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:35.843323 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:35.779116 17040 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:35.760265 17094 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:35.849149 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71018.8us
I20260709 12:26:35.852320 17041 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:35.852824 17041 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.853245 17041 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:35.829119 17090 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:35.856953 16739 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.857682 16710 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:35.841432 17121 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.858388 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:35.872484 10018 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:35.829774 17108 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.862986 16959 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:35.869748  9842 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.879223 16993 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:35.905038 10018 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
W20260709 12:26:35.881662 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52584.6us
I20260709 12:26:35.883827 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:35.885269 17178 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:26:35.887516 16941 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:35.887882 17037 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:35.889487 17120 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:35.857702 16760 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.893669  9945 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:26:35.894704 17121 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.866256 17020 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.899075 16710 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.867695 17090 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:35.909494  9842 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 18, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 18 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:35.918679  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:35.920635  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 135078us
I20260709 12:26:35.940335 10643 leader_election.cc:400] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:35.940912 10643 leader_election.cc:403] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:35.881343 17185 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:35.935215  9945 pending_rounds.cc:85] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 12
I20260709 12:26:35.948137  9945 pending_rounds.cc:107] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:26:35.948400  9945 raft_consensus.cc:2889] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:35.879935 17094 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.936815 10024 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:26:35.949399  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.950299 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:35.933573 16993 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:35.954422 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47595.6us
I20260709 12:26:35.933910 16959 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:35.957036  9948 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:35.954169  9842 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 16, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 16 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:35.956224 10018 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:35.957305 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:35.958114 10017 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:26:35.960460 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.950445 17037 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:35.942147 16874 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.874s	user 0.002s	sys 0.011s
I20260709 12:26:35.959491 16363 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:35.942997 17090 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:35.984395 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:35.985200  9842 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:35.962826  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:35.962711 17182 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:35.974467 16661 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:35.975521 16941 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:35.975836 17108 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:35.947213 16836 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:35.981487 17040 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.982689 16874 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:35.988299 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:35.985253 10020 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:36.007616  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:35.953550 17100 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:35.956969 16956 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:35.943169 16710 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.008738  9944 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:36.007561 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.010313  9947 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:36.010494  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47747.5us
I20260709 12:26:36.019613 10017 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:35.999274 17037 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:35.987708 17185 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:36.015586 10016 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:36.011026 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.018558 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.027690 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:35.991452 16661 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.028178  9842 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 15, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 15 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:35.962173 17186 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 12:26:35.983549 16959 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.029165 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.032477 10671 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:36.037525 10091 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:36.011988 17120 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:36.023147 16888 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.004s
I20260709 12:26:36.033526 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.010856 17100 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.034039 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.034113 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.048808 10097 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:36.033545 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.031345 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.049729 10093 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:36.034479  9842 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:36.026777  9946 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:36.029132 16661 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.038344 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.038858 17176 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:36.039535 17186 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:36.039681 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.019809 17099 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:36.018730 16956 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:36.053695  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.055140 10095 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:36.039558 17189 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:26:36.057092 10021 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:36.051031 16993 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.060176 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.077102 10095 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:36.077503 10092 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:26:36.059137 10094 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:36.050729 17120 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.079401 17099 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:36.059525 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.080704 10097 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:36.080453 17099 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.081135 17120 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.081707 10090 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.056521 17176 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.082788 17176 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:36.083475 16956 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.050241 17100 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.063016 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.059672 17186 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:36.078351 10095 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 12:26:36.059223  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.079747 10094 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:36.069217  9987 leader_election.cc:400] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:36.063494 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.062703 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.097741 10093 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.095026  9842 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 14, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 14 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.067760 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.063122 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.043617 17025 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.096544  9987 leader_election.cc:403] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Cancelling election due to peer responding with higher term
I20260709 12:26:36.106524 10096 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:36.071378 17197 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:26:36.097098 10016 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:36.062299 16959 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.103250 17194 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:26:36.048429 16888 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.133419  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:36.124715 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.060930 16710 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:36.063196 17121 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.119063 10093 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:36.094345 17040 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:36.074820 17020 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:36.124540 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83373.2us
W20260709 12:26:36.135998  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66978us
I20260709 12:26:36.115288 17178 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:26:36.036875 17041 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.133036 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.146782 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.134232 10095 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:36.114897  9842 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 7 to 12, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 12 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:36.143747 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.148082 10093 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:36.148154  9949 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:36.064596 17183 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.093s
I20260709 12:26:36.147024 10094 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:36.109932 17005 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.138664 16874 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:36.136520 17025 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.165421 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.184880 10093 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:36.166517  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:36.164842  9842 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:36.187899  9842 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 14, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 14 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:36.172796  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.066978 17185 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.156337 17120 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:26:36.177320 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.189800  9947 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:36.163585 10090 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:36.192844 10098 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:36.192906 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.198246 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.199712 17192 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 12:26:36.201217 17188 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 16. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:36.201958 16800 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:36.206691 16874 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.207110 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.208443 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.209055 10097 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:36.209167 16874 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:36.210698 17040 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:36.211269 17040 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.214469 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.216137  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.217159  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:36.217936 17020 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:36.218376 17020 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:36.218989 17020 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:36.219668 17040 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:36.220703 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.221309 10095 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:36.222174 16874 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:36.218339  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.223424 16898 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.225239 17041 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.226053 17121 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:36.229385  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.230804  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:36.234081  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:36.209897 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:36.226384 17179 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.503s	user 0.000s	sys 0.005s
I20260709 12:26:36.243474 10096 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:36.194496  9842 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:36.213783 16772 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.259140  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:36.259428  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.262740  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.226529 17121 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.263645  9943 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:36.265594 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.200841 16661 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.270722  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.272255  9842 catalog_manager.cc:5697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:36.183429 17109 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.083s
I20260709 12:26:36.281914  9940 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:36.266222 16874 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:36.293533 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.227262 17020 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:36.298990 17200 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
W20260709 12:26:36.307103  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 115934us
I20260709 12:26:36.307922 16661 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:36.307978  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:36.308712 16661 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:36.309036 17188 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:36.309623 17188 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:36.309772  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:36.265239 16739 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:26:36.310985  9945 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:26:36.310695 17188 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.270380 17041 raft_consensus.cc:2764] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
W20260709 12:26:36.312377  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 80459.9us
I20260709 12:26:36.313591  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:36.265136 17090 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.309230 16661 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.270951 17121 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:36.315826 17090 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:36.193662 17025 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.318183 17025 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.319777 17100 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.321951 17100 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.322477 16661 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:36.323212 17020 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:36.333971  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.335007  9940 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:26:36.315027 16888 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.348912 10023 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:36.337958  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:36.343808  9944 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:36.353734  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.177538 17186 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.306010 17176 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.355646 10093 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
W20260709 12:26:36.360857 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 376787us
I20260709 12:26:36.365672  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.369491 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.373201  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:36.351956 16661 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.284801 16800 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:26:36.371562  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.395669  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:36.396970 10097 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:36.408181 16800 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:36.408838 16800 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:36.412846  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.413885  9942 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:36.418974  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.363826 17192 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:36.420264  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:36.420354 17192 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.358415 17191 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:36.376050 16888 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:36.425712 16888 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.426177 16888 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.384007 16739 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:36.409914 16800 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.426939 16739 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:36.383199 16941 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:36.358760 17094 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:26:36.428280 16739 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:36.429420 17100 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:36.429791 17100 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:36.383055 17150 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:36.430943 17150 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.430696 17100 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.432205 17100 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.433020 17176 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:36.433728 16920 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:36.434262 16920 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:36.435143 16920 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:36.440618  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 127110us
I20260709 12:26:36.441385  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:36.431528  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.431349 17150 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.451455  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:36.453754 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.454876  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:36.458307 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.465504  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:36.359086 17020 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:26:36.487040 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.487846 10093 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:36.488157  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:36.488807  9942 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:36.484936  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.488989 16917 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.495810 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.505177 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.505537 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.507879  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:36.508234 10092 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:36.508725 10092 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:36.504752  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73303.3us
I20260709 12:26:36.466693 17105 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:36.508785  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:36.466730 16870 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:36.513943  9942 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:36.515537 10096 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:36.389917 16661 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.497898 16760 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:36.404487 16956 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:36.522550 10093 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:26:36.528774  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 43907us
I20260709 12:26:36.530040 16800 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.538931 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.510836 17108 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:36.553090 10096 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:36.555356  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100564us
I20260709 12:26:36.556946  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:36.516577 17105 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.496300 17194 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
W20260709 12:26:36.585343 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.584909  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.588954 17028 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:36.516839 16870 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:36.495494 17191 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:26:36.526391 17185 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:36.525315 17188 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:36.595183 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.596411  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:36.542238 10092 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.541039 16956 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.600589 10093 raft_consensus.cc:2368] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:36.555671 17108 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.556207 16760 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:36.592495 17178 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:36.595543 16870 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:36.624923 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:36.625771 17221 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:26:36.629282  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:36.630885 17187 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:36.631371 17187 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.631728 17187 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.635587 16760 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.640115 17190 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:36.640704 17190 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.641132 17190 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.640429 17121 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.620895 17041 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:36.643493 17041 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:36.643975 17041 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.647179 17191 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:36.647647 17191 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:36.648612 17191 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.650096 17191 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:36.651307 17204 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:26:36.609261 16956 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:36.655088  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58806.1us
I20260709 12:26:36.617175  9942 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:36.585251 17192 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.591390 17105 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:36.602522 16772 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:36.668638 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:36.618323 17108 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:36.670801 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.674455  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.688818  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 2752) took 1456 ms. Trace:
I20260709 12:26:36.689283  9946 rpcz_store.cc:276] 0709 12:26:35.232804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.232975 (+   171us) service_pool.cc:224] Handling call
0709 12:26:36.688636 (+1455661us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54001b8354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:36.688749 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1777831296}
I20260709 12:26:36.697026 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.698302 10096 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:36.608134 17176 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:26:36.716820 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.717988  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:36.719359  9945 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:36.673508 17028 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:36.722899 10090 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:36.723366  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 215557us
I20260709 12:26:36.730901  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:36.741098 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.731489 17209 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:36.717142 16920 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:36.741962 10096 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:36.696396 16888 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.659598 17178 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.732043 16772 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:26:36.755316 17209 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.759876 16772 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:36.760356 16772 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:36.754740 16874 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:36.757969 16920 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.761317 16772 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:36.771800 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:36.772713 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:36.775015  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:36.774109  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2538) took 1293 ms. Trace:
I20260709 12:26:36.741852 16661 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:36.750355 16917 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:36.776042  9948 rpcz_store.cc:276] 0709 12:26:35.480876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.568309 (+ 87433us) service_pool.cc:224] Handling call
0709 12:26:36.773930 (+1205621us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b540003f554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:36.774054 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":727850240}
W20260709 12:26:36.763461 17213 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
I20260709 12:26:36.761790 17209 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:36.781329 17121 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.781797 17192 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:36.782436 10097 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:36.783011 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.784559 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.788596 16888 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:36.790911 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.791600 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.810812 10099 raft_consensus.cc:2368] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:36.798935 16917 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:36.764127 16920 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:36.807792 10095 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:36.764240 17224 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:26:36.797226 17214 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:26:36.815971 17185 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:36.928073 10676 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:36.809275 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:36.888764  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 87977.3us
I20260709 12:26:36.894006 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.930634  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:36.895552 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.932293  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:36.898197 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:36.938369 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:36.938740 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.938887 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:36.939323 10017 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:36.939616 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.939724 10670 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:36.940533 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.941128 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:36.942349 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:36.943851 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.946817 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.949236 10019 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:36.949514 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.950206 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:36.947618 10020 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:36.953145 10021 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:26:36.953824 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.955415 10025 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 17: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:36.956466 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:36.957141 10672 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:36.956477 10674 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:36.960008 10017 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:36.963349 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:36.964160 10674 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:26:36.964214 10019 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:36.968760 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.838368 17094 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.142s
W20260709 12:26:36.974351  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 317538us
I20260709 12:26:36.977033 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:36.978883 10022 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:36.979300 17185 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.980345 10022 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:36.981750 10022 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:26:36.982837 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:36.983618 10016 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:36.966002 16870 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:36.982868 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:36.985746 10022 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:36.990841 10674 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:36.988070 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:36.994478 10024 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 24 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:26:36.995172 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:36.996133 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:36.996181  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 321804us
I20260709 12:26:36.999864  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:36.998477 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:36.997062 10025 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:26:36.999670  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:37.003360 10098 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:37.003392 10671 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
W20260709 12:26:36.987653 16739 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.558s	user 0.019s	sys 0.002s
I20260709 12:26:37.008630 16739 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.011004 16739 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.005888 10018 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: [18.15-19.16]
I20260709 12:26:37.012965 10019 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:37.013132 10020 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:37.017678 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:37.018455 10024 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.020112 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.021798 10094 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:37.024940  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:37.007339 17099 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.018s	sys 0.022s
I20260709 12:26:37.026669 10671 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [16.14-17.15]
I20260709 12:26:37.031304 10016 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:37.064411 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.067773 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:37.068574 10672 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:26:37.069799  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44977.5us
I20260709 12:26:37.073673 17237 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:37.021695 17231 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:26:37.079893 10024 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
W20260709 12:26:37.040498 17099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.018s	sys 0.023s
I20260709 12:26:37.085456  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:37.071825 17100 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.639s	user 0.018s	sys 0.005s
I20260709 12:26:37.065752 17230 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:37.089991 10671 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:37.071784 17028 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:37.091545  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:37.101485 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:37.104326 16870 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:37.109381 17150 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.613s	user 0.008s	sys 0.000s
I20260709 12:26:37.109825 17150 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.128690  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:37.130231 10094 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:37.130781 10094 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:37.126825 17231 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:37.134177 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:37.134987 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:37.092041 17185 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:37.135758 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:37.136406 10673 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:37.136445 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:37.137218 10668 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:37.138055 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:37.138229 17187 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.506s	user 0.010s	sys 0.000s
I20260709 12:26:37.099798 17209 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.138912 10669 raft_consensus.cc:2410] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 20 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:26:37.138863 17187 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.135125 10674 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:37.134984 10676 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:37.135669 10670 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:37.141144 17187 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:37.148689 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:37.152169  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2767) took 1650 ms. Trace:
I20260709 12:26:37.152225 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:37.153342 10676 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:26:37.152899  9949 rpcz_store.cc:276] 0709 12:26:35.502104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.701992 (+199888us) service_pool.cc:224] Handling call
0709 12:26:35.731127 (+ 29135us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:36.151346 (+420219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.151944 (+1000598us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b5400015ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:37.152116 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":958656768}
I20260709 12:26:37.099654 17230 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:37.154784  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51737.7us
I20260709 12:26:37.156910  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:37.158527 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:37.168913 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.169793 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.170459 10096 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:37.157677 17209 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.173410 17209 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:37.174095 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.185370 10099 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:37.187208 16363 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.187693 16363 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.188688 10024 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:26:37.189280 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:37.190160 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.159438 10094 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.189899 10097 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:37.157140 17230 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.197044 10020 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:26:37.193529 10093 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:26:37.193554 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.202348 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:37.182358 10096 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:37.203500 10016 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:37.202358 10094 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.202529 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:37.192194 17190 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.014s
I20260709 12:26:37.205538 10025 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:26:37.185796 17231 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.201756 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:37.207521 16363 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:37.217073  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:37.219986 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:37.192241 17185 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.025s
W20260709 12:26:37.215477 17108 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.511s	user 0.006s	sys 0.016s
I20260709 12:26:37.208724 17231 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.250864 17231 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.254459 17234 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:37.255004 17230 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.255445 17230 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.255203 17234 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.255522  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:37.239786 17105 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.560s	user 0.006s	sys 0.005s
I20260709 12:26:37.240129 16898 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:37.246938 17233 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.247507 16861 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:37.248261 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:37.248936 16710 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:37.249315  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 246705us
I20260709 12:26:37.249539 17199 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:37.248304 16917 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:37.255882 17230 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.237864 10025 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:26:37.261090 17028 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:37.222460 17005 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:37.213196 17190 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:37.279561 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 86417.2us
I20260709 12:26:37.281441 16888 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:37.305773  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:37.293277  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:37.340775  9948 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:37.296609  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:37.275979 17249 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 12:26:37.299505  9942 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 15. Current term is 16. Ops: []
W20260709 12:26:37.300050 16956 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.646s	user 0.023s	sys 0.020s
I20260709 12:26:37.307000 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:37.303208 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55088.6us
I20260709 12:26:37.265581 16898 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.347298 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:37.231094 17224 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:37.304484  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.353097 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:37.257958 17234 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.304100 10025 pending_rounds.cc:85] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 10
I20260709 12:26:37.354355 10025 pending_rounds.cc:107] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:26:37.354684 10025 raft_consensus.cc:2889] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.356163 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:37.357465 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:37.335301  9945 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:37.291214  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:37.275444 17109 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.360360 17109 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.361462 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:37.362471 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:37.362994 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:37.365595  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:37.366086 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59183.6us
I20260709 12:26:37.262356 17108 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.368069 10025 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:37.369506 10023 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
W20260709 12:26:37.376549  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.336457 16917 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:37.382118 10016 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:26:37.380636 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.383968  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:37.363445 17203 raft_consensus.cc:2764] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:37.398757 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:37.399995 17150 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.400055 10016 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:37.363708 16772 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.401078 16772 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.349642 16888 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.402356 17186 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.404336 16888 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.365468 16956 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.407007 16956 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.407735  9944 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
W20260709 12:26:37.409354 17202 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.185s	user 0.004s	sys 0.016s
W20260709 12:26:37.410277 17191 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.760s	user 0.000s	sys 0.013s
I20260709 12:26:37.304531 17199 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.410772  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2769) took 1467 ms. Trace:
I20260709 12:26:37.410892 17224 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.411211  9941 rpcz_store.cc:276] 0709 12:26:35.943463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:35.950583 (+  7120us) service_pool.cc:224] Handling call
0709 12:26:35.971334 (+ 20751us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:35.997564 (+ 26230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:35.997596 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:36.025009 (+ 27413us) log.cc:844] Serialized 26 byte log entry
0709 12:26:36.176820 (+151811us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:36.176861 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:36.176924 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:36.961714 (+784790us) spinlock_profiling.cc:243] Waited 4.34 ms on lock 0x7b48002a2f58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:26:37.386489 (+424775us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:37.410514 (+ 24025us) raft_consensus.cc:1685] finished
0709 12:26:37.410536 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:37.410710 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":276277,"raft.queue_time_us":14022,"raft.run_cpu_time_us":2738,"raft.run_wall_time_us":963417,"spinlock_wait_cycles":22915712,"thread_start_us":128135,"threads_started":1,"wal-append.queue_time_us":190333}
I20260709 12:26:37.412823 17199 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:37.414263 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46225.8us
I20260709 12:26:37.392858 10092 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.393195 17108 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.431555  9940 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:37.394004 10022 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:37.395301 16800 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:37.396793 17190 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.397390 17187 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.267210 17233 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.400460 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:37.302850 16710 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.398380 17203 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:26:37.372212 17041 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.728s	user 0.013s	sys 0.000s
I20260709 12:26:37.432209 10024 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:37.364665 17109 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.371577 16661 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:37.446415  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 514164us
W20260709 12:26:37.448421 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85994.8us
I20260709 12:26:37.450719  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:37.453738  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:37.265102 17105 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.436059 16800 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:37.443249 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:37.474977 10025 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:26:37.471551  9946 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:26:37.439435 17187 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.452517 16888 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:37.452853 16661 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:37.478480  9940 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:37.480527 10090 raft_consensus.cc:2368] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:37.438452 17150 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:26:37.476214 17187 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.439394 17040 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:26:37.483894 17040 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:37.487995 17233 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.445976 17041 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.494534 16888 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:37.498612 16888 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.499711 16710 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:37.501189 17192 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:37.501811 17192 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.505836 17192 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:37.506994 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54532.9us
I20260709 12:26:37.489235 16661 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.510142 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:37.510846 10094 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 12:26:37.514564 16898 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:37.516037 17105 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:37.516407 17041 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:37.476488 16917 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.511248 10094 pending_rounds.cc:85] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 15
I20260709 12:26:37.524652 10094 pending_rounds.cc:107] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:26:37.528240 10094 raft_consensus.cc:2889] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:37.529546 16920 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:26:37.530148  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 76498.9us
I20260709 12:26:37.306254 17176 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:37.528793 17005 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:37.534157 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:37.527529 10092 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 12:26:37.563812 10092 pending_rounds.cc:85] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 14
W20260709 12:26:37.534185 17252 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.092s	user 0.003s	sys 0.000s
W20260709 12:26:37.531477 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 148511us
I20260709 12:26:37.563477  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:37.564174 10092 pending_rounds.cc:107] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:26:37.532447 16898 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:37.564616 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 186610us
I20260709 12:26:37.567028 10092 raft_consensus.cc:2889] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:37.564388 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 149065us
I20260709 12:26:37.570379  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:37.570832 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:37.572770  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:37.573575 16959 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:26:37.575748 16920 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:37.576566 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1262) took 1103 ms. Trace:
I20260709 12:26:37.584272 10092 rpcz_store.cc:276] 0709 12:26:36.469998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.486427 (+ 16429us) service_pool.cc:224] Handling call
0709 12:26:36.508098 (+ 21671us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:37.392347 (+884249us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b540023d320. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cba1d 00007f4e094b21a4 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419
0709 12:26:37.567834 (+175487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:37.568013 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284631936}
I20260709 12:26:37.563289 17005 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:37.593791 16772 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:37.593123  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:37.591749 16920 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:37.528920 16800 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.602859  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:37.563695 17231 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.577868 16956 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:37.603873  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:37.611647 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:37.612207  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:37.621891  9942 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:37.603622  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:37.583985 17264 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:37.626963 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57369.9us
I20260709 12:26:37.624897  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:37.599578 17271 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.629523 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:37.637714 10020 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.613929 16874 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.634358 10021 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:37.630152  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:37.652117  9942 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:37.641399 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:37.655869 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:37.656785  9941 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 22 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:37.639164 16861 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:37.639560 17267 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.657225  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:37.647225 17273 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:37.647536 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:37.647637 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:37.671124  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:37.623361 16956 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:37.623471 16920 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:37.688859 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:37.672689  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:37.668954 10022 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:37.691110  9945 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:37.674415 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:37.680887 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.680945 10097 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:37.692004 10091 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:37.680928 10095 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:37.629382 17251 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.627117 17264 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:37.647152 17262 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:26:37.687058 16956 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:37.730310  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.730930 17041 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:37.733075 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:37.758304  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62074us
I20260709 12:26:37.742098 10020 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:37.759071 10093 raft_consensus.cc:2368] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:37.651207 17271 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.734025 17263 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:26:37.652055 16917 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.087s
I20260709 12:26:37.669775 17105 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:26:37.635655 16772 consensus_queue.cc:1175] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd983415389444e292b2330ef630850a" responder_term: 23 status { last_received { term: 22 index: 16 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:37.671247 17186 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:26:37.779575 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:37.690419 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:37.630621 16800 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:37.693683  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:37.751411  9942 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:37.730042 17230 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.736840 16661 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.692126 16898 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:37.649423 17176 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 12:26:37.754069 16888 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:37.686704 17267 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.783182 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.802775 10098 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:37.783528 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 171965us
I20260709 12:26:37.785832 10097 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:37.756327 17264 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.752948 16861 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:37.791625 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.806267 10093 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:37.627535 17005 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:37.798220 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 219694us
I20260709 12:26:37.738543 17187 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:37.764467 17025 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:26:37.813478 10642 leader_election.cc:400] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:37.788561  9942 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:37.806773 10024 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: [18.10-18.10]
W20260709 12:26:37.801229  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 107644us
I20260709 12:26:37.815186 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:37.805445 17121 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:37.770283 17234 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:26:37.768076 17041 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.800648 16888 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:37.842446 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 152116us
I20260709 12:26:37.823644 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:37.846838 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.858654 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.860301 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:37.925989 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:37.940310  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:37.775374 17251 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:37.973129 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:37.975571 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 155143us
I20260709 12:26:37.977136 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:37.853587 16888 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:37.983233 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:37.843808 17234 consensus_queue.cc:1243] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:37.857661 17194 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:37.993001 17280 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:26:38.005123 10097 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:38.008487 17207 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.253s
I20260709 12:26:38.009415 10097 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:26:38.012750 17258 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.252s
I20260709 12:26:38.015596 16661 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.017760 17041 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.019927 17271 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:38.020010 10098 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:38.022697 10098 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:26:37.995340 17194 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:38.014369 17261 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.253s	user 0.002s	sys 0.000s
I20260709 12:26:38.036536 10097 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:38.027464 10091 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:26:38.029054 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:38.031076 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 210179us
I20260709 12:26:38.046618 10090 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.047130 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:38.047425 10093 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
W20260709 12:26:37.995066 17202 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.138s	user 0.000s	sys 0.000s
I20260709 12:26:38.052342 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:37.825384 16920 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.051708 10099 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:38.003954 17005 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:38.069094 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 225100us
I20260709 12:26:38.074942 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.075757 10671 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:38.076211 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.079115 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.079584 10676 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:38.123293 10669 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:38.049170 17263 consensus_queue.cc:1243] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 18.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:26:38.041414 17260 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:38.029583 17278 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.133302 10062 leader_election.cc:400] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:38.138159 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.051039 16661 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.003424 17269 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.450s
I20260709 12:26:38.068817 16898 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:38.149158 10062 leader_election.cc:403] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:38.149935 10090 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:38.059693 17271 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 12:26:38.141620 17289 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:38.176566 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 106184us
I20260709 12:26:38.135169 17176 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:26:38.181715 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:38.183926 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:38.184839 10020 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:38.187077 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:38.157444 17268 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:38.157166 16917 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.189328 10023 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:38.191205  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:38.165266 17258 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:38.175921 17178 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.173331 17289 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:38.199863 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:38.201153 17284 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:26:38.072544 16861 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.230454 10090 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:38.205057 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:38.208001  9945 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.000s
I20260709 12:26:38.199182 10021 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:38.209286 10668 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:26:38.210646 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:38.176311 17296 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.176102 17230 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.182008 17278 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:38.215440 10016 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:38.219050 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:38.276347 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.219771 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:38.220453 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:38.202831 17287 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:38.222087 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.225108  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:38.226016 17150 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:38.226781 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:38.227334 17231 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.227680 17292 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.228111 17288 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:38.228826 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.299410 10669 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:38.209252 16917 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:38.229300 17279 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.425s	user 0.002s	sys 0.000s
I20260709 12:26:38.229449 17295 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:38.182379 17260 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18.
I20260709 12:26:38.141485 17194 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.267104 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:38.281035 17278 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.307066 10020 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:38.269940 10023 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:38.308854 10023 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:38.270557  9945 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:38.310470 10097 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:38.271276 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:38.278261 10672 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.221261 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:38.279803 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:38.213229 17187 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:38.284168  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59154.4us
I20260709 12:26:38.290416 17041 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:38.242071 17267 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.289749 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:38.211786 17258 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.270152 16956 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.294730 17282 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:38.284171 16661 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.289880 10016 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
W20260709 12:26:38.300487  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.003s	sys 0.012s
I20260709 12:26:38.214445 17281 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 12:26:38.298532 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:38.331372  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.004s	sys 0.012s
W20260709 12:26:38.272917 17285 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:26:38.317559  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:38.290340 16941 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:26:38.318734 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47452.2us
I20260709 12:26:38.299072 17288 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:38.318686 16710 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:38.311367 17278 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:38.329157 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 352119us
I20260709 12:26:38.335600 10021 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:38.343801 10672 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
W20260709 12:26:38.346413  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.355114  9941 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:38.351768 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:38.355674 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.014s	sys 0.038s
W20260709 12:26:38.356012 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.014s	sys 0.038s
W20260709 12:26:38.356314 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 727874us
I20260709 12:26:38.353621  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:38.280726 17296 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.336207 17121 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:38.327766 17251 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.319089 17271 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:26:38.364480 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:38.298202 16898 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.302048 17292 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.350185 16941 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:38.365115 17150 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:38.357123 17267 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.021s
I20260709 12:26:38.306139 17295 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.354827 17108 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:38.386457 10642 leader_election.cc:400] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:38.386929 10642 leader_election.cc:403] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:26:38.389631 10064 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.002s	sys 0.019s
W20260709 12:26:38.390149 10064 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.003s	sys 0.019s
I20260709 12:26:38.390118  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:38.316864 17289 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.376248 17291 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:38.344020 16661 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.380414 17271 consensus_queue.cc:1243] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:38.429375 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:38.433377  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:38.436242 10672 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.435192 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.436973 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:38.437752 10098 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:38.437325 10669 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:38.383708 17108 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:38.447458 17306 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:26:38.449509 16874 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.781s	user 0.007s	sys 0.002s
I20260709 12:26:38.449977 16874 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:38.450736  9943 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:38.451488 17258 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.451795 17296 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.452051 17258 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:38.452708  9914 leader_election.cc:400] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:38.452625 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:38.453121  9914 leader_election.cc:403] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:38.453433 10090 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:38.453840 17258 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:38.387115 17150 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:38.392850 17269 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:38.393366 16710 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:38.444912 10672 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:38.439955 17268 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:38.467262 17300 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:26:38.456707 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:38.385032 17288 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.483814 17295 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.459499 17257 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:38.489678 17121 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:38.505983 17274 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.773s	user 0.000s	sys 0.007s
I20260709 12:26:38.495870 10093 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:38.507400 10093 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:38.458629  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:38.460139 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:38.509187 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49139.6us
I20260709 12:26:38.462201 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:38.384137 16888 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.515493 17269 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:38.489826 16710 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.517320 16888 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.457028 16941 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.518189 16941 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:38.459746 17289 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.518862 17237 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:38.519229 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:38.519760 17237 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:38.520022 10677 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:38.520143 17237 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:38.503415 16917 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.503204 16956 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.521324 17237 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:38.457589 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 76456us
W20260709 12:26:38.458809 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.524780  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66252.5us
I20260709 12:26:38.524899  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:38.526242  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.456385 17108 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:38.526811  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:38.546430 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 84322.8us
W20260709 12:26:38.500555 17266 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.774s	user 0.002s	sys 0.001s
I20260709 12:26:38.549451 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:38.551620 10097 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:38.498818 17292 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.554834 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:38.555030 16661 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.555620 16870 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:38.556301 16870 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:38.557430 16870 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.558956 17150 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.562714 17288 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.553167 10675 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:38.551139 10098 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:38.579854 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:38.580053 10096 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:38.547906 10093 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.587251 17292 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:38.518872 16710 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.588498 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:38.588965 10090 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:38.589618 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:38.615384 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:38.589653  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1557) took 2195 ms. Trace:
I20260709 12:26:38.616940 10097 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:38.529507 16800 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.735s	user 0.006s	sys 0.005s
I20260709 12:26:38.617086  9944 rpcz_store.cc:276] 0709 12:26:36.380223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.421177 (+ 40954us) service_pool.cc:224] Handling call
0709 12:26:36.421494 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:37.408141 (+986647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.575434 (+1167293us) spinlock_profiling.cc:243] Waited 986 ms on lock 0x7b54000b7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:38.575731 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2267608960}
I20260709 12:26:38.544902 17257 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:38.522532 17237 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:38.529369 17230 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.529148 17041 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.636405 17041 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.618322 17005 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:26:38.639093  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 15 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:38.640087 17185 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.640648 17185 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.643129 17230 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.645301 17230 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.646226 17185 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:38.647171 17041 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.650192 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:38.618276 17199 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:26:38.651419 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:38.652640 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:38.656551 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.662079 10093 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.672588 10676 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:38.673041 10676 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:38.588156 16661 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.673791 10676 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.674726  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:38.677412  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:38.678239  9949 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:38.682580 17194 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:38.683799  9943 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:26:38.684417 10676 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:38.688587  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:38.678417 17304 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.237s	user 0.002s	sys 0.000s
W20260709 12:26:38.697018 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 108604us
I20260709 12:26:38.653261 10095 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:38.523463 16959 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:38.705001 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:38.705814 10095 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:38.707070 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:38.709697  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 184869us
I20260709 12:26:38.635365 17268 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:38.714305  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:38.721561 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:38.715809  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:38.700356 17264 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.887s	user 0.006s	sys 0.005s
W20260709 12:26:38.699522 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 110011us
I20260709 12:26:38.725884 10093 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 12:26:38.726261 10093 pending_rounds.cc:85] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 14
I20260709 12:26:38.726473 10093 pending_rounds.cc:107] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:26:38.726786 10093 raft_consensus.cc:2889] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.728186 17278 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.730188 17278 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:38.723191 10095 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:38.732532 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.735644  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:38.701094 17186 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:38.736522 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:38.727157 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:38.743271 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:38.739053 10090 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:38.548790 16888 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.151s
I20260709 12:26:38.709270 17005 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:38.743338 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:38.738019 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:38.767561 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:38.771096 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2716) took 1084 ms. Trace:
I20260709 12:26:38.771559 10093 rpcz_store.cc:276] 0709 12:26:37.686194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.054006 (+367812us) service_pool.cc:224] Handling call
0709 12:26:38.054364 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.495755 (+441391us) spinlock_profiling.cc:243] Waited 86.9 ms on lock 0x7b54001d77a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:26:38.770855 (+275100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.771044 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":199969152}
I20260709 12:26:38.772248 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:38.767951 10019 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:38.768795 10025 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:38.733155 16959 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.747181 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:38.708504 17288 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:38.748626  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:38.780313 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 53236us
I20260709 12:26:38.780826  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:38.722126 17257 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:38.721262 16772 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:38.780848  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:38.787418 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:38.789466 10023 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:38.792059 10676 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:26:38.792403 10676 pending_rounds.cc:85] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 12
I20260709 12:26:38.794346 10676 pending_rounds.cc:107] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:26:38.794641 10676 raft_consensus.cc:2889] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.740166 16941 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:38.807946 10097 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:38.733284 16800 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.805761 16363 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:38.771274 17185 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:38.744890 16861 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:38.769510 16917 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:38.767560 17188 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:26:38.735939 17109 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:38.740463 17199 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.076s
I20260709 12:26:38.769783 17121 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:26:38.812933 10642 leader_election.cc:400] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:38.867826 10642 leader_election.cc:403] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:38.814186 10019 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:38.778880 16959 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.737600 17264 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:38.805260 17278 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:38.787374 17288 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:38.812568 17040 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 12:26:38.788635 17041 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:38.851914 16861 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.793195 16898 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:38.826370 17005 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:38.829629 17268 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.831154 17316 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:38.832242 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 85178.8us
I20260709 12:26:38.912915 10094 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:38.811133 17280 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
W20260709 12:26:38.789064 17105 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 18 current term: 18 is higher.
I20260709 12:26:38.810106 16941 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:38.809324 17295 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:38.896813 16363 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18.
I20260709 12:26:38.841866 17025 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.805922 16956 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:26:38.805876 16772 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.832391 17231 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:38.868996 16917 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:38.921355 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:38.910998 16861 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:38.912953 17316 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:38.895089 17230 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:26:38.871553 17109 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.852515 16800 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:38.947728  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 167012us
W20260709 12:26:38.947395  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 143865us
I20260709 12:26:38.951390 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:38.915047 16898 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:38.937021 16941 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:38.967478 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 161286us
I20260709 12:26:38.915277 17268 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:38.968811  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 188033us
I20260709 12:26:38.969381 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:38.936146 17199 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.025s
W20260709 12:26:38.974370  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 140095us
I20260709 12:26:38.970816 10022 pending_rounds.cc:85] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 14
I20260709 12:26:38.951197 17316 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:38.976258 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:38.976514 10022 pending_rounds.cc:107] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:26:38.955984 17231 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.974712  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:38.960062 17186 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:38.978742 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:38.979315 16959 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:38.978834 16898 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.974630 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:38.983819 10022 raft_consensus.cc:2889] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:38.998520 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:38.988322  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:38.988224 10673 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:38.992535 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:38.992892 10668 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:39.009716 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:38.994915  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:38.950704 17295 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.001808  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:39.004206  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:39.033967 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:39.038137 10669 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:39.042501 10016 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:39.045225  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:39.050050  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55250.8us
W20260709 12:26:39.055917  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.089876 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.114274  9990 leader_election.cc:400] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17.
I20260709 12:26:38.956393 17288 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.972226 17188 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:26:39.029984 17185 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:38.818222 17257 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.046277 16920 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:39.003633 16710 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:39.195338  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 144283us
I20260709 12:26:38.722860 17276 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.319s
I20260709 12:26:39.198721 10677 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:39.011931 17041 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:39.012945 17207 raft_consensus.cc:2764] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:39.172145 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:39.237217 10671 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:38.960758 16917 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.045333 17186 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:39.039992 16739 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.146s
I20260709 12:26:39.031194 16959 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.206s
I20260709 12:26:39.238498 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:39.311872 10674 raft_consensus.cc:2368] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:39.217833 16920 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:39.276697 17321 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.225s	user 0.000s	sys 0.000s
I20260709 12:26:39.276881 17109 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.284125 17258 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:39.221364 17185 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:39.362545  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 320572us
I20260709 12:26:39.170082 17224 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.105s
W20260709 12:26:39.350883 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.360371  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 124099us
W20260709 12:26:39.284152 17312 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.328s	user 0.001s	sys 0.000s
I20260709 12:26:39.379438 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:39.351027 17317 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.404s	user 0.000s	sys 0.001s
I20260709 12:26:39.385913 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:39.362778 17108 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.778s	user 0.007s	sys 0.003s
I20260709 12:26:39.386031  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:39.351181 16920 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:39.344115 16772 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.327750 17288 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.362640 17267 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:39.396042  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:39.371344 16800 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.413511  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:39.363317 17258 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:39.411054 10016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
W20260709 12:26:39.369760 17237 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.002s
I20260709 12:26:39.428316 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:39.285193 17323 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:26:39.430140 10671 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:39.432590 10675 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:39.409690 17041 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:39.430177 17109 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 12:26:39.395890 17108 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.448179 17041 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:39.449359 17108 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.449571 17041 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.424206 16772 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.408053 17186 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.359011 16941 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.454411 10671 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:39.460462 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:39.462071 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:39.411841 17267 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:39.443171 16800 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.411109 17327 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:26:39.455848 17276 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:39.451960 17295 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:39.506570 10097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.000s	sys 0.008s
W20260709 12:26:39.513278 10097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.000s	sys 0.009s
I20260709 12:26:39.513721 10675 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 12:26:39.514823 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1321) took 1274 ms. Trace:
I20260709 12:26:39.528381 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:39.529404 10671 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.462880 10674 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
W20260709 12:26:39.464782 17331 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:26:39.515533 10020 rpcz_store.cc:276] 0709 12:26:38.240790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.319429 (+ 78639us) service_pool.cc:224] Handling call
0709 12:26:38.319720 (+   291us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:38.349216 (+ 29496us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.349268 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:38.357181 (+  7913us) log.cc:844] Serialized 52 byte log entry
0709 12:26:38.697663 (+340482us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:38.697704 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.697891 (+   187us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.514553 (+816662us) raft_consensus.cc:1685] finished
0709 12:26:39.514574 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.514764 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7047,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":13319,"thread_start_us":315493,"threads_started":1,"wal-append.queue_time_us":534783}
I20260709 12:26:39.515460 10675 pending_rounds.cc:85] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 14
I20260709 12:26:39.536550 10675 pending_rounds.cc:107] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:26:39.537036 10675 raft_consensus.cc:2889] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:39.537643 10675 pending_rounds.cc:107] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 12:26:39.537988 10675 raft_consensus.cc:2889] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:39.464409 17186 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.544302 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:39.538437 17269 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.022s	user 0.004s	sys 0.008s
W20260709 12:26:39.555956 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.582227 10674 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:39.515693 17289 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:39.602079 10671 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
W20260709 12:26:39.606900 10642 leader_election.cc:336] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.560s (SENT)
W20260709 12:26:39.616245 17328 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:26:39.616679 17186 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:39.629550 17318 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.798s	user 0.008s	sys 0.040s
W20260709 12:26:39.634018 17314 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.814s	user 0.003s	sys 0.008s
I20260709 12:26:39.634810 17267 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.636270 17267 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:39.641040 17207 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:39.641659 17207 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:39.642294 16941 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.642710 17207 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.644174 17207 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:39.647274 17292 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:39.647903 17292 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 15 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:39.648824 17292 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.650315 17292 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:26:39.652751 17194 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.970s	user 0.011s	sys 0.000s
I20260709 12:26:39.592844 17025 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.658084 17025 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.514150 17258 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.663067 17258 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.588035 17269 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:39.671232 17319 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.309s	user 0.000s	sys 0.000s
I20260709 12:26:39.672583 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3880) took 1149 ms. Trace:
I20260709 12:26:39.672739 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3891) took 1028 ms. Trace:
I20260709 12:26:39.673087 10091 rpcz_store.cc:276] 0709 12:26:38.523484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.544953 (+ 21469us) service_pool.cc:224] Handling call
0709 12:26:38.545312 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.586124 (+ 40812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.586181 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.665035 (+ 78854us) log.cc:844] Serialized 26 byte log entry
0709 12:26:38.840968 (+175933us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:38.841007 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.843522 (+  2515us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.672233 (+828711us) raft_consensus.cc:1685] finished
0709 12:26:39.672269 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.672504 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27224,"raft.queue_time_us":32205,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":175491,"threads_started":1,"wal-append.queue_time_us":532019}
I20260709 12:26:39.676123 17185 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.594918 16710 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.677412 16710 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:38.993281 17231 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.689020 17231 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.513801 17276 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.691946 17276 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:39.708611 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.711722 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.720239 10670 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
W20260709 12:26:39.722430 10642 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.906s (SENT)
I20260709 12:26:39.673211 10098 rpcz_store.cc:276] 0709 12:26:38.644482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.661192 (+ 16710us) service_pool.cc:224] Handling call
0709 12:26:38.661637 (+   445us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.731242 (+ 69605us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.731278 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.769210 (+ 37932us) log.cc:844] Serialized 26 byte log entry
0709 12:26:38.985449 (+216239us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:38.985559 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.985660 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.672509 (+686849us) raft_consensus.cc:1685] finished
0709 12:26:39.672533 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.672678 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69152,"raft.queue_time_us":74249,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5359,"thread_start_us":152078,"threads_started":1,"wal-append.queue_time_us":606684}
W20260709 12:26:39.732295 10642 leader_election.cc:336] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.985s (SENT)
I20260709 12:26:39.733311 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:39.734894 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.032s (SENT) [suppressed 22 similar messages]
W20260709 12:26:39.739233 17316 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.741s	user 0.003s	sys 0.008s
I20260709 12:26:39.728900 17207 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.763077 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3874) took 2088 ms. Trace:
I20260709 12:26:39.777611 16920 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:39.779183 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.763934 10094 rpcz_store.cc:276] 0709 12:26:37.674356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.678869 (+  4513us) service_pool.cc:224] Handling call
0709 12:26:37.679180 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.913510 (+1234330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.762585 (+849075us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400094554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:39.762978 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2836759296}
W20260709 12:26:39.782061 10017 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.992s	user 0.004s	sys 0.016s
W20260709 12:26:39.785413 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:39.792316 10017 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.002s	user 0.004s	sys 0.016s
I20260709 12:26:39.793490 10016 raft_consensus.cc:606] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:39.793951 10016 raft_consensus.cc:614] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:39.794446  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.042s	sys 0.045s
W20260709 12:26:39.795559  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.042s	sys 0.045s
W20260709 12:26:39.796686  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 383250us
I20260709 12:26:39.798339  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:39.800243 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:39.801076  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:39.792165 17289 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.802325 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2470) took 1598 ms. Trace:
I20260709 12:26:39.766403 17316 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.768352 17207 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.806478  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:39.805081  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:39.803583 16661 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:39.839279 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:39.803486 17292 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:39.805485 17185 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:39.806810 10017 rpcz_store.cc:276] 0709 12:26:38.204306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.204522 (+   216us) service_pool.cc:224] Handling call
0709 12:26:38.205086 (+   564us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:38.212210 (+  7124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.212256 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:38.212993 (+   737us) log.cc:844] Serialized 52 byte log entry
0709 12:26:38.392624 (+179631us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:38.789922 (+397298us) log.cc:844] Serialized 14 byte log entry
0709 12:26:39.792781 (+1002859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.793311 (+   530us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.793364 (+    53us) raft_consensus.cc:1685] finished
0709 12:26:39.793376 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.802266 (+  8890us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":88821,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":49982,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":56417,"spinlock_wait_cycles":1026432,"thread_start_us":1459913,"threads_started":3,"wal-append.queue_time_us":183985}
I20260709 12:26:39.805465 17295 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.851428 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:39.851449 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3669) took 1117 ms. Trace:
I20260709 12:26:39.852727 10094 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:39.841089 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:39.843788  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:39.855829  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:39.812158 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.801s	user 0.012s	sys 0.011s
W20260709 12:26:39.859014 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.848s	user 0.012s	sys 0.011s
W20260709 12:26:39.859871 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 850219us
I20260709 12:26:39.846263 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:39.848245 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:39.865072 10019 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:39.852427 10022 rpcz_store.cc:276] 0709 12:26:38.733663 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.733806 (+   143us) service_pool.cc:224] Handling call
0709 12:26:38.767362 (+ 33556us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:39.040616 (+273254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.040654 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:39.044646 (+  3992us) log.cc:844] Serialized 52 byte log entry
0709 12:26:39.533555 (+488909us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:39.533612 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:39.623995 (+ 90383us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:39.624053 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.846736 (+222683us) raft_consensus.cc:1685] finished
0709 12:26:39.846765 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.851305 (+  4540us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":448,"raft.queue_time_us":185563,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":21614,"spinlock_wait_cycles":467379072,"thread_start_us":315860,"threads_started":1,"wal-append.queue_time_us":541923}
I20260709 12:26:39.871011 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:39.871939 17105 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:39.874836 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:39.809433 17289 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.880718 17289 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:39.877470 16920 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:39.855911 17292 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:39.883638 17289 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:39.884330 17289 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:39.884867 17292 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:39.851045 17327 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:26:39.841351 16710 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.840890 16941 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:39.886451 17292 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
W20260709 12:26:39.902856 16861 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.953s	user 0.013s	sys 0.000s
I20260709 12:26:39.903348 16861 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.905572 16861 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:39.849720 17185 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.906334 17185 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:39.907148 17209 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:26:39.907716 17209 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:39.908799 17209 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.910125 17209 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:39.848917 16661 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.912776 16661 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:39.913036 17320 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.093s	user 0.024s	sys 0.020s
I20260709 12:26:39.913596 17121 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:39.916769 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:39.888176 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:39.920214 10671 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:39.921528 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:39.922320 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3892) took 1277 ms. Trace:
I20260709 12:26:39.850922 17224 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:39.926257 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:39.922839 10096 rpcz_store.cc:276] 0709 12:26:38.644779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.663393 (+ 18614us) service_pool.cc:224] Handling call
0709 12:26:38.663811 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:38.708912 (+ 45101us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.708944 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:38.709418 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 12:26:38.795790 (+ 86372us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:38.798952 (+  3162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.799024 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:39.632547 (+833523us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:26:39.912450 (+279903us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:39.922049 (+  9599us) raft_consensus.cc:1685] finished
0709 12:26:39.922081 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:39.922252 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44713,"raft.queue_time_us":50974,"raft.run_cpu_time_us":11835,"raft.run_wall_time_us":911576,"spinlock_wait_cycles":56072832,"thread_start_us":85955,"threads_started":1,"wal-append.queue_time_us":110327}
I20260709 12:26:39.931779 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:39.932413 10090 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:39.933650 10097 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 16: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:39.868520 17194 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.869621 17025 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:39.891609 16941 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:39.884939 17289 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:39.857172 16739 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 12:26:39.924664 17230 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.892921 17207 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:39.894129 17316 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:39.927767 17224 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:39.948457 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 213594us
I20260709 12:26:39.949028 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:39.959411 10090 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:39.869012 17295 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:39.960474 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:39.944547 16941 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:39.983671 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.002863  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 133104us
I20260709 12:26:39.953478 17224 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.043679 10098 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:26:40.045195 17268 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:40.045969 17268 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:40.012281 10016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:26:39.944334 17194 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.011098 17207 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:40.026803 17315 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.210s	user 0.028s	sys 0.023s
W20260709 12:26:40.029597  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.916s	user 0.028s	sys 0.083s
W20260709 12:26:40.064471  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.029s	sys 0.084s
I20260709 12:26:40.034880 17041 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:40.035382 17326 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:39.950632 17230 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:40.065177  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.018420  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:40.071312 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.063624  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:40.088644 10671 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 15. Current term is 16. Ops: []
I20260709 12:26:40.092772 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.094089 10671 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:40.058830 17231 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.119999 17231 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:40.121230 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1536) took 1143 ms. Trace:
I20260709 12:26:40.121608 10668 rpcz_store.cc:276] 0709 12:26:38.978077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.978257 (+   180us) service_pool.cc:224] Handling call
0709 12:26:38.982618 (+  4361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.008571 (+ 25953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.121001 (+1112430us) spinlock_profiling.cc:243] Waited 9.87 ms on lock 0x7b54004d35d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:40.121174 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22751232}
I20260709 12:26:40.122152 17231 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:40.123955 16861 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:40.060385 17335 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:26:40.124852 16861 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:40.125126 17335 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:40.125558 17335 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:40.125910 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.100522 17257 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:40.126370 17335 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:40.126960 17257 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.127537 17335 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:40.129448 17257 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.100649 17337 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:40.097332 16959 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:40.126063 16861 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.131007 17185 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.132135 10021 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:40.130234 17337 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.134366 17353 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:40.134977 17353 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.136229 17295 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.136495 17337 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.136973 17260 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:26:40.137362 17260 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:40.137457  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 119150us
I20260709 12:26:40.138217 17260 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.138980 17194 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.139343 17260 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:40.140341 17339 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:40.140892 17339 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.099242 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.144814 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.145273 10674 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:40.145591 10094 raft_consensus.cc:2410] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:40.103579  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:40.080454 17326 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.108018  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.147239  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:40.147768  9948 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.108891 16739 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:40.150838 10020 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:40.150738 10097 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:40.109776  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:40.152037 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.152396  9942 raft_consensus.cc:2410] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:26:40.153422  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:40.154145  9942 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:40.098079 17316 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:40.144656  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:40.157918 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:40.158859 10019 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:40.112421  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.080561 17041 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:40.066505 17336 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:40.116743  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:40.117676 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.180415 10098 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:40.118113  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.120332 17264 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:40.124060 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1481) took 1636 ms. Trace:
I20260709 12:26:40.130267 16959 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:40.101538  9949 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:26:40.099455 10677 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 12:26:40.102382 17230 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:40.148320 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.188376 10024 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:40.189535 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.148553 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.183065 10672 rpcz_store.cc:276] 0709 12:26:38.488118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.498996 (+ 10878us) service_pool.cc:224] Handling call
0709 12:26:38.499383 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:38.499786 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:38.499814 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:38.500485 (+   671us) log.cc:844] Serialized 52 byte log entry
0709 12:26:38.708221 (+207736us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:38.736787 (+ 28566us) log.cc:844] Serialized 14 byte log entry
0709 12:26:38.736992 (+   205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:38.737058 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.026056 (+1288998us) spinlock_profiling.cc:243] Waited 98.6 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:40.123772 (+ 97716us) raft_consensus.cc:1685] finished
0709 12:26:40.123866 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.124006 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17819,"prepare.queue_time_us":23479,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":23363,"raft.queue_time_us":5714,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":24409,"spinlock_wait_cycles":226707200,"thread_start_us":198569,"threads_started":1,"wal-append.queue_time_us":315814}
I20260709 12:26:40.046478 17268 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.067181 17207 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:40.155431 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.156185  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.143128 17353 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.111516  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.160614  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.160871 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.161204 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.201843  9948 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:40.163949 17185 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.175127 10016 raft_consensus.cc:606] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:40.202944 10016 raft_consensus.cc:614] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:40.158941  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:40.180889  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.204041  9947 raft_consensus.cc:2410] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:40.182390 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:40.204495 10023 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.205866 17268 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:40.206283  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.206755 10018 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:40.207484 17339 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.207749  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.208276 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.208950 16959 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:40.211035 17207 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.196493 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.212231 10024 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:40.190929 10017 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:40.196563  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.213536  9944 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:40.214469  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.216176  9944 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 12:26:40.179930 17341 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:40.216279 17326 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.157930 16739 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.220801 16739 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.224422 17289 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:40.225306  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66433.7us
I20260709 12:26:40.226564  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:40.227253 17316 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:40.227716 17316 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:40.227988  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:40.228098 17316 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.207518 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:40.229310  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:40.230983 10018 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:26:40.229861 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.207518  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.233887  9949 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:40.238098 10023 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:40.242106  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:40.247911 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:40.241658 16920 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:26:40.251521 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 406765us
I20260709 12:26:40.189877 17295 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:40.253813 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:40.230736 17260 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.262115 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:40.206423 17268 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:26:40.265199 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:40.262742  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:40.186460 17336 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:40.247468 16959 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:40.271862 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.243031 17341 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:40.269551  9943 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:40.252455 17326 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:40.270563 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.258603 17207 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.242672 17150 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:26:40.284072 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:40.307722 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:40.261304 17289 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.263484 16898 raft_consensus.cc:2764] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:40.317960 17335 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:40.320149  9945 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:40.321949 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:40.278743 17194 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:40.272081 17264 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:40.358543 17264 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.359054 17264 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.289464 16710 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:40.260854 17295 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.327916 17025 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:40.291507 16959 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:40.368580 17041 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:40.371783  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 293229us
I20260709 12:26:40.347519 17257 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:40.337062 17326 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:40.374308 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1537) took 1389 ms. Trace:
I20260709 12:26:40.286295 17260 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.381974 10676 rpcz_store.cc:276] 0709 12:26:38.985526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:38.985735 (+   209us) service_pool.cc:224] Handling call
0709 12:26:38.987668 (+  1933us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:38.987881 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:39.044911 (+ 57030us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.023542 (+978631us) spinlock_profiling.cc:243] Waited 5.36 ms on lock 0x7b18003b25a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:26:40.373964 (+350422us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.374012 (+    48us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:40.374066 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.374130 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.374254 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42209,"prepare.queue_time_us":64082,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"spinlock_wait_cycles":12327168,"thread_start_us":350216,"threads_started":1}
I20260709 12:26:40.377415 17257 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:40.394703 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.373312 16959 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:40.375573 17276 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:40.381397 17326 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.328097 17207 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.336256 17224 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:40.425359 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 303039us
I20260709 12:26:40.397753 16888 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.373816 17230 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:26:40.403731 17257 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:40.423882 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 102017us
I20260709 12:26:40.428376 16959 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:40.426779 17339 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:40.428498 17268 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.392486 17341 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:40.491597 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.495430 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.444363 16888 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.499195 16888 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.430974 17326 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:40.508572  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 280636us
I20260709 12:26:40.510977 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:40.511710 10098 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:40.439045  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 168772us
W20260709 12:26:40.460353  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 231105us
I20260709 12:26:40.512794 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.513796 10672 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 14: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:40.514442 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.515209 10671 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:40.432021 17352 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:40.430500 17349 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.466235  9941 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:26:40.442332 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1542) took 1410 ms. Trace:
I20260709 12:26:40.526436 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:40.526289 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:40.451671 17258 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.060s
W20260709 12:26:40.499148  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 17 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:40.534642 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.535593 10097 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:40.537460 10672 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:26:40.480306  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 217609us
I20260709 12:26:40.521327  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.539958  9940 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:40.522210  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:40.522610  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 120545us
I20260709 12:26:40.523999 10676 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:40.530085  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.467255 16959 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:40.459530 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:40.542785  9948 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:40.543602 10096 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:40.544417 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.544631  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.544034  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.550632 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.527135 10669 rpcz_store.cc:276] 0709 12:26:39.032779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.048047 (+ 15268us) service_pool.cc:224] Handling call
0709 12:26:39.219252 (+171205us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:39.219551 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:39.780049 (+560498us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.436576 (+656527us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:40.436648 (+    72us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:40.436714 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.442050 (+  5336us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.442248 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":996,"prepare.queue_time_us":111048,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":214993,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7631,"thread_start_us":1050206,"threads_started":3,"wal-append.queue_time_us":534249}
I20260709 12:26:40.459575 16941 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:40.553565 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:40.554692 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 95225.4us
I20260709 12:26:40.556829 10095 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:26:40.562738 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:40.563598 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:40.564251 10671 raft_consensus.cc:2376] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 19. Re-sending same reply.
I20260709 12:26:40.566706 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:40.567368 10672 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:40.570926 10095 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:40.572113 10676 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:40.573336 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.574395 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.579632 10671 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 13. Current term is 14. Ops: []
I20260709 12:26:40.519922 17352 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:40.580721 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.573994 10097 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:40.604799 10096 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:40.605499 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.587273 10098 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:40.534974 17280 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.589329 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.616276 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:40.603035 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.550127 16959 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:40.625092 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:40.625890 10669 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:40.623391 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:40.610862 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:40.629457 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:40.629637 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:40.550889 17209 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.001s
I20260709 12:26:40.503325 17268 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.533352 17349 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.604033 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.621778 10672 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:26:40.622696 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.623474 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:40.641382 10675 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:40.640147 10676 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:40.630477 10095 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:26:40.645285 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.581696 16920 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:40.631428 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.569952 17335 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:40.580902 17289 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:26:40.645870 10092 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:40.624462 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.640846 10668 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:40.603052 17336 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:40.659869 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:40.663888 10670 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:40.669365 10098 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:40.679097 10096 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: [20.14-22.15]
I20260709 12:26:40.580371 17371 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.097s
I20260709 12:26:40.690618 10677 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:40.612346 17269 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:26:40.695564 10099 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:26:40.652465 16920 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:40.658336 17289 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:40.690948 17351 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
I20260709 12:26:40.688635 17350 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:40.675169 17335 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:40.631305 17280 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:40.721787 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:40.712529  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 166979us
I20260709 12:26:40.724678  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:40.718156 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.699427 17303 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.706103 17289 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:40.757768 10675 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:40.739114  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:40.789202 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:40.795204  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56190.6us
W20260709 12:26:40.750141 17355 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:26:40.820502 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.754225 17194 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:40.835044 10674 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:40.827876 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.836809 10020 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:40.827944 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.841403 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.800002 17292 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:40.783757 17377 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:40.852208  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.858345  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2806) took 1857 ms. Trace:
I20260709 12:26:40.872364  9946 rpcz_store.cc:276] 0709 12:26:39.001320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.002343 (+  1023us) service_pool.cc:224] Handling call
0709 12:26:39.013710 (+ 11367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.041557 (+ 27847us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.041593 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.797980 (+756387us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b2c00012928. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:26:39.802849 (+  4869us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.230918 (+428069us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:40.230967 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.231304 (+   337us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.832721 (+601417us) raft_consensus.cc:1685] finished
0709 12:26:40.832739 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.858249 (+ 25510us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":97,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":124853,"spinlock_wait_cycles":5426048,"thread_start_us":389018,"threads_started":1,"wal-append.queue_time_us":522254}
I20260709 12:26:40.853252 10018 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:40.853906  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:40.851727  9948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "840d8356270745f492c7d2c7099794c1"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:26:40.878098  9948 raft_consensus.cc:606] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:40.855221 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:40.862112 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.869477 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.870883  9949 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:40.879213 10099 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
W20260709 12:26:40.869187 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 225681us
I20260709 12:26:40.891706 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:40.831064 17258 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:40.893139 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:40.899839  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:40.901669 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.736858 17376 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.121s
I20260709 12:26:40.846261 17231 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:40.719240 17350 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:40.871510 17316 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.610s	user 0.006s	sys 0.006s
I20260709 12:26:40.706368 17352 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.888531 17260 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.943010 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2728) took 1022 ms. Trace:
I20260709 12:26:40.943392 10091 rpcz_store.cc:276] 0709 12:26:39.920811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.922517 (+  1706us) service_pool.cc:224] Handling call
0709 12:26:39.922835 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.923166 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:39.923194 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:39.928371 (+  5177us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.298551 (+370180us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:40.298612 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.298680 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:40.942786 (+644106us) raft_consensus.cc:1685] finished
0709 12:26:40.942808 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:40.942949 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2944,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7729,"thread_start_us":348366,"threads_started":1,"wal-append.queue_time_us":565451}
W20260709 12:26:40.911489  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 115219us
I20260709 12:26:40.806921 16739 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.065s
I20260709 12:26:40.745534 17303 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:40.847955 17209 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:40.854681 17391 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.907876 17378 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.957620 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:40.941581 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50009.1us
I20260709 12:26:40.949131  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:40.928812 17357 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 12:26:40.996833 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:40.899718 17258 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:40.958021 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:40.916450 17207 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:40.991214  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:40.931535  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:40.940567  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 150200us
I20260709 12:26:40.956225 17292 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:40.989283 17378 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.008991 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:41.018002 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:41.032071 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:40.887234 17335 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:40.950783 16941 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:41.020596 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:41.060207 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:40.933894 17380 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.110s
I20260709 12:26:41.004263 17406 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:40.892473 17377 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.076591 10669 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:26:41.141551 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 150544us
I20260709 12:26:41.143652 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:41.005904 17258 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:41.082196  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 182444us
I20260709 12:26:40.935563 17231 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.005505 17268 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:41.028713 16898 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:26:41.061846 17368 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.386s	user 0.001s	sys 0.000s
W20260709 12:26:41.146023 17394 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:41.005074 16739 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.092s
I20260709 12:26:41.016497 17398 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:41.081638  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 139733us
I20260709 12:26:40.941202 17316 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.176494 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:40.996891 17209 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.060480 17388 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:41.036329 17349 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:41.195331 10016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
W20260709 12:26:41.144289 17360 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:26:41.214915 10016 raft_consensus.cc:606] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:41.153270 17041 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:41.218886  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 206860us
I20260709 12:26:41.144424 17362 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:41.156947  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:41.150254 17366 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:41.190685 17367 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:26:41.220757 10676 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:26:41.241035 10676 pending_rounds.cc:85] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 14
I20260709 12:26:41.241324 10676 pending_rounds.cc:107] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:26:41.241555 10676 raft_consensus.cc:2889] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:40.997956 17391 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:41.214349 17257 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.762s	user 0.010s	sys 0.002s
W20260709 12:26:41.209678 17373 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.483s	user 0.000s	sys 0.001s
W20260709 12:26:41.196622 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 136499us
I20260709 12:26:41.190361 17269 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:26:41.209471 17295 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.271310 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:41.232611 17207 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:41.294310 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3683) took 1117 ms. Trace:
I20260709 12:26:41.292936  9912 leader_election.cc:400] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:41.303030 10019 rpcz_store.cc:276] 0709 12:26:40.176866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.177047 (+   181us) service_pool.cc:224] Handling call
0709 12:26:40.182852 (+  5805us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.222859 (+ 40007us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.222889 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.223352 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.434894 (+211542us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:40.434934 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.523310 (+ 88376us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b540011ba54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:40.523365 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.259252 (+735887us) raft_consensus.cc:1685] finished
0709 12:26:41.259272 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.294231 (+ 34959us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20877,"raft.queue_time_us":25078,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":43066112,"thread_start_us":211161,"threads_started":1,"wal-append.queue_time_us":379226}
I20260709 12:26:41.239754 16898 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:41.239880 17349 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:41.253118 17366 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:41.309391 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48272.5us
W20260709 12:26:41.304373  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 79646us
I20260709 12:26:41.311556 17366 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:41.311991  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:41.313217  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:41.316136  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:41.321091 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:41.318285  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:41.372550 17367 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:26:41.380611 17316 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.380898 17185 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:41.381414 17316 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:41.381558 17185 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:41.382668 17360 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:26:41.383502 17400 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:26:41.384686 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1483) took 2378 ms. Trace:
I20260709 12:26:41.385080 10673 rpcz_store.cc:276] 0709 12:26:39.006554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.893064 (+886510us) service_pool.cc:224] Handling call
0709 12:26:39.893462 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.083799 (+190337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.083827 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.084307 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.439202 (+354895us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:40.452390 (+ 13188us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.491382 (+ 38992us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.491474 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.384443 (+892969us) raft_consensus.cc:1685] finished
0709 12:26:41.384467 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.384624 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41943,"prepare.queue_time_us":12968,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":201776,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":22174,"thread_start_us":544449,"threads_started":2,"wal-append.queue_time_us":750733}
I20260709 12:26:41.259089 17428 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.304967 17336 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 12:26:41.297019 17416 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:41.297228  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 140360us
I20260709 12:26:41.239732 17041 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.434254  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:41.393276  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.017s	sys 0.042s
I20260709 12:26:41.301900 16917 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:41.184386 17406 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.225518 17388 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:41.390782 17373 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:26:41.439851 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:41.437067  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:41.388482 16941 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:41.442775 10673 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:41.437002  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.017s	sys 0.042s
I20260709 12:26:41.259852 17257 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.214998 17292 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.443602  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:41.461954 17391 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:41.462193 17415 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:26:41.437621 16917 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:41.463090 16917 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.437819 17436 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:41.445135 17433 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.465463 17433 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:41.466044 17433 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.468286 17433 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:41.388499 17372 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.009s	sys 0.029s
W20260709 12:26:41.465562 17364 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.731s	user 0.000s	sys 0.001s
I20260709 12:26:41.209949 17398 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.438611 17316 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:41.491719  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:41.499447  9948 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:41.492719  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:41.462126 17185 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.440387 17350 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:41.520260  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:41.578006 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:41.557842 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 236841us
I20260709 12:26:41.471549 16739 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:41.440341 16898 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:41.591504 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:41.557052 10096 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.007s	sys 0.012s
W20260709 12:26:41.607825 10096 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.008s	sys 0.013s
W20260709 12:26:41.481634 16959 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.841s	user 0.012s	sys 0.000s
I20260709 12:26:41.593890 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:41.602092 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3681) took 1448 ms. Trace:
I20260709 12:26:41.602180 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:41.609428 10676 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:26:41.610246 10020 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
W20260709 12:26:41.486233 17372 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.009s	sys 0.030s
W20260709 12:26:41.618156 17401 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
I20260709 12:26:41.619948 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:41.622385 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:41.623278 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:41.625528 10019 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:41.629189 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:41.499776 17353 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:41.608740 10025 rpcz_store.cc:276] 0709 12:26:40.153279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.153403 (+   124us) service_pool.cc:224] Handling call
0709 12:26:40.153677 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.159499 (+  5822us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.159543 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.160071 (+   528us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.422319 (+262248us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:40.422357 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.422428 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.465012 (+1042584us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:41.584311 (+119299us) raft_consensus.cc:1685] finished
0709 12:26:41.584375 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.602012 (+ 17637us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5462,"raft.queue_time_us":25967,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":5535,"spinlock_wait_cycles":49087744,"thread_start_us":261920,"threads_started":1,"wal-append.queue_time_us":540404}
I20260709 12:26:41.638572 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2730) took 1708 ms. Trace:
I20260709 12:26:41.638967 10093 rpcz_store.cc:276] 0709 12:26:39.929717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.929839 (+   122us) service_pool.cc:224] Handling call
0709 12:26:39.930133 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:39.930512 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:40.251257 (+320745us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.481408 (+230151us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.548046 (+ 66638us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.548243 (+   197us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:40.548291 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.548377 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.638406 (+1090029us) raft_consensus.cc:1685] finished
0709 12:26:41.638424 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.638526 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":737,"prepare.queue_time_us":501166,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":6566,"raft.queue_time_us":478,"raft.run_cpu_time_us":304,"raft.run_wall_time_us":18034,"thread_start_us":491426,"threads_started":2,"wal-append.queue_time_us":374219}
I20260709 12:26:41.554286 17436 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:41.497613 17429 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:41.578356 17389 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.092s
I20260709 12:26:41.582583 17391 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:41.593919 17336 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 12:26:41.594013 17295 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:26:41.535614 17231 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.579435 16941 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:41.571936 17378 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:41.499480 17416 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.164s
W20260709 12:26:41.684073 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61824.8us
I20260709 12:26:41.634158  9941 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:26:41.618629 17209 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.638098 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3909) took 1034 ms. Trace:
I20260709 12:26:41.662477 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2472) took 1570 ms. Trace:
I20260709 12:26:41.585752 17409 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:41.612852 17264 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.253s	user 0.009s	sys 0.005s
I20260709 12:26:41.603401 17428 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.644170 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:41.667879 10669 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:41.723459 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:41.785610 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 92271us
I20260709 12:26:41.585232 17384 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:41.789471 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 145398us
I20260709 12:26:41.771467 16941 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.498622 17316 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:41.607388 16898 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.629098 17276 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:41.737004 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:41.679777 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3912) took 1018 ms. Trace:
I20260709 12:26:41.804956 10098 rpcz_store.cc:276] 0709 12:26:40.661383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.467177 (+805794us) service_pool.cc:224] Handling call
0709 12:26:41.467469 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.467685 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:41.467939 (+   254us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.679293 (+211354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.679371 (+    78us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:41.679434 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.679529 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.679698 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":551,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":145,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":3999,"thread_start_us":210798,"threads_started":1}
I20260709 12:26:41.661804 17447 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:41.743897 17289 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.951s	user 0.010s	sys 0.000s
I20260709 12:26:41.747999 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3901) took 1228 ms. Trace:
I20260709 12:26:41.749811 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3900) took 1230 ms. Trace:
I20260709 12:26:41.813481 10094 rpcz_store.cc:276] 0709 12:26:40.519230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.520780 (+  1550us) service_pool.cc:224] Handling call
0709 12:26:40.521157 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.521564 (+   407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.521605 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.531068 (+  9463us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.741725 (+210657us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:40.741781 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.741866 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.749570 (+1007704us) raft_consensus.cc:1685] finished
0709 12:26:41.749595 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.749751 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6327,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":28595,"thread_start_us":114652,"threads_started":1,"wal-append.queue_time_us":1165363}
W20260709 12:26:41.750384 17258 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.552s	user 0.011s	sys 0.000s
I20260709 12:26:41.725993 17391 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.755754 17406 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:41.687505 17436 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:41.757074 16920 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.052s	user 0.009s	sys 0.000s
I20260709 12:26:41.634734 16661 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:41.759171 17257 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:41.765240 17326 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.265s	user 0.011s	sys 0.000s
I20260709 12:26:41.716010 17433 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:41.629835 17185 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:41.781700 10022 rpcz_store.cc:276] 0709 12:26:40.092320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.094929 (+  2609us) service_pool.cc:224] Handling call
0709 12:26:40.113587 (+ 18658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.113953 (+   366us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:40.392184 (+278231us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.793099 (+400915us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.803371 (+ 10272us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.805647 (+  2276us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:40.808298 (+  2651us) log.cc:844] Serialized 14 byte log entry
0709 12:26:40.831927 (+ 23629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.832000 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.662250 (+830250us) raft_consensus.cc:1685] finished
0709 12:26:41.662270 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.662417 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22716,"prepare.queue_time_us":411905,"prepare.run_cpu_time_us":103,"prepare.run_wall_time_us":6336,"raft.queue_time_us":338798,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":180,"thread_start_us":635775,"threads_started":2,"wal-append.queue_time_us":877893}
I20260709 12:26:41.781546 10096 rpcz_store.cc:276] 0709 12:26:40.604040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.709019 (+104979us) service_pool.cc:224] Handling call
0709 12:26:40.729273 (+ 20254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.729859 (+   586us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:40.793211 (+ 63352us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.637446 (+844235us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.637523 (+    77us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:41.637597 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.637840 (+   243us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.638021 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14926,"prepare.queue_time_us":69630,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":45,"raft.queue_time_us":53669,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"thread_start_us":814814,"threads_started":1,"wal-append.queue_time_us":818749}
I20260709 12:26:41.727639 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3911) took 1096 ms. Trace:
I20260709 12:26:41.688117 17339 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:41.795976 17428 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:41.801594  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 444038us
I20260709 12:26:41.628243 16959 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:41.811616 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74770.8us
I20260709 12:26:41.812321 10090 rpcz_store.cc:276] 0709 12:26:40.519503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.519685 (+   182us) service_pool.cc:224] Handling call
0709 12:26:40.550520 (+ 30835us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:40.553515 (+  2995us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:40.553561 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:40.554275 (+   714us) log.cc:844] Serialized 52 byte log entry
0709 12:26:40.741281 (+187006us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:40.741322 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.741397 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.747713 (+1006316us) raft_consensus.cc:1685] finished
0709 12:26:41.747743 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.747926 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2476,"raft.queue_time_us":7523,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":10413,"thread_start_us":108950,"threads_started":1,"wal-append.queue_time_us":1142600}
I20260709 12:26:41.784842 17409 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:41.824885 10094 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:41.804504 16898 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:41.829025 17436 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:41.816206 17447 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.793287 17264 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.817476 17209 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:41.851815 17433 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:41.797289  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 353790us
I20260709 12:26:41.824551 17419 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:26:41.829566 17276 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.815300 17289 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.835961 17041 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:41.836793 16920 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.905740 10094 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:41.866555 17420 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.909014 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2742) took 1019 ms. Trace:
W20260709 12:26:41.886269 17402 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:26:41.825229 17258 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.831720 17424 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:41.854619 17339 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:41.881390 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:41.900710 17452 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.882179 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:41.855986 17185 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.883759 17414 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:41.884652 17398 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:41.900238 17453 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:41.895506  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:41.864643 17409 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:41.835556 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3915) took 1110 ms. Trace:
W20260709 12:26:41.887353 17399 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.251s	user 0.000s	sys 0.001s
I20260709 12:26:41.848306 17326 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.850091 10099 rpcz_store.cc:276] 0709 12:26:40.631034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.204516 (+573482us) service_pool.cc:224] Handling call
0709 12:26:41.228047 (+ 23531us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.228262 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:41.254520 (+ 26258us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.727142 (+472622us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.727217 (+    75us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:41.727287 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.727385 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.727565 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33367,"prepare.queue_time_us":67853,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":10584,"raft.queue_time_us":110935,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":87,"thread_start_us":407155,"threads_started":2,"wal-append.queue_time_us":496291}
I20260709 12:26:41.914216 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:41.887717 17391 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.882331 17433 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:41.899703 17428 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:41.898203 17289 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:41.882014 10097 raft_consensus.cc:2368] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:41.935366  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:41.897609 17372 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:41.897060 17264 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:41.837395 17316 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.973034 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:41.919297 10669 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:41.883574 17375 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.883617 17418 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:26:41.898095 17419 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:41.864997 17335 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:41.888455 16917 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.924943 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:41.909854 10098 rpcz_store.cc:276] 0709 12:26:40.890020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.806937 (+916917us) service_pool.cc:224] Handling call
0709 12:26:41.807306 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.807548 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:41.828041 (+ 20493us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.907612 (+ 79571us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.907701 (+    89us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:41.907797 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.907928 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.908934 (+  1006us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21872,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":34,"raft.queue_time_us":857,"thread_start_us":78775,"threads_started":1}
I20260709 12:26:41.925648 10093 rpcz_store.cc:276] 0709 12:26:40.726007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.672141 (+946134us) service_pool.cc:224] Handling call
0709 12:26:41.672568 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.672839 (+   271us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:41.730522 (+ 57683us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.834655 (+104133us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.834708 (+    53us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:41.834758 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:41.834820 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.835498 (+   678us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4646,"prepare.queue_time_us":58089,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":4752,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":8442,"thread_start_us":99031,"threads_started":1,"wal-append.queue_time_us":186889}
I20260709 12:26:41.997035 10090 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:41.997466 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:41.917245 17424 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:41.895428 17257 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:26:41.893741 17440 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:26:41.943881 16920 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:41.925627 17414 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.947582 17390 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:26:41.948818 17258 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.953614 17444 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:41.912657 17420 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.952322 17326 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.952983 10023 raft_consensus.cc:2368] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24.
I20260709 12:26:41.952927 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1299) took 1376 ms. Trace:
I20260709 12:26:41.958236 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:41.958585 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:41.958755 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.010974 10675 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:42.011113 10668 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:41.959332 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:41.973470  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 70383.1us
W20260709 12:26:41.994571  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 121454us
I20260709 12:26:41.997555 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:42.014281 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.015391 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.016647 10093 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:41.891002 17276 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:41.998241 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:42.002383 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.004052 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:41.937136 17372 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:41.950932 17185 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:41.972499 17398 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.006494 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.001346 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.006754 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:42.025576 10090 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:42.000316 17424 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.022265 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.005889 17429 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.009474 10095 rpcz_store.cc:276] 0709 12:26:40.576281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.661972 (+ 85691us) service_pool.cc:224] Handling call
0709 12:26:40.662512 (+   540us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:40.675401 (+ 12889us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:40.675440 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:40.676083 (+   643us) log.cc:844] Serialized 26 byte log entry
0709 12:26:40.804814 (+128731us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:40.804869 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:40.804958 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:41.952719 (+1147761us) raft_consensus.cc:1685] finished
0709 12:26:41.952740 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:41.952868 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12239,"raft.queue_time_us":27344,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":12064,"thread_start_us":105483,"threads_started":1,"wal-append.queue_time_us":1014699}
I20260709 12:26:42.030054 10677 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:42.031334 10098 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:41.922536 17452 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.033046  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:42.035768 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.036641 10675 raft_consensus.cc:2376] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 20. Re-sending same reply.
I20260709 12:26:42.033466 10676 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.18-21.18]
I20260709 12:26:42.039819 10671 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:42.002235 17335 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:41.929522 17453 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.047254 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:42.048120 10095 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:42.022446 17423 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:42.024185 17416 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.050745  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:42.041335 16861 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:26:42.000090 17375 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.060747 17375 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.063127 17447 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.067049 17280 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:42.068759 17397 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.320s	user 0.001s	sys 0.003s
I20260709 12:26:42.071792 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3916) took 1061 ms. Trace:
W20260709 12:26:42.072499 17459 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:26:42.072861 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2744) took 1182 ms. Trace:
I20260709 12:26:42.073357 10096 rpcz_store.cc:276] 0709 12:26:40.890654 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.849751 (+959097us) service_pool.cc:224] Handling call
0709 12:26:41.862328 (+ 12577us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:41.862882 (+   554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:41.862928 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:41.865280 (+  2352us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.002774 (+137494us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:42.002817 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.002883 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:42.072649 (+ 69766us) raft_consensus.cc:1685] finished
0709 12:26:42.072669 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.072803 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1763,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":24163,"thread_start_us":81680,"threads_started":1,"wal-append.queue_time_us":177253}
I20260709 12:26:42.035130 17398 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.075349 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1490) took 1514 ms. Trace:
I20260709 12:26:42.075736 10674 rpcz_store.cc:276] 0709 12:26:40.560750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.562668 (+  1918us) service_pool.cc:224] Handling call
0709 12:26:42.075179 (+1512511us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b540045ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:42.075289 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398887808}
I20260709 12:26:42.076442 17398 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.028537 17444 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:42.077911 17444 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.078699 17419 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.079324 17185 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:42.079763 17185 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.080127 17185 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:42.082427  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49458.6us
I20260709 12:26:42.082646 10668 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:42.072404 10094 rpcz_store.cc:276] 0709 12:26:41.010577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.858487 (+847910us) service_pool.cc:224] Handling call
0709 12:26:41.858787 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.906156 (+ 47369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.071438 (+165282us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b5400164754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:42.071706 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107258112}
I20260709 12:26:42.086688  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:42.088322 16959 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.088896 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2729) took 2167 ms. Trace:
W20260709 12:26:42.091727 10097 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1302) took 1514 ms (client timeout 1583 ms). Trace:
I20260709 12:26:42.094492  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.094884  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.099298 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:42.099376 10092 rpcz_store.cc:276] 0709 12:26:39.921083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:39.921228 (+   145us) service_pool.cc:224] Handling call
0709 12:26:42.088741 (+2167513us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b540015bdd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:42.088843 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1663918336}
I20260709 12:26:42.095193  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.101480  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.095304 17303 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.095563  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.096149  9946 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:42.099462 10097 rpcz_store.cc:269] 0709 12:26:40.576985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.605173 (+ 28188us) service_pool.cc:224] Handling call
0709 12:26:42.091518 (+1486345us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54002fc954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:42.091641 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2933590912}
I20260709 12:26:42.108126  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.097577 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:42.108955  9948 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:26:42.104174  9945 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:42.105003 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.103227 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.109613 10018 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:42.110035 10094 raft_consensus.cc:2368] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:42.107285  9940 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:42.114748 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:42.115569 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.116122 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.116959 10095 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:42.116961  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.121737  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:42.122088 10090 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:42.123343 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.126003 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.132179 10016 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:26:42.132509 10096 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:42.133406 10098 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: [15.13-16.14]
I20260709 12:26:42.071852 17420 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.122925 17194 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:42.128755 10022 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:42.127251 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:42.047636 17327 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:26:42.111470 16941 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:42.158403  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:42.136904  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:42.166071  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 116133us
I20260709 12:26:42.166877  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:42.123145 17378 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:42.111814 17335 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:42.137007 17341 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.641s	user 0.009s	sys 0.006s
W20260709 12:26:42.180518 17341 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.685s	user 0.009s	sys 0.006s
W20260709 12:26:42.164036  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 75871.8us
I20260709 12:26:42.081774 17416 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.181208 17477 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.136483 16959 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:42.183390 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.184551 10018 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:42.034830 17429 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.157424 17303 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:42.163636 17194 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:42.164323 17425 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.189460  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:42.135711 17398 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
W20260709 12:26:42.202379 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 91893.6us
I20260709 12:26:42.184607 17427 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.135458 17447 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:26:42.168254 17264 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.195716  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:42.181806 17477 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.217445 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.218097 10025 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:42.218793 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.219116 17477 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.220108 17442 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:42.220221 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.220602 17442 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.221009 17442 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.221345 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.222368 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.191303 17230 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:26:42.201668  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64846.3us
I20260709 12:26:42.223522 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.177623 17257 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:26:42.192222 17420 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:42.168923 17390 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:26:42.199651 17194 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.227272 10017 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 16: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:42.193447 16959 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:42.233629  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.102059 17372 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.236159 10016 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:42.237108 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.201886 17303 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:42.239118 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.262562 10017 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:42.214650 16941 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:42.257354  9944 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:26:42.225028 17264 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:42.233992 16917 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.266896 17448 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:42.266958 17303 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:42.269745 17467 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:26:42.270942 17462 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.271492 17462 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.273552 17462 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.267588 17448 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:42.284194 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 80760.3us
I20260709 12:26:42.226164 17427 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.256548 17267 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.256238 17025 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.294862 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:42.307816  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:42.256659 17420 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:42.316521 10642 leader_election.cc:400] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:42.317018 10642 leader_election.cc:403] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:26:42.315418 17464 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:26:42.287601 17372 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:42.206727 17425 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.314896  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:42.386798  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72001us
I20260709 12:26:42.387548  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:42.339809 17374 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.390864 17257 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:42.391546 17257 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.391979 17257 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.392529 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:42.393983 17257 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.328761 17453 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:42.294102 16861 consensus_queue.cc:1175] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 22 status { last_received { term: 20 index: 17 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.18-21.18]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:42.399056 17414 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:42.401080 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1330) took 1512 ms (client timeout 1731 ms). Trace:
W20260709 12:26:42.401499 10023 rpcz_store.cc:269] 0709 12:26:40.889042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.889282 (+   240us) service_pool.cc:224] Handling call
0709 12:26:42.400848 (+1511566us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b540012c854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:42.400998 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2435495296}
I20260709 12:26:42.316609 17267 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.393985 10095 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:42.411017 17409 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.333006 17416 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:42.416994 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.423581 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.296363 17443 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.274461 17424 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.407142 17267 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.354259 17433 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.431921 17194 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.425470 17289 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.286803 17276 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.333639 17268 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:42.346237 17352 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:42.311019 16941 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:42.347787 17339 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.348415 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:42.349051  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.330322 17406 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:42.328616 17418 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:26:42.318419 17444 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:42.318059 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:42.349646 17442 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.339653 16898 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:42.483647 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 165678us
I20260709 12:26:42.352672 17362 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:26:42.331408 17468 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
I20260709 12:26:42.308454 17452 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.336740 17429 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.506235 17339 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.470856 17406 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.360112  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.360939  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:42.510023  9940 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.361300  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.361649  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:42.362059  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.362390  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.512995  9942 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:42.328573 17411 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:42.330865 17280 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.364040 17185 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.522920 17280 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.524359 17185 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.318102 17436 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.526002 17280 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.364813 16959 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.527281 17436 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.332649 17427 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.374255 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.376641 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:42.376698 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:42.530042 10022 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.530158 10092 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.377807 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.530872 10018 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.531045  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.531402 10023 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:42.531754  9946 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:26:42.287747 17264 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.532174 10019 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:42.532558  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:42.384565 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.446918 17289 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.533627 17427 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:42.414091 17409 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.445845 10099 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:42.279250 17449 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:42.534883 17449 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.535390 17449 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.530951 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:42.295754 17326 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.454085 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:42.457888  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:42.460733 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.318102 17025 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.544657 17289 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.545540 17025 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.430197 16861 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 12:26:42.433109 17443 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.519491 10097 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:42.520591 17406 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:42.547043 16941 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.548341  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.549080 17442 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.549549  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:42.550102  9946 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.447517 17420 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.552176 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.552474 16898 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.493268 17268 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.533521  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.489217 17362 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:42.558987  9948 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:42.534497  9942 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 22 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:42.449697 10671 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:42.470299 17444 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:42.537566  9949 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:42.447634 17336 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:26:42.542332 10669 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:42.543598  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.521014 17194 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.553376 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.557616 10092 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:42.533679 10016 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:42.545590 17326 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.568478 17362 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.569322  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2815) took 2434 ms. Trace:
I20260709 12:26:42.569617 17362 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:42.571106 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.572227 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.572726 10669 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.573318 10018 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:42.571146  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:42.575006 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.575433 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.576144 10092 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:42.576890  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:42.578037  9943 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:26:42.558765 17268 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.579881  9944 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:42.576190 10016 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:42.584151 10022 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:42.580794 10676 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:42.585175 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.585527  9945 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 16.
I20260709 12:26:42.585834 10022 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:42.540175 17433 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:42.593928 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.596875 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.569703  9947 rpcz_store.cc:276] 0709 12:26:40.134443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.230080 (+ 95637us) service_pool.cc:224] Handling call
0709 12:26:40.233754 (+  3674us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:40.364840 (+131086us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:40.722993 (+358153us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.015028 (+1292035us) spinlock_profiling.cc:243] Waited 5.17 ms on lock 0x7b48002a2f58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:26:42.568408 (+553380us) spinlock_profiling.cc:243] Waited 3.71 ms on lock 0x7b1800267fa0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287
0709 12:26:42.568565 (+   157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:42.568610 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:42.568650 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.569133 (+   483us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540009b5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:42.569164 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:42.569272 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94217,"prepare.queue_time_us":504563,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":322711,"raft.run_cpu_time_us":3370,"raft.run_wall_time_us":787334,"spinlock_wait_cycles":320178048,"thread_start_us":824594,"threads_started":3,"wal-append.queue_time_us":1026726}
I20260709 12:26:42.602332 10090 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:42.604278 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.575141 17444 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:42.591076 17268 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.610563 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.614105 10676 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:42.615113 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.616297 10676 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:42.610828 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.610837 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.608886  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:42.612010 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.617161 10092 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:26:42.618294 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.603232 17393 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:42.620887 10090 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:42.620967  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:42.600668 17280 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:42.625959 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:42.629416 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:42.631692 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:42.630451  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:42.627872  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:42.634039 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.613621 17416 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:42.634832 10022 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.628489  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:42.660931 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.652591 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:42.635376  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:42.662185  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:42.654482 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.664749 10024 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 14.
W20260709 12:26:42.666716 10677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.023s	sys 0.018s
W20260709 12:26:42.667984 10677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.023s	sys 0.018s
I20260709 12:26:42.686936 10669 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:42.625586 17336 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:42.615633 17409 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:42.691538 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.691560 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.701812 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:42.702200 10096 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:26:42.703104 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.703884 10096 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 13.
I20260709 12:26:42.703758 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.704454 10090 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:42.704710 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.705533  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:42.657289 17433 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:42.693239 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:42.707257 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:42.659940 17194 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:42.689152 17452 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:42.627959 17372 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:42.700738 17406 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:42.690227 17339 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.666404 16861 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:26:42.748607 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55484.7us
I20260709 12:26:42.694875 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.689641 17416 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:42.642423 17280 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.666118 16898 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:42.660429 17442 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:42.754101 10022 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:42.812570 10093 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:42.797657  9940 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:42.840934 10020 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:42.707888 17376 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:42.883646 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.822300 10673 raft_consensus.cc:2410] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:26:42.899276 10022 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:42.900961  9946 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:42.794950 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.745539 17433 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.884380 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:26:42.772069 17395 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:26:42.745905 17452 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.760411 17425 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.887267 17224 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.694744 17336 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.777832 17372 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:42.889501 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.882959 10669 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.919286 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.821244 10676 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:42.921137 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.925681 10676 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:42.905608 17433 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:42.928937 10094 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.019s	sys 0.053s
W20260709 12:26:42.929586 10094 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.019s	sys 0.054s
I20260709 12:26:42.760016 17025 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:26:42.889240 17339 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.740988 17257 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:42.759538 17194 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:42.699872 17448 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:42.741993 17409 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.895705 16898 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:42.950839 17416 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:42.903919 17376 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.960346 16898 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:42.961133 17376 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.922751 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.923457 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.906301 17452 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:42.896880 17443 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:42.924877 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.746299 17374 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:42.910133 17405 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:42.914494 17336 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:42.941783 10669 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:42.976790 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:42.931680 17484 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:26:42.884188 17442 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.948669 10676 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:42.964025 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1567) took 1002 ms. Trace:
I20260709 12:26:42.979250 10668 rpcz_store.cc:276] 0709 12:26:41.961237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.012068 (+ 50831us) service_pool.cc:224] Handling call
0709 12:26:42.012433 (+   365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.083000 (+ 70567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.963818 (+880818us) spinlock_profiling.cc:243] Waited 69.9 ms on lock 0x7b54007b0cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:42.963972 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160661888}
I20260709 12:26:42.979713 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.980301 10676 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:42.980551 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.980940 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.953474 17339 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.983395 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.981081 10668 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:42.984196 10022 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:42.946151 17025 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:42.986668 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:42.987124 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.989854 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.991382 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:42.992403 10669 raft_consensus.cc:2410] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 21 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
W20260709 12:26:43.003037  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.005357 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.974475 17448 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:43.004547 16959 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.008004 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:42.956866 17257 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.008744 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:42.983146 17442 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:43.023476 10676 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:43.067057 10676 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 12:26:43.031809 17485 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:26:42.986356 17209 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.035512 10017 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:42.993207 17025 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:43.059496 10022 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:43.068732 10669 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 12:26:42.988078 17335 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:42.977710 17405 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.070924 10674 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:43.065518 17406 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:43.078575 10022 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:42.978295 17372 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.090999 10016 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:26:43.090642 17337 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.696s	user 0.018s	sys 0.022s
I20260709 12:26:43.066190 17448 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.105090 10017 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:43.065115 16959 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.076531 17335 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.034895 17194 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.083489 17280 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:43.090680 17388 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.090139 17426 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:43.108379 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.111553 10017 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:26:43.104489 17337 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.717s	user 0.018s	sys 0.022s
I20260709 12:26:43.075268 17209 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:43.109795 16941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.009s	sys 0.009s
W20260709 12:26:43.108562 17465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.002s	sys 0.015s
I20260709 12:26:43.093060 17406 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.137844 17224 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:43.133251 16941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.010s	sys 0.009s
W20260709 12:26:43.133649 17465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.002s	sys 0.015s
I20260709 12:26:43.143028 10017 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:43.148626 10668 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:43.149726 10676 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
W20260709 12:26:43.139174 17490 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:26:43.151707 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:43.149909  9830 thread.cc:641] client-negotiator [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.020s	sys 0.024s
W20260709 12:26:43.153891  9830 thread.cc:608] client-negotiator [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.020s	sys 0.024s
W20260709 12:26:43.154452 17489 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
I20260709 12:26:43.143633 17224 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.157572 17224 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.129571 17426 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.151686 17377 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:43.165712 17449 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.002s
I20260709 12:26:43.212133 10017 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:43.212831 10020 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 12:26:43.153972 17409 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:43.214672 17495 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:26:43.215185 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 1329) took 1117 ms. Trace:
I20260709 12:26:43.219295 10674 rpcz_store.cc:276] 0709 12:26:42.097519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.444279 (+346760us) service_pool.cc:224] Handling call
0709 12:26:43.215008 (+770729us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400544f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:43.215122 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003019520}
I20260709 12:26:43.219853 10020 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:43.215857 17209 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:43.218478 17490 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:26:43.231680 17406 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.218148 17257 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.223773 17414 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:43.240617  9946 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:43.243829  9946 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:43.216558 17449 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.248570  9946 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.252311  9946 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.253244  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.254499 17433 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.256218 17376 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.258710 17376 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.260569 10095 raft_consensus.cc:2368] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
W20260709 12:26:43.262439 17268 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.640s	user 0.012s	sys 0.000s
I20260709 12:26:43.263650 17426 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.264747 17433 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:43.269621  9946 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:43.270948 17448 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:43.276080 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3910) took 2662 ms. Trace:
I20260709 12:26:43.277690 10092 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:26:43.276598 10091 rpcz_store.cc:276] 0709 12:26:40.614840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.945120 (+330280us) service_pool.cc:224] Handling call
0709 12:26:40.945404 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:41.240316 (+294912us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:41.333274 (+ 92958us) log.cc:844] Serialized 14 byte log entry
0709 12:26:41.481536 (+148262us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:41.481600 (+    64us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:41.481658 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.275831 (+1794173us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400304654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:43.275876 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.275988 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14234,"prepare.queue_time_us":143515,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":6423,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":21385,"spinlock_wait_cycles":677608192,"thread_start_us":187112,"threads_started":2,"wal-append.queue_time_us":413663,"wal-append.run_cpu_time_us":1048,"wal-append.run_wall_time_us":1009863}
I20260709 12:26:43.280090 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:43.281824 10091 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:43.280633 17482 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.827s	user 0.009s	sys 0.000s
I20260709 12:26:43.284243 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1330) took 1003 ms. Trace:
I20260709 12:26:43.284729 10020 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:26:43.284670 10676 rpcz_store.cc:276] 0709 12:26:42.280519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.073191 (+792672us) service_pool.cc:224] Handling call
0709 12:26:43.149402 (+ 76211us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:43.150126 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.284065 (+133939us) spinlock_profiling.cc:243] Waited 69.8 ms on lock 0x7b540073afd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:43.284189 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160645888}
I20260709 12:26:43.285844 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.286075 10676 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:43.287297  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.287617  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:43.291803 17444 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.676s	user 0.007s	sys 0.005s
I20260709 12:26:43.292984 17339 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.298497 17335 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.316538 10091 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:43.247793 17414 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.318298 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:43.254469 17336 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:43.325897  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.339700 17316 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:43.339745  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:43.328325 17390 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.340631 17390 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.318050 17444 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.342957  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.344084 17444 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.273504 17268 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.345824 17326 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.346334 17268 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:43.331133 17264 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.799s	user 0.007s	sys 0.004s
I20260709 12:26:43.347029 17264 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:43.319773 17362 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.750s	user 0.013s	sys 0.001s
I20260709 12:26:43.348879 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3701) took 1128 ms. Trace:
I20260709 12:26:43.332159 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.334507 10671 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:43.334451 17449 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.316372 17442 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.351468 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.352087 10097 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:43.337136 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3689) took 1120 ms. Trace:
I20260709 12:26:43.336938 17025 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.337620 17414 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.338135 17339 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.330314 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.346982 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3695) took 1128 ms. Trace:
I20260709 12:26:43.358091  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.358748  9940 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:43.359551  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.359745  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.359874  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.360169  9948 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:43.360453  9945 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:43.360785  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.349272 10022 rpcz_store.cc:276] 0709 12:26:42.220382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.088833 (+868451us) service_pool.cc:224] Handling call
0709 12:26:43.112223 (+ 23390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.112432 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:43.127466 (+ 15034us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.348558 (+221092us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:43.348608 (+    50us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:43.348651 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.348714 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.348824 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125411,"prepare.queue_time_us":25643,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":125784,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":95141,"threads_started":1,"wal-append.queue_time_us":102993}
I20260709 12:26:43.352881 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.352650 10019 rpcz_store.cc:276] 0709 12:26:42.216759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.216873 (+   114us) service_pool.cc:224] Handling call
0709 12:26:43.336870 (+1119997us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540038e0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:43.337052 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":625142656}
I20260709 12:26:43.356737  9946 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:43.360502  9949 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:43.366811  9949 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:43.358377 10025 rpcz_store.cc:276] 0709 12:26:42.218743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.218875 (+   132us) service_pool.cc:224] Handling call
0709 12:26:42.219180 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:42.232988 (+ 13808us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:42.455882 (+222894us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.886978 (+431096us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:42.887965 (+   987us) log.cc:844] Serialized 52 byte log entry
0709 12:26:42.888111 (+   146us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:42.893929 (+  5818us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.322390 (+428461us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b18003ab640. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:26:43.322874 (+   484us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.323073 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.323248 (+   175us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.346725 (+ 23477us) raft_consensus.cc:1685] finished
0709 12:26:43.346755 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.346922 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68569,"prepare.queue_time_us":45262,"prepare.run_cpu_time_us":141,"prepare.run_wall_time_us":129,"raft.queue_time_us":469319,"raft.run_cpu_time_us":418,"raft.run_wall_time_us":4006,"spinlock_wait_cycles":8809344,"thread_start_us":619283,"threads_started":2,"wal-append.queue_time_us":506901}
I20260709 12:26:43.367354  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:43.369247  9949 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:43.377640  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.371652 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3703) took 1110 ms. Trace:
I20260709 12:26:43.370659 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:43.379851 10097 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:43.382302 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.383323 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:43.383802 10020 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:26:43.350193 17336 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.377614 17326 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:43.385464 17150 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:43.385604  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.386278 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:43.386566  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.387367 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.378993 10016 rpcz_store.cc:276] 0709 12:26:42.260928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.103346 (+842418us) service_pool.cc:224] Handling call
0709 12:26:43.112691 (+  9345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.112823 (+   132us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:43.228680 (+115857us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.371251 (+142571us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:43.371315 (+    64us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:43.371367 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.371450 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.371584 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134469,"prepare.queue_time_us":41824,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":126946,"thread_start_us":114536,"threads_started":2}
I20260709 12:26:43.393567 10095 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.367992 17264 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.366549 17442 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.395082 17452 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.400755 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.401798 10022 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:43.402266 17452 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.402861 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.416126  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.417696  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.418756  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.418890  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.419118  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.419538  9947 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 17: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:43.420353  9948 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:26:43.420737  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.420384  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.425140  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.426124 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.427160  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.419459  9944 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:43.392247 17388 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:43.392484 17326 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:43.398619 17390 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.440148 10097 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:43.445823 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.447480  9947 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:43.448530 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.449103 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:43.449777 10097 raft_consensus.cc:2410] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:26:43.436688 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:43.434839 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:43.451247 10025 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:43.449862 10016 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.433800 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.435722 17041 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:43.454952  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.455196 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3927) took 1078 ms. Trace:
I20260709 12:26:43.466280 10020 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:26:43.455763 10094 rpcz_store.cc:276] 0709 12:26:42.376913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.377967 (+  1054us) service_pool.cc:224] Handling call
0709 12:26:42.378290 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.378523 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:43.355612 (+977089us) spinlock_profiling.cc:243] Waited 2.44 ms on lock 0x7b18004b1cc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf
0709 12:26:43.355752 (+   140us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.449246 (+ 93494us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.454733 (+  5487us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:43.454807 (+    74us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:43.454873 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.454962 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.455114 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5183,"prepare.queue_time_us":532227,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":5442,"spinlock_wait_cycles":5609856,"thread_start_us":594167,"threads_started":2}
I20260709 12:26:43.469164 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.469969 10091 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:43.472049 10091 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 14. Current term is 15. Ops: []
I20260709 12:26:43.475137 17424 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.479187 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:43.479930 10672 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 15. (term mismatch)
I20260709 12:26:43.479943 10022 raft_consensus.cc:2368] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:26:43.480384 10672 pending_rounds.cc:85] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 14
I20260709 12:26:43.480885 10672 pending_rounds.cc:107] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:26:43.481208 10672 raft_consensus.cc:2889] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:43.447234 17388 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:43.490658 10095 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
W20260709 12:26:43.482913 10673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 1328) took 1576 ms (client timeout 2024 ms). Trace:
W20260709 12:26:43.491853 10673 rpcz_store.cc:269] 0709 12:26:41.906967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.907121 (+   154us) service_pool.cc:224] Handling call
0709 12:26:43.479469 (+1572348us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b54004d49d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:43.482813 (+  3344us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2048444672}
I20260709 12:26:43.482973 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:43.487051  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.493299  9947 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:43.458082 17041 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:43.491401  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 151995us
I20260709 12:26:43.487979  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:43.496340  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:26:43.497037  9944 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 22 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:43.495255 10019 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:43.509584  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:43.511114 17231 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:43.511631 17231 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:43.512068 17231 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.516753 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1573) took 1551 ms. Trace:
I20260709 12:26:43.526922 10016 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:43.525862  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:43.530956 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 646665us
I20260709 12:26:43.527297 10065 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:43.531279 10671 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:43.534339 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:43.535279  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:43.535837 10091 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:43.536031 10671 raft_consensus.cc:2410] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:43.534309 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:43.538065  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:43.517352 10677 rpcz_store.cc:276] 0709 12:26:41.965778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.047531 (+ 81753us) service_pool.cc:224] Handling call
0709 12:26:42.094970 (+ 47439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.095320 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.095354 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.095831 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.669270 (+573439us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:42.707026 (+ 37756us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.515952 (+808926us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b1800450ca0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:26:43.516189 (+   237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.516293 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.516357 (+    64us) raft_consensus.cc:1685] finished
0709 12:26:43.516478 (+   121us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.516670 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9717,"prepare.queue_time_us":39513,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":1116,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":11474,"spinlock_wait_cycles":74984320,"thread_start_us":572990,"threads_started":1,"wal-append.queue_time_us":729487}
I20260709 12:26:43.547164 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:43.549242  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.529192 17420 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:43.550385 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 267086us
I20260709 12:26:43.556757 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:43.551203  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:43.554725 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:43.530161 17474 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.535411 17433 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:43.561674 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:43.537724 17150 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:43.538307 17425 raft_consensus.cc:2764] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:43.541225 17442 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:43.565910  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:43.537978 16959 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.570257 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:43.588817 10671 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:43.583472 17506 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:26:43.570299 17474 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.584128 17433 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:26:43.585218 17481 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.139s	user 0.002s	sys 0.001s
I20260709 12:26:43.566843 17371 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:43.544459 17339 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:26:43.585687 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:43.555296 17025 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.588519 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:43.610188 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:43.526746 17326 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.571895 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:43.617209 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:43.560751 17335 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:43.561256 17420 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:43.544358 17336 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:26:43.544683 17327 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:26:43.543011 17295 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:26:43.612447 10671 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:43.588481 10094 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:43.612725 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:43.612716 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:43.611232 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:43.587951 17264 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:43.617441 10677 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:43.620707 17335 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:43.644023 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:43.603200 17505 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:26:43.626217 17448 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:43.593129 17231 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.591539 17409 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:43.636024 17025 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:26:43.635675 10673 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:43.555514 17303 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:43.641774 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:43.648168 17335 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:43.604074 17362 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:43.643733 10021 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:26:43.645251 10671 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:43.642304 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:43.595623 17442 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:43.616087 17433 consensus_queue.cc:1243] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 22.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:26:43.643791  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:43.630976 17420 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:43.568269 17388 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 15.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.594099 17461 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:26:43.594722 17374 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:43.634001 17336 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:43.571183 17041 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.638569 17264 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:43.664870 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:43.701735 17280 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:43.671244 17508 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:26:43.673235 17474 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:43.723742 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3711) took 1132 ms. Trace:
I20260709 12:26:43.724375 10017 rpcz_store.cc:276] 0709 12:26:42.591631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.218581 (+626950us) service_pool.cc:224] Handling call
0709 12:26:43.218969 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:43.370914 (+151945us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.370960 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:43.379874 (+  8914us) log.cc:844] Serialized 52 byte log entry
0709 12:26:43.545596 (+165722us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:43.545710 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.545849 (+   139us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.723429 (+177580us) raft_consensus.cc:1685] finished
0709 12:26:43.723461 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.723654 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":194480,"raft.queue_time_us":158217,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":7770,"thread_start_us":123308,"threads_started":1,"wal-append.queue_time_us":247399}
I20260709 12:26:43.679198  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:43.680470  9941 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:43.727596  9941 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 16 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:43.682662 10099 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2749) took 1780 ms (client timeout 1731 ms). Trace:
W20260709 12:26:43.729454 10099 rpcz_store.cc:269] 0709 12:26:41.901713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.977396 (+ 75683us) service_pool.cc:224] Handling call
0709 12:26:43.682341 (+1704945us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b540014a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:43.682545 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":976165760}
I20260709 12:26:43.649050 17398 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:43.647722 17268 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:43.694387 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1572) took 1729 ms. Trace:
W20260709 12:26:43.696818 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:43.735061 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.728597  9941 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.736459  9941 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.710656 17336 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:43.698848  9947 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:26:43.703465 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61279.5us
I20260709 12:26:43.716953 17264 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:43.706640  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 62944.8us
I20260709 12:26:43.706077 17425 raft_consensus.cc:2764] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:26:43.743947 10673 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 12:26:43.715143 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50407.3us
I20260709 12:26:43.746614 10640 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:43.597097 16941 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.706202 17433 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:43.693079 17025 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:43.760612 17025 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.700563 17362 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.762814 17362 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.699856 17209 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:43.764303 17209 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.765722 17405 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.704799 17150 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:43.766727 17510 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:26:43.766868 17150 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.767874 17433 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.770231 17433 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:43.770790 16941 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:43.772930 16941 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.772579 17362 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.710362 17372 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:43.708413 17352 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:43.703142 17442 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.774709 17372 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:43.774935 17352 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.775409 17372 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:43.777496 17352 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:43.708276 17374 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.778872 17374 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:43.722096 17041 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:43.740253 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:43.783933 17336 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:43.784425 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44271us
I20260709 12:26:43.785198 17336 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:43.785673 17504 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 12:26:43.786011 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:43.792377 10673 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:26:43.752991 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:43.753643  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.733155 10670 rpcz_store.cc:276] 0709 12:26:41.965422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.033731 (+ 68309us) service_pool.cc:224] Handling call
0709 12:26:42.034034 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:42.034368 (+   334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:42.034395 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:42.034837 (+   442us) log.cc:844] Serialized 26 byte log entry
0709 12:26:42.472611 (+437774us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:42.493829 (+ 21218us) log.cc:844] Serialized 14 byte log entry
0709 12:26:42.579464 (+ 85635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:42.579534 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:43.584619 (+1005085us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:43.660145 (+ 75526us) raft_consensus.cc:1685] finished
0709 12:26:43.694081 (+ 33936us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:43.694303 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85327,"prepare.queue_time_us":21036,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":90704,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":21784,"spinlock_wait_cycles":93467392,"thread_start_us":369449,"threads_started":1,"wal-append.queue_time_us":393780}
I20260709 12:26:43.749854 16661 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
W20260709 12:26:43.801138 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48260us
W20260709 12:26:43.784577 17486 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.839s	user 0.002s	sys 0.000s
I20260709 12:26:43.741786  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:43.720827 17365 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:43.786669 17477 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:43.758846 17444 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:43.770381 17150 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.739687 17479 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:43.810137  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 68426.3us
I20260709 12:26:43.832257  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.833133  9949 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:43.833559  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.834399  9947 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:43.836114  9940 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:26:43.761045 17231 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:43.774874 17268 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:43.766578 17209 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:43.808795 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:43.838903 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.841084 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:43.847352 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:43.843837  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:43.848152 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:43.782465 17448 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:43.807814 17365 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:43.818811 17425 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:43.851491  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.818650 17398 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:43.847987 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:43.820711 17461 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:43.864452 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:43.852542 17352 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:43.855675 17406 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:43.859074 17365 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:43.852953 17280 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:43.872618 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:43.856812 17231 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:43.844619 17350 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:43.846180 17479 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.855047 17503 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:26:43.858111 17371 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:26:43.874135 17448 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:43.859514 17433 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:26:43.872227 17461 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:43.930068  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.936574  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.937942  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.930636  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:43.939857  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.941183  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.942178  9942 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 23 because replica has last-logged OpId of term: 21 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 12:26:43.931059  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.882516 17231 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:43.943001 17231 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:43.943041  9945 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:43.948364  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:43.948616  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2593) took 1587 ms. Trace:
I20260709 12:26:43.951200 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:43.948992  9943 rpcz_store.cc:276] 0709 12:26:42.360885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.385292 (+ 24407us) service_pool.cc:224] Handling call
0709 12:26:42.385626 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.578460 (+192834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.948399 (+1369939us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b540009bad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:43.948561 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":441625344}
W20260709 12:26:43.954274  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:43.957057 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3920) took 2033 ms. Trace:
I20260709 12:26:43.957580 10098 rpcz_store.cc:276] 0709 12:26:41.923671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.039006 (+115335us) service_pool.cc:224] Handling call
0709 12:26:42.133076 (+ 94070us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:42.133910 (+   834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.956820 (+1822910us) spinlock_profiling.cc:243] Waited 93.8 ms on lock 0x7b5400161f50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:43.956990 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":215678208}
I20260709 12:26:43.920245 17362 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:43.931771 17025 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:43.878240 17352 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:43.962994 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 87561.1us
I20260709 12:26:43.910183 17280 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:43.968690 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:43.978928 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:44.036260 17479 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:44.037618 17514 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 12:26:43.960052 17519 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.086s
I20260709 12:26:44.038902 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1577) took 1543 ms. Trace:
I20260709 12:26:44.040014 17452 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:44.039260 10668 rpcz_store.cc:276] 0709 12:26:42.495182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.150197 (+655015us) service_pool.cc:224] Handling call
0709 12:26:43.150681 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:43.549423 (+398742us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.549451 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:43.549908 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.706757 (+156849us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:43.706793 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.706867 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.038663 (+331796us) raft_consensus.cc:1685] finished
0709 12:26:44.038702 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.038843 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":398269,"raft.queue_time_us":399206,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":9243,"thread_start_us":133779,"threads_started":1,"wal-append.queue_time_us":287820}
W20260709 12:26:43.989028 17512 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:26:44.040477 17452 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:44.041677 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3938) took 1433 ms. Trace:
I20260709 12:26:44.041452 17452 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.043183 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1576) took 1990 ms. Trace:
I20260709 12:26:44.042177 10097 rpcz_store.cc:276] 0709 12:26:42.608609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.456829 (+848220us) service_pool.cc:224] Handling call
0709 12:26:43.457271 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:43.457800 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.457844 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:43.494121 (+ 36277us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.639642 (+145521us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:43.639702 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.639806 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.041404 (+401598us) raft_consensus.cc:1685] finished
0709 12:26:44.041434 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.041596 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":13136,"thread_start_us":142097,"threads_started":1,"wal-append.queue_time_us":225701}
I20260709 12:26:44.045796  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.046509  9944 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:44.033773 17374 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.046051 17264 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.052852 17503 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:44.053661 17503 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:44.054550 17530 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:44.055387 17264 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:44.054688 17374 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:44.043612 10669 rpcz_store.cc:276] 0709 12:26:42.053129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.017774 (+964645us) service_pool.cc:224] Handling call
0709 12:26:43.068383 (+ 50609us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:43.076403 (+  8020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.042971 (+966568us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b54007d3cd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:44.043121 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71968128}
I20260709 12:26:44.060324  9941 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 12:26:44.060890  9941 pending_rounds.cc:85] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 13
I20260709 12:26:44.061448  9941 pending_rounds.cc:107] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 12:26:44.054018 17503 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:43.966625 17474 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:44.062942  9941 raft_consensus.cc:2889] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:44.065971 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:43.975633 17428 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:44.061980 17406 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:44.063827 17452 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.070420 17503 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:44.019872 17520 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:26:44.009289 17025 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:44.080482 17280 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:44.081729 17517 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:26:44.081811 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.082036  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.082469 17150 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:44.056622 17515 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.188s	user 0.000s	sys 0.000s
I20260709 12:26:44.098441 10022 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:44.097694  9944 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:44.095743 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 117062us
I20260709 12:26:44.103204 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:44.092520 17258 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:44.091369 17474 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:44.096707 17428 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.129854  9949 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:44.110857 17258 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:44.134399  9948 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:44.136878 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1340) took 1853 ms (client timeout 1921 ms). Trace:
I20260709 12:26:44.100807 17025 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.101960 17257 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
W20260709 12:26:44.139930 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 158998us
W20260709 12:26:44.138144 10023 rpcz_store.cc:269] 0709 12:26:42.283386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.402469 (+119083us) service_pool.cc:224] Handling call
0709 12:26:44.136686 (+1734217us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54002d4e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:44.136821 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245143936}
I20260709 12:26:44.140786 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1579) took 1645 ms. Trace:
I20260709 12:26:44.125861 17474 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:44.141153 10674 rpcz_store.cc:276] 0709 12:26:42.495850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.224185 (+728335us) service_pool.cc:224] Handling call
0709 12:26:43.224608 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:43.587801 (+363193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.587830 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:43.598671 (+ 10841us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.818712 (+220041us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:43.934784 (+116072us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.936962 (+  2178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.937036 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.140581 (+203545us) raft_consensus.cc:1685] finished
0709 12:26:44.140602 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.140728 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":374407,"prepare.queue_time_us":221379,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":368087,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":10679,"thread_start_us":273213,"threads_started":2,"wal-append.queue_time_us":242484}
I20260709 12:26:44.064761 17461 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.164896  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.165752  9940 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 23 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:26:44.167291 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:44.167886 10671 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:44.168398 10671 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:44.105123 17150 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.146215 17257 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:44.147892 17372 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.184624  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:44.154366 17264 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:44.155427 17289 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:44.156183 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.157001 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.202981 10097 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:44.158573 17433 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:44.158963 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.206712 10095 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:44.159704 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.160672 10675 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:44.162071 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1565) took 2201 ms. Trace:
I20260709 12:26:44.210884 10672 rpcz_store.cc:276] 0709 12:26:41.960177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:41.960329 (+   152us) service_pool.cc:224] Handling call
0709 12:26:43.479599 (+1519270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:43.481693 (+  2094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.161860 (+680167us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54004d49d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:44.162012 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3493370240}
I20260709 12:26:44.213256 10672 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:44.163327 17352 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:44.136750 17534 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:44.219928 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:44.144055 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1581) took 1645 ms. Trace:
I20260709 12:26:44.172161 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:44.178877 17366 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:44.178156 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:44.230665 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52597.2us
I20260709 12:26:44.232158 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:44.169361 10671 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.233491 10671 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:44.234131 10023 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:44.195381 17477 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:26:44.204715 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59139.6us
I20260709 12:26:44.129125 17448 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:44.221032 10676 rpcz_store.cc:276] 0709 12:26:42.498289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.297196 (+798907us) service_pool.cc:224] Handling call
0709 12:26:43.311072 (+ 13876us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:43.550800 (+239728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:43.550861 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:43.551437 (+   576us) log.cc:844] Serialized 26 byte log entry
0709 12:26:43.708736 (+157299us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:43.933927 (+225191us) log.cc:844] Serialized 14 byte log entry
0709 12:26:43.937877 (+  3950us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:43.938574 (+   697us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:44.143869 (+205295us) raft_consensus.cc:1685] finished
0709 12:26:44.143889 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:44.144005 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":239269,"prepare.queue_time_us":273725,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":251486,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":107,"spinlock_wait_cycles":5854592,"thread_start_us":332371,"threads_started":2,"wal-append.queue_time_us":258434}
I20260709 12:26:44.173431 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:44.181895 17257 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:44.116930 17335 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.238722 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:44.244685 10670 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:44.241124 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:44.201851 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:44.206173 17433 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:44.258464 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:44.266022 10671 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:44.249940 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.252509 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.268231 10095 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:26:44.136909 17406 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.230018 17534 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:44.280493 10022 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:44.274993 10097 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:44.266362 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:44.277894 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.242475 17405 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:44.238729 17374 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:44.279351 10091 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:44.289877 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:44.291217 10674 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:26:44.217149 17289 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.247990 17538 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:44.278208 16661 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:26:44.217351 17264 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.275341 17194 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:44.298403 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:44.286633 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:44.238693 17536 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 12:26:44.291229 17209 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:44.269814 17425 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:44.308265 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:44.231019 17258 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.287124 17534 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.304263 10674 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.312638 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3921) took 1937 ms. Trace:
I20260709 12:26:44.314015 10093 rpcz_store.cc:276] 0709 12:26:42.374869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.380406 (+  5537us) service_pool.cc:224] Handling call
0709 12:26:42.380718 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:42.885792 (+505074us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.312398 (+1426606us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b54001928d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:44.312582 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992680192}
I20260709 12:26:44.314973 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:44.315572 10677 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.303509 17531 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:44.323448 17531 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.324050  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.325001 17428 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:44.325995 17025 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:44.311390 17209 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:44.327629 17209 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.306318 17257 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.333885  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.309443 17532 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:44.340793 17289 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.337682 17231 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.330966 17150 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.342919 17264 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.364228 17231 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:44.310143 17461 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:26:44.366952 17461 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:44.367075 10674 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:44.368312 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:44.368745 17365 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.369408 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:44.370059 10673 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:44.370886 10674 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:44.355899 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.334955 17257 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:44.372252 10090 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:44.372869 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1320) took 1886 ms. Trace:
I20260709 12:26:44.373155 10090 rpcz_store.cc:276] 0709 12:26:42.486351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.794300 (+307949us) service_pool.cc:224] Handling call
0709 12:26:44.371657 (+1577357us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:26:44.372043 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.372644 (+   601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.372816 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8310528}
I20260709 12:26:44.373775 17365 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:44.317431 17544 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:44.350636 10677 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:44.302852 17538 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.379097 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.315122 17448 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.382287 10677 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:26:44.382848 10677 pending_rounds.cc:85] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 12
I20260709 12:26:44.383131 10677 pending_rounds.cc:107] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:26:44.383394 10677 raft_consensus.cc:2889] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:44.381877 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74016.5us
I20260709 12:26:44.386068 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.388053 17372 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.390576 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2763) took 1286 ms. Trace:
I20260709 12:26:44.391062 10091 rpcz_store.cc:276] 0709 12:26:43.104638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.536615 (+431977us) service_pool.cc:224] Handling call
0709 12:26:43.560695 (+ 24080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:44.289198 (+728503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:44.390308 (+101110us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b5400162454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:44.390511 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651768704}
W20260709 12:26:44.395643 10021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3694) took 2177 ms (client timeout 2223 ms). Trace:
W20260709 12:26:44.396095 10021 rpcz_store.cc:269] 0709 12:26:42.218413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:42.218527 (+   114us) service_pool.cc:224] Handling call
0709 12:26:44.395445 (+2176918us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54002666d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:44.395584 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3274418048}
I20260709 12:26:44.400506 17358 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:44.404831 17465 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:44.405121 16959 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:44.405319 17465 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:44.405608 16959 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:44.406746 17544 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:44.407773 17544 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.409093 17362 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:44.409639  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.410252 17374 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:44.410646 17374 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:44.411427 17374 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.412473  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2604) took 1125 ms. Trace:
I20260709 12:26:44.412838  9946 rpcz_store.cc:276] 0709 12:26:43.287294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:43.287413 (+   119us) service_pool.cc:224] Handling call
0709 12:26:44.412265 (+1124852us) spinlock_profiling.cc:243] Waited 68.4 ms on lock 0x7b5400189cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:44.412419 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":157356928}
I20260709 12:26:44.414821 17536 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:44.415194  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:44.416299 17150 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:44.417740 17150 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:44.418263 17150 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:44.406481 16959 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:44.419148  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.419934 17150 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:44.410204  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.424976 17350 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:26:44.425426 17350 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:44.309144 17433 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.402530 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:44.427336 17533 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:44.427449 17433 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:44.428825 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:44.431128  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.434425  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:44.434984 17533 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:44.435398 17543 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.105s	user 0.002s	sys 0.000s
I20260709 12:26:44.426219 17350 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.377200 17461 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:44.436939 17350 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:26:44.437666 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:44.438210 17335 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:44.437757 17461 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.438567 17533 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:44.439111 17533 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:44.440263 17533 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:44.401593 17524 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:26:44.443693 16959 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:44.444097 17420 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.733s	user 0.012s	sys 0.001s
I20260709 12:26:44.439153 17461 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:44.444588 17420 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.445880 10017 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 23 because replica has last-logged OpId of term: 21 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 12:26:44.446883  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:44.447482  9944 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:44.448240 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.448675 17531 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.451066 17538 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.450911 17531 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:26:44.406360 17465 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.452430 17465 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:44.453563  9948 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:44.455696  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:44.456691  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:44.459154 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:44.460621 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.461076 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:44.462862 17280 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:44.463315 17280 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:44.463721 17280 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:44.402510  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.465672 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.465948 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.431299 17258 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:44.468008 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:44.468576  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:44.476052 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:44.464543 17534 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.468894  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:44.470366 16661 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:44.470446 17531 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:44.474543 10674 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:26:44.470108 17538 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:26:44.478307  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 296241us
W20260709 12:26:44.489140 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 80266.1us
I20260709 12:26:44.497298  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:44.478718  9990 leader_election.cc:400] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
I20260709 12:26:44.498063  9990 leader_election.cc:403] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:44.385128 17365 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:44.499822 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:44.500092 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:44.501700 10673 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:26:44.510005  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:44.488606  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.510470 10670 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [15.12-17.13]
I20260709 12:26:44.488101  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:44.503098  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:44.507304  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.511719 10640 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.514721 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.515090  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:44.515987 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.515878 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.518106 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.520108 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.523821 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.528106 10098 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:26:44.529978 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.531206 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.533685  9940 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17. Ops: []
W20260709 12:26:44.535519  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48364.5us
I20260709 12:26:44.534981  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:44.539615 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.541917 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.543303 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.543960 10099 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:44.546252  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.547932 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:44.477473 17530 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:26:44.567443  9944 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:44.551783 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:44.549768  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:44.471958 17420 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.564133  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.574164 10091 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: [20.14-23.15]
I20260709 12:26:44.575428 10090 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:44.500703 17461 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:44.480703 17335 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:44.508798 17418 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:44.512147 17378 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:44.607892 17534 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.510109 17519 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:44.608261  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:44.608599 17519 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.609377 17389 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:44.610668 17519 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:44.533599 17536 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:44.612401 17536 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.568261 17539 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:44.534799 17532 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.615577 17532 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.616530  9940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37576
I20260709 12:26:44.617318  9940 raft_consensus.cc:606] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Received request to transfer leadership
I20260709 12:26:44.617877 17532 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:44.619807 17503 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.516s	user 0.006s	sys 0.004s
I20260709 12:26:44.620283 17503 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.610689  9912 leader_election.cc:400] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:26:44.625036  9912 leader_election.cc:403] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:26:44.625283  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.584172  9946 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:44.586421 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:44.635040 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63139.2us
I20260709 12:26:44.635963 10674 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:44.533545 17390 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
I20260709 12:26:44.589339 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:44.592038 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:44.592780 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:44.596345 17541 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:26:44.596184 17420 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:44.596577 17444 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 12:26:44.513698 17352 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.084s
I20260709 12:26:44.533471 17371 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:44.573871 17393 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:44.599116 17316 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:44.599762 17336 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:44.600309 17526 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:26:44.606125 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:44.704046 17474 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.508s	user 0.007s	sys 0.003s
I20260709 12:26:44.639775  9940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37576
I20260709 12:26:44.719039  9940 raft_consensus.cc:606] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Received request to transfer leadership
I20260709 12:26:44.641898  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:44.642225  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.643075  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:44.698184 17503 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:44.665172  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54790.1us
I20260709 12:26:44.663447 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.666539 10668 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:44.534451 17362 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:44.477662 17425 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.567361 17520 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.098s
I20260709 12:26:44.614545 17536 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:44.700596  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 185565us
I20260709 12:26:44.701567 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:44.635538 17530 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:44.705268 17280 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.635150 17461 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:44.737890 10669 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:44.738442 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.742985 10091 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:44.738798 10099 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:26:44.708149  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 205131us
W20260709 12:26:44.712147 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 74082.4us
I20260709 12:26:44.739537  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:44.746754  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.666538 17442 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:26:44.573556 17264 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:44.663775 10675 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:44.623054 17525 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.087s
I20260709 12:26:44.725847  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:44.667815 17025 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:26:44.710184 17335 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:44.733974 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.741256 10095 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:26:44.748726 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.747431  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:44.761859  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:44.750175 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:44.764081 10094 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:44.750283  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:44.767268 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.767578 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.768988 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.769308 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:44.769603 10019 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:44.770278 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.770526 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:44.771325 10016 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:44.772480 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:44.774937 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:44.753448 10097 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:44.726457 17316 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:44.778966 17316 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.781937 17316 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:44.771341 10023 raft_consensus.cc:2376] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20. Re-sending same reply.
I20260709 12:26:44.785784 17041 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:44.790576 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:44.795818 17452 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:44.819450 17544 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:44.808911 17316 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.821376 17390 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:44.822032 17390 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:44.821460 17316 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.806501 17362 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:44.824553 17362 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.801330 10094 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:26:44.823386 17390 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.799041  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:44.791657  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:44.835119 17316 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:44.838135 10094 pending_rounds.cc:85] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 12
I20260709 12:26:44.839362 10094 pending_rounds.cc:107] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:26:44.840747 10094 raft_consensus.cc:2889] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:44.737666 17257 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:44.752974 17371 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:44.726140 17474 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.733681 17209 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:44.904803 10676 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:44.913698 17526 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:26:44.924197 17550 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:26:44.924744 17362 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:44.925680 17551 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:26:44.932741 17526 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:44.936133 17526 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:44.943189 17474 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:44.916145 17461 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:44.979589  9830 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for a7196ffc873e4e7e84c76068d5b95904 already in progress [suppressed 10 similar messages]
W20260709 12:26:44.983909  9842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 324) took 32156 ms (client timeout 15000 ms). Trace:
W20260709 12:26:44.992246  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 266311us
W20260709 12:26:44.994938  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.003319  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 165591us
W20260709 12:26:45.003957  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.004940  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.006351  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.006980  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 217210us
I20260709 12:26:45.009805  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:44.984683  9842 rpcz_store.cc:269] 0709 12:26:12.827736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:12.828020 (+   284us) service_pool.cc:224] Handling call
0709 12:26:44.983588 (+32155568us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:36.789184 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:36.789966 (+   782us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:36.790028 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:26:36.790080 (+    52us) tablet.cc:662] Decoding operations
0709 12:26:36.792794 (+  2714us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:36.792897 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:36.797704 (+  4807us) tablet.cc:685] Acquiring locks for 35 operations
0709 12:26:37.119694 (+321990us) tablet.cc:701] Row locks acquired
0709 12:26:37.119754 (+    60us) write_op.cc:260] PREPARE: finished
0709 12:26:37.119974 (+   220us) write_op.cc:270] Start()
0709 12:26:37.120169 (+   195us) write_op.cc:276] Timestamp: P: 1783599997119892 usec, L: 0
0709 12:26:37.120218 (+    49us) op_driver.cc:348] REPLICATION: starting
0709 12:26:37.121194 (+   976us) log.cc:844] Serialized 13868 byte log entry
0709 12:26:37.579452 (+458258us) op_driver.cc:464] REPLICATION: finished
0709 12:26:37.813779 (+234327us) write_op.cc:301] APPLY: starting
0709 12:26:37.844139 (+ 30360us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:38.923288 (+1079149us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b18002a67c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e092950ec 00007f4e09297af7 00007f4e09298fb0 00007f4e0cfad1bb 00007f4e0cfad532 00007f4e0cff4b93 00007f4e0cea1de3 00007f4e0cea16ab 00007f4e0cea2e77
0709 12:26:39.686434 (+763146us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:39.686486 (+    52us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:41.885639 (+2199153us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:41.886218 (+   579us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=70,key_file_lookups=70,delta_file_lookups=70,mrs_lookups=0
0709 12:26:41.886300 (+    82us) write_op.cc:312] APPLY: finished
0709 12:26:42.037883 (+151583us) log.cc:844] Serialized 300 byte log entry
0709 12:26:42.213118 (+175235us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:42.257234 (+ 44116us) write_op.cc:454] Released schema lock
0709 12:26:42.306558 (+ 49324us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:42.409409 (+102851us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:42.409446 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12184,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":209018,"apply.run_cpu_time_us":101442,"apply.run_wall_time_us":4609984,"cfile_cache_hit":142,"cfile_cache_hit_bytes":150497,"num_ops":35,"prepare.queue_time_us":144894,"prepare.run_cpu_time_us":21845,"prepare.run_wall_time_us":429009,"raft.queue_time_us":176889,"raft.run_cpu_time_us":13978,"raft.run_wall_time_us":173834,"replication_time_us":459008,"spinlock_wait_cycles":54563968,"thread_start_us":572006,"threads_started":5,"wal-append.queue_time_us":489520,"wal-append.run_cpu_time_us":21419,"wal-append.run_wall_time_us":2247412}]]}
W20260709 12:26:45.015413  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.016198  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.020486 10017 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:45.027442 17280 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:45.029474 17041 raft_consensus.cc:2764] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:26:45.032541 10096 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:45.035439 17209 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:45.088359 17326 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:45.090204 17371 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:26:45.091248  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 97324.8us
I20260709 12:26:44.917172 17335 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:45.032564 17041 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:45.103629  9830 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 124337us
W20260709 12:26:45.107939  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 98367.9us
W20260709 12:26:45.131366  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.133577  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 124488us
I20260709 12:26:45.147297  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:45.150039 17316 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:26:45.152868 17316 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.157874 17041 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:45.158743 17231 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:45.159644 17041 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:45.174201  9948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.013s	sys 0.006s
W20260709 12:26:45.174765  9948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.014s	sys 0.006s
I20260709 12:26:45.175561 17575 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:45.176038 17575 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:45.213618 17358 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:45.214573 17358 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.217373 17374 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:45.219498  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78472.4us
I20260709 12:26:45.177379 17575 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:45.233595  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 86640.7us
I20260709 12:26:45.271373 17358 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:45.278210 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 122995us
I20260709 12:26:45.287478 10674 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:26:45.287570 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:45.290843 10022 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:45.290702 17316 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:26:45.291321  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 281862us
I20260709 12:26:45.291553  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:45.292132 10016 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:26:45.271225 17566 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:45.298458 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:45.299489 10676 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:45.301720  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:45.304000 17526 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.304936 17526 raft_consensus.cc:993] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: : Instructing follower 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:26:45.305550 17526 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:45.306982 17587 raft_consensus.cc:1081] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Signalling peer 9aa345bcd43b44088114b7b6f2830b17 to start an election
I20260709 12:26:45.307614 17442 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:45.308120 17442 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:45.310565 17566 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:45.311184 17566 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:45.320561 17326 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.328902 10097 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:26:45.329419 10097 pending_rounds.cc:85] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 12
I20260709 12:26:45.329741 10097 pending_rounds.cc:107] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:26:45.330109 10097 raft_consensus.cc:2889] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:45.331890  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:45.333405 17257 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:45.334141 17257 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.334462 17587 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:45.335268 17565 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:45.336761 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.337744 10091 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:45.339608 17390 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:45.342612 17531 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:45.309717 17442 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:45.354149  9940 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:37576
I20260709 12:26:45.354984 17544 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.355185  9940 raft_consensus.cc:606] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Received request to transfer leadership
I20260709 12:26:45.355851 17544 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.356766 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:45.357200 17574 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:45.357615 17574 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:45.360318 17575 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:45.355010 10674 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:45.369508 17587 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:45.370157 17587 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:45.344411 17531 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:45.371595 17587 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.372077 17531 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:45.373402 17587 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.374040 17388 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:45.374225 17531 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:26:45.344532 17571 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 12:26:45.376129 10092 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:45.376745 17577 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 12:26:45.375658  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:45.344537 17358 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:45.343068 17372 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:26:45.379110 17544 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:45.395239  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:45.402662 17358 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:45.426846 16661 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.930s	user 0.005s	sys 0.020s
I20260709 12:26:45.427430 16661 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:45.438856  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57819.7us
I20260709 12:26:45.451676 10676 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:45.453164 17358 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:45.459105  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83605.6us
I20260709 12:26:45.470849  9947 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:45.474937 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.475286  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:45.478035 17582 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 12:26:45.478998 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:45.480702 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:45.481932  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:45.482311 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:45.483112 10668 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:45.484311 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:45.485059 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:45.485982 17209 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:45.487126 17336 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:45.487924 17336 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.494899 17526 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:45.497287 17526 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:45.498946 17295 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:45.500670 17295 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 12:26:45.504691 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.514187 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:45.529577  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 86377.6us
I20260709 12:26:45.507150  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:45.502527 17295 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:45.502393 17461 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:45.490479 17336 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:45.514740  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:45.556703 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:45.554133  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:45.555111  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:45.552901 17258 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.010s	user 0.005s	sys 0.019s
W20260709 12:26:45.562863 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 76372.3us
I20260709 12:26:45.560141  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:45.538786 17295 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:45.540926 17461 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.561913  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:45.564275 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.539088 17041 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:45.568172  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61185.2us
I20260709 12:26:45.576547 10671 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:45.561069 17326 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.572052 17461 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.593997  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:45.594710 17584 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:26:45.596552 17444 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.603072  9949 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
W20260709 12:26:45.604079  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 66795.4us
I20260709 12:26:45.604827 17452 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:45.605660 17557 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:45.605772 17452 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.606832 17371 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:26:45.607831 17570 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.422s	user 0.000s	sys 0.002s
I20260709 12:26:45.622722 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:45.609634 17420 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:45.606420 17557 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:45.610842 17326 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:45.573348 17531 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:45.615061  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 120560us
I20260709 12:26:45.586858 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:45.574715 17258 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:45.605966 17295 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.619028 17231 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:45.621022 17587 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:45.622525  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.622907 17452 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:45.622982 10091 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:45.594183 17418 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.071s
I20260709 12:26:45.616513 10671 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:45.626397 17378 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.614082 17566 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.628317 17041 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:45.646834 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71045.9us
I20260709 12:26:45.635398 17371 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:45.650709  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.652592 17295 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.654873 17425 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:45.656226 17566 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:45.656234  9943 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 17. Current term is 18. Ops: []
W20260709 12:26:45.661687 17581 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:26:45.643914 17358 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:45.663144 17358 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.664892 10017 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:45.665429 17358 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:45.655561 17425 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.651304 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:45.671765  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 89455.3us
I20260709 12:26:45.672267 17326 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:45.672943 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 99784.1us
I20260709 12:26:45.673650 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:45.674293 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:45.676219 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:45.679598 17587 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.680534 17474 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:45.680948 17474 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:45.639058 17316 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.679733 17041 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:45.681917  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.702965 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52890.2us
I20260709 12:26:45.702816 17593 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:45.705755 17374 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.706705 17534 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:45.707319 17534 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.707717 17534 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.708209 10019 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:45.708715 17405 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:45.709530 10020 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:45930
I20260709 12:26:45.709998 17405 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.713560 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2769) took 1199 ms. Trace:
I20260709 12:26:45.714015 10096 rpcz_store.cc:276] 0709 12:26:44.514628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.515626 (+   998us) service_pool.cc:224] Handling call
0709 12:26:45.713293 (+1197667us) spinlock_profiling.cc:243] Waited 506 ms on lock 0x7b54003ae3d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:45.713473 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1164871424}
I20260709 12:26:45.717197  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2866) took 1251 ms. Trace:
I20260709 12:26:45.717654  9948 rpcz_store.cc:276] 0709 12:26:44.465559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.465690 (+   131us) service_pool.cc:224] Handling call
0709 12:26:44.467672 (+  1982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:44.470444 (+  2772us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:44.471004 (+   560us) log.cc:844] Serialized 14 byte log entry
0709 12:26:45.373001 (+901997us) spinlock_profiling.cc:243] Waited 20 ms on lock 0x7b1800365b00. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:26:45.626104 (+253103us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:45.671173 (+ 45069us) log.cc:844] Serialized 26 byte log entry
0709 12:26:45.671348 (+   175us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:45.671383 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:45.671466 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:45.716831 (+ 45365us) raft_consensus.cc:1685] finished
0709 12:26:45.716958 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:45.717112 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93593,"prepare.queue_time_us":153,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":123651,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":26942,"spinlock_wait_cycles":46045056,"thread_start_us":955022,"threads_started":2,"wal-append.queue_time_us":778884}
I20260709 12:26:45.693801 17209 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:45.722080  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 337470us
I20260709 12:26:45.725524 17587 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:45.706506 17374 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.722479 17557 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:45.723603 17371 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 21.18, Last appended by leader: 18, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.723907 17574 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:45.736123 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48364.8us
I20260709 12:26:45.736698 17574 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:45.737797  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:45.651149 17378 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.724602 17425 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:45.721999 17257 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:45.681303 17474 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.681293 17584 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:45.730319 10017 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:45.691787 17358 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.738569 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:45.743762 17393 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:45.741623 17378 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:45.743944 17025 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:45.741434 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:45.777416 17393 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:45.678011 17231 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.732551 17452 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:45.781028  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 174785us
I20260709 12:26:45.798434 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:45.750873 17575 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:45.744277 17257 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:45.778138 17025 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:45.748898 17374 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:45.710534 17405 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.823697 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 17 candidate_status { last_received { term: 15 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:45.823957 10092 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:45.825750 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:45.827239 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:45.836654  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49065.9us
I20260709 12:26:45.836908 10676 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:45.832442 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.738798 17316 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.833257 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.741794 17209 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.080s
W20260709 12:26:45.843156 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 104681us
W20260709 12:26:45.846796  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 95483us
I20260709 12:26:45.821667 17025 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.819295 17575 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:45.773430 17418 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:26:45.857718 17538 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:26:45.858613  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.859368 17389 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:45.860519 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.861711 17578 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:45.855650 17584 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.862126 17578 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:45.863257 17578 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.864992 17578 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:45.865089 17584 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:45.865737 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:45.867264 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.867865 17231 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:45.867980 10090 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:45.868710 17593 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.869446 17593 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:45.870675 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47231.5us
I20260709 12:26:45.872265 10020 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:26:45.872699 10020 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:45.873126 17531 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:45.874918 17442 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:45.875383 17316 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:45.876061 17405 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:45.879505  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2617) took 1391 ms. Trace:
I20260709 12:26:45.853256 17452 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.880779 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:45.882984 17520 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:45.883598 17520 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:45.883021 17393 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.884040 17520 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:45.884397 17393 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:45.884895 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:45.878885 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:45.890254 10024 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
W20260709 12:26:45.853718  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49644.1us
I20260709 12:26:45.818841 17358 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.879981  9949 rpcz_store.cc:276] 0709 12:26:44.488034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:44.488318 (+   284us) service_pool.cc:224] Handling call
0709 12:26:45.879262 (+1390944us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540003e154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:45.879398 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2416184064}
I20260709 12:26:45.892073 10024 pending_rounds.cc:85] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 14
I20260709 12:26:45.907194 10024 pending_rounds.cc:107] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:26:45.907445 10024 raft_consensus.cc:2889] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:45.917462  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.856406 17209 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:45.923590 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:45.905838 17452 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:45.936415  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:45.937428  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:45.938503 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:45.940282 17374 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:45.940927 16661 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:45.942178 16661 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:45.942546 16661 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:45.943493 16661 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.944730 16661 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:45.947091 17559 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:45.947402 10020 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.948275 17559 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.948681 10025 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:45.949143 10016 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:45.948913 17559 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.950944 17418 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:45.951088 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.919755 17388 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:26:45.940326 17525 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:45.922443 17209 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:45.955153 17420 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:45.955705 17420 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.956003 17209 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:45.956877 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:45.957283 17209 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:45.957818 17531 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:45.958092 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46148.7us
I20260709 12:26:45.954739 17150 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:45.951436 17418 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:45.959019 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:45.959928 17418 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:45.960589 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:45.961512 17418 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:45.962922 17335 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:45.963686 17335 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:26:45.965406 17595 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:26:45.967044 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:45.860608 17389 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:26:45.931241 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:45.974016 17531 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:45.974467 17559 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.007905  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.012914  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.008863  9947 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:46.020603  9943 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 13. (term mismatch)
I20260709 12:26:46.022228  9946 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:26:46.022919 10098 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:46.015566 17378 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 12:26:46.012835 16661 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:46.024327 16661 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.026533 16661 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.001166 17258 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:46.040791  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.021155  9943 pending_rounds.cc:85] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 12
I20260709 12:26:46.043179  9943 pending_rounds.cc:107] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:26:46.043520  9943 raft_consensus.cc:2889] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:45.954746 17525 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.051668  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.045440 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:46.064512 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:46.065412 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:46.013226 17374 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:26:46.078953  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.083634 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.091567 10017 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:46.094373  9946 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:46.112651 17525 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.115643 17474 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.118045 17474 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.118734 17414 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.119007 17420 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.119470 17414 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.120635 17289 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.096982 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:46.076242 17425 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:46.122395 17289 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:26:46.071498 17566 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:46.066160 17531 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.040s
I20260709 12:26:46.122714  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.141026  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.138635  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.141779 17041 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.143646 17041 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.146173 17041 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.146703  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.147405  9942 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:46.148110  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.148767  9946 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
W20260709 12:26:46.151374 17602 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:26:46.126926 17425 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:46.155000 17425 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.157302 17520 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.159238 17534 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.160823  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:46.092908 17258 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.161491 17534 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:46.161814  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.162618  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:46.163126 17258 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.125448 17358 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.163650 17442 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:46.164659 17593 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:46.167212 17461 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.519s	user 0.011s	sys 0.000s
I20260709 12:26:46.167354 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:46.167727 17461 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.168574 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.169962  9940 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 17. (term mismatch)
I20260709 12:26:46.170110 10098 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:26:46.170354  9940 pending_rounds.cc:85] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 16
I20260709 12:26:46.170693  9940 pending_rounds.cc:107] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:26:46.170866 17534 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.171653 17534 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.172394 17566 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.170948  9940 raft_consensus.cc:2889] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:46.176411 17531 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:46.177158 17520 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.177047 17531 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.177449 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.134843 17378 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:46.180123  9945 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:46.170439 10098 pending_rounds.cc:85] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 9
I20260709 12:26:46.183193 10098 pending_rounds.cc:107] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:26:46.182890 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.183466 10098 raft_consensus.cc:2889] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:46.184428 10097 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:26:46.172803 17461 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.191674 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.193197 10090 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:46.141286 17414 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.189988 17335 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:46.204950  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.191543 17420 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:46.206286  9949 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:26:46.210361  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49642us
I20260709 12:26:46.204923  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.212466  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:46.218436 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 137174us
I20260709 12:26:46.219888 10091 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:46.220322  9947 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:46.196655 17452 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.174003 17025 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.229516 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.222265 17603 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:46.262390 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:46.214604 17209 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:46.271842  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54145.1us
I20260709 12:26:46.259567  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.227918 17414 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.272970  9949 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:46.214215 17335 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.219408 17593 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:26:46.278520 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:46.231096 17566 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.263289 17603 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.291988 10093 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:46.291231 10094 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: [19.16-19.16]
I20260709 12:26:46.304193  9949 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:46.304436  9944 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:46.259622 17452 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:46.310060 17609 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:26:46.292735 17603 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:46.290788 17414 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.328873 17414 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:46.329011 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:46.291644 17209 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.331519 17209 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:46.331640 17614 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:26:46.334749 17618 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:46.335623 17618 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:46.336189 17618 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.343437 17025 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.329052 17452 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:46.340409 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 108951us
I20260709 12:26:46.378188 17414 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:46.378767 17414 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:46.379216 17414 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.381660 17603 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.383980 17603 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.387235 17358 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.403229 17358 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.405351 17358 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.371830 17544 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:46.406162 17544 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.408703 17544 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.415449 17619 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:46.416224 17619 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:46.422986 17621 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:26:46.417979 17414 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.418383 17619 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:46.425792 17025 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:26:46.463500 10673 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:46.467136 10673 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:46.467437 17389 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.468284 17389 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.469909 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:46.470786 17566 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.471451 17566 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:46.474713 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.447662 17358 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.479203 17358 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.481613 17358 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:46.485532 10673 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:46.493657 10020 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.499444 17374 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:46.501570 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:46.447856 17619 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.512033 17442 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:46.514364  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.557677  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.558353  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.558921  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.559437 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57978us
I20260709 12:26:46.560593 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:46.541406  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:46.551921  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 279057us
W20260709 12:26:46.549595 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 163263us
I20260709 12:26:46.548930 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:46.561619  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:46.573377 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.573720 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.574424 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:46.522485 17624 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.215s	user 0.000s	sys 0.001s
W20260709 12:26:46.572892  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 255764us
I20260709 12:26:46.580287  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.573935 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.589504  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.549085 17474 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.590513  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:46.581212  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:46.574405 10093 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:46.537979 17534 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:46.549221 17544 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.592633  9944 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:46.581889  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.601015  9949 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 20 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:26:46.545460 17335 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:46.594256 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.601835 17538 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.599366 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.602519 17538 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:46.602968  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:46.603302 10093 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:46.574218 17378 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:46.611184  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:46.625561  9946 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:46.556255 17566 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.604091  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:46.518975 17425 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.618487  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:46.620018 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71184.6us
I20260709 12:26:46.577901 17374 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.567641 17418 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:46.624163 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.577955 17405 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:46.629532  9843 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 17, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 17 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:46.597782 17619 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.631656  9843 catalog_manager.cc:5697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 18, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 18 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:46.633396 17619 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.633222  9843 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 16 to 17, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 17 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:46.634959  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.635520 17414 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.635845  9948 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:46.636300  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:46.636580  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:46.636940 17150 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.637503 17150 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.637784  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:46.637995 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:46.639916 17538 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:46.640612 17420 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:46.640668 17538 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:46.641094 17465 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:46.641855 17465 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:46.642194 17465 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.642536 17465 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:46.642665 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.643618 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.646171 17425 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.646780 17558 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:46.646970 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.647832 10099 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:46.648629 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.649173 10099 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:46.650025 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.650918 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.651515 10677 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:26:46.651600 10099 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 15.
I20260709 12:26:46.594213 17258 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.652772 10091 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:46.652997 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.653131 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.653762 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:46.654035 17531 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:46.581166 17442 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.655735 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.656634 10096 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:46.656773 17442 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.582160 17257 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.075s
I20260709 12:26:46.577863 17593 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.080s
I20260709 12:26:46.540172 17575 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.583611 17452 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.646857 17425 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:46.640002 17150 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.662985  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:46.667447 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:46.710955 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.711828 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.710260  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.654125 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.713371 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:46.746984 10098 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:26:46.711472 10021 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:26:46.711421 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:46.658522 10096 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:46.665110 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.759655  9948 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 23 because replica has last-logged OpId of term: 21 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 12:26:46.759523 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:46.761266  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 84652.1us
I20260709 12:26:46.710973 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:46.710186 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.593848 17389 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.653785 10671 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:46.759950 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:46.768590  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:46.769558  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80167.9us
I20260709 12:26:46.648629 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.745610  9843 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:46.763888 10018 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:46.766008 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:46.800866 10096 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:46.768217 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100892us
I20260709 12:26:46.802204 10094 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:26:46.803960  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:46.779765  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.780179  9941 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:46.780987 10022 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:46.659535 17627 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.293s	user 0.002s	sys 0.000s
I20260709 12:26:46.660108 17597 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:46.817091 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.816644 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.659584 17405 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.663267 17531 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.821261 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.822502  9949 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:46.743266 17425 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.825960 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.816792  9843 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 17 to 20, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 20 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:46.693339 17628 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:46.802021 17150 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 12:26:46.808974 17557 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:26:46.767765 17452 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.659859 17418 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.652839 17258 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.836182 10024 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:46.847345 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.882628 10671 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:46.838155 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.618384 17474 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.837392 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:46.812170 17358 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:46.869231 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:46.781159 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:46.840473  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.872897 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:46.844858 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:46.846877 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:46.847122 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.849398 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.870242 10020 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 14.
I20260709 12:26:46.851078 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:46.842152 10017 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:46.840866 10021 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:46.848217  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:46.854925 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:46.871599 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:46.851845 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:46.825446 17414 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.887753 10093 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:26:46.834865 17316 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:46.801132 17461 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:26:46.880496  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:46.894071 10097 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:46.895298 10677 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.872268 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:46.663213 17544 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.899545  9843 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 16 to 18, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 18 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:46.847210 17425 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.841198 17619 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:46.810691 17618 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:46.913417  9945 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:46.847600 17628 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:46.871347 17418 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:46.845346 17405 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:46.918396  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 250616us
I20260709 12:26:46.869553 17452 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:46.840021 17597 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:46.887737 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50523.9us
I20260709 12:26:46.951135 10670 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:46.890292 17358 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:46.952306 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:46.907927 17316 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:46.901834 17534 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:46.902299 17630 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:46.950251  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:46.957010 10096 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:46.878742 17531 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:46.967346 10674 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:46.963788  9940 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:46.890976 17025 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:26:46.846166 17632 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.108s
W20260709 12:26:46.965530  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 185250us
I20260709 12:26:46.987624  9843 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 19, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 19 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:46.989925 10672 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:26:46.996510  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 162123us
I20260709 12:26:46.954905 17597 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:46.956687 17358 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:46.957800 17316 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:46.958140 17630 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.016440 10099 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:47.018316  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 214446us
I20260709 12:26:47.015287  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:46.948808 17414 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:47.027002 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.033684  9843 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 15, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 15 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:26:47.046316  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 203099us
I20260709 12:26:47.048278 10093 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:47.022517 17575 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:47.024636 17630 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:46.984408 17619 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:47.094141 10097 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:47.030062 17372 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:26:47.094012  9843 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 17, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 17 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:26:47.025178 17371 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:26:47.098785  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:47.099632 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.104079 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:47.068603 17257 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:47.069684 17374 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:47.109819 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.071244 17452 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.111631  9843 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 16 to 17, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 17 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:47.104986 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:47.117816 10099 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:47.121783 10019 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:47.122736 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:47.094172 17559 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.128139 10094 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 15: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:47.101387 17558 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:47.132894 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 180692us
W20260709 12:26:47.133411  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.134698 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:47.135480  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.135950  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.128640 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 234637us
I20260709 12:26:47.138121 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.111349 17257 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.139087  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 145673us
W20260709 12:26:47.138242 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 266099us
I20260709 12:26:47.141604  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:47.147904  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:47.128993 17374 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:47.165380 17372 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:47.165951 17372 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:47.154019  9843 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 17 to 22, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 22 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:47.171627  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.071803 17628 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:47.143899 17258 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:47.152424 17257 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:47.178544  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:47.183624  9942 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:47.181047  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:47.186084 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.129532 17538 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:26:47.133937 17452 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:47.178776 17634 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:26:47.161154 17374 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:47.167430 17618 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:47.209167 17418 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:47.210060 17474 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:47.211082 17258 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:47.211757 17578 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:47.212591  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:47.213742 17442 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:47.143275 17558 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:47.218024 17629 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:26:47.219101 17638 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:26:47.219755 17654 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:26:47.220100  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.290630  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:47.178515 17642 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:47.232252 17372 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:47.194510 17150 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:26:47.191363 17640 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:26:47.242848 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:47.141357 17559 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:47.244886 10017 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 12:26:47.211015 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.245692 17531 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:47.274539 17636 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.239s	user 0.001s	sys 0.001s
I20260709 12:26:47.285732 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.260864 17578 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:47.349786 10016 raft_consensus.cc:2410] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 12:26:47.286502  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74004.7us
I20260709 12:26:47.271355 17442 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:47.191774 17617 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.135s
I20260709 12:26:47.298636 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.239534 17374 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:47.262912  9946 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.635s	user 0.010s	sys 0.000s
I20260709 12:26:47.367000 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.367431  9946 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:47.284085 10022 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:47.316862 17150 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.370368 10023 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:47.273046 17629 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.179507 17648 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.143s
I20260709 12:26:47.350584  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:47.278023  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 290506us
I20260709 12:26:47.352563 10090 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:47.374517 10018 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:47.355396  9843 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 11 to 13, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 13 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.359647  9945 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:47.377660  9946 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:26:47.378201 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:47.374359  9949 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
W20260709 12:26:47.377225 10098 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.001s
I20260709 12:26:47.345912 17653 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:47.365262 10677 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:47.361912 10019 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:47.296065 17371 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:47.371480 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 224468us
I20260709 12:26:47.379304  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:47.395470 10675 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:47.395993 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:47.397411 10091 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.000s
I20260709 12:26:47.397899 10091 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:47.357815 17649 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:47.363649 17414 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:47.377960 17629 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:47.385766 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:47.409548 10016 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:47.311429 17642 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:47.380740 10098 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:47.399953  9843 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 16, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 16 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:47.381691 17645 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.380556 17150 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:47.424834 17559 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:47.426267 17610 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:47.430864 17653 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:47.431567 17654 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:47.450513 17442 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:47.488193  9941 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 12:26:47.488721  9941 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:47.394377 17632 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.479199  9843 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 21, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:47.505007 10677 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 19. (index mismatch)
I20260709 12:26:47.515846 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:47.405432 17649 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:47.527403  9843 catalog_manager.cc:5697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 9 to 15, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 15 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:26:47.544973 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 356905us
W20260709 12:26:47.549654 17425 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.589s	user 0.005s	sys 0.005s
I20260709 12:26:47.550500 17425 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:47.553139 17425 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:47.555660  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.513289 17461 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:47.498481 17371 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:47.556907  9843 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:26:47.543318 17442 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:47.527384 17649 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:47.496102 17653 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:47.575117 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 168281us
I20260709 12:26:47.559531 17474 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:47.501869 17632 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:47.496451 17645 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:47.577852 17358 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.549s	user 0.005s	sys 0.006s
I20260709 12:26:47.588546  9843 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 16 to 19, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 19 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:26:47.592088  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.585388  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 206144us
I20260709 12:26:47.593550  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:47.582125 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:47.595695 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 424326us
I20260709 12:26:47.599028  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:47.600107  9941 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:47.604571 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:47.605429  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 230878us
W20260709 12:26:47.588075 17465 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.945s	user 0.013s	sys 0.000s
I20260709 12:26:47.563274 17257 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:47.586480 17452 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:47.646454  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:47.670563 10095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.007s	sys 0.021s
I20260709 12:26:47.591559 17619 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:47.633266  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:47.671510 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89490.4us
I20260709 12:26:47.782246 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:47.783365 10095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.120s	user 0.007s	sys 0.031s
W20260709 12:26:47.698853 10093 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.635s	user 0.011s	sys 0.000s
I20260709 12:26:47.722045  9843 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:26:47.698169  9940 raft_consensus.cc:2368] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
W20260709 12:26:47.784540 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 238541us
I20260709 12:26:47.671559  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:47.788870  9949 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:47.621930 17662 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:47.672465 10016 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
W20260709 12:26:47.819029  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 189903us
I20260709 12:26:47.787760 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:47.820510  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:47.827527 10093 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:26:47.823173  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 151610us
I20260709 12:26:47.823411  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:47.621685 17672 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:47.833587 17662 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:47.759408 17660 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 12:26:47.903913 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:47.837904  9843 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 16 to 19, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 19 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:26:47.894394  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:47.915531  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:47.629179 17610 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:47.768971 17667 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:47.918583 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:47.724922 17378 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:47.930852  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:47.836788 17452 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:47.765965 17666 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:26:47.671500 17465 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:47.935374  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:47.936014  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 112684us
I20260709 12:26:47.925575  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:47.610662 17358 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:47.940987 10668 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:47.765182 17683 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:47.941864  9948 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:47.768332 17645 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:47.839583 17393 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:47.914770 17672 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:47.927194 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 426252us
I20260709 12:26:47.835754 17619 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:47.918500 17531 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:48.001230 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:48.004984  9949 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:48.010142  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:47.954726  9843 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 19 to 20, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 20 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:26:47.931105 17558 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:47.952361 17041 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:47.945652 17653 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.049103 10025 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:26:48.006901 17603 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:47.945729 17642 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:48.026624 17374 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.627s	user 0.007s	sys 0.004s
I20260709 12:26:47.960544 17452 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.142233 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2806) took 1158 ms. Trace:
I20260709 12:26:48.022480 17672 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:48.141302 17418 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:47.953547 17378 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:48.163407  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 153396us
I20260709 12:26:48.051949 17558 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 12:26:48.052310 17619 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:26:48.152931 17405 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.193s	user 0.013s	sys 0.007s
I20260709 12:26:48.062354 17257 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.157719 10096 rpcz_store.cc:276] 0709 12:26:46.906964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.992694 (+ 85730us) service_pool.cc:224] Handling call
0709 12:26:46.993061 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.003689 (+ 10628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.003740 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.021119 (+ 17379us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.196927 (+175808us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:47.196999 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.198775 (+  1776us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.065177 (+866402us) raft_consensus.cc:1685] finished
0709 12:26:48.065200 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.065382 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2729,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":9154,"thread_start_us":109865,"threads_started":1,"wal-append.queue_time_us":352647}
I20260709 12:26:48.066844 17371 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.021553 17680 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.162s
I20260709 12:26:48.142196 17393 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:48.172351 17041 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:48.023626 17610 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.128s
I20260709 12:26:48.180271 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3979) took 1627 ms. Trace:
I20260709 12:26:48.029964 17531 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.068370 17231 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.113s
W20260709 12:26:48.169176  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 145949us
I20260709 12:26:48.169914 17632 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.142122 17696 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:26:48.228221 17670 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:26:48.228655 17677 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:48.230006 17663 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:26:48.231346 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:48.145262 17374 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.233043  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:48.061786 17683 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.247095 17538 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.149433 17668 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.492s
W20260709 12:26:48.368880  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 135918us
I20260709 12:26:48.221860 10095 rpcz_store.cc:276] 0709 12:26:46.553199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.553305 (+   106us) service_pool.cc:224] Handling call
0709 12:26:46.572985 (+ 19680us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.580370 (+  7385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:46.580420 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.589126 (+  8706us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.784036 (+1194910us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:47.784098 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.784197 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.175153 (+390956us) raft_consensus.cc:1685] finished
0709 12:26:48.175180 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.180203 (+  5023us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22288,"raft.queue_time_us":7094,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":10440,"thread_start_us":1120209,"threads_started":1,"wal-append.queue_time_us":972615}
I20260709 12:26:48.388528 17326 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:48.242535 17414 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.390575 17682 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.427287 17326 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:48.430970  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2898) took 1139 ms. Trace:
I20260709 12:26:48.426879 17531 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:48.357548 17676 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:48.434965 17534 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:48.436273 17465 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:48.437577 17575 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.367s	user 0.010s	sys 0.002s
I20260709 12:26:48.438109 17701 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.438549 17378 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:48.439190 17699 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.258s	user 0.000s	sys 0.001s
W20260709 12:26:48.416139  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 490608us
I20260709 12:26:48.557150  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:48.415042 17538 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.428017 17358 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.422029 17619 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.424753 17041 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.420634 17393 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.453616 10677 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:48.391613  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2901) took 1023 ms. Trace:
W20260709 12:26:48.483011  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71498.4us
W20260709 12:26:48.571014 10641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.015s	sys 0.039s
W20260709 12:26:48.597878 10641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.910s	user 0.015s	sys 0.039s
I20260709 12:26:48.599191  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2897) took 1308 ms. Trace:
W20260709 12:26:48.433802 17630 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.345s	user 0.010s	sys 0.001s
I20260709 12:26:48.249178 17653 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:48.318185 17632 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.361524 17628 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.191378 17405 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.527451  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2643) took 1814 ms. Trace:
I20260709 12:26:48.595830  9945 rpcz_store.cc:276] 0709 12:26:47.368482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.382722 (+ 14240us) service_pool.cc:224] Handling call
0709 12:26:47.407693 (+ 24971us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.508734 (+101041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.508798 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.528000 (+ 19202us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.938783 (+410783us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:47.938855 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.938950 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.391388 (+452438us) raft_consensus.cc:1685] finished
0709 12:26:48.391410 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.391555 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36283,"raft.queue_time_us":69263,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":15930,"thread_start_us":355067,"threads_started":1,"wal-append.queue_time_us":832370}
I20260709 12:26:48.545365 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:48.643365  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:48.545583  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:48.595888 17442 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.015s	user 0.012s	sys 0.000s
I20260709 12:26:48.485471  9942 rpcz_store.cc:276] 0709 12:26:47.291121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.352945 (+ 61824us) service_pool.cc:224] Handling call
0709 12:26:47.353284 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.834526 (+481242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.834561 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:47.935636 (+101075us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.179234 (+243598us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:48.179320 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.179521 (+   201us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.430705 (+251184us) raft_consensus.cc:1685] finished
0709 12:26:48.430734 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.430907 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58781,"raft.queue_time_us":154688,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":12315,"thread_start_us":566772,"threads_started":2,"wal-append.queue_time_us":348659}
I20260709 12:26:48.426909 17683 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:48.533301 17663 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:26:48.490545 17326 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:48.568245 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:48.483983 17680 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:48.576383 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.004s	sys 0.012s
W20260709 12:26:48.663491 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.004s	sys 0.012s
I20260709 12:26:48.579015 17566 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.581959 17673 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.875s	user 0.007s	sys 0.010s
I20260709 12:26:48.585987 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2529) took 1516 ms. Trace:
I20260709 12:26:48.155792 17662 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:48.547435 17641 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.024s	sys 0.071s
I20260709 12:26:48.537817 17603 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.537655 17531 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:48.610955 17678 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.611415 17681 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:48.612882  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:48.613405 17656 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:48.614383 17707 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:26:48.662870 17461 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:48.563884 17701 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.604770 17358 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.619202 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1554) took 1119 ms. Trace:
I20260709 12:26:48.620500 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1364) took 1970 ms. Trace:
I20260709 12:26:48.620975 17474 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.621678 17694 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.622440 17371 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:48.623417 17657 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:48.624612 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2545) took 1282 ms. Trace:
W20260709 12:26:48.625437  9940 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2644) took 1488 ms (client timeout 1522 ms). Trace:
I20260709 12:26:48.599726  9943 rpcz_store.cc:276] 0709 12:26:47.290819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.291581 (+   762us) service_pool.cc:224] Handling call
0709 12:26:47.342035 (+ 50454us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.368201 (+ 26166us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:47.577795 (+209594us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.932018 (+354223us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.932178 (+   160us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.587621 (+655443us) spinlock_profiling.cc:243] Waited 5.57 ms on lock 0x7b2c00012928. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:26:48.588160 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.588355 (+   195us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:48.588402 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.588495 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.598933 (+ 10438us) raft_consensus.cc:1685] finished
0709 12:26:48.598957 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.599114 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":123253,"prepare.queue_time_us":498151,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":346337,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":26663,"spinlock_wait_cycles":48681344,"thread_start_us":463733,"threads_started":2,"wal-append.queue_time_us":630259}
I20260709 12:26:48.627260 17686 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:48.636039 17669 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.939s	user 0.008s	sys 0.012s
I20260709 12:26:48.636698 17618 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:48.637588 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2533) took 1561 ms. Trace:
I20260709 12:26:48.638126 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:48.369558 17677 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:48.645372 10676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.046s	sys 0.066s
I20260709 12:26:48.486363 17414 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:48.701061 17629 consensus_meta.cc:287] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.269s	user 0.011s	sys 0.002s
I20260709 12:26:48.601894 17534 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:48.734894 17534 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.677498 17680 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.539601 17682 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.737306 17534 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.737478 17680 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.737891 17534 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.707785 17461 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:48.738135 17682 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.738791 17682 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.604382 17465 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.695313 17668 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.739354 17629 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.739784 17465 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.740041 17668 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.741703 17465 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.742689 10672 raft_consensus.cc:2368] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:26:48.742337 17686 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.721411 10677 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:48.643414  9944 rpcz_store.cc:276] 0709 12:26:46.713077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.845922 (+132845us) service_pool.cc:224] Handling call
0709 12:26:46.891116 (+ 45194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:46.902644 (+ 11528us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:46.902681 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:46.939885 (+ 37204us) log.cc:844] Serialized 26 byte log entry
0709 12:26:47.501182 (+561297us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:47.501249 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.501335 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.527134 (+1025799us) raft_consensus.cc:1685] finished
0709 12:26:48.527163 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.527368 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3574,"raft.queue_time_us":153943,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":16915,"thread_start_us":485248,"threads_started":1,"wal-append.queue_time_us":946515}
I20260709 12:26:48.668704 10019 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:48.604619 17610 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:48.666219 10018 rpcz_store.cc:276] 0709 12:26:47.069886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.070043 (+   157us) service_pool.cc:224] Handling call
0709 12:26:47.097966 (+ 27923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.375031 (+277065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.585581 (+1210550us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540032aad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:48.585875 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618463616}
W20260709 12:26:48.678496 17150 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.175s	user 0.008s	sys 0.007s
I20260709 12:26:48.758956  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:26:48.760380 10672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1556) took 1259 ms (client timeout 1671 ms). Trace:
I20260709 12:26:48.678690  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:48.684032 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 452762us
I20260709 12:26:48.762849 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:48.764676 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:48.640731 17696 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.603838 17630 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.560431 17676 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.686743 10671 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
W20260709 12:26:48.694437  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81642.1us
I20260709 12:26:48.712693 10024 rpcz_store.cc:276] 0709 12:26:47.341989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.361281 (+ 19292us) service_pool.cc:224] Handling call
0709 12:26:47.381932 (+ 20651us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:47.706009 (+324077us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.706151 (+   142us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:47.750628 (+ 44477us) log.cc:844] Serialized 52 byte log entry
0709 12:26:47.954668 (+204040us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:47.954711 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.954773 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.624368 (+669595us) raft_consensus.cc:1685] finished
0709 12:26:48.624392 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.624553 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14,"raft.queue_time_us":28371,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":82,"thread_start_us":409676,"threads_started":2,"wal-append.queue_time_us":859652}
I20260709 12:26:48.768381  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:48.684712 17566 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:48.683645 17641 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.025s	sys 0.072s
I20260709 12:26:48.699267 10669 rpcz_store.cc:276] 0709 12:26:47.499862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.499975 (+   113us) service_pool.cc:224] Handling call
0709 12:26:47.523570 (+ 23595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.597105 (+ 73535us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.597137 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:47.597178 (+    41us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:47.597211 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.618789 (+1021578us) spinlock_profiling.cc:243] Waited 73.3 ms on lock 0x7b540050a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:48.618838 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.618988 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168649088}
I20260709 12:26:48.700410 10675 rpcz_store.cc:276] 0709 12:26:46.650212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.946114 (+295902us) service_pool.cc:224] Handling call
0709 12:26:47.395286 (+449172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.398443 (+  3157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.620262 (+1221819us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b54007602d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:48.620435 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010503552}
I20260709 12:26:48.643975 17632 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.642963 17628 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.642397 17653 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:48.721220 10676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.815s	user 0.046s	sys 0.066s
I20260709 12:26:48.698741 17603 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.725275 17414 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.725620 17718 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.784111 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1357) took 2133 ms. Trace:
I20260709 12:26:48.726289 17677 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.729022  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2903) took 1306 ms. Trace:
I20260709 12:26:48.682840 17578 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:48.706434 17316 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.673s	user 0.011s	sys 0.003s
I20260709 12:26:48.701488 17714 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:48.738437 17534 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:48.667865  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 122411us
W20260709 12:26:48.668381 10674 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.022s	sys 0.032s
W20260709 12:26:48.722405  9940 rpcz_store.cc:269] 0709 12:26:47.138696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.138901 (+   205us) service_pool.cc:224] Handling call
0709 12:26:48.625211 (+1486310us) spinlock_profiling.cc:243] Waited 549 ms on lock 0x7b5400222ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:48.625373 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1263581568}
W20260709 12:26:48.760803 10672 rpcz_store.cc:269] 0709 12:26:47.500615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.500717 (+   102us) service_pool.cc:224] Handling call
0709 12:26:48.743828 (+1243111us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54006a0254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:48.760276 (+ 16448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2819177344}
I20260709 12:26:48.769666  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:48.614943 17644 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:26:48.703035 17531 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.722208 17618 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:48.777719 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:48.821875 10675 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:48.706943 17371 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:48.712311 17657 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:48.779570  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:48.721624 10021 rpcz_store.cc:276] 0709 12:26:47.075828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.075956 (+   128us) service_pool.cc:224] Handling call
0709 12:26:47.099116 (+ 23160us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:47.133675 (+ 34559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.133721 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:47.152143 (+ 18422us) log.cc:844] Serialized 52 byte log entry
0709 12:26:47.490425 (+338282us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:47.697385 (+206960us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.708120 (+ 10735us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.778487 (+ 70367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.778621 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.637368 (+858747us) raft_consensus.cc:1685] finished
0709 12:26:48.637391 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.637531 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":289656,"prepare.queue_time_us":342697,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":85067,"raft.run_cpu_time_us":459,"raft.run_wall_time_us":75545,"thread_start_us":444767,"threads_started":2,"wal-append.queue_time_us":1404540}
I20260709 12:26:48.768064 17610 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.781984 16661 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:48.744256 17452 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.783041 10673 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:48.761950 17575 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:48.779357 17597 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.762s	user 0.008s	sys 0.006s
I20260709 12:26:48.761776 17680 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:48.728319 17405 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:48.788728 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:48.768770 17150 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.707208 17358 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.773902 17257 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:48.711071 17474 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.796132 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:48.791096  9946 rpcz_store.cc:276] 0709 12:26:47.422729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.478977 (+ 56248us) service_pool.cc:224] Handling call
0709 12:26:47.479379 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.562001 (+ 82622us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:48.030557 (+468556us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.575295 (+544738us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:48.575845 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 12:26:48.575983 (+   138us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:48.576018 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.576086 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.728798 (+152712us) raft_consensus.cc:1685] finished
0709 12:26:48.728822 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.728965 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80172,"prepare.queue_time_us":254887,"prepare.run_cpu_time_us":122,"prepare.run_wall_time_us":18167,"raft.queue_time_us":50429,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":11176,"thread_start_us":816004,"threads_started":2,"wal-append.queue_time_us":661314}
I20260709 12:26:48.793699 17557 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:48.798799 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:48.799335 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:48.695526 17678 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.782302 17420 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:48.739178 17461 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.804474 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:48.802079 10674 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.022s	sys 0.033s
I20260709 12:26:48.811558 17714 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.722000 17694 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.719229 17681 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.739276 17682 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:48.811594 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 200034us
I20260709 12:26:48.702332 17372 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:48.788231 10676 rpcz_store.cc:276] 0709 12:26:46.648233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.712613 (+ 64380us) service_pool.cc:224] Handling call
0709 12:26:46.801000 (+ 88387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:46.823453 (+ 22453us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:26:46.982990 (+159537us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.906046 (+923056us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.616062 (+710016us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b480002ea58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:26:48.635199 (+ 19137us) spinlock_profiling.cc:243] Waited 29.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:48.781456 (+146257us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:48.781516 (+    60us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:48.781569 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.781648 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.781777 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22601,"prepare.queue_time_us":269284,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":7331,"raft.queue_time_us":503662,"raft.run_cpu_time_us":14395,"raft.run_wall_time_us":997329,"spinlock_wait_cycles":72467584,"thread_start_us":1894921,"threads_started":4,"wal-append.queue_time_us":620669}
I20260709 12:26:48.864040 17257 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:48.771991 17696 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.842113 17701 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.844583 17603 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.846551  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:48.848263 17630 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.798926 17718 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.820060 17648 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:26:48.856003 17628 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.856755 17566 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.857443 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:48.816578 17650 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:48.799726 17713 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:48.858934 17632 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:48.830612 17578 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:48.908232 10018 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:48.799620 17316 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.909060 10017 raft_consensus.cc:2368] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:26:48.800617 17414 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.072s
I20260709 12:26:48.908596 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1555) took 1408 ms. Trace:
I20260709 12:26:48.911342 10677 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:48.929163 17714 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:48.932358 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2802) took 2265 ms. Trace:
I20260709 12:26:48.934520 17694 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.936693 17662 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:48.936981 17682 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.831317  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:49.001142 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:49.001762  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 170563us
I20260709 12:26:49.002934  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:48.938520 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1367) took 1034 ms. Trace:
I20260709 12:26:48.942713 10016 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
W20260709 12:26:48.951525 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:48.904906 17557 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:48.983291 17722 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:48.862035 17597 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:48.983989 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:49.012137 10024 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:26:48.945029 17667 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:48.944876 17257 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:48.942615 17657 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.021899  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:48.903148 17680 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.910912 17420 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.981243 17686 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.010113 17566 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:48.998493  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 196835us
I20260709 12:26:49.026535  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:48.996143 10093 rpcz_store.cc:276] 0709 12:26:46.684672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.939609 (+254937us) service_pool.cc:224] Handling call
0709 12:26:46.939940 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.828073 (+888133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.932013 (+1103940us) spinlock_profiling.cc:243] Waited 108 ms on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:48.932267 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":248763776}
I20260709 12:26:48.987666 17650 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:49.034866 17641 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:49.035696 17641 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:49.036921 17641 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.861770 17538 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.992305 10674 rpcz_store.cc:276] 0709 12:26:47.500250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.500357 (+   107us) service_pool.cc:224] Handling call
0709 12:26:47.524114 (+ 23757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.524400 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:47.546566 (+ 22166us) log.cc:844] Serialized 14 byte log entry
0709 12:26:48.908119 (+1361553us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:48.908196 (+    77us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:48.908257 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.908345 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.908520 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55218,"prepare.queue_time_us":21979,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":441684,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":147,"thread_start_us":1295729,"threads_started":2,"wal-append.queue_time_us":1048178}
I20260709 12:26:49.004472 10668 rpcz_store.cc:276] 0709 12:26:47.904243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.016060 (+111817us) service_pool.cc:224] Handling call
0709 12:26:48.043579 (+ 27519us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:48.561511 (+517932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:48.561548 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:48.590650 (+ 29102us) log.cc:844] Serialized 52 byte log entry
0709 12:26:48.788332 (+197682us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:48.788366 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:48.788429 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.938277 (+149848us) raft_consensus.cc:1685] finished
0709 12:26:48.938300 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:48.938460 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":340397,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":22317,"thread_start_us":588816,"threads_started":2,"wal-append.queue_time_us":325754}
I20260709 12:26:48.996296 17534 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:48.861107 17575 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:48.996851 17632 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:49.040877 17632 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.041867 17575 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:48.941543 17644 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:49.043015 17644 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.043764  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.044606  9944 raft_consensus.cc:2410] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:26:49.045207 17644 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:49.021142 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 256556us
I20260709 12:26:49.047921 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:49.009675 17372 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:48.941509 17371 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.029119 10022 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:49.051465 17371 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:49.055401 17534 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:49.057066 17538 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:49.058411 17538 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:49.058848 17538 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:49.050096 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:49.064603 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:49.069398 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:49.012429 17682 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.079638 17682 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.080135 17372 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:49.080818 17372 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.011742 17713 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:49.082103 17713 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.083763 17372 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:48.970487 17618 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.084739 17618 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.078688 17657 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.085333 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:49.092126 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.006192 17326 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.092720 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:49.071424 17650 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.097281  9987 leader_election.cc:400] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:26:49.098011  9987 leader_election.cc:403] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:49.025887 17694 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:49.090945 17452 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:49.088224 17686 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.090711 17729 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:49.095434 17619 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:49.118345 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.107249 17452 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:49.112172  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 82479.8us
W20260709 12:26:49.106545 17672 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.916s	user 0.006s	sys 0.006s
I20260709 12:26:49.113265 17728 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:49.103008 17644 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:49.093576 17676 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.100224 17722 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:49.122138 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.095551 17566 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:49.108036 17694 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.023138 17667 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.126617  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.126418 17452 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.102564 17372 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:49.106953 17617 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:49.106999 17653 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:49.132748 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.115826 17739 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:49.119688 17664 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.414s	user 0.009s	sys 0.006s
I20260709 12:26:49.135429 10090 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:49.136749 10093 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:49.160740 10093 raft_consensus.cc:606] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:26:49.110847 17531 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.140252 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.140585  9945 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 20 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:49.161471 10097 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:49.127511 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.141923 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.162349 10099 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:49.162276 10098 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:49.145505 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.163455 10094 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:49.132506 17358 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.133744 17672 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.138330 17557 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.152252 17372 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.156454 17694 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.157406 17718 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.184641  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2652) took 1582 ms. Trace:
I20260709 12:26:49.184993 17372 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:49.185441  9894 auto_rebalancer.cc:893] Removing replace marker failed: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:26:49.185911  9894 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for a7196ffc873e4e7e84c76068d5b95904 already in progress
W20260709 12:26:49.186220  9894 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for a7196ffc873e4e7e84c76068d5b95904 already in progress
I20260709 12:26:49.186465 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.186429 17566 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.187804  9940 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:49.188233 17372 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:49.157948 17610 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.158710 17696 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.188760 17372 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.138284 17474 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:49.140372 17722 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.189786 17722 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.185151  9941 rpcz_store.cc:276] 0709 12:26:47.602568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.670152 (+ 67584us) service_pool.cc:224] Handling call
0709 12:26:47.670563 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:47.678723 (+  8160us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:48.618126 (+939403us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b18003982c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:26:48.798090 (+179964us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.008590 (+210500us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.014938 (+  6348us) log.cc:844] Serialized 26 byte log entry
0709 12:26:49.020029 (+  5091us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:49.020072 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.020170 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:49.184264 (+164094us) raft_consensus.cc:1685] finished
0709 12:26:49.184412 (+   148us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.184576 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7594,"prepare.queue_time_us":216994,"prepare.run_cpu_time_us":131,"prepare.run_wall_time_us":53596,"raft.queue_time_us":52301,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":30929,"spinlock_wait_cycles":15073280,"thread_start_us":368519,"threads_started":2,"wal-append.queue_time_us":294252}
I20260709 12:26:49.192363  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.193084  9941 raft_consensus.cc:2368] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:26:49.117239 17686 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.194763 17672 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.195896 17566 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.161049 10093 raft_consensus.cc:614] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:49.182487 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.197506 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.195899 17686 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.198632  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.199918  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.201086  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.198632 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.202203  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.193879  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.202776 10021 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:49.203843 10065 leader_election.cc:400] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:26:49.208699 10065 leader_election.cc:403] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:49.208148  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.209487  9945 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:49.212116 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.212558 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.213250 10093 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:49.207803  9949 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:49.126889 17729 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.213397 10668 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:49.221431  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.222380  9941 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
W20260709 12:26:49.224287  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.143517 17644 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.227118  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.228075  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.233656  9941 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:49.234193  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.234862  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.235396  9945 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:26:49.177251 17653 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.231277 10674 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:49.239377  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.174911 17617 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:49.242964  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.214269 17316 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.250783  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:49.255234 17694 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:49.231452 17619 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:26:49.232188 17442 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:49.231950 17656 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.274106 17722 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.279219  9945 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:49.282289 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.288564 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.289628 10093 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:49.291038 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.295506 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.296276 10674 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
W20260709 12:26:49.296216 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 203570us
I20260709 12:26:49.297181 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:49.291649 10095 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:49.301438 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:49.247118 17653 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.307744 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.308674 10093 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:49.299304 17372 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.310470 17544 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.326663 17672 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:26:49.328786 17739 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:49.339843 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.338960 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.299703 17316 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.298977 17657 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.345088 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.280665 17694 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.375761 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.362563 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.362849  9945 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:49.352756 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.362434 10093 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:49.376873 10097 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.351199 10021 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:49.361800 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.384312 10090 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:49.305639 17718 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:49.385152 10096 raft_consensus.cc:2368] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21.
W20260709 12:26:49.351233 17743 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:26:49.388000  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:49.390363  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:49.387326  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 128940us
I20260709 12:26:49.308321 17680 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.361690 17372 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.326414 17668 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.338685 17646 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:49.399413 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2530) took 2319 ms. Trace:
I20260709 12:26:49.386068 17025 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.399020 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.418952 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.290740 17656 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.421528 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.422145 10090 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:26:49.419760 10092 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:49.423641  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.400235 17646 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:49.424979 17729 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:49.426061 17726 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.309s	user 0.002s	sys 0.000s
I20260709 12:26:49.425346 17646 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.427009 17646 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:49.427915 17374 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.427976 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.428936 10091 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:49.431819 17150 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.432601 17656 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:49.433636  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.433480  9941 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:49.409801 10020 rpcz_store.cc:276] 0709 12:26:47.074352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.074477 (+   125us) service_pool.cc:224] Handling call
0709 12:26:47.103762 (+ 29285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:47.137427 (+ 33665us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:47.137458 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:47.147540 (+ 10082us) log.cc:844] Serialized 52 byte log entry
0709 12:26:47.407550 (+260010us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:26:47.512811 (+105261us) log.cc:844] Serialized 14 byte log entry
0709 12:26:47.513031 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:47.513109 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:48.990507 (+1477398us) spinlock_profiling.cc:243] Waited 5.34 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:49.384799 (+394292us) raft_consensus.cc:1685] finished
0709 12:26:49.393244 (+  8445us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:49.393436 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46751,"prepare.queue_time_us":172900,"raft.queue_time_us":38097,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":20077,"spinlock_wait_cycles":17864192,"thread_start_us":248171,"threads_started":1,"wal-append.queue_time_us":418491}
I20260709 12:26:49.434911 10096 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:49.335237 17442 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.388526 17657 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.438895 17653 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.439744 17722 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:49.444860 17538 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.448166  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:49.450071 10017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2544) took 2160 ms (client timeout 1522 ms). Trace:
W20260709 12:26:49.450381 17744 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:26:49.452006  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.452621  9946 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:49.453441  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:49.453961  9946 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:49.455843 17597 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.454277 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.397739 17680 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.458812 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.464088 10024 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:49.365237 17316 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.459517 10096 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:49.472525 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.468804 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:49.474005 10643 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:49.450490 10017 rpcz_store.cc:269] 0709 12:26:47.289739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.349899 (+ 60160us) service_pool.cc:224] Handling call
0709 12:26:49.449824 (+2099925us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54001c5054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:49.449969 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3544289536}
I20260709 12:26:49.478847 10099 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:26:49.479671 10093 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [20.14-24.15]
I20260709 12:26:49.480392  9946 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:49.481207  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.482124  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:49.482577  9946 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.485805  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.487778 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.488620 10020 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:49.489517  9946 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:49.510357  9947 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:49.523581 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.459549 17538 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.559096 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.559580  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:49.559849 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.542214 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.541824 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.560880 10020 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:26:49.542435  9948 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:49.398655 17372 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.545004 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.551018 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.552977  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.556279 17657 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.556699 17442 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.484759 17653 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:49.466054 17708 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 15. Current term is 16.
I20260709 12:26:49.466353 17667 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:49.560326  9946 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:49.561810 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.586701 10096 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:49.486820 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:49.588495 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 101761us
I20260709 12:26:49.565265 10017 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:49.565279  9942 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:49.435642 17025 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.557951 17713 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.476812 17680 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.589502 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.591619 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.591812 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.544111 17425 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:49.384049 17617 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.562435 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2537) took 2431 ms. Trace:
I20260709 12:26:49.595958  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:49.596288  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:49.596514 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:49.566725 17374 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:26:49.603202 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.604805 10099 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:49.605178 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.607540 10020 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:49.604182 10023 rpcz_store.cc:276] 0709 12:26:47.131294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.131454 (+   160us) service_pool.cc:224] Handling call
0709 12:26:47.370192 (+238738us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:47.373108 (+  2916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.491962 (+2118854us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b540038d6d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:49.562395 (+ 70433us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503066368}
I20260709 12:26:49.609884 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.609800 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.612464  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.614140 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.613332 10668 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:49.617192  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:49.618440  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:49.619078  9946 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:49.607260 17618 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.518s	user 0.006s	sys 0.006s
I20260709 12:26:49.621160  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.595391 17722 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:49.594976 17557 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:26:49.566869 17650 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.626950 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.627874 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.628125 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.628973 10021 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:49.628783 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.632687 10020 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:49.625622  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:49.622255 17578 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.000s
I20260709 12:26:49.640321  9945 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:49.630465 17722 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:49.644060 17722 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.647320 10024 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:49.648917  9944 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:49.649747  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.651301 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.652505 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.653257 10097 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:49.636188  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:49.653991  9949 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:49.626870 17713 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:49.654558 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.626535 17644 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.658609 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.659876 10098 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:49.656436 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.631393 17603 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:49.658762 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.665988 10023 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:49.698992 17025 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:49.700073 17752 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:26:49.705406 17619 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:26:49.707677 17644 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:49.708895 17644 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:49.711172 17753 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:26:49.649588 17150 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:49.714900 17150 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:49.715292 17150 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:49.715694 17746 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:26:49.717940 17618 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:49.718348 17618 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.718719 17618 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:49.720439 17372 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:49.724581 17662 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.718s	user 0.011s	sys 0.000s
I20260709 12:26:49.725477 17644 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:49.726976 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65414.5us
I20260709 12:26:49.728226 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.729760 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.730405 17718 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:49.730937 17718 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:49.731472 17662 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.731882 17718 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.732049 17662 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.737042 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.737619 10092 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:49.738250 10098 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:26:49.739631 10090 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 16.
I20260709 12:26:49.740748 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:49.661392  9946 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:49.744531 10099 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
W20260709 12:26:49.657645 17257 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.628s	user 0.011s	sys 0.000s
I20260709 12:26:49.657884  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:49.746539 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:49.749926 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1379) took 3079 ms. Trace:
I20260709 12:26:49.741312 10090 raft_consensus.cc:2376] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25. Re-sending same reply.
I20260709 12:26:49.657011 17713 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:49.755182 10025 rpcz_store.cc:276] 0709 12:26:46.670477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.670653 (+   176us) service_pool.cc:224] Handling call
0709 12:26:46.711357 (+ 40704us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.061006 (+1349649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.749736 (+1688730us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54001c7354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:49.749880 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3075978496}
I20260709 12:26:49.765798 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.662374 17578 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:49.750509 17371 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.698s	user 0.010s	sys 0.003s
I20260709 12:26:49.662940 17676 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:26:49.753871 17452 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.000s
I20260709 12:26:49.777761 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.779269 10668 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:49.779646 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:49.756340 17649 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 2.170s	user 0.016s	sys 0.000s
I20260709 12:26:49.787485 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.793288 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.794421 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.756544 17729 raft_consensus.cc:2764] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:26:49.768239 17713 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:49.800541 10676 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:49.771804 17644 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:49.803104 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.770982 17694 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:49.811874 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.812749 10668 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:49.806553 10674 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:49.774773 17578 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.786975 17603 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:49.825413  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.826865 17650 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.827167 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.828246 17680 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.828395 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.829039 10676 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:49.830022 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.831115 17662 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.832201 17316 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.796327 17649 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.837893 10018 raft_consensus.cc:2368] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:26:49.839097 17649 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.839479 17150 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:49.796456 17452 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.841787 17150 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.786753 17676 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:49.842567 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.844188 16661 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:49.852209 16661 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.845065 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1543) took 3196 ms. Trace:
I20260709 12:26:49.846971 10024 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:49.848434  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:49.861493 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.853327 10671 rpcz_store.cc:276] 0709 12:26:46.648862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.896018 (+247156us) service_pool.cc:224] Handling call
0709 12:26:46.896432 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:48.768071 (+1871639us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.844762 (+1076691us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400528fd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:49.845000 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4116755840}
I20260709 12:26:49.861040  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.885124 10675 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:26:49.893467  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:49.896020  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.896938  9941 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:49.900053 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:49.900974 10674 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:49.764601 17257 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.903723 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.908723 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.822857 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:49.918520 10091 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:49.920264 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:49.825938 17578 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:49.923383 10023 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:49.920706  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:49.873283 17676 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:49.929984 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.932049 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:49.932576 17700 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:49.933193 17700 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:49.933599 17374 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:49.933692 17700 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.927012 17619 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:49.938410 17619 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:49.939174 17619 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:49.906986 17442 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:26:49.943320 17697 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:49.943715 17697 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:49.944519 17697 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.945752 17697 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:49.886687 17372 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:26:49.959708 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 172857us
I20260709 12:26:49.825629 17603 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:49.949575  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:49.949476  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.888267 17649 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:50.015097 17681 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.014958 17649 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.016830 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.017663 10092 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:50.017802 17649 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.824397 17644 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.018543 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.027837 10092 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:26:50.025048 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1388) took 1240 ms. Trace:
I20260709 12:26:50.011365 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:49.964315 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:49.972900 16661 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:49.954779 17685 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.024s	user 0.054s	sys 0.064s
I20260709 12:26:49.959553 17372 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:50.039376 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 75146.2us
I20260709 12:26:49.988377 17420 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.988646 17452 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.040995 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:49.990578 10673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 1359) took 3341 ms (client timeout 3615 ms). Trace:
I20260709 12:26:49.990717 10670 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:49.991765 17326 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:49.992825 17618 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:49.919785 17641 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:49.937549 17678 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:49.959425 17566 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:49.997994 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:50.048759 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50850.2us
I20260709 12:26:50.048899 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:50.049082  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 391286us
I20260709 12:26:50.049474 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:49.999547 17632 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.006114 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.008529 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:50.054512 10642 leader_election.cc:400] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21.
I20260709 12:26:50.054908 10020 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:50.011335  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.019364  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:49.966841  9948 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:50.036368 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.030579 10019 rpcz_store.cc:276] 0709 12:26:48.784967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.785397 (+   430us) service_pool.cc:224] Handling call
0709 12:26:50.024730 (+1239333us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54001f7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:50.024940 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902235520}
I20260709 12:26:50.023636 10095 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:49.986685 17603 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:50.019654 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.041090 17372 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:49.902371 17316 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:50.043390 10673 rpcz_store.cc:269] 0709 12:26:46.648862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.850726 (+201864us) service_pool.cc:224] Handling call
0709 12:26:49.990317 (+3139591us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b5400723654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:49.990484 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4339804032}
W20260709 12:26:50.038458 17685 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.115s	user 0.055s	sys 0.065s
I20260709 12:26:50.049546 10021 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.003456 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:50.069118 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 65736.1us
I20260709 12:26:50.069468 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.070231 10093 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:50.051283 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.052033 16661 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:50.026387 17644 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:50.068706  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 292019us
I20260709 12:26:50.110040 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.054920 10642 leader_election.cc:403] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:26:50.040252 17534 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:50.062003 10091 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:50.041316 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.063897 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.026482 17729 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:26:50.128739 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:50.060813 10062 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.656s (SENT) [suppressed 7 similar messages]
I20260709 12:26:50.023895 17722 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.049958 17656 raft_consensus.cc:2764] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:26:50.149400 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.109963  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:50.108244  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58524.9us
I20260709 12:26:50.151216  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:50.124874 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 70490.8us
I20260709 12:26:50.152899 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:50.153839 16661 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:50.154637 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:50.125348 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:50.048050 17748 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:50.046507 17641 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.096503 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.061470 17378 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:50.150413 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:50.161628  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2655) took 1035 ms. Trace:
I20260709 12:26:50.160360 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1390) took 1373 ms. Trace:
I20260709 12:26:50.159344 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.063083 17662 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:50.062498 17694 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:50.131323 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:50.158847 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 98080.1us
I20260709 12:26:50.151541  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:50.181041 10672 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:50.174000  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:50.181324 10017 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:50.173452 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1385) took 2804 ms. Trace:
I20260709 12:26:50.174145  9940 rpcz_store.cc:276] 0709 12:26:49.126147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.126351 (+   204us) service_pool.cc:224] Handling call
0709 12:26:50.161459 (+1035108us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b54001a36d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:50.161566 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103297920}
I20260709 12:26:50.186749 10022 rpcz_store.cc:276] 0709 12:26:47.369261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:47.393242 (+ 23981us) service_pool.cc:224] Handling call
0709 12:26:47.393583 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.051338 (+1657755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.173180 (+1121842us) spinlock_profiling.cc:243] Waited 1.64 s on lock 0x7b5400137ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:50.173394 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3761033088}
I20260709 12:26:50.174840 10093 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:50.175232 10016 rpcz_store.cc:276] 0709 12:26:48.786765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.788545 (+  1780us) service_pool.cc:224] Handling call
0709 12:26:50.160149 (+1371604us) spinlock_profiling.cc:243] Waited 42.6 ms on lock 0x7b54001f7a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:50.160289 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":97875712}
I20260709 12:26:50.188093 10093 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:50.188925 10093 raft_consensus.cc:606] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:26:50.189185 10017 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:50.173273 17641 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.190311 17374 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:50.190754 17374 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:50.187390  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.191150 17374 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.192148  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.192345 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:50.193001 10091 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:50.193245  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.194249  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.189292 10093 raft_consensus.cc:614] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:50.061337 17025 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.107s
I20260709 12:26:50.195170  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.196219  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:50.196743  9943 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.197299  9940 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:26:50.199400 17751 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.632s	user 0.006s	sys 0.000s
I20260709 12:26:50.200335 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.201215 10018 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:50.123260 17316 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.212744 17316 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.216238 17378 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.182181 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.224258 17713 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.225653 10099 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:50.214188 17641 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.227388 17650 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:50.228387 17722 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:50.228168 17650 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.230405 17641 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.184615 17662 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.129920 17685 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.236063  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2666) took 1044 ms. Trace:
W20260709 12:26:50.236323  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62406.8us
I20260709 12:26:50.236943 17678 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:50.237349 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2820) took 1066 ms. Trace:
I20260709 12:26:50.237970 17618 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.238651 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.243772 10098 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:50.128422 17425 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:26:50.235316  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83843.5us
I20260709 12:26:50.257215 10020 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:50.172981 17420 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:50.253904  9949 rpcz_store.cc:276] 0709 12:26:49.191535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.217042 (+ 25507us) service_pool.cc:224] Handling call
0709 12:26:50.235551 (+1018509us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b540009c4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:50.235892 (+   341us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939449728}
I20260709 12:26:50.266413  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:50.259479 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73164.5us
I20260709 12:26:50.273432 17629 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 16 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:50.264917 17257 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.275207 17425 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:50.276268 17425 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.282725 17597 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.282552 17578 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.283257 17597 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.283774  9944 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:50.284917 17578 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.285279 17597 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:50.261183 10094 rpcz_store.cc:276] 0709 12:26:49.170704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.170846 (+   142us) service_pool.cc:224] Handling call
0709 12:26:49.179971 (+  9125us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.247278 (+ 67307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.247327 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:49.247398 (+    71us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:49.247445 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.236969 (+989524us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b5400093154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:50.237129 (+   160us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.237290 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154160384}
I20260709 12:26:50.291532 10674 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:50.252912 16661 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:50.282357 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:50.300776  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:50.303783 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:50.302193 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:50.308344 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:50.311687 16661 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:50.254103 17662 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.317181 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.316814 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.315840 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.324520 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:50.349040 17566 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:50.267411 17713 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:50.354552 10677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 1370) took 1657 ms (client timeout 1656 ms). Trace:
I20260709 12:26:50.269455 17420 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.340962  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:50.351387 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.332691  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:50.349869 17682 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:50.286970 17678 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:50.322527  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:50.354511 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.359230 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:50.304922 17150 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:50.363895 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:50.371667 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:50.372274 10017 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:50.356037 10677 rpcz_store.cc:269] 0709 12:26:48.697528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.778543 (+ 81015us) service_pool.cc:224] Handling call
0709 12:26:50.354333 (+1575790us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b540058e654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:50.354469 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116751232}
I20260709 12:26:50.313102 17650 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:50.366915  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44484.4us
I20260709 12:26:50.318022 17654 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:50.376963  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:50.263278 17685 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.336033 17748 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:50.312501 17538 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:26:50.378069  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45484.4us
I20260709 12:26:50.335845 17648 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:50.382965 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:50.389461 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:50.390355  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.300449 17667 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21.
I20260709 12:26:50.366426 17672 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.366773 17316 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.366647 17566 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:50.368139 17420 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.394649 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:50.381558 17418 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:50.390748 17465 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:50.476990  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1646) took 1125 ms. Trace:
W20260709 12:26:50.477244  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100360us
I20260709 12:26:50.397433 17713 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:26:50.468325 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73812.4us
I20260709 12:26:50.467388 10017 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:50.474206  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.478024  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:50.482532  9942 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:50.379376 17656 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 12:26:50.477419  9945 rpcz_store.cc:276] 0709 12:26:49.352092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.390117 (+ 38025us) service_pool.cc:224] Handling call
0709 12:26:50.476807 (+1086690us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b540042de54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:50.476929 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":466139520}
W20260709 12:26:50.425010  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 475490us
I20260709 12:26:50.480792 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.483424  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:50.483564  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:50.483999  9945 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.396229 17729 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:50.543231  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59751.5us
I20260709 12:26:50.556073  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:50.397993 17682 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:50.476893 17650 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:50.551052 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:50.476704 17465 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:50.479938 17713 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:50.559213 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:50.560267 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:50.565949  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:50.576279  9946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.029s	sys 0.092s
I20260709 12:26:50.477185 17674 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:50.481767 17680 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:50.481580 17538 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:50.597004  9945 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:50.414911 17566 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:50.579486 10094 raft_consensus.cc:2410] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 21 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
W20260709 12:26:50.580260 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 425698us
I20260709 12:26:50.607192 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:50.608340 16661 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:50.608891 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:50.400514 17150 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:50.585289 10092 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
W20260709 12:26:50.585876  9946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.029s	sys 0.092s
I20260709 12:26:50.614310 17566 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.619834  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2922) took 1168 ms. Trace:
I20260709 12:26:50.620204  9946 rpcz_store.cc:276] 0709 12:26:49.451015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.743789 (+292774us) service_pool.cc:224] Handling call
0709 12:26:49.780740 (+ 36951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.780933 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:26:50.069269 (+288336us) log.cc:844] Serialized 14 byte log entry
0709 12:26:50.619391 (+550122us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:50.619471 (+    80us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:50.619536 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.619633 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.619779 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":438626,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":69,"thread_start_us":793573,"threads_started":2}
I20260709 12:26:50.425580 17316 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.551927  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.603818  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:50.626967  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:50.605242 17676 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.616s	user 0.010s	sys 0.002s
I20260709 12:26:50.612314 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:50.551927 17378 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:50.580616 17657 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:50.584309 17465 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:50.589600 17713 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:50.607249 17656 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:50.400476 17648 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.628476  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.575215 17682 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.629292  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:50.631815 17316 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:50.634974  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:50.657172  9946 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:50.431761 17619 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.640569 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:50.624696 16661 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:50.617110 17538 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:50.582804 17650 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.640568 17657 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.599154 17674 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.678529  9943 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:50.693449 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:50.575716 17678 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.626116 17629 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:26:50.713282  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:50.714155 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73699.6us
I20260709 12:26:50.654685 17378 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:50.648617 17656 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:50.888164 17656 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:50.651958 17718 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:50.890889 17718 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.696897 17657 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.892045 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:50.720956 10021 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.652s	user 0.004s	sys 0.005s
I20260709 12:26:50.897815 10021 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:50.898437 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3768) took 1069 ms. Trace:
I20260709 12:26:50.898819 10021 rpcz_store.cc:276] 0709 12:26:49.828881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.040028 (+211147us) service_pool.cc:224] Handling call
0709 12:26:50.898361 (+858333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:50.841557 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:50.661473 17763 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 12:26:50.901193  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2674) took 1430 ms. Trace:
I20260709 12:26:50.901607  9948 rpcz_store.cc:276] 0709 12:26:49.470424 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.890220 (+419796us) service_pool.cc:224] Handling call
0709 12:26:49.890708 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.057200 (+166492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.900944 (+843744us) spinlock_profiling.cc:243] Waited 75.9 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:50.901135 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174490368}
I20260709 12:26:50.646443 17722 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:50.907526  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1647) took 1541 ms. Trace:
I20260709 12:26:50.641566 17676 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.862515 17150 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.864463  9949 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:50.867002 17629 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:50.715566 17257 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:26:50.869634 17768 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.222s	user 0.000s	sys 0.000s
I20260709 12:26:50.870384 17650 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:50.903950 17680 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:50.875771 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:50.877496 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:50.879520 17771 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.212s	user 0.000s	sys 0.000s
W20260709 12:26:50.880226 17372 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.757s	user 0.009s	sys 0.001s
I20260709 12:26:50.881726 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:50.962023 10677 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:50.882913 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:50.883499 17654 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:26:50.887353 17374 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.696s	user 0.006s	sys 0.005s
I20260709 12:26:50.887467 17772 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:50.896880 17672 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:50.700222 10674 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:50.652190 17025 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:50.971836  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 493847us
I20260709 12:26:50.660365 17682 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:50.697104 17578 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:50.908250  9947 rpcz_store.cc:276] 0709 12:26:49.366161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.646773 (+280612us) service_pool.cc:224] Handling call
0709 12:26:49.861479 (+214706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.863361 (+  1882us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.863396 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:49.863471 (+    75us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:49.863517 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:49.863593 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.907299 (+1043706us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540021dc50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:50.907454 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492952320}
I20260709 12:26:50.928211 17729 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:50.903548 17767 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.281s	user 0.000s	sys 0.000s
I20260709 12:26:50.868288 17538 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.874816  9943 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
W20260709 12:26:50.938977 10022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.046s	sys 0.072s
W20260709 12:26:50.993160 10022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.047s	sys 0.072s
I20260709 12:26:50.893069 17718 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:50.958909 10671 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:50.929462 17442 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.029s	sys 0.008s
I20260709 12:26:50.908582 17774 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:50.991587  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:50.997583 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3756) took 1864 ms. Trace:
I20260709 12:26:50.967262 17372 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.916324  9946 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:50.973519 10096 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:50.932622 17653 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:26:50.919432 17722 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.969736 17374 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:50.990911  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.035s	sys 0.081s
W20260709 12:26:51.015380  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.035s	sys 0.082s
I20260709 12:26:50.913414 17566 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:51.000903  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.002127  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:50.918470 17685 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:50.929368 17629 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:51.021988  9945 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
W20260709 12:26:50.996358 17442 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.030s	sys 0.009s
I20260709 12:26:50.997979 10025 rpcz_store.cc:276] 0709 12:26:49.133064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.767034 (+633970us) service_pool.cc:224] Handling call
0709 12:26:49.782250 (+ 15216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:49.782942 (+   692us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:49.783006 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:49.806569 (+ 23563us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.314408 (+507839us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:26:50.314470 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.314555 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:50.997363 (+682808us) raft_consensus.cc:1685] finished
0709 12:26:50.997383 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:50.997527 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12460,"raft.queue_time_us":13002,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":6102,"thread_start_us":392428,"threads_started":1,"wal-append.queue_time_us":697295}
W20260709 12:26:51.001173 17700 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.065s	user 0.014s	sys 0.000s
I20260709 12:26:51.026947 17700 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.028051 17372 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.029548 17700 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:50.971264 17772 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.029990 17374 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.031005 17674 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.031716 17772 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.031818 17566 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.034076 17722 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:51.035035 17770 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:26:50.998446 17774 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:50.977635 17025 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.987429 17465 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:50.996477 17578 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:50.996354 17729 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.059986 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3754) took 1933 ms. Trace:
I20260709 12:26:51.064052 10024 rpcz_store.cc:276] 0709 12:26:49.127030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.648131 (+521101us) service_pool.cc:224] Handling call
0709 12:26:49.648410 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:49.853429 (+205019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.059687 (+1206258us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54001c6954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:51.059908 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":456225664}
I20260709 12:26:51.040107 17685 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:51.070323  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:51.070451 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:51.108175 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:51.041785 17774 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:51.073309  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.043393 17653 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:26:51.064570 17418 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.010s	sys 0.032s
I20260709 12:26:51.106230 10018 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:51.114110 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:51.121919 17632 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.123303 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:26:51.063390 17619 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:51.123312  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:51.116559  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46330.4us
I20260709 12:26:51.074503 17378 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:51.117419 17770 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:26:51.102041 17685 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.115736 17718 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:51.117996 17653 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:51.119871 17418 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.010s	sys 0.032s
I20260709 12:26:51.132444 17531 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:51.136092  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.125165 17566 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:51.119648 17672 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:26:51.144083 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 252125us
I20260709 12:26:51.133441 17326 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:51.148029 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.146077  9949 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:51.128575  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:51.138717 17685 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:51.145900 17531 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:51.160682 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3991) took 1611 ms. Trace:
W20260709 12:26:51.165056  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.153661  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:51.145682 17603 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.038s	user 0.006s	sys 0.006s
I20260709 12:26:51.144033 17566 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:51.171118 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.172230  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:51.169039 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.173357  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:51.174111  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 172059us
I20260709 12:26:51.161108 10093 rpcz_store.cc:276] 0709 12:26:49.549486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.196221 (+646735us) service_pool.cc:224] Handling call
0709 12:26:50.241546 (+ 45325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.241761 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:50.242047 (+   286us) log.cc:844] Serialized 14 byte log entry
0709 12:26:50.699757 (+457710us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:50.699835 (+    78us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:50.699899 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.699971 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.160431 (+460460us) spinlock_profiling.cc:243] Waited 45 ms on lock 0x7b5400164250. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:51.160590 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3788,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":7077,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":68146,"spinlock_wait_cycles":111859968,"thread_start_us":456526,"threads_started":1,"wal-append.queue_time_us":730817}
I20260709 12:26:51.146567 17653 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:51.169773 17739 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.981s	user 0.000s	sys 0.023s
I20260709 12:26:51.177024 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.196309 10668 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:51.138005 17775 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:26:51.175249 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1557) took 2689 ms. Trace:
I20260709 12:26:51.146248 17418 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:51.175665 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 334215us
I20260709 12:26:51.188077 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:51.190339  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:51.153076 17782 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:51.170058 17648 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.154699 17326 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:51.202333 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:51.159221 17785 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:51.175710 17566 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.171026 17575 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.205065 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:51.205339 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:51.206975  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 69764.7us
I20260709 12:26:51.209132 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:51.157107 17531 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.195313 17674 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:51.194868 17372 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:26:51.137741 17629 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.203286 17641 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:51.212136 17739 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.000s	sys 0.024s
I20260709 12:26:51.199810 10669 rpcz_store.cc:276] 0709 12:26:48.485695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:48.795386 (+309691us) service_pool.cc:224] Handling call
0709 12:26:48.795728 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:48.927925 (+132197us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:26:49.030489 (+102564us) log.cc:844] Serialized 14 byte log entry
0709 12:26:49.224706 (+194217us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:49.224772 (+    66us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:49.224807 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:50.147821 (+923014us) spinlock_profiling.cc:243] Waited 4.6 ms on lock 0x7b18000078a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419
0709 12:26:51.175021 (+1027200us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:51.175080 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.175195 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":152763,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":95,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":3331,"spinlock_wait_cycles":313954304,"thread_start_us":233985,"threads_started":2,"wal-append.queue_time_us":283397,"wal-append.run_cpu_time_us":2324,"wal-append.run_wall_time_us":1500869}
I20260709 12:26:51.205559 17782 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:51.240020 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3767) took 1505 ms (client timeout 1804 ms). Trace:
W20260709 12:26:51.240592 10023 rpcz_store.cc:269] 0709 12:26:49.735055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:49.764956 (+ 29901us) service_pool.cc:224] Handling call
0709 12:26:51.234062 (+1469106us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540038dbd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:51.239937 (+  5875us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328109312}
I20260709 12:26:51.176285 17603 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.237942 10671 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:51.209914 17785 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:51.246536 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:51.251411 10098 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:51.252576 10099 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [20.14-24.15]
I20260709 12:26:51.261674 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.211410 17791 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:51.262403 10677 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:51.265118 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.266098 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.266847 10677 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:51.268800 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.269017 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:51.271275 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.264264 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:51.271994 10099 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:51.263073  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:51.273581 10097 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:51.274935 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:51.277019 10669 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:51.270982 10023 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:51.273264 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:51.280467 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.279747 10091 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:51.196738 17378 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.245146 17782 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:51.259181 17372 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:51.259845 17465 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.262842 17667 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:51.285808 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:51.293111 10025 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:51.289198  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.267869 17641 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:51.263366 17025 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.285823 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.263661 17785 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:51.266505 17788 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:51.270717 17531 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:51.296129 10677 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:51.273010 17757 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:51.309633 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.285557 17729 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:51.288861 17780 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.155s	user 0.000s	sys 0.001s
I20260709 12:26:51.289513 17629 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:51.285017 17425 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:51.282723 17648 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.019s
I20260709 12:26:51.305313  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.306347  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.308035  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.273262 17578 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.309150  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.262970 17656 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.335475 17656 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.303625  9949 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:51.338261 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.339540 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.340222 17603 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.340227 10676 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:51.341813 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.319145 10677 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:51.321962  9946 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:51.342901  9948 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:51.322204  9947 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:51.300562 17452 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:51.299515 17788 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.291131 17418 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.296841 17372 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:51.328204 17420 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.844s	user 0.007s	sys 0.003s
W20260709 12:26:51.329885 17776 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 12:26:51.331400 17657 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.372247  9942 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.333055 17425 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:51.319748 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:51.362495  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 172274us
I20260709 12:26:51.363874  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.300549 17790 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.366513 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:51.314437 17777 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.319s	user 0.001s	sys 0.000s
I20260709 12:26:51.318831 17371 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:51.375783  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:51.379057  9941 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:51.373384 10669 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:26:51.411724  9942 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:51.300551 17326 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.368221 17788 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:51.383229 10676 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:26:51.386336  9946 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
W20260709 12:26:51.388123 17694 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.168s	user 0.013s	sys 0.001s
I20260709 12:26:51.388425 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.373574 17452 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:26:51.379516 17662 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.007s	user 0.007s	sys 0.005s
I20260709 12:26:51.393906 17425 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.398559 17657 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.402907 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2555) took 1575 ms. Trace:
I20260709 12:26:51.436695 10022 rpcz_store.cc:276] 0709 12:26:49.827346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.187713 (+360367us) service_pool.cc:224] Handling call
0709 12:26:50.188013 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:50.236192 (+ 48179us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:26:50.236556 (+   364us) log.cc:844] Serialized 14 byte log entry
0709 12:26:50.996437 (+759881us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:50.999695 (+  3258us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.999863 (+   168us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:51.001122 (+  1259us) log.cc:844] Serialized 14 byte log entry
0709 12:26:51.170074 (+168952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.170133 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:51.402669 (+232536us) raft_consensus.cc:1685] finished
0709 12:26:51.402692 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.402840 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47708,"prepare.queue_time_us":1981,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":79,"raft.queue_time_us":314941,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":3076,"spinlock_wait_cycles":10242048,"thread_start_us":925273,"threads_started":2,"wal-append.queue_time_us":892811}
I20260709 12:26:51.373457 17420 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.440533 17420 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.388976 17371 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.447472 17372 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.448992 17685 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.450023 17420 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.450840 17452 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.451233 17662 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.405543 17785 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:51.452661 17713 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.795s	user 0.007s	sys 0.004s
I20260709 12:26:51.443845 17694 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:51.454792  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1661) took 1506 ms. Trace:
W20260709 12:26:51.457727 10124 heartbeater.cc:646] Failed to heartbeat to 127.8.137.254:38721 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.8.137.254:38721 timed out after 15.000s (SENT)
I20260709 12:26:51.462217 10668 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:51.430058 17257 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:51.472788  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.410570 17566 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.474431 17025 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.008s
I20260709 12:26:51.484140 17566 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.475858 10065 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:51.478135 10024 raft_consensus.cc:606] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:51.479461  9947 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:51.478859  9940 rpcz_store.cc:276] 0709 12:26:49.947934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.199119 (+251185us) service_pool.cc:224] Handling call
0709 12:26:50.199444 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:50.284223 (+ 84779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:50.284252 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:50.284788 (+   536us) log.cc:844] Serialized 26 byte log entry
0709 12:26:50.606400 (+321612us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:50.606439 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.454362 (+847923us) spinlock_profiling.cc:243] Waited 84.4 ms on lock 0x7b540009b5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:51.454416 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:51.454465 (+    49us) raft_consensus.cc:1685] finished
0709 12:26:51.454479 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:51.454726 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1381,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":194157312,"thread_start_us":321242,"threads_started":1,"wal-append.queue_time_us":672442}
I20260709 12:26:51.480197  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:51.482569 10675 raft_consensus.cc:2368] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19.
I20260709 12:26:51.476702  9946 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:51.487600  9912 leader_election.cc:400] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:26:51.487941  9912 leader_election.cc:403] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:51.489012  9949 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:51.490185 17657 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:26:51.491760 17657 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:26:51.491991 17644 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:51.492770 17644 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:51.493685 17644 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.494971 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:51.488363 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:51.459277 17782 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.502031  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:51.511956 17371 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.512475 17787 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:51.514571 17662 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.519392 17805 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:51.520099 17785 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.481746 17713 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.481282 17257 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:51.520787  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.521790 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1551) took 4688 ms. Trace:
I20260709 12:26:51.538945  9943 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:51.517931 17788 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.475004 17538 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:26:51.473456 17420 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.489391 17566 raft_consensus.cc:2764] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:26:51.534614  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.534567 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:51.504894 17025 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:51.569399 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.570218 10668 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.574206 10675 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:51.579814 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:51.581171 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.581753 10098 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:51.531916  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:51.538793 10670 rpcz_store.cc:276] 0709 12:26:46.833709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:46.957947 (+124238us) service_pool.cc:224] Handling call
0709 12:26:49.990470 (+3032523us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:50.045291 (+ 54821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.521554 (+1476263us) spinlock_profiling.cc:243] Waited 2.99 s on lock 0x7b5400723650. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:51.521701 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6871281024}
I20260709 12:26:51.546105 17782 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.551554 17685 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.536602 17774 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.552529 10094 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:51.553007 17672 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:51.553495 17682 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.557s	user 0.007s	sys 0.004s
I20260709 12:26:51.523033 17694 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.536180 17790 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:26:51.555801 17713 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.532580 17656 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:51.536394 17805 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.538409 17775 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:26:51.538028 17729 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:26:51.557528 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:51.558329 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.536872 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:51.561473 10095 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:51.563946  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.643726 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:51.585878  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54053.8us
I20260709 12:26:51.644546 10099 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:51.645022  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:51.558007 17257 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:51.607439 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:51.608556 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.562033 17644 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:51.611999 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.561419 17779 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.611120 17785 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:26:51.621776 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:51.656098 10094 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:51.611436 17672 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.539464 17787 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:51.623560  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2932) took 1151 ms. Trace:
I20260709 12:26:51.624470 17682 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.577740 17025 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.625687 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.672914 10676 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:51.626694 10668 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:51.627413 10671 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:51.646183 17656 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:51.638968 10677 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:51.640196 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:51.642467  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.650511  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:51.659662  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.686700 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:51.565593 17654 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:26:51.660313 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 123506us
I20260709 12:26:51.684715  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:51.663017  9945 rpcz_store.cc:276] 0709 12:26:50.471816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.649884 (+178068us) service_pool.cc:224] Handling call
0709 12:26:50.689153 (+ 39269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.022721 (+333568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.623291 (+600570us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540003eb54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:51.623492 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764841216}
I20260709 12:26:51.663892 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.672950 10097 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:51.636536 10098 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:51.680157 10091 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:51.659813 17650 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.688127 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:51.688730  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.694494  9912 leader_election.cc:400] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19.
I20260709 12:26:51.671821 17632 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:51.700276 17632 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:51.701108  9912 leader_election.cc:403] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 12:26:51.702150  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.674736 17025 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:51.716306 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.717749 17700 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:51.717628 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.718128 17700 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:51.659703 17653 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.722093 17672 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:51.704277 17372 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:51.723254 17372 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:51.723665 17372 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:51.710389  9941 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:51.701252 17393 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:26:51.701376 17538 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:51.712021 17420 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.712790 17656 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.749249 10099 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
W20260709 12:26:51.714905 17766 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.317s	user 0.002s	sys 0.007s
I20260709 12:26:51.715003 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.667948 17779 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.703490 17722 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:26:51.715596 17650 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:51.723263 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1408) took 1402 ms. Trace:
I20260709 12:26:51.726877 10674 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
W20260709 12:26:51.739112  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44768.4us
I20260709 12:26:51.739702  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.722184 17653 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:51.692953 17785 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:51.690337 17654 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.689930 17729 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.750262 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:51.758826 10091 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:51.760396  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:51.773970 10099 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:51.758877 10018 rpcz_store.cc:276] 0709 12:26:50.320662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.378783 (+ 58121us) service_pool.cc:224] Handling call
0709 12:26:50.398816 (+ 20033us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.125960 (+727144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.723046 (+597086us) spinlock_profiling.cc:243] Waited 704 ms on lock 0x7b540038ccd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:51.723205 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1619294592}
W20260709 12:26:51.775813 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 113829us
I20260709 12:26:51.776283  9949 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:51.778890  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.779954  9943 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:51.744832 17804 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:51.783828  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.784858  9949 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:51.785830  9943 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:51.754274 17672 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:51.772194 17534 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:51.772226 17785 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:51.777382 17531 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:51.777232 17654 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:51.776885 17802 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:26:51.783509 17641 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:51.751610 17538 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.795583 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:51.793306 17534 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:51.795706 17785 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:51.783391 17729 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.809242 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.816263 10675 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:51.809336 17629 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:51.818147 17815 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 12:26:51.822114 17814 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:26:51.827102 17534 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.828495 17534 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:51.830161 17779 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.801231 17788 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.839026 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.787369 17789 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:26:51.841118 10669 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:51.846285 10018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8453bea911c4756add721edebd5d124"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:26:51.843664 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.794523 17672 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.785012 17700 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.799110 17531 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.846904 10018 raft_consensus.cc:606] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:51.787370 17787 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:51.807693 17641 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:51.851662 10098 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:51.853302 17676 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.790840 17632 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.907232  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.908107  9949 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:51.910274 17700 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:51.914740 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.917048 17757 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:51.917457 17757 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:51.918756 17650 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:51.919761 17650 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:51.920117 17650 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:51.920928 17650 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.921542 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.922094 17650 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:51.922304 10668 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:51.921680 17641 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.923269  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.928058 17578 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:51.928481 17578 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:51.929407 17578 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.928431  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.861578 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:51.832231 17774 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.949054  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.940954 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 3997) took 2380 ms. Trace:
I20260709 12:26:51.862500 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:51.951028 10090 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:51.873245 17531 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:51.876920 17025 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.890504 10675 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:51.886296 17805 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:51.895913 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:52.100447 10092 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:52.100838 10676 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:51.897042 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:52.101516 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:51.898850 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:51.932116 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1377) took 1451 ms. Trace:
I20260709 12:26:52.102807 10096 rpcz_store.cc:276] 0709 12:26:50.480395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.480545 (+   150us) service_pool.cc:224] Handling call
0709 12:26:51.931960 (+1451415us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b540022ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:51.932065 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1127400704}
I20260709 12:26:51.933336  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.104732  9942 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:52.105834  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.948108 10671 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:51.934988  9944 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:51.947896 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:51.950477 10095 rpcz_store.cc:276] 0709 12:26:49.560766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.240599 (+679833us) service_pool.cc:224] Handling call
0709 12:26:50.240896 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.648506 (+1407610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:51.940721 (+292215us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b540002dd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:51.940894 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3036645120}
I20260709 12:26:51.935663  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:52.108158  9942 raft_consensus.cc:2368] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21.
I20260709 12:26:52.108974  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.949393  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:51.894562 17538 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:51.984182  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:51.960995 17682 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.102053 17618 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:51.936187 17672 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:52.126726 17805 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.127246 17805 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.918293 17757 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.930576 17578 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.132943 17452 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:52.133553 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.133466 17452 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.134229 10025 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:52.134853 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.135169 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.135442 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.136104 10017 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:52.136174 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.137037 17757 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.137997  9948 raft_consensus.cc:2368] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20.
I20260709 12:26:52.138413 17452 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:51.931433 17713 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:52.139554 17713 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:52.140532 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.141134 10017 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.141353 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.142064 10091 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:52.142817 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.143293 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.143023 17713 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.144379 17713 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.083570  9946 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:26:52.089185  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.150887 17025 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.151101 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:52.151705  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.151937 10019 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:26:52.154215  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:51.949765 17393 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:52.155217 17393 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.155679 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.156674 17654 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:51.941102 17566 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:51.953557  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.198072 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.198856 10098 raft_consensus.cc:2368] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19.
I20260709 12:26:52.109231 10090 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:26:52.115918 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.200476 10096 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:52.117551 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1405) took 1800 ms. Trace:
I20260709 12:26:52.200925 10016 rpcz_store.cc:276] 0709 12:26:50.316731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:50.316962 (+   231us) service_pool.cc:224] Handling call
0709 12:26:52.117379 (+1800417us) spinlock_profiling.cc:243] Waited 881 ms on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:52.117497 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2025203072}
I20260709 12:26:52.119007 17662 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:52.119557 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.120286 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.101159 17774 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:26:52.101483 10670 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:52.124505  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:52.124858 10668 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:52.125888 17618 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.133734 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.209564 10022 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:52.209672 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.134929 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.150086 10017 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:52.211625 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.212000 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.212669 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.213689 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.157292 10021 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:51.938760 17534 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:26:51.931023 17792 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:26:52.093379 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 129956us
I20260709 12:26:52.216770 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:51.954124 17775 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:52.160033 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.218546 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:52.218755  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 94338.6us
I20260709 12:26:52.160398 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:52.161605 16661 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.219811  9949 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:52.219942 16661 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.220290 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.143365 10091 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:52.162182 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:52.222090 10024 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:52.165247  9943 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:26:52.178942 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.179804 17804 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19.
I20260709 12:26:52.182961 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:52.158437 17393 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.182435 10019 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:26:52.227030 10676 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:26:52.187857 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:52.174317 17718 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.153425 17257 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.196518  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:52.213884 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:52.225380 10025 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:52.217873 17775 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.230960 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.257498 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.231964 17618 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:52.232496 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.260416 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.195923 17700 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:26:52.233196 16661 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:52.215270 17792 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:52.237025 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.236511  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:52.237322 10024 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
W20260709 12:26:52.245963 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83853.4us
I20260709 12:26:52.263870 10643 leader_election.cc:400] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21.
I20260709 12:26:52.264557 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.264819 10643 leader_election.cc:400] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20.
I20260709 12:26:52.235982 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.265149 10025 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.265697 10019 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:52.248409 10091 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:52.175732 17805 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.245952  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:52.266685  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 323423us
I20260709 12:26:52.255065 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.256659 10022 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:52.258297 10016 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:52.207722 17531 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:52.258872  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62442us
I20260709 12:26:52.257949 10095 raft_consensus.cc:2376] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 20. Re-sending same reply.
I20260709 12:26:52.266769  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.273751  9949 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:26:52.265199 10643 leader_election.cc:403] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:26:52.267933 10023 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:52.261719 17629 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:52.279821 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.281047 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.282469  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.283179 10021 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:52.283756 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:52.285292 10016 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:52.270483 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.251710 10090 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 16.
I20260709 12:26:52.275159 10019 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:52.275331  9914 leader_election.cc:400] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19.
I20260709 12:26:52.288136  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:52.246047 17619 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:52.257184 17653 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:52.265844 17804 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:52.211155 17566 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.278430 10023 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:52.174952 17372 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.283653  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.289311  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:52.277123  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:52.293341  9940 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:52.313320  9944 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:52.314497  9940 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:52.235585 17708 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:52.298728 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.302301 10093 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:52.275309 17618 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:26:52.296885 17619 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:52.304903  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.305907 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:52.309765  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.311336 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:52.335376 10099 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 16.
I20260709 12:26:52.336185 10099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:52.336721 10099 raft_consensus.cc:606] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:52.301659  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:52.258742 17718 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.321312  9946 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:52.321892 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.336345  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:52.323235  9947 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:26:52.249445 17418 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:26:52.326320 17657 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:52.327486 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:52.329317  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52300.3us
I20260709 12:26:52.330319 10098 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:26:52.353597  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:52.354455 10098 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:52.332096  9945 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:52.337057 10099 raft_consensus.cc:614] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:52.300048 17757 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.376372 17610 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:52.377816 17610 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:52.377992  9948 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:52.379665  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:52.381610  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:52.382421 17676 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:52.383455 10091 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:26:52.383714 17676 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:52.384142 17676 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:52.384553 17676 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.320798 10096 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:52.346693 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.385076 10091 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
W20260709 12:26:52.385504 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.376397  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:52.386997  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 85452.2us
I20260709 12:26:52.347288  9941 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:52.357311 17461 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:26:52.388041  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:52.389127  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:52.389371 17461 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:52.389650  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:52.390765 17420 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:52.391098  9947 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:52.391295 17578 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:52.391454 17420 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:52.391711 17578 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:52.392225 17578 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.392795 10093 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:52.393342  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:52.308362 17653 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:52.394783 17418 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:52.395462 17653 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.320598 17708 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.396605 17708 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.400352 17578 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.319379 17729 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:52.361933 10095 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
W20260709 12:26:52.363682 17785 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.532s	user 0.008s	sys 0.001s
I20260709 12:26:52.365103 17372 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.324872 17629 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:52.402630 17785 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.403414 17629 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.404937 17785 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.407029 17578 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:52.410224  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57255.1us
I20260709 12:26:52.353650 17617 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:52.299821 17782 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:26:52.300601 17316 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:26:52.352885 17393 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:52.406932 17654 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:52.375701 17231 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:52.390357  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:52.391004  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:52.402138  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:52.390158 17461 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:52.366976 17718 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.297825 17805 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:52.420899 10096 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:26:52.375581 17775 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:52.378327 17610 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:52.419013 17617 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.410197 17676 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.424026  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:26:52.425521  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.426800  9948 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:52.427001  9946 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:52.473584  9941 raft_consensus.cc:2410] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 21 because replica has last-logged OpId of term: 19 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:26:52.427523 17793 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:52.419791 17761 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:52.430963 10669 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.039s	sys 0.076s
I20260709 12:26:52.383581 17257 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.419626 17807 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:52.449230  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58933.1us
I20260709 12:26:52.419337 17799 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.213s
I20260709 12:26:52.436510 17420 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:52.451984 10098 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
W20260709 12:26:52.474464 10097 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.013s	sys 0.019s
W20260709 12:26:52.476070 10669 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.039s	sys 0.076s
I20260709 12:26:52.457486 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:52.494658 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:52.495268 10670 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.450194 17657 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:52.481920 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:52.496956 10671 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:52.472813  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:26:52.431075 17231 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:52.488376  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:52.385118 17682 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:52.470980 17617 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:52.499676 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:52.493669 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:52.505956 10097 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.013s	sys 0.019s
I20260709 12:26:52.437198 17760 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:26:52.513217 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.483592 17775 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.515604 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.518740 10096 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.519026 10093 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:52.475415 17793 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:52.516268  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:52.524020 16661 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:52.459003 17804 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:26:52.499337 17656 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:26:52.518170 17657 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:52.540365 17779 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:52.540822 17657 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.495424 17739 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:26:52.488505 17452 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:26:52.507045 17761 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:26:52.546710 17761 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:52.547086 17761 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:52.489301 17785 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:26:52.548352 17785 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:26:52.548714 17785 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:52.494998 17025 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 12:26:52.486398 17619 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.492035 17420 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:52.496567 17795 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:52.537571 17566 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.576505 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:52.577180 17829 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:26:52.577503 10090 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:52.521955 17807 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:52.553381 16661 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.584023 16661 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.532137 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:52.520006 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 572211us
W20260709 12:26:52.535151 17828 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:26:52.587692 17722 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:52.588536 17722 consensus_queue.cc:1243] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:26:52.589030 17722 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:52.589367 17722 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:52.505365 17788 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:52.491160 17718 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:26:52.504025 17682 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:52.552315 17775 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.592178 17682 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.593523 17682 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:52.593701 17775 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:52.594340 17775 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:52.530882 17793 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.590253 17722 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.573760 17757 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20.
I20260709 12:26:52.596918 17025 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:52.597909  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:52.605427 17722 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.614068 17231 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.580224  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:52.600033 17708 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.616940 17420 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.569316 17739 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:52.618574 17420 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.618993 17739 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.529881 17662 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:52.621840 17662 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.622001 17739 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.601504 17785 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.605701 17795 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:52.611630 17775 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:52.647469 10025 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:52.611884 17676 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.616030 17231 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:52.622397 17662 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:52.618320 17708 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:52.663414  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 83323.2us
I20260709 12:26:52.598155 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:52.633745  9946 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:26:52.532040 10671 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:26:52.662250 10093 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:52.668916 10096 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:52.669917 16661 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:52.669595  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:52.550932 17425 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.101s
I20260709 12:26:52.651038 17757 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.652323 17619 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.677260  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:52.678745 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80663.5us
I20260709 12:26:52.671043 17641 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:52.684051 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:52.675769 17785 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:52.693496 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:52.682413 17656 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:52.685276 17739 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:52.688630 17644 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:26:52.729064 10062 leader_election.cc:336] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.522s (SENT)
I20260709 12:26:52.730904 10096 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:52.690243 17641 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.736673 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:52.743445 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:52.744167 10090 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.737919 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:52.583020 17610 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.748032 16661 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:52.759651 16661 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:52.762454  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 343662us
I20260709 12:26:52.767256 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:52.768365 10095 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:52.762313 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:52.775655 10090 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:52.776964 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:26:52.777637 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.781046  9949 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:26:52.783551  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:52.786830  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:52.789290 10092 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:26:52.732991 17694 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:52.734279 17316 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:52.688863 17757 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.760854 17641 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:52.814049 16661 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:52.815477 16661 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:52.816094 16661 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:52.697485 17838 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.109s
I20260709 12:26:52.760139 17656 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:52.825984 16661 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.774277 17644 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:52.734213 17700 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.075s
W20260709 12:26:52.837776 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 308730us
W20260709 12:26:52.839844  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.842223  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.833657 10670 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:52.688053 17775 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.844707  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.848397 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:52.852200 10670 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:52.809347 17025 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:52.834136 17629 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:26:52.834038 17713 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.634s	user 0.004s	sys 0.007s
I20260709 12:26:52.860009 10094 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 21 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:26:52.856665  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:52.815112 17694 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:26:52.853214  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 321865us
I20260709 12:26:52.858629 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:52.856760  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 454675us
I20260709 12:26:52.863590  9949 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:52.858275 17788 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:52.864035 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:52.850701 17610 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:52.880746 10670 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:52.864068 17629 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.875200 10675 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:52.864583 17757 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:52.850906 17739 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:52.835482 17617 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:52.891702  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:52.797495 17795 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.884965 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:52.907011  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:52.861605 17425 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:26:52.907913  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:52.866671 17025 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:52.874114  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.908836 17846 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:52.910586 17708 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:52.915869 17799 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:26:52.899672 17629 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.957523 17629 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:52.945873  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:52.951069 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:52.960203 10669 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:52.932024 17847 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:52.900221 17760 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
W20260709 12:26:52.928927 17837 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:26:52.952908 10676 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:52.905180 17739 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.961932 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2850) took 1249 ms. Trace:
I20260709 12:26:52.932705 17708 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:52.959980 17799 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:52.974325 10017 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:52.977583 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.927686 17807 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:26:52.976168 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.903676 17644 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:52.980571 10025 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 17: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:52.980693 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.977332 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:52.979077 10017 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:26:52.983848 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:52.985817 10019 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:52.986873 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:52.996757 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 213264us
I20260709 12:26:52.964917 17531 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:53.001245 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:53.001430 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:52.975663 10097 rpcz_store.cc:276] 0709 12:26:51.712444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:51.736674 (+ 24230us) service_pool.cc:224] Handling call
0709 12:26:51.761056 (+ 24382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:51.779431 (+ 18375us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:26:51.908330 (+128899us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b18002f9440. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:26:52.506801 (+598471us) log.cc:844] Serialized 14 byte log entry
0709 12:26:52.961572 (+454771us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:52.961630 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:52.961700 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:52.961765 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:52.961878 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25731,"prepare.queue_time_us":130120,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":18024,"raft.run_cpu_time_us":227,"raft.run_wall_time_us":9058,"spinlock_wait_cycles":7870720,"thread_start_us":895534,"threads_started":2}
I20260709 12:26:52.963379 17846 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:52.914508 17656 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.005954 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:53.008616 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:53.009402 10021 raft_consensus.cc:2376] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 19. Re-sending same reply.
I20260709 12:26:52.929308 17694 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:53.029920 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.032248 17785 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:53.033156 10096 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:26:52.996904 17840 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.167s	user 0.000s	sys 0.001s
I20260709 12:26:53.034701 17025 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:52.998725 17629 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:53.039155 17629 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.041379 17629 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:53.043440 17708 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.044545 17708 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:52.962464 17788 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.053360 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:53.013074 10675 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:26:53.014998 17841 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:26:53.018568 10023 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:53.019559 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:53.012013 10674 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:53.055335 10676 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:52.972273 17372 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:52.980171 17799 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:53.014341 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:53.027763 17739 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.028439 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:53.082562 10094 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4025) took 1165 ms (client timeout 1522 ms). Trace:
W20260709 12:26:53.100041 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 194462us
W20260709 12:26:53.106240 10094 rpcz_store.cc:269] 0709 12:26:51.917182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.091601 (+174419us) service_pool.cc:224] Handling call
0709 12:26:53.082335 (+990734us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b540034add4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:53.082503 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1511570176}
I20260709 12:26:53.092713 10091 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:26:53.110142 10091 pending_rounds.cc:85] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 15
I20260709 12:26:53.110496 10091 pending_rounds.cc:107] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:26:53.111013 10091 raft_consensus.cc:2889] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:53.084851  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.080739  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1673) took 1125 ms. Trace:
W20260709 12:26:53.112418  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.072028 17851 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:26:53.068800 17857 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:26:53.066898 17844 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
W20260709 12:26:53.069298 17848 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
I20260709 12:26:53.061957 17372 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.112164  9942 rpcz_store.cc:276] 0709 12:26:51.955713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.108915 (+153202us) service_pool.cc:224] Handling call
0709 12:26:52.109241 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.158755 (+ 49514us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:52.484209 (+325454us) log.cc:844] Serialized 14 byte log entry
0709 12:26:52.854402 (+370193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:26:52.854479 (+    77us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:52.854529 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.080502 (+225973us) spinlock_profiling.cc:243] Waited 49.3 ms on lock 0x7b540009b5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:53.080547 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:53.080680 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15814,"prepare.queue_time_us":49171,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":117,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":7440,"spinlock_wait_cycles":113319808,"thread_start_us":659019,"threads_started":2,"wal-append.queue_time_us":417976}
I20260709 12:26:53.068307 17618 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:53.118868 10674 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 15. (term mismatch)
I20260709 12:26:53.177870 10674 pending_rounds.cc:85] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 14
I20260709 12:26:53.178208 10674 pending_rounds.cc:107] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 24.15
I20260709 12:26:53.178519 10674 raft_consensus.cc:2889] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:53.119431 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:53.180194 10021 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:26:53.132272  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.108242 17793 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.505s	user 0.009s	sys 0.002s
I20260709 12:26:53.134181 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:53.134552 10669 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:53.136113 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.140367 10676 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:53.075719 17802 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:53.073091 17617 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:53.114660 17682 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.000s
I20260709 12:26:53.070744 17838 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:53.182515 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46515.5us
I20260709 12:26:53.186010 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:53.187316  9942 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:53.116741 17676 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.195621 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.201913 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:53.217757 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.185268 17722 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:53.184631 17802 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:53.203158 17739 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:53.067518 17846 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.228874 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:53.210831 17617 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:53.193404 17618 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:53.238107  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 259171us
I20260709 12:26:53.221740 17718 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:53.201802 17676 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.244474  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:53.242726  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:53.244388 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:53.197311 17793 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.233215 17682 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.200218 17838 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:53.248386 17619 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.560s	user 0.011s	sys 0.000s
I20260709 12:26:53.260689 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.265717  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:53.242219 17722 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:53.284543 17393 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:53.286664 17393 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:53.261026 17644 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:53.290385 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:53.290455  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:53.292133 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.292742 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:53.293373 10022 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:53.293550 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:53.293820 10019 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:53.294371 10017 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:26:53.294756 17694 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.295261 17694 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.297369 17694 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:53.299367 17372 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:53.300880 17676 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:53.305147 10024 raft_consensus.cc:2368] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20.
I20260709 12:26:53.252722 17420 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:26:53.308461 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.309481 10025 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:26:53.309506 10022 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
W20260709 12:26:53.307472  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50433.9us
I20260709 12:26:53.310890  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:53.314352 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:53.315433 10022 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:53.316450  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:53.311745 17372 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:53.321318 17372 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:53.321731 17372 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:53.322872 17880 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:53.292167 17617 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:26:53.324890 17617 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:53.317368 17656 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:53.325871 17866 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:53.291914 17644 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:53.326462 17866 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:53.326745 17644 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.327392 17866 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.312188 17799 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.329506 10021 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:53.334331  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:53.351454  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:53.355499 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50672.6us
W20260709 12:26:53.341619 17662 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.000s
W20260709 12:26:53.356397 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48045.4us
I20260709 12:26:53.330495 17799 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.306618 17722 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:26:53.358644  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:53.267468 17802 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.369480 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:53.369866  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:53.370821 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.373417 10024 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:53.364907 10022 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:53.317575 17865 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:53.326483 17838 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:53.357424 17656 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:53.360493 17713 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:53.365839 17641 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.536s	user 0.006s	sys 0.005s
I20260709 12:26:53.390861 17641 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:53.391592 17843 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.495s	user 0.000s	sys 0.001s
I20260709 12:26:53.393069 17641 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.393896  9944 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:53.367790 17870 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:53.396533 17792 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.402065  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:53.402851  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:53.387691 17793 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.387490 17761 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.387952 17838 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:53.388067 17802 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.456909 17761 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:53.377259 17864 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:53.389904 17722 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:53.392699 17393 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.460371 17393 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.473479 17802 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.474094 17618 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:53.474218 17802 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.475986 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:53.477149 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:53.478178 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:53.357767 17708 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.014s
I20260709 12:26:53.445294 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:53.480666 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:53.483119 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:53.445606 10023 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:53.445916 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:53.491711  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 121886us
I20260709 12:26:53.447288  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:53.493034  9942 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:26:53.365623 17662 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.494616 10024 raft_consensus.cc:2410] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:26:53.481527 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.404686 17792 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.510855  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:53.513223  9942 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:53.512378  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:53.514895 10640 leader_election.cc:400] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20.
I20260709 12:26:53.531898 10672 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:26:53.469941 17372 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.535122 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:53.535975 10090 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:53.537436 10095 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:26:53.519230 17879 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:26:53.516987 17682 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:53.483412 17802 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.528075 17838 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:53.518303 17774 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:53.553194 17420 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:53.558099 17653 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:53.524456 17870 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:26:53.556326 17739 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:53.526947 17641 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:53.569243 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48963.8us
I20260709 12:26:53.570164 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:53.478484 17644 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.571585 10094 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:53.559485 17694 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:53.518101 17864 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:53.531915 17649 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:26:53.563987 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:53.564101 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:53.571090 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:53.524595 17793 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
W20260709 12:26:53.591706 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 220948us
I20260709 12:26:53.589133 10092 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.589242 10097 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
W20260709 12:26:53.590695  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 223873us
I20260709 12:26:53.565428 17802 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:53.564996 17713 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:53.569347 17875 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:53.596719  9948 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:53.590219 17884 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:53.620397 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:53.647608 17618 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.647043 17874 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:26:53.652385 17618 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:53.649641  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:53.650326 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 133794us
I20260709 12:26:53.622948 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:53.654048  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 130775us
I20260709 12:26:53.663794 10092 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:53.606696 17884 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:53.609915 17802 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.647210 17729 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:53.717268 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55177.1us
I20260709 12:26:53.731511 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4017) took 1918 ms. Trace:
I20260709 12:26:53.652868 17874 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:53.737169 10098 rpcz_store.cc:276] 0709 12:26:51.812970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.356449 (+543479us) service_pool.cc:224] Handling call
0709 12:26:52.451813 (+ 95364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:52.492271 (+ 40458us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.727968 (+1235697us) spinlock_profiling.cc:243] Waited 95.1 ms on lock 0x7b5400164750. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:53.731448 (+  3480us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218631424}
W20260709 12:26:53.742012  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 148667us
I20260709 12:26:53.742817  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:53.736830 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.575426 17641 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.751735 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.733381 17713 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.706189 17782 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:53.731828 17316 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:53.734876 17729 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:53.666534 17682 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.731638 17884 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:53.782667 17316 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.741165 17372 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.783382 17896 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:53.780416 17782 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:53.736418 17774 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:53.739588 17718 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:53.801741 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.804057 17729 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:53.739637 17874 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:53.774950 17662 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:53.734018 17757 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:26:53.851532 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:53.810563 17641 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.653903 17875 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.816717 10093 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:53.874985 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:53.876168 10093 raft_consensus.cc:2368] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
I20260709 12:26:53.877197 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:53.877894 10093 raft_consensus.cc:2376] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18. Re-sending same reply.
I20260709 12:26:53.816658 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:53.820580  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77846.7us
I20260709 12:26:53.880403 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:53.881013  9912 leader_election.cc:400] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:53.799763 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1399) took 1061 ms. Trace:
I20260709 12:26:53.801921 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:53.882025 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:53.832947 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.801028 17775 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:53.882658 10096 rpcz_store.cc:276] 0709 12:26:52.738235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.771751 (+ 33516us) service_pool.cc:224] Handling call
0709 12:26:52.816831 (+ 45080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.033538 (+216707us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.799466 (+765928us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54002fc954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:53.799682 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496776576}
I20260709 12:26:53.680545 17838 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.781847 17713 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:53.883204 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81382.1us
I20260709 12:26:53.885852 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:53.891680  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:53.822912 17889 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.904066 10099 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:53.704807 17864 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.826167 17870 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.851585 17372 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.909112 10020 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:26:53.823376 17898 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:26:53.906054  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:53.885937 17644 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:53.873241 17883 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.902881 17896 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:53.917359 10093 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:53.819644 17682 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.920084 10099 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:53.948982 10021 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:53.912031 17618 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:53.874841 17774 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.950639 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:53.916522 17870 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.981230 17802 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:53.983083 17531 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:53.984508 17893 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 12:26:53.985160 17888 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
W20260709 12:26:53.985927 17887 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
W20260709 12:26:53.986765 17897 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:26:53.912462 17889 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.990908 17682 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:53.992427 17713 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:53.993161 17900 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:26:53.995442 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.088733 10020 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:54.058157 17619 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.912842 17662 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:53.993405 17881 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.617s	user 0.002s	sys 0.000s
I20260709 12:26:54.002980 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.095775 10021 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:54.005168 17875 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:54.015981 10098 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:26:54.019591 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:54.097112  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:54.020960 17874 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:53.940280 17372 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:53.939536 17718 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:54.025321 17792 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.505s	user 0.006s	sys 0.000s
I20260709 12:26:54.033277 17461 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:54.034039 17896 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:53.928145 17775 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 12:26:53.947391 17902 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.119s
I20260709 12:26:54.036161 10092 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:54.039762 10668 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:54.042472  9948 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:54.043658  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:53.999126  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 84110us
I20260709 12:26:54.117491  9940 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:54.044780  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.117751  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:54.056378  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:53.994020 17892 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:53.905587 17894 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.093s
I20260709 12:26:54.124132  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:54.066395 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.127918 10018 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:54.030422 17653 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:54.005220 17774 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:54.076879 10095 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:26:54.078823 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4036) took 1566 ms. Trace:
I20260709 12:26:54.079468  9942 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:54.003822 10099 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: [19.12-21.13]
W20260709 12:26:54.082341  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.134213  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.134728  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.084179 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1396) took 1228 ms. Trace:
I20260709 12:26:54.136144 10672 rpcz_store.cc:276] 0709 12:26:52.855438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.855546 (+   108us) service_pool.cc:224] Handling call
0709 12:26:52.877368 (+ 21822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.546731 (+669363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.075224 (+528493us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b5400610dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:54.084097 (+  8873us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1498974464}
I20260709 12:26:54.057585 17531 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:54.092823  9990 leader_election.cc:400] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
I20260709 12:26:54.104300 17739 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:26:54.098361 17662 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.105379 10097 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:54.115404 17682 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:26:53.951926 17883 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.097015 17420 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:26:54.123287 10673 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:26:54.125119  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.096139 17644 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.138391  9949 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:54.131992 10091 rpcz_store.cc:276] 0709 12:26:52.512753 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.512872 (+   119us) service_pool.cc:224] Handling call
0709 12:26:53.092220 (+579348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:53.111565 (+ 19345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.078564 (+966999us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540034add0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:54.078762 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285061376}
I20260709 12:26:54.143445 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:54.169237 10671 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:54.170068  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.144238 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.145010 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:54.145653 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:54.148744  9990 leader_election.cc:403] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:26:54.175215  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84798.5us
I20260709 12:26:54.176002 10023 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:54.151867 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:53.950780 17782 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.153864  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:54.113040 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 93516.4us
I20260709 12:26:54.113317 17891 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:54.181972  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:54.115649 17461 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:54.157373  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.151923 17892 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:54.123311 17718 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.162325 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.191598 10669 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:54.125633 17641 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:54.165539 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:54.193212 10672 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:54.121280  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:54.194155  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72958.2us
I20260709 12:26:54.166623 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:54.129894 17792 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.180600 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:54.178784 17231 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
I20260709 12:26:54.098946 17619 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.159823 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:54.196156 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:54.187201 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.197016 10097 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:26:54.187601  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:54.188581 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116697us
I20260709 12:26:54.161373  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.152396 17904 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:54.196246 10671 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:26:54.197013 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:54.194854  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:54.146804 17653 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.200133  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:54.157451 17578 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:26:54.212666 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.213755 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.129727 17896 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.190017 17891 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.227100 17231 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.227208 10023 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:54.202345 17793 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:54.207442 17682 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:54.202144 17649 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.212803 17875 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:54.233603 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.242499 10670 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:54.202795 17641 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:54.207505 17879 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:26:54.214917 17802 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:54.209977 17785 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:54.232265 17231 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.185241 17884 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.219096 17904 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:54.238752 17906 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:26:54.222134 17902 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:26:54.178946 17644 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.188463 17025 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:26:54.240649 17729 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.247073 17641 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:54.241815 17682 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.212146 17792 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.243541 17418 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:54.242151 17875 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.249365 17894 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.250361 17802 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:54.266500 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:54.267661 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.269485 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.314558 10670 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
W20260709 12:26:54.325603 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59211.7us
W20260709 12:26:54.341306 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 142898us
I20260709 12:26:54.281419 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.246306 17649 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.347199 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:54.259436 10672 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:54.348325 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:54.349095 10675 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
W20260709 12:26:54.340045 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 143093us
I20260709 12:26:54.342476 10669 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:54.350781 10676 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:26:54.350910 10677 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
W20260709 12:26:54.273561 17873 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.675s	user 0.010s	sys 0.001s
I20260709 12:26:54.355052 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:54.351419 10643 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:54.278344 10094 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:54.273680 17025 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:54.271652 17785 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:54.274401 17662 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.463393 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:54.275404 17870 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.340057 17418 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.346698 17372 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.288092 17889 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.345590 17420 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.316620 17802 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:54.300961 17629 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:54.288354 17875 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:54.464857 17025 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:54.341686 17910 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:54.286126 17682 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:54.288640 17231 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.286145 17578 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:54.276321 17896 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.521008  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2974) took 1334 ms. Trace:
I20260709 12:26:54.461249 17847 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.275633 17729 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:54.474498 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.275477 17891 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.269677 17879 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:54.277187 17644 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.489693 17420 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.523870 10090 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:26:54.492890 17864 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:54.288611 17894 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.502681 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.269958 17461 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.264778 17866 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.492982 17629 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:54.299401 17904 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.273680 17718 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.547307 17718 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:54.547927 17718 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.548506 17718 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.524897 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:54.492476 17662 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:54.551134 17883 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.503077 17025 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:54.544497 17911 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.538904 17847 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.552114 17911 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.483084 17418 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.521458  9943 rpcz_store.cc:276] 0709 12:26:53.186163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.188412 (+  2249us) service_pool.cc:224] Handling call
0709 12:26:53.188797 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:53.189194 (+   397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:53.189234 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:53.189815 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 12:26:53.262208 (+ 72393us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:53.262261 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:53.262346 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:54.240318 (+977972us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:54.511810 (+271492us) raft_consensus.cc:1685] finished
0709 12:26:54.511849 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:54.520935 (+  9086us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":649,"raft.queue_time_us":51049,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":7357,"spinlock_wait_cycles":3918592,"thread_start_us":40049,"threads_started":1,"wal-append.queue_time_us":359965}
I20260709 12:26:54.554826 17884 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:54.541855 17891 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.555474 17891 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.556232 17418 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.557579 17718 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.557734 17891 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.547348 17629 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.567724 17629 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:54.567572 17231 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:54.544499 17653 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.573364  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.524451 17578 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.559341 17879 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.563503 17792 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.564175 17619 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:54.565126 17316 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.740s	user 0.008s	sys 0.004s
I20260709 12:26:54.618757 17316 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.620296  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.620838 17316 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.620980  9945 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:54.539047 17785 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.622754 17889 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:54.623925 10023 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:54.565889 17641 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.551185 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.540705 17864 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.538453 10676 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:54.625185 10090 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:26:54.570439  9943 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 17 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 16. Current term is 17. Ops: []
I20260709 12:26:54.626170 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:54.626505 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.547977 17461 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:54.577139  9942 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:54.554286 17847 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.612416  9944 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:26:54.626806 10090 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
W20260709 12:26:54.629460 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 278102us
I20260709 12:26:54.630072 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.630641 10090 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:26:54.631079 10643 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:54.623298 17889 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.554340 17911 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.316187 17892 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:54.635769 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.637066 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.572230 17257 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:54.649119 10090 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:54.653672 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.655710 10090 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:54.658061 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.658396 10090 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:26:54.659000 10096 raft_consensus.cc:2410] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 21 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
I20260709 12:26:54.570272 17866 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.645582 17649 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.669005 10090 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:54.669896 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:54.651777 17792 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:54.612886 17915 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.090s
W20260709 12:26:54.693154  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 165709us
I20260709 12:26:54.695196 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.696146 10676 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:54.698277  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:54.702450 17793 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:54.712882  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.718969  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.719018  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.719753  9945 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
W20260709 12:26:54.721938 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 258673us
I20260709 12:26:54.722862 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:54.727564 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.727881 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.728850 10675 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:54.727886 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:54.656656 17257 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.735946 10676 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:54.677138 17914 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:54.754246 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:54.686112 17653 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:54.703063 17793 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.661238 17531 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.758921  9947 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:54.663602 17025 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.771526 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2867) took 1232 ms. Trace:
I20260709 12:26:54.788869 10093 rpcz_store.cc:276] 0709 12:26:53.539473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.881716 (+342243us) service_pool.cc:224] Handling call
0709 12:26:54.771315 (+889599us) spinlock_profiling.cc:243] Waited 34.3 ms on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:54.771471 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":78888320}
W20260709 12:26:54.775933  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:54.764854  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.025s	sys 0.005s
I20260709 12:26:54.774847 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1398) took 1897 ms. Trace:
W20260709 12:26:54.790925  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.026s	sys 0.006s
I20260709 12:26:54.777812 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:54.791536  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 591449us
I20260709 12:26:54.784227 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:54.781450  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:54.792223 10091 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:54.793205 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.793792 10091 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:54.793367  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:54.760119 17729 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:54.791177 10673 rpcz_store.cc:276] 0709 12:26:52.877823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.878002 (+   179us) service_pool.cc:224] Handling call
0709 12:26:52.894318 (+ 16316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.167965 (+1273647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.774569 (+606604us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54005267d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:54.774775 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2825274240}
I20260709 12:26:54.796726 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.790441 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:54.760774 17846 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.379s	user 0.013s	sys 0.009s
I20260709 12:26:54.768513 17578 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.797485  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:54.769877 17864 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.774238 17866 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.762212 17894 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.811681 10093 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:54.812279 10096 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:54.841413  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.854486  9942 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:54.805698 10091 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:26:54.745164 17662 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:26:54.843456  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:54.789996 17896 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:54.858889 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.844228  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.775435 17889 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.846043  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:54.785636 17793 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:54.811543  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:54.863646  9949 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:54.777493 17682 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.786178 17805 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.066s
I20260709 12:26:54.805176 17641 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:54.850188 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 67102.4us
I20260709 12:26:54.803014 17870 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:54.893151 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:54.894477 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:54.860949 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.895702 10093 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:54.896716 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.897151 10093 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:54.897857 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:26:54.895342  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 83878.6us
I20260709 12:26:54.899847  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:54.863057 10091 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
W20260709 12:26:54.839491 17846 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.458s	user 0.013s	sys 0.014s
I20260709 12:26:54.803143 17922 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:54.771034 17875 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:54.851928  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.803808 17653 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.088s
I20260709 12:26:54.919402 10098 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:26:54.804958 17729 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:54.891570 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.795940 17316 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
I20260709 12:26:54.859746 17896 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.929607 17896 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:54.901973 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.903409 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.811254 17915 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 12:26:54.932440 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.933036 10093 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:54.933355 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.933985 10675 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:54.857779 17617 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:54.779716 17914 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:54.945972 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:54.945907 10025 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:54.912532 17785 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:54.848088 17879 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:26:54.811092 17847 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 12:26:54.843149 17919 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:54.893682 17870 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:54.926471 17257 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:54.836676 17718 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:26:54.867781 17420 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:54.938370 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.939146 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.965543 10019 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:54.940348 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.941639 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.932749 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.967646 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:54.968318 10676 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:54.893095 17891 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.944485 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.917364  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:54.874223 17231 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:54.917943 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:54.949994 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:54.950896 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:54.919945 17575 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:54.911913 17894 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:54.912106 17838 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:54.951922 10097 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:54.924021 17713 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:54.931607 17729 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:54.960542 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:54.973740 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 68995.4us
I20260709 12:26:54.952342 17785 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:54.978804  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61594.2us
I20260709 12:26:54.969087 17870 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:55.008566 17531 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.009342 17718 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:55.011556 17875 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:55.012403 17025 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:55.021085 10017 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:55.023146 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1419) took 1725 ms. Trace:
I20260709 12:26:55.001770 17932 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.964876 17910 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:54.989895 17805 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:55.024374 17932 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.025075 17805 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.025467 17805 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.026780 10670 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:54.994633 17662 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.031093 17662 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:54.995548 17785 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.001780 10018 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:26:54.975235 17575 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:54.970690 17316 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:55.006734 10090 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:55.007155 17931 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:26:55.007655 17849 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:55.050647 17849 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:55.051770 17849 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.053409 17849 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.008062 17682 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:54.990460  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:54.977465 17231 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.057878 17231 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.033051 10674 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:55.060079 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:55.060477  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 70106.3us
I20260709 12:26:55.023520 10017 rpcz_store.cc:276] 0709 12:26:53.304970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:53.310307 (+  5337us) service_pool.cc:224] Handling call
0709 12:26:53.333503 (+ 23196us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.021595 (+1688092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.022902 (+  1307us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b54001f6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:55.023088 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3880696192}
I20260709 12:26:55.037344 10672 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:26:55.038007 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45787.8us
I20260709 12:26:54.963903 17919 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:55.049325 10090 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: [14.12-16.13]
I20260709 12:26:55.061411  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:55.071368 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.072086 10672 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:55.072935 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.073541 10672 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
W20260709 12:26:55.074898 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.076917  9946 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:55.078485 17911 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:55.079006 17911 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:55.080333 17911 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:55.081758  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.082129 17911 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:55.083813 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.088711 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.977134 17838 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:26:55.093920 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:54.980417 17420 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:54.981141 17928 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:54.977097 17894 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.067691 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:55.089598 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:54.977818 17713 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.095028 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.115931 10090 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:55.101513 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:55.115109  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 180797us
I20260709 12:26:55.116464 10020 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:54.975502 17891 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.117405 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:55.101764 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:55.118813 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.119397 10025 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:55.120208 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:55.120689 10025 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.102157 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.047861 17575 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.078999 17933 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.109954 10016 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:26:55.114264 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:55.091035 17912 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:55.094558 17892 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:55.105724  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:55.114778  9991 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:55.094707 17618 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:55.178002  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72400.1us
W20260709 12:26:55.133518 17461 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.502s	user 0.005s	sys 0.010s
I20260709 12:26:55.134786 17913 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:55.135003 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.180864 10025 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:55.182641 10018 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:26:55.062162 17316 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.137897 17793 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:55.185659  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 70953.3us
I20260709 12:26:55.115236 17452 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.125008 17713 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:55.140177 17802 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.648s	user 0.010s	sys 0.000s
I20260709 12:26:55.140731 10090 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22. Ops: []
W20260709 12:26:55.124380 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56754us
I20260709 12:26:55.141935 17891 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:55.094978 17644 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:55.142306 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:55.142762 17932 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.181908 17718 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:55.111008 17838 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.263599 10019 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:55.147787 17662 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.106961 17865 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.114135 17894 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.150442 17907 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:55.112876 17420 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.153021 17785 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.156208 17578 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:55.157233  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:55.111371 17928 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:55.091213 17682 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.123080 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:55.182018 17946 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 12:26:55.173489 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:55.178126  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 50382.1us
I20260709 12:26:55.294107 17891 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:55.178808  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:55.294651 17793 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.183287 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.294884 17682 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.288196 17953 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
I20260709 12:26:55.165691 17575 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.296021 17420 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.288221 17928 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.297096 17682 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.294615 17891 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.297783 17575 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.297982 17891 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.253674 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:55.298947 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45361.3us
I20260709 12:26:55.299095 10024 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:26:55.181623 17795 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:55.299568 17795 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.300020 17795 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.257032 17452 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.302806 17662 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:55.255800 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.171897 17899 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:26:55.303799 17452 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.303916 10018 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:26:55.304407 17899 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:55.272842 17718 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.182160 17913 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:55.305266 17913 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:55.308743 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:55.090898 17919 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.169880 17805 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.310914 17662 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:55.312672 17805 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.263013 10090 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:26:55.271943 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.176304 17933 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.315968 10090 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:26:55.317417 10090 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:55.256783 17708 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:55.318094 17708 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.317901 17729 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.318537 17708 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.323626 17953 consensus_queue.cc:1243] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:26:55.324120 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.324828 10677 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
W20260709 12:26:55.325830  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 147107us
I20260709 12:26:55.327443  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:55.328261 17785 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.328472 17919 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.329483 10025 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:55.329936 10090 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:26:55.330178 17949 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:55.330665 17949 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.330988 17949 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:55.331207 10090 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
W20260709 12:26:55.265134 17943 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:26:55.287137  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 129993us
I20260709 12:26:55.180328 17892 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.173192 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.332691  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:55.332922 17849 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:55.259825 17644 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:55.172935 17912 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.342681 17793 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.342952 17912 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.304809 17899 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:55.343245 17793 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.344033 17899 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.345249 17899 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.345487 17793 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.181759 17618 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.256930 17802 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.269857 17393 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:55.288465 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 165480us
I20260709 12:26:55.357638 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.357764 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:55.358299 10095 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.290846 17904 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:55.334766 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:55.338369  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.333853  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:55.333348 17892 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.347424 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.296051 17316 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.333330 17849 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.268889 17870 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.355113 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.317471 17933 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.277717 17865 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.370283 17919 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.370841 17802 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.357270 17805 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 12:26:55.371423 17921 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:55.371271 17662 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:55.372557 17922 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:55.373291 17891 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.375875 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:55.376012 17913 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.377516 17910 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:55.338534 17785 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.380035 17774 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:26:55.339085 17911 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.334502 17729 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.346657 17644 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.383589 17682 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:55.366258 17904 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:55.388430 17849 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.364282 17656 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
W20260709 12:26:55.418040 10668 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1661) took 2936 ms (client timeout 1543 ms). Trace:
I20260709 12:26:55.394969 17921 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:55.367743 17393 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.405378 17925 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:55.406121 17870 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.407141 17911 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.403362 17910 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:55.408802 17838 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.410466 17618 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:55.401695 17922 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:55.423777 17896 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.441689 17896 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.425503 17949 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.434892 17910 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:55.444947 17644 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.416898 17904 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.447552 17644 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.433130 17894 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.449900 17847 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:26:55.450434 17847 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:55.451452 17894 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.419685 17682 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:55.451296 17847 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.452838 17847 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:26:55.452548 17682 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.439433 17922 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.444335 17949 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.433372 17713 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.428334 17718 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:55.426906 17891 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.464231 17922 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.459246 17921 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.466116 17874 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.467366 17913 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.469355 17914 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:55.471129 17928 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.466142 17865 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.444370 17911 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:55.494362 17921 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.371907  9941 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:55.535861  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 144225us
W20260709 12:26:55.541306  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.541507 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2592) took 1189 ms. Trace:
I20260709 12:26:55.384584 10095 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
W20260709 12:26:55.542570 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.398092 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.544914 10090 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:55.535738  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 170186us
W20260709 12:26:55.536377  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 202453us
I20260709 12:26:55.547533 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1404) took 1194 ms. Trace:
I20260709 12:26:55.549160  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:55.549865 10093 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:26:55.542326  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:55.535769  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 277936us
I20260709 12:26:55.548152 10091 rpcz_store.cc:276] 0709 12:26:54.353229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.021642 (+668413us) service_pool.cc:224] Handling call
0709 12:26:55.022018 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:55.022375 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.022407 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:55.032250 (+  9843us) log.cc:844] Serialized 26 byte log entry
0709 12:26:55.186716 (+154466us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:55.186782 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.186894 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.546832 (+359938us) raft_consensus.cc:1685] finished
0709 12:26:55.546875 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.547236 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9443,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":153740,"threads_started":1,"wal-append.queue_time_us":330598}
W20260709 12:26:55.551811  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.544936 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.564841 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.565828 10095 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:55.418685 10668 rpcz_store.cc:269] 0709 12:26:52.481916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:52.482403 (+   487us) service_pool.cc:224] Handling call
0709 12:26:55.417785 (+2935382us) spinlock_profiling.cc:243] Waited 1.51 s on lock 0x7b54006126d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:55.417948 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3470513920}
I20260709 12:26:55.566632 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.567745 10090 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.569811 10671 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:26:55.571043 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.569643 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.573504  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:55.574924 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:55.576015 10676 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.542145 10021 rpcz_store.cc:276] 0709 12:26:54.352094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.353175 (+  1081us) service_pool.cc:224] Handling call
0709 12:26:54.353458 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.353784 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:54.353809 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:54.364740 (+ 10931us) log.cc:844] Serialized 26 byte log entry
0709 12:26:54.673514 (+308774us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:54.787811 (+114297us) log.cc:844] Serialized 14 byte log entry
0709 12:26:54.788496 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:54.788594 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:55.484937 (+696343us) spinlock_profiling.cc:243] Waited 8.66 ms on lock 0x7b180010d800. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:55.540792 (+ 55855us) raft_consensus.cc:1685] finished
0709 12:26:55.541017 (+   225us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:55.541304 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":197294,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":9779,"raft.queue_time_us":6546,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":104753,"spinlock_wait_cycles":19917056,"thread_start_us":299306,"threads_started":2,"wal-append.queue_time_us":333169}
I20260709 12:26:55.575003 10675 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:55.579355 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:55.580296 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:55.583210  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:55.584223 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.585747  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:55.588126 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:55.553643 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.591192 10091 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:55.591334 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.592500 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.607227 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.608560  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 2977) took 1037 ms. Trace:
I20260709 12:26:55.608745 10671 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:55.592314 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.599268 17257 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:55.609094  9941 rpcz_store.cc:276] 0709 12:26:54.571318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.571481 (+   163us) service_pool.cc:224] Handling call
0709 12:26:54.571862 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.534333 (+962471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.608026 (+ 73693us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b5400016dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:55.608331 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1839520512}
I20260709 12:26:55.607379  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:55.611058 17257 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:55.611820 17257 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:55.596153 17393 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.613387 17257 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.615437 17393 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:55.467217 17649 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:55.651160 10676 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:55.725014 17649 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:55.660413 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.731091 17644 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:55.731601 17644 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.732007 17644 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.733142 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:55.664964 10673 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:26:55.665486 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2875) took 1562 ms. Trace:
I20260709 12:26:55.736197 10096 rpcz_store.cc:276] 0709 12:26:54.103382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.671384 (+568002us) service_pool.cc:224] Handling call
0709 12:26:54.671789 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.854081 (+182292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.664956 (+810875us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b5400305f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:55.665312 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322349312}
I20260709 12:26:55.665894 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:55.654290 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.667470 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.669418 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.663414 17713 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.671160 17949 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:55.672189 17954 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:55.673110 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4047) took 1572 ms. Trace:
I20260709 12:26:55.678915 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1590) took 1333 ms. Trace:
I20260709 12:26:55.684203 17418 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:26:55.673187  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62785.7us
I20260709 12:26:55.470402 17708 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.766937 17954 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.767513 17954 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:55.769330 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.769780 17708 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.699348 10090 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:55.770993 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.774456 10669 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:55.775887 17418 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:55.775705 17708 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:55.776849  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:55.778165  9941 raft_consensus.cc:2393] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:55.780982  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:55.781517 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.781823  9947 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:55.783809  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:55.786192  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:55.763296 10670 rpcz_store.cc:276] 0709 12:26:54.344998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.345313 (+   315us) service_pool.cc:224] Handling call
0709 12:26:54.382441 (+ 37128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.027211 (+644770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.678154 (+650943us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:55.678621 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481384832}
I20260709 12:26:55.704078 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.478561 17804 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:55.789690 17804 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:55.711351 17875 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:55.790711  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:55.790819 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.791529 17804 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.792659 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.793437 17804 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:55.793530 10096 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:55.712462 17838 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.790853 17875 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.795172 17875 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.796149 17418 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.797407 17418 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:55.713092 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2868) took 2173 ms. Trace:
I20260709 12:26:55.798663 17617 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.798853 10094 rpcz_store.cc:276] 0709 12:26:53.539767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.074261 (+534494us) service_pool.cc:224] Handling call
0709 12:26:54.074555 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.457414 (+382859us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.712574 (+1255160us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:55.712941 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468256000}
I20260709 12:26:55.799443 17617 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.800487 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.801661 10094 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:55.649504 17578 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:55.714293 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3804) took 1479 ms. Trace:
I20260709 12:26:55.802738 17393 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:55.802961 17968 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:55.803146 10023 rpcz_store.cc:276] 0709 12:26:54.235174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.236721 (+  1547us) service_pool.cc:224] Handling call
0709 12:26:54.237027 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.625938 (+388911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.713781 (+1087843us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b540038d6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:55.714087 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":889387904}
I20260709 12:26:55.804185 17968 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.671073 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4053) took 1464 ms. Trace:
I20260709 12:26:55.806207 10097 rpcz_store.cc:276] 0709 12:26:54.206754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.702470 (+495716us) service_pool.cc:224] Handling call
0709 12:26:54.702975 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:54.975627 (+272652us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.670468 (+694841us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540014a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:55.670844 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":561695104}
I20260709 12:26:55.807646 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.808507 10097 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:55.809489 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.810207 10097 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:55.734115 17619 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.811059 17619 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.811627  9947 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:26:55.811905 17919 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.812748 17838 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.813216  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:55.813651 17619 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.813409 17838 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.671051 17936 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:55.497921 17928 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.749032 17316 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:26:55.814502 17907 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:55.814975 17316 consensus_queue.cc:1243] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:26:55.802966 17578 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.815163 17907 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.815744 17838 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.816356 10091 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:55.817116 17578 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.817330 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.723966 17912 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.755216 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.822283 17393 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.823314 10025 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:53.869290  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:26:55.810222  9948 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:26:55.755918 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.759116 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:55.651998 17916 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.108s
I20260709 12:26:55.827553 17782 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.828955 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.829340 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:55.754166  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:55.829805 10097 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:55.831400  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 77351.4us
I20260709 12:26:55.832828  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:55.833755 17936 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:55.754448 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:55.835042 17915 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:55.835121 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108375us
I20260709 12:26:55.835490 17915 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.835969 17915 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.726686 17649 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.837163 17649 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.838975  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:55.838800 17461 consensus_queue.cc:1175] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "cd983415389444e292b2330ef630850a" responder_term: 25 status { last_received { term: 25 index: 18 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:55.812660 17919 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:55.841806 17936 consensus_queue.cc:1175] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" responder_term: 25 status { last_received { term: 24 index: 17 } last_committed_idx: 17 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:26:55.842630 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.776933 17708 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:55.843319 17919 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.843664 17708 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:55.844233 17316 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:55.780984 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.752192 17795 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.763150 10098 rpcz_store.cc:276] 0709 12:26:54.100991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.101126 (+   135us) service_pool.cc:224] Handling call
0709 12:26:54.771472 (+670346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:54.919817 (+148345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.298729 (+378912us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:55.672527 (+373798us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b5400162e50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:55.672845 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1880583936}
I20260709 12:26:55.804700 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:55.827204 10099 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:55.830192 10018 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:55.869936  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 296709us
I20260709 12:26:55.847075 10094 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:55.853185 17708 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.844923 17316 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:55.817451 17907 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.817272 17928 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.858922 17913 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.859221 17968 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.860482  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:55.861742 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56140.1us
I20260709 12:26:55.818466 10091 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:26:55.862792 17838 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.903929 10092 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:55.864892  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:55.865069 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:55.865283 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:55.865999  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:55.866110 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 133104us
I20260709 12:26:55.866503 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:55.904996  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:55.867370  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:55.868532  9940 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:26:55.868597  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:55.845451 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:55.763098 10095 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:55.898851 17915 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.863724 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:55.866755 17619 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:55.909075 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:55.912973  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44504.8us
I20260709 12:26:55.907645 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:55.915436 10098 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:55.910404  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:55.916661 10671 raft_consensus.cc:2368] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:55.917531 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2880) took 1072 ms. Trace:
I20260709 12:26:55.917958 10098 rpcz_store.cc:276] 0709 12:26:54.845474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.904751 (+1059277us) service_pool.cc:224] Handling call
0709 12:26:55.915092 (+ 10341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:55.915885 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.917445 (+  1560us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.918298  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:55.923226  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:55.923873  9948 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.928946 10674 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:55.904609 17461 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:26:55.930434  9945 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:55.930480  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:55.929544  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:55.935724 10096 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:55.927306 17913 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.942231 17849 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.942692 17793 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:26:55.944825 17849 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.944947 17913 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.946928 17916 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:55.947408 17916 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:55.947731 17916 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:55.949231 17785 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.537s	user 0.003s	sys 0.012s
I20260709 12:26:55.949753 17785 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:55.950841  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:55.951234 17963 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:55.951665 17963 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:55.953024 17963 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.953867 17919 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:55.954453 17919 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:55.954972 10099 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: [24.12-24.12]
I20260709 12:26:55.955158 17907 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:55.955729 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2878) took 1115 ms. Trace:
I20260709 12:26:55.956262 17907 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:55.956166 17970 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:55.956840 17907 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:55.957374 17970 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:55.957492  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:55.958019 17970 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:55.925547 17928 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.962891  9947 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:55.965451 10095 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:55.958503 10091 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.16-21.16]
I20260709 12:26:55.971314 10090 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:26:55.966979 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50085.9us
I20260709 12:26:55.956126 10099 rpcz_store.cc:276] 0709 12:26:54.839855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.894289 (+1054434us) service_pool.cc:224] Handling call
0709 12:26:55.930684 (+ 36395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:55.955447 (+ 24763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:55.955656 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:26:55.965821 17912 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:55.974963 10098 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:55.955610 17919 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:55.978929 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 109630us
I20260709 12:26:55.959460 17970 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.980500 17970 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.965736  9949 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:55.983928  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:55.985378 17708 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:55.932657 17968 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:55.987090 17968 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:55.988371 17968 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:55.963913 17904 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.517s	user 0.017s	sys 0.000s
I20260709 12:26:55.991195 17904 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:55.993330 17904 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.995122 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:55.995419 17617 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:55.997043 17795 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:55.924162 17838 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.999483 17619 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:55.999701 17838 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.001055 17875 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.963472 17779 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:55.981302 17316 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:55.997787 10062 leader_election.cc:400] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:56.009487 10062 leader_election.cc:403] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
W20260709 12:26:56.003659  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:55.971102  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:55.934152 17257 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:56.010130  9943 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:56.014865  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:55.962441 17847 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.509s	user 0.006s	sys 0.006s
I20260709 12:26:56.017314 10093 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:26:56.015770  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 77659.9us
I20260709 12:26:56.017771 10018 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:26:56.025616  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.018571 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 17.
I20260709 12:26:56.029604 10094 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:55.958086 17907 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:56.033370 10090 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:26:56.036664  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:56.037601 10099 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:26:56.034929  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.036454 17779 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.039364 10097 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:26:56.041675  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.044955  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.046607  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.048024  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.043794  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:56.028604 17257 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:56.030105 17913 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.031834 17838 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:56.031596 17849 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:26:56.059091  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.032204 17847 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:56.066458  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 159075us
I20260709 12:26:56.064693 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:56.070737 10668 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:56.075456  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.075685 17911 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.107s
I20260709 12:26:56.077258  9945 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:26:56.078809 17849 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:56.079934 17849 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.083242 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:56.038679 17795 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:26:56.086467 10096 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:26:56.030791 17644 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.092727 10674 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
W20260709 12:26:56.095978 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 93603.5us
I20260709 12:26:56.097388 10668 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:56.116262 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:56.067718  9914 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.506s (SENT) [suppressed 12 similar messages]
I20260709 12:26:56.120822 10096 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:56.122222 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:56.066437 17928 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:56.123690  9945 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:56.125341 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:56.128109 10672 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 21 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:56.132369  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:56.133761 10097 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:56.110805 17911 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:56.138665 10670 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:26:56.138816 17461 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.140048  9941 raft_consensus.cc:2368] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:56.132162 10094 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:56.140605 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:56.093926 17913 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.154384 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:56.129184 17907 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:26:56.156483  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 199072us
W20260709 12:26:56.159062  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 91377.2us
I20260709 12:26:56.120188 17954 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:56.164546 10097 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
W20260709 12:26:56.170508 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73856.7us
I20260709 12:26:56.169327 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:56.151767 17915 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.177141 10671 raft_consensus.cc:2410] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 18 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:26:56.177981 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:56.140257 17847 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:56.170318 17875 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.190840 17874 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:56.192883 17971 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.295s	user 0.003s	sys 0.000s
W20260709 12:26:56.189911 17974 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:26:56.174189 17919 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:26:56.190332 17910 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.691s	user 0.016s	sys 0.000s
W20260709 12:26:56.214807 17972 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.318s	user 0.000s	sys 0.003s
I20260709 12:26:56.214455 17910 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.217840 10096 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
W20260709 12:26:56.186087 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83194.6us
I20260709 12:26:56.173064 17804 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:56.208741 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:56.208942  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.213310  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 97718.9us
I20260709 12:26:56.171437 17461 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.208745 10022 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:26:56.224777  9948 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:26:56.224849 17694 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:56.227766 17899 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.228343 17899 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.228866 17847 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.229969 17954 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.230677 17788 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:56.232203 17393 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.230664 17899 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.217486 17919 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:56.254019 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 82715.7us
I20260709 12:26:56.254884 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:56.227124 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:56.216094 17779 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:56.268260 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59585.5us
I20260709 12:26:56.276216 10096 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:26:56.278049 10643 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:56.281476 17899 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:56.281634 17757 raft_consensus.cc:2764] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:26:56.282348 17949 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:26:56.282418 17757 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:56.282860 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:56.282986 17949 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:56.183990 17257 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.259862 17919 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:56.290856  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:56.282455 17774 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:26:56.266958 17875 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:56.293951 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:56.293962 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2890) took 1182 ms. Trace:
I20260709 12:26:56.296715 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:56.295034 10098 rpcz_store.cc:276] 0709 12:26:55.105650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.975845 (+870195us) service_pool.cc:224] Handling call
0709 12:26:55.976240 (+   395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:55.976501 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:55.976526 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:55.977254 (+   728us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.157637 (+180383us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:26:56.157677 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.157741 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.285385 (+127644us) raft_consensus.cc:1685] finished
0709 12:26:56.285410 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.287843 (+  2433us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":115016,"threads_started":1,"wal-append.queue_time_us":214964}
W20260709 12:26:56.302927 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83790.1us
I20260709 12:26:56.285679 17713 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:56.259382 17804 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:56.305831  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46771.1us
I20260709 12:26:56.294463 17757 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:56.315107  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:56.317332 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:56.301777 17874 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:26:56.300474 17418 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:56.336895 17846 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.337533 17846 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.338771 10096 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:26:56.340655 10096 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
W20260709 12:26:56.343220 17976 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.443s	user 0.000s	sys 0.003s
I20260709 12:26:56.343542 17875 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:56.346973 17949 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.348513 17949 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:56.355549 17461 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:56.355937 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63524us
I20260709 12:26:56.356411 17910 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.357286 17910 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.357887 17910 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.359506 17916 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.360095 17804 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.361279 17874 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:56.361786 17916 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.358304 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:56.321498 17788 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:56.316576 17757 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:56.332988 17418 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:56.390165 17979 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:56.398927  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 63954.5us
I20260709 12:26:56.399322 17644 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.399852  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:56.401081 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:56.402086 10099 raft_consensus.cc:2368] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:56.404915 17899 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:26:56.406656  9940 raft_consensus.cc:2410] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
W20260709 12:26:56.410063 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 101841us
I20260709 12:26:56.411074 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:56.412139  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 97155.6us
I20260709 12:26:56.362393 17874 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.420487 17775 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:56.421378 17775 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:56.407898 17418 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:26:56.416577 17649 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.579s	user 0.010s	sys 0.003s
I20260709 12:26:56.436287 17757 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.437793 17757 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:56.443061 17617 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:56.443498 10065 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:56.444805 17875 raft_consensus.cc:2764] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:26:56.445001 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:56.445896 17914 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:56.446349 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:56.393426 17846 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.398022 17919 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.457592 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:56.459091 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 149593us
I20260709 12:26:56.447752 17907 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:26:56.465456  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52468.4us
I20260709 12:26:56.466306  9991 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:56.456544 17779 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:56.484189 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:56.462523 17875 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:56.468072 17788 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:56.490686 10099 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:56.462612 17907 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.496836 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:56.501010 17919 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.504365 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2888) took 1399 ms. Trace:
I20260709 12:26:56.508713 17910 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:56.513371  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.514315  9949 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:56.515055 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2895) took 1175 ms. Trace:
I20260709 12:26:56.515353 10097 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:26:56.516863 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:56.485031 17649 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:56.515623 10093 rpcz_store.cc:276] 0709 12:26:55.339288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.781167 (+441879us) service_pool.cc:224] Handling call
0709 12:26:56.514859 (+733692us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b54002fc954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:56.514997 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":529735552}
I20260709 12:26:56.468297 17954 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:26:56.504989 10091 rpcz_store.cc:276] 0709 12:26:55.104916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.973460 (+868544us) service_pool.cc:224] Handling call
0709 12:26:56.038174 (+ 64714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.093828 (+ 55654us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:26:56.294950 (+201122us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.418987 (+124037us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.422498 (+  3511us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.423099 (+   601us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:56.424456 (+  1357us) log.cc:844] Serialized 14 byte log entry
0709 12:26:56.444092 (+ 19636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.444211 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:56.504069 (+ 59858us) raft_consensus.cc:1685] finished
0709 12:26:56.504098 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:56.504266 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19125,"prepare.queue_time_us":247424,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":58724,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":9687,"thread_start_us":227368,"threads_started":2,"wal-append.queue_time_us":151028}
I20260709 12:26:56.527957 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:56.514936 17907 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:56.533102  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.538916  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.543910 10095 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: [20.14-25.15]
W20260709 12:26:56.544152  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 83805.6us
I20260709 12:26:56.520520 17928 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:56.523409 17257 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
W20260709 12:26:56.566583  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.567094  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:56.567675  9943 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:56.545243  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79022us
I20260709 12:26:56.524183 17788 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.550880 17921 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:56.575282  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:56.575946  9948 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:56.551469  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:56.552192 17644 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:26:56.523160 17979 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.556169  9949 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
W20260709 12:26:56.569752  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.520956 17779 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:56.624240 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46272.4us
I20260709 12:26:56.583693 17994 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:56.584970  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:56.634999 17617 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.635151 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:56.636126 10091 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:26:56.637235 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:56.637727 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:56.638761 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:56.639324  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:56.640911  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.530674 17875 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:56.623370 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2604) took 1040 ms. Trace:
I20260709 12:26:56.708909 10021 rpcz_store.cc:276] 0709 12:26:55.583341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.583847 (+   506us) service_pool.cc:224] Handling call
0709 12:26:56.595239 (+1011392us) spinlock_profiling.cc:243] Waited 433 ms on lock 0x7b54001f8e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:56.623313 (+ 28074us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":996385536}
I20260709 12:26:56.582688 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:56.623416 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:56.578752  9944 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: [24.12-24.12]
I20260709 12:26:56.579092  9940 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:56.591315 17418 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.622027 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:56.712245 10669 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:26:56.627323 17899 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:56.628005 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2887) took 1523 ms. Trace:
I20260709 12:26:56.628805  9943 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:56.635684 17617 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.638415 10097 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:56.577229  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:56.643261 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:56.713608 10095 rpcz_store.cc:276] 0709 12:26:55.104603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:55.967918 (+863315us) service_pool.cc:224] Handling call
0709 12:26:56.543633 (+575715us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:56.561713 (+ 18080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.627776 (+ 66063us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54002fdd50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:56.627938 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1192965888}
I20260709 12:26:56.717175 10676 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:56.586055 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 58168.1us
I20260709 12:26:56.638387 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:56.639359 10094 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:56.621743 17393 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:56.671651 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46377.8us
W20260709 12:26:56.671705 10092 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4051) took 2549 ms (client timeout 1506 ms). Trace:
I20260709 12:26:56.682045 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:56.683036 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:56.686120  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.688191 17779 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:56.691056 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83098.3us
I20260709 12:26:56.691565 17910 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:56.692006 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52062.9us
I20260709 12:26:56.692513 17775 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:56.695653 17788 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.670444 17994 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.768191 17994 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:56.768988 17775 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:56.769407 17775 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:56.770285 17775 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.771600 17775 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:56.700819 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:56.775905 10024 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:56.635596 16661 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:56.776794 16661 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:56.777649 16661 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.778832 16661 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:56.704874 10672 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:26:56.779452 10672 pending_rounds.cc:85] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 12
I20260709 12:26:56.780560 10094 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:56.704886  9948 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:26:56.703569 10022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3809) took 1715 ms (client timeout 2176 ms). Trace:
I20260709 12:26:56.711023 10670 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 17.
I20260709 12:26:56.724174 10642 leader_election.cc:400] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:56.783188 10642 leader_election.cc:403] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:26:56.719784 10092 rpcz_store.cc:269] 0709 12:26:54.102440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.146666 (+ 44226us) service_pool.cc:224] Handling call
0709 12:26:56.595007 (+2448341us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b540002dd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:56.651682 (+ 56675us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3740380416}
I20260709 12:26:56.727602 10093 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:56.728772 10674 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:56.733942  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.727068 10675 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:56.746879 17914 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:56.747605  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 179170us
I20260709 12:26:56.748191 17865 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:56.748951 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:56.746381 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:56.756052 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:56.791429 10096 raft_consensus.cc:2376] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 21. Re-sending same reply.
I20260709 12:26:56.790692 10669 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:26:56.792768 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73279.9us
I20260709 12:26:56.756299 17418 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.756517 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:56.779691 10672 pending_rounds.cc:107] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:26:56.757059 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:56.795260 10672 raft_consensus.cc:2889] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:56.795571 10018 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:56.757160 17910 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.758100 17393 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:56.758219 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:56.703678 17979 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:56.759099 17757 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:56.759238 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:56.762104 10020 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:56.709800 17936 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:26:56.765913 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:56.785847 10097 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:26:56.782902 10022 rpcz_store.cc:269] 0709 12:26:54.936328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:54.946567 (+ 10239us) service_pool.cc:224] Handling call
0709 12:26:56.594825 (+1648258us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540038e0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:56.652036 (+ 57211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2757961472}
I20260709 12:26:56.805608  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:56.806417 10016 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:26:56.793483 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:56.797087 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:56.813056 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:56.814285 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:56.817062 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:56.821954 10676 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:56.824054  9943 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20. Ops: []
W20260709 12:26:56.832198 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:56.833889 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:56.802736 17994 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:56.800849 17913 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:56.874719  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63141.1us
I20260709 12:26:56.804812 17838 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:26:56.876467  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67305.6us
W20260709 12:26:56.873895 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 76981.7us
W20260709 12:26:56.879791  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 328454us
I20260709 12:26:56.875337  9942 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:56.912827 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:56.913739 10091 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 22
W20260709 12:26:56.875279  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 310194us
I20260709 12:26:56.916126  9945 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:26:56.798472 17954 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:26:56.915197  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:56.798259 17914 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:56.922989 17922 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:56.923441 17922 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.923787 17922 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:56.924513 17914 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:56.927139 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 178329us
I20260709 12:26:56.928758 10065 leader_election.cc:400] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:56.929066 10065 leader_election.cc:403] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:26:56.930744 17774 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:26:56.936038 17644 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:26:56.937345 17644 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:56.937770 17644 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:56.937939 17393 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.881656 17708 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:26:56.941720 17617 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:56.881455 17899 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:56.942870 17899 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:56.943306 17899 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:56.813843 17779 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.105s
I20260709 12:26:56.823724 17757 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:56.943243 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 130291us
I20260709 12:26:56.934126 17418 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:56.956866 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:56.826501 17788 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:56.969995 17979 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:56.967475 10097 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
W20260709 12:26:56.973655  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58553.7us
I20260709 12:26:56.921078 17968 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:26:56.990520 17708 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:56.991454 17779 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:26:56.991778 17913 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:26:57.030380  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:56.995338 17996 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:26:56.995877 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:56.995740 18007 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:56.996969 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:57.034678 10092 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:56.957434 17418 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:56.999555 17644 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.000025 17907 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.002573 17910 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:56.974673 17985 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:26:57.006006  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48019.5us
I20260709 12:26:57.011281 17708 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:57.012668 10094 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:57.030081 17779 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:56.968502 17774 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:57.022442 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:57.047139 10097 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:26:57.045429  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 58380.3us
I20260709 12:26:57.034523 17911 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:57.029188 10093 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:56.955317 17838 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:56.963995 17788 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.039964 10092 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:26:57.022570 17994 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19.
I20260709 12:26:57.043429  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:57.016956 17899 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.033388 17968 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:57.027086 17954 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 12:26:57.063968 17644 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:57.064498 17774 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.066413 17649 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:57.066294 17708 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.050019 17617 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.038064 17979 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:26:57.070880 10097 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
W20260709 12:26:57.068094 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 146125us
I20260709 12:26:57.051611 17788 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:57.078958 17617 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.082192 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:57.084407 17907 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:57.085280 17757 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:57.086201 17997 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.090s	user 0.003s	sys 0.000s
I20260709 12:26:57.106871 18002 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:57.070696 17911 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.107543 18002 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.108079 18002 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:57.086911 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1422) took 1431 ms. Trace:
I20260709 12:26:57.110759 10098 rpcz_store.cc:276] 0709 12:26:55.655361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.300670 (+645309us) service_pool.cc:224] Handling call
0709 12:26:56.316558 (+ 15888us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.364708 (+ 48150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.364752 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.377694 (+ 12942us) log.cc:844] Serialized 26 byte log entry
0709 12:26:56.523131 (+145437us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:56.523166 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.523235 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:57.086673 (+563438us) raft_consensus.cc:1685] finished
0709 12:26:57.086696 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.086850 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":84267,"threads_started":1,"wal-append.queue_time_us":388129}
I20260709 12:26:57.087952 17793 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 12:26:57.093132 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 348748us
W20260709 12:26:57.113338 17928 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.000s
I20260709 12:26:57.113785 17928 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:57.115760 17963 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.115978  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:57.116918 17865 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:26:57.117074  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:57.118163  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:57.095729 17949 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.118957 17949 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.121206 17949 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:57.121580  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.121856 17949 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:26:57.122287 17949 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.124608 17846 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:57.125218 17846 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.125670 17846 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:57.125743 17418 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.126390 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.127102 10672 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:57.129000 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.130966 17968 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.133129 17257 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:57.133551 17257 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:57.136945  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:57.096211 17649 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:57.138584  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:57.139959 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.105113 10016 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 24. Ops: []
I20260709 12:26:57.106009 17617 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:57.105746 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:57.145236 10091 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:57.117784 17865 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:57.153540 10097 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:57.157941 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.159065 10669 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:57.159561 18011 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:57.160183 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.161003 10676 raft_consensus.cc:2368] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
W20260709 12:26:57.162768 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57133.6us
I20260709 12:26:57.160895 10677 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:57.163762 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:57.073168 17921 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:26:57.167824 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:57.179059 18019 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:26:57.180971 18017 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:57.180891 17954 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:57.181625 18017 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:57.182089 18017 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:57.182472 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.182477 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.183370 10675 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:57.183534 10670 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 12:26:57.139832 17949 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:57.184968 10672 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 22. Ops: []
I20260709 12:26:57.205889 10098 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:57.207115 10095 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:26:57.211088 17682 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22.
I20260709 12:26:57.215404 10016 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:26:57.217051 10020 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:26:57.139313 17649 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:57.219856  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.186342 10092 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:57.227946 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.229664 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:57.227020  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 179535us
I20260709 12:26:57.231630 10018 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:26:57.232685 10677 raft_consensus.cc:2410] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
W20260709 12:26:57.235605  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.240891 10021 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:26:57.248610 10676 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:57.242734  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:57.259747  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 198958us
I20260709 12:26:57.184994 17713 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:26:57.260872 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.155946 17865 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:57.262755 10022 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:26:57.279103  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:57.263126 10016 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:26:57.263345 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:57.205056 17617 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:26:57.265292 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:57.134478 17257 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.278074  9943 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:57.283921  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.285429 10091 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:57.286651  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:57.287220  9944 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:57.290920 17936 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.295378 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.300525  9942 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:26:57.301635 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.302973 10677 raft_consensus.cc:2410] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:26:57.303946 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:57.305076 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 130945us
I20260709 12:26:57.307219 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:57.315145 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.313241 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:57.321338  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.284721 17874 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.322505  9943 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:57.322686 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:57.320161  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57151.8us
I20260709 12:26:57.324584 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.324946  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.304155 17919 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:57.335856 17911 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:57.340710 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:57.341533 10093 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.341852 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.342679 10669 raft_consensus.cc:2410] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 21 because replica has last-logged OpId of term: 20 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:26:57.233471 17838 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.326453 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.328229 10097 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:26:57.328434  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:57.327379 10670 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:57.330924 10675 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:26:57.331061  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.331670  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1703) took 1246 ms. Trace:
I20260709 12:26:57.334692 17954 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:57.330302 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.340816  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.345038  9945 rpcz_store.cc:276] 0709 12:26:56.085280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.130046 (+ 44766us) service_pool.cc:224] Handling call
0709 12:26:56.152948 (+ 22902us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.916553 (+763605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.331395 (+414842us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b54001a36d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:57.331584 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1754548608}
I20260709 12:26:57.312348 18019 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:57.319897 17713 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:26:57.355274  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 112618us
I20260709 12:26:57.361603 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:57.357970  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:57.362372 10095 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:57.363521 16661 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:57.363832 16661 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:57.364416 16661 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.363148  9940 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:26:57.373200  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:57.373952 16661 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.376775  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.376981 10020 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:26:57.379266  9944 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:57.379532 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.382197 10095 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:26:57.322901 18011 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:57.389109 10095 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:57.351473 17644 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:57.390551 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.391363 10670 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:57.390939 17644 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.392115 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.393762 17644 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:57.361858 17954 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.394902  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:57.356089 17257 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:57.399256 10093 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:57.360941 17713 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:57.372592 17899 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:57.400822  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:57.315611 10094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:57.382364 17904 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:57.371474 18016 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.427299 18016 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.429538 18016 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:57.402550 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.403370 10094 raft_consensus.cc:606] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:26:57.430791 10094 raft_consensus.cc:614] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:57.432650 17999 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:57.433709 10022 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
W20260709 12:26:57.403522  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.347738 17788 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:57.414026 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.383764 17919 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:57.400535 18003 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.454035 18003 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.455240 10670 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:26:57.456266 18003 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:57.399914 17865 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.456413 18011 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.457507 17865 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.460366 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.460969 10673 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.463137 18019 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.455276 10669 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:26:57.423789  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.402918 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:57.469452  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.346138 17874 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.417721 18014 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:57.443454 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:26:57.446169 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.450331 17999 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:57.451471 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.452872 17788 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:57.453361 18009 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:57.470690  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:57.471935  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.473801  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:57.487020  9943 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:57.395646 18013 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:26:57.479838 10673 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:57.489061  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.402913 17713 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:57.483819 10675 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:57.411001 17904 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:57.492126 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:57.492880  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.411235 18018 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:57.499285 18002 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:57.502724 18000 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 12:26:57.504102 18020 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.504441 17804 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:57.504958 17804 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:57.504696 18020 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.493534 18014 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:57.505868 17804 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.506263 18014 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.507791 10670 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:57.508813  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:57.509379 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1431) took 1080 ms. Trace:
I20260709 12:26:57.515616  9941 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
W20260709 12:26:57.522286  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 131762us
I20260709 12:26:57.401022 18022 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.098s
I20260709 12:26:57.442801 17393 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.073s
I20260709 12:26:57.515283 17804 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:57.559523 10018 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:26:57.537426  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.516613 10099 rpcz_store.cc:276] 0709 12:26:56.428587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.496016 (+ 67429us) service_pool.cc:224] Handling call
0709 12:26:56.520773 (+ 24757us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:56.559306 (+ 38533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.559340 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:56.572776 (+ 13436us) log.cc:844] Serialized 52 byte log entry
0709 12:26:56.879676 (+306900us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:26:56.879715 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:56.879772 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:57.509092 (+629320us) raft_consensus.cc:1685] finished
0709 12:26:57.509123 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:57.509307 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1964,"raft.queue_time_us":29995,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":13158,"thread_start_us":244651,"threads_started":1,"wal-append.queue_time_us":401002}
I20260709 12:26:57.562058  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.523173 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:57.523744 16661 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.583179 16661 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.538923  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:57.539681 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:57.544226 18008 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:57.544888 17999 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.546698 17968 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:26:57.480686 17899 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:26:57.508206 18018 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:57.562929  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.563511 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.564658 10016 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:57.524286  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:57.572391 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.567390 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:57.576148 17713 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:57.555930 17922 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.632s	user 0.010s	sys 0.002s
I20260709 12:26:57.580251 17874 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:57.581190  9912 leader_election.cc:400] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:57.522837 17963 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:57.590113 18008 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.619670  9943 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
W20260709 12:26:57.587276 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.589591 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66514.2us
I20260709 12:26:57.580621 17846 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.572685 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:57.599997  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 75798.6us
I20260709 12:26:57.556260 17936 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:57.611135 10099 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:57.581624 18003 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:57.596200 17996 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:26:57.614722 16661 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:57.614930  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.508028 18009 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.614949  9912 leader_election.cc:403] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:26:57.617027 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.602511 18020 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:57.600376 18018 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:57.598563 18002 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:57.603245 17788 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:57.644202 17846 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:57.644505 18009 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:57.633795 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:26:57.619125  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:57.647037  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66030.7us
I20260709 12:26:57.633791 17649 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 12:26:57.612823 18022 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:57.640996 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.648307 10019 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.643252 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.649464 10675 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:26:57.641392 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:57.626751 18008 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:57.650622 10099 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
W20260709 12:26:57.649650  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.651197 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.652400 18009 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:57.653424 18009 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:26:57.653743 18009 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:57.654793 18009 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.655961 18009 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:57.660015 17968 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18.
I20260709 12:26:57.602945 17919 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.685421  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.647127 17418 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:57.619642 17922 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.697962 10099 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 12:26:57.720610  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.727439 17393 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:26:57.728017 17393 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:26:57.728577  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.728917 17393 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.750262 17779 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.752072 17779 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:57.753582 10092 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:26:57.730532 17393 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:57.754779 17911 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:26:57.755396 17911 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:57.756568 17911 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.758190 17911 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:26:57.761513 18025 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.328s	user 0.003s	sys 0.000s
I20260709 12:26:57.637243 18003 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.824009 10093 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:26:57.797091  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.797219 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.798020 18039 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:57.832367  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 184608us
W20260709 12:26:57.798890 17949 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.584s	user 0.003s	sys 0.008s
W20260709 12:26:57.801029  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:57.802305 17919 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:26:57.804136 17968 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:26:57.806912 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:57.808034 17907 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.796020 17846 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.811168 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:57.811753 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:57.935074 10676 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:57.935226 10091 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:57.815454  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.816211  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:57.817715 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:57.820138 17682 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:57.795321 18030 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.938192  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:57.822721 10672 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:26:57.823410 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:57.697026 18020 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.835386 10097 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:57.945516 10097 raft_consensus.cc:606] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:57.945943 10097 raft_consensus.cc:614] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:57.836095 10019 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:57.841655  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:57.850400 18010 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.727s	user 0.001s	sys 0.003s
I20260709 12:26:57.853928 17979 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:57.876427 18013 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:26:57.818351 17617 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:57.930143 17963 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:57.918383 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:26:57.951871 18007 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.812s	user 0.014s	sys 0.028s
I20260709 12:26:57.717331 18014 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.933449 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:57.933336 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.965526 10023 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:57.934104 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:57.822427 17954 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:57.939914 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 116586us
I20260709 12:26:57.946985 17619 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:57.940057  9944 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
W20260709 12:26:57.940559 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 122907us
I20260709 12:26:57.970471  9944 pending_rounds.cc:85] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 12
I20260709 12:26:57.949854 17899 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:57.943892  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:57.944397 10022 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:26:57.832578 18002 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:57.954440 10091 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:57.916879  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:57.931125 17949 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.926939 10677 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 19 index: 17. (index mismatch)
I20260709 12:26:57.969660 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:57.649034 18017 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.970798  9944 pending_rounds.cc:107] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:26:58.002123  9944 raft_consensus.cc:2889] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:57.970932 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:57.937638  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:57.946523 18022 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:57.965777 10017 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
W20260709 12:26:57.997644  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 80847.3us
I20260709 12:26:57.961524 10675 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:58.013914 16661 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:26:57.939049 17393 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:26:57.956019 18013 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:58.031381 10676 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:26:57.944732 17846 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.937963 17907 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:57.927781 18039 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.001004  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:26:58.025388 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55837.1us
W20260709 12:26:57.973958 18007 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.014s	sys 0.029s
I20260709 12:26:57.952425 17979 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:57.943269 18030 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.948798 18020 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:57.997746 17899 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.146366 17899 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:58.147722 18020 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:26:58.149724 18010 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 12:26:58.038180 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 214328us
I20260709 12:26:58.008150 18014 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.151722 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:58.152508 18014 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:58.152635 18038 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 12:26:57.972354 17921 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.116947 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:58.165022 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1432) took 1597 ms. Trace:
I20260709 12:26:58.176920 10065 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:26:58.177790  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.178423  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.200522 10023 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:58.119925 10024 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:26:57.962510 17617 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.027388 18022 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.127714 10016 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:58.128723 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:26:58.131433  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 130487us
I20260709 12:26:58.132611 17708 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:58.132892  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:58.135416 18044 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:26:58.140717 17846 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.141860 18013 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.145624 18030 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.158003 17874 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.033998 17682 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:58.154084 10018 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:26:58.155390  9940 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:26:58.156119 10021 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 12:26:58.201910  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.314344  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.217628 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 201104us
I20260709 12:26:58.315294 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:26:58.153362 17257 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.752s	user 0.009s	sys 0.000s
I20260709 12:26:58.372793 17904 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.374176 17904 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:58.374352 18003 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.376235 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.376492 17949 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.377151 10016 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:26:58.378243 10016 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:26:58.165584 10090 rpcz_store.cc:276] 0709 12:26:56.568147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.667934 (+ 99787us) service_pool.cc:224] Handling call
0709 12:26:56.668400 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:56.878057 (+209657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:56.878101 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:56.958168 (+ 80067us) log.cc:844] Serialized 26 byte log entry
0709 12:26:57.074080 (+115912us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:26:57.078411 (+  4331us) log.cc:844] Serialized 14 byte log entry
0709 12:26:57.078650 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:57.078729 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:57.849764 (+771035us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:26:58.156426 (+306662us) raft_consensus.cc:1685] finished
0709 12:26:58.164735 (+  8309us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.164945 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3989,"prepare.queue_time_us":4185,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":17120,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":2217,"spinlock_wait_cycles":57142400,"thread_start_us":247360,"threads_started":2,"wal-append.queue_time_us":139589}
I20260709 12:26:58.275775 10017 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 18: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:58.382387 10017 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:26:58.280685 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.386991 10023 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:58.024461 18002 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.392650 18002 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.283711 10095 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: [24.12-24.12]
W20260709 12:26:58.155929 17865 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.698s	user 0.019s	sys 0.013s
I20260709 12:26:58.155066 17774 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.394184 17774 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.396713 17774 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:58.302023 10096 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4086) took 1653 ms (client timeout 2101 ms). Trace:
W20260709 12:26:58.397382 10096 rpcz_store.cc:269] 0709 12:26:56.636854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:56.995575 (+358721us) service_pool.cc:224] Handling call
0709 12:26:58.282927 (+1287352us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b5400192dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:58.288237 (+  5310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2155788032}
I20260709 12:26:58.025859 17713 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.224881 17963 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.184618 17968 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:26:58.281896 17708 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.352450 16661 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:58.400310 17708 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:58.400899 16661 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:58.404462 17788 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.761s	user 0.002s	sys 0.010s
I20260709 12:26:58.370636 17682 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:58.405064 17788 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:58.275986 17914 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:58.407557 18017 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.409162 18013 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:58.424762 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.425923 10096 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:58.427166 17914 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.427968 17914 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.438735 17914 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.164688 17838 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.441604 17838 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.452591 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3821) took 1171 ms. Trace:
I20260709 12:26:58.455962 10021 rpcz_store.cc:276] 0709 12:26:57.281166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.281430 (+   264us) service_pool.cc:224] Handling call
0709 12:26:58.452332 (+1170902us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54001f8954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:58.452511 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957536128}
I20260709 12:26:58.285940 18047 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:26:58.469420  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 525600us
I20260709 12:26:58.260254 17921 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.473438 18011 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:58.291455 17779 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.170s
W20260709 12:26:58.483778  9915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.035s	sys 0.036s
W20260709 12:26:58.484411  9915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.035s	sys 0.036s
W20260709 12:26:58.486665  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.017s	sys 0.026s
W20260709 12:26:58.487706  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.017s	sys 0.026s
I20260709 12:26:58.490270  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:58.491212  9947 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:58.314520 17619 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.161s
I20260709 12:26:58.495249  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:58.497084  9943 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:26:58.496366  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:58.355829 17874 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.485105 18014 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.511291 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 334477us
W20260709 12:26:58.524732  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.018s	sys 0.021s
I20260709 12:26:58.502633  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:58.500010  9940 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:26:58.499891  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:58.498991 18033 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.552s	user 0.000s	sys 0.001s
I20260709 12:26:58.532996  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:58.533638  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:26:58.527237 10098 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.008s	sys 0.032s
I20260709 12:26:58.303058 17979 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.529316 10021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59744
W20260709 12:26:58.529951  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.018s	sys 0.022s
I20260709 12:26:58.491361  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:58.495078 17922 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:58.532274  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:58.535991 10098 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.008s	sys 0.032s
W20260709 12:26:58.536197  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 703511us
I20260709 12:26:58.550045  9943 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
W20260709 12:26:58.536972  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45693.8us
I20260709 12:26:58.551494  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:58.498951 18059 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.556602  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.557394  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.536099 10021 raft_consensus.cc:606] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
W20260709 12:26:58.557976  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.557940 10021 raft_consensus.cc:614] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:58.502079 17788 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.566213  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:58.566561  9942 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:58.566852  9949 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:26:58.531078 17999 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.009s	sys 0.031s
W20260709 12:26:58.574486  9843 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 325) took 21961 ms (client timeout 15000 ms). Trace:
W20260709 12:26:58.575476 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 260255us
I20260709 12:26:58.491907 18011 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.495700 18017 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:26:58.580189 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:58.580757 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:58.581028 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:58.531149 18063 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:58.577324  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:58.578024 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.580837 10675 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:58.590334 10090 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
W20260709 12:26:58.571175 17999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.009s	sys 0.032s
I20260709 12:26:58.565498 17774 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:58.595045 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:58.583038 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:58.583225 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:58.583241  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80728us
W20260709 12:26:58.588711  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.581593 10674 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:26:58.528469 18014 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.551678 17914 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.562928 17649 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:58.564247 18030 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:58.526827 18020 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:58.576009 17911 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.685s	user 0.012s	sys 0.000s
I20260709 12:26:58.596662 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:26:58.598352 10096 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:26:58.569264 17919 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:26:58.603700  9942 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:26:58.576216 17846 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.605486 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:58.605679  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:58.585501 17907 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.564895 17708 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.616633 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.570901 17788 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.597536 18002 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:58.596352 18063 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.616930 17617 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:58.568046 18059 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.629292 10674 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:58.631565 16661 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:26:58.633634  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:58.599570 17922 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:26:58.636304 10022 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:58.636410 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.639950 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:26:58.611343 18020 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.640770 16661 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:58.641304 16661 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:58.641760 16661 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:58.645112 18058 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.645705 18058 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.647881 17708 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.648481 18060 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:26:58.577154  9843 rpcz_store.cc:269] 0709 12:26:36.203633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:36.203914 (+   281us) service_pool.cc:224] Handling call
0709 12:26:58.156896 (+21952982us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:26:49.102319 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:26:49.158217 (+ 55898us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:26:49.158277 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:26:49.158295 (+    18us) tablet.cc:662] Decoding operations
0709 12:26:49.183482 (+ 25187us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:26:49.183600 (+   118us) write_op.cc:641] Partition lock acquired for write op
0709 12:26:49.183634 (+    34us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:26:49.185310 (+  1676us) tablet.cc:701] Row locks acquired
0709 12:26:49.185361 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:26:49.185528 (+   167us) write_op.cc:270] Start()
0709 12:26:49.185674 (+   146us) write_op.cc:276] Timestamp: P: 1783600009185460 usec, L: 0
0709 12:26:49.185705 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:26:49.190018 (+  4313us) log.cc:844] Serialized 7855 byte log entry
0709 12:26:49.829592 (+639574us) op_driver.cc:464] REPLICATION: finished
0709 12:26:50.424029 (+594437us) write_op.cc:301] APPLY: starting
0709 12:26:50.481927 (+ 57898us) tablet.cc:1367] starting BulkCheckPresence
0709 12:26:51.415148 (+933221us) tablet.cc:1370] finished BulkCheckPresence
0709 12:26:51.415273 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:26:52.046561 (+631288us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:26:52.288483 (+241922us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=40,mrs_lookups=0
0709 12:26:52.323999 (+ 35516us) write_op.cc:312] APPLY: finished
0709 12:26:52.529207 (+205208us) log.cc:844] Serialized 180 byte log entry
0709 12:26:52.894499 (+365292us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:52.908315 (+ 13816us) write_op.cc:454] Released schema lock
0709 12:26:52.925706 (+ 17391us) write_op.cc:341] FINISH: Updating metrics
0709 12:26:52.980570 (+ 54864us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:26:52.995409 (+ 14839us) write_op.cc:454] Released schema lock
0709 12:26:53.065210 (+ 69801us) spinlock_profiling.cc:243] Waited 30.5 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
Metrics: {"mutex_wait_us":115659,"reported_tablets":47,"spinlock_wait_cycles":9088,"child_traces":[["op",{"apply.queue_time_us":550079,"apply.run_cpu_time_us":55640,"apply.run_wall_time_us":2577662,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87767,"num_ops":20,"prepare.queue_time_us":337440,"prepare.run_cpu_time_us":33025,"prepare.run_wall_time_us":254226,"raft.queue_time_us":201076,"raft.run_cpu_time_us":54054,"raft.run_wall_time_us":322935,"replication_time_us":643645,"spinlock_wait_cycles":70177152,"thread_start_us":1021348,"threads_started":5,"wal-append.queue_time_us":653971,"wal-append.run_cpu_time_us":116339,"wal-append.run_wall_time_us":2492520}]]}
I20260709 12:26:58.649029 18060 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.650828  9949 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:26:58.651296 18060 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.651862 18060 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:58.653035  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:58.654228 18053 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:58.654812 18053 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:58.655763 18053 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.637357 10677 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:26:58.658816 18016 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:26:58.661432 18016 raft_consensus.cc:2764] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:26:58.663599 17921 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.615523 18039 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.609843 17914 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.734756 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2911) took 1098 ms. Trace:
I20260709 12:26:58.610950 17793 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:26:58.805388 17928 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.806070 17928 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.736600 18059 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.737771 18011 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.612534 17779 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.089s
I20260709 12:26:58.701042 17911 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.738916 17919 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.812880 17911 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.815982 17911 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.740053  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:58.817422  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77448.8us
W20260709 12:26:58.818077 17804 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.236s	user 0.010s	sys 0.000s
I20260709 12:26:58.818200  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:58.740789 17921 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:26:58.819278 17874 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.619784 17846 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.743283 17914 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.744009  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:58.746089 18063 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.821257 18031 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:58.821895 17846 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:58.822714  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78824.4us
I20260709 12:26:58.823594 18004 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:58.823974 17788 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:58.668162 17393 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.157s
W20260709 12:26:58.749585 18043 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 12:26:58.632823 18002 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.750053 18066 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:26:58.755227 18055 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:26:58.826967 18002 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:58.708088 17418 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.989s	user 0.011s	sys 0.000s
I20260709 12:26:58.750546 16661 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.829110 16661 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.756106 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4103) took 1322 ms. Trace:
I20260709 12:26:58.757843 10094 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:26:58.830678 17922 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.831271 10094 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:58.831166 17922 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:58.761085  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.833592 18057 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:26:58.834560  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:58.834959  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:58.766150 18009 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.110s	user 0.008s	sys 0.001s
I20260709 12:26:58.768146 18014 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.757934 10099 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:26:58.766333 17979 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.755613 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3825) took 1437 ms. Trace:
I20260709 12:26:58.751807 18075 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.874706  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:58.875492  9946 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:58.875845  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:58.876616  9943 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:26:58.834383 17838 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:26:58.777681  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:26:58.882367  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:58.787742 17617 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.792376 18030 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.731653 18056 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.141s
I20260709 12:26:58.812988 17919 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.739611 17968 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:26:58.822003 18031 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:58.837371  9948 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:26:58.835100 10025 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 25 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
W20260709 12:26:58.761015 18008 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.102s	user 0.005s	sys 0.003s
I20260709 12:26:58.830085 10093 rpcz_store.cc:276] 0709 12:26:57.434159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.447055 (+ 12896us) service_pool.cc:224] Handling call
0709 12:26:57.447801 (+   746us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:57.824424 (+376623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.755856 (+931432us) spinlock_profiling.cc:243] Waited 376 ms on lock 0x7b5400093154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:58.756048 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":864660864}
I20260709 12:26:58.844976  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:58.824995  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:58.849917 10090 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:26:58.862352 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.864588  9945 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:26:58.799237 10098 rpcz_store.cc:276] 0709 12:26:57.636416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.741592 (+105176us) service_pool.cc:224] Handling call
0709 12:26:57.742066 (+   474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:57.822435 (+ 80369us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:26:57.822726 (+   291us) log.cc:844] Serialized 14 byte log entry
0709 12:26:58.438005 (+615279us) spinlock_profiling.cc:243] Waited 78.5 ms on lock 0x7b1800380200. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:26:58.551879 (+113874us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:58.552521 (+   642us) log.cc:844] Serialized 52 byte log entry
0709 12:26:58.563376 (+ 10855us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:58.563417 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.563483 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:58.734371 (+170888us) raft_consensus.cc:1685] finished
0709 12:26:58.734493 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:58.734680 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91827,"prepare.queue_time_us":2650,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":2544,"raft.queue_time_us":107101,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":9993,"spinlock_wait_cycles":180616448,"thread_start_us":727434,"threads_started":1,"wal-append.queue_time_us":799829}
I20260709 12:26:58.926659 10096 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
W20260709 12:26:58.798642 18022 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.002s
I20260709 12:26:58.872797 10024 rpcz_store.cc:276] 0709 12:26:57.317931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.318316 (+   385us) service_pool.cc:224] Handling call
0709 12:26:57.318713 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:58.202039 (+883326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.755315 (+553276us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54002652d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:26:58.755531 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1672101760}
I20260709 12:26:58.777210 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2914) took 1082 ms. Trace:
I20260709 12:26:58.930166 10095 rpcz_store.cc:276] 0709 12:26:57.695328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.821603 (+126275us) service_pool.cc:224] Handling call
0709 12:26:58.283454 (+461851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.393476 (+110022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.776943 (+383467us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b5400192dd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:58.777130 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061397248}
I20260709 12:26:58.931245  9947 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 18: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:26:58.824000 18004 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:58.878140 17899 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.726s	user 0.002s	sys 0.008s
W20260709 12:26:58.911180  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 328788us
I20260709 12:26:58.910441 18013 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.914538 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.915835 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:58.916622  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:58.920514 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.924193 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.928831 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.928565 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.880612  9942 pending_rounds.cc:85] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 13
W20260709 12:26:58.939029  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 351985us
I20260709 12:26:58.943814  9942 pending_rounds.cc:107] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:26:58.944190  9942 raft_consensus.cc:2889] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:58.920370 17846 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.876268 17393 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.055s
I20260709 12:26:58.928550 18002 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.911336 18030 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.937824  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:58.925128 18057 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:58.955691  9944 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
W20260709 12:26:58.956276 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.007s	sys 0.010s
I20260709 12:26:58.914150 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.921304 17418 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.959473 17846 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.960539 17418 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.928763 18039 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.962427 17846 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.963201  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:58.962918 18039 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.963941 10021 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:58.964848 17804 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:26:58.965494 18020 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.965972 17804 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:26:58.944934 17899 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.968258 17804 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:58.968741 17804 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.968871 17899 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:58.969215 17804 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:58.970103 18058 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.935751 18004 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:58.944517 18056 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:58.972282 17907 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:26:58.972815 17907 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 17 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:58.973062 18057 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:58.934093 18008 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.973907 17907 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:58.976068 18008 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:58.976580 17907 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:58.943225 10099 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:58.978950 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.979913 10099 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 18: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:58.980700 10099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:58.981319 10099 raft_consensus.cc:606] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:26:58.981693 10099 raft_consensus.cc:614] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:26:58.943516 10016 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:58.984571 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.941190 10023 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:26:58.987061 10023 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:26:58.942221 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.942189  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:58.948964 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:58.988401 10024 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:26:58.988654 10018 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:26:58.942253  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:58.989557  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47383.5us
I20260709 12:26:58.962427 17418 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:58.990615 17418 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:26:58.990959 17418 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:58.991678 17418 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:26:58.950268 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.949915 18002 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.941614 18022 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:58.951390 18068 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.952633 17914 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.940466 17936 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:26:58.939316  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 114466us
I20260709 12:26:58.957543  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:59.009830  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 67741.8us
W20260709 12:26:58.956624 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.007s	sys 0.010s
I20260709 12:26:58.958834 10017 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:58.959477 18071 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:58.959810 17393 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:26:58.921136 17617 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:58.997301 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 400690us
I20260709 12:26:58.990980  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:58.953841 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:58.882457 18075 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:59.010820  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 53369.7us
I20260709 12:26:58.931921 17257 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:26:58.957297 18019 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:26:59.068807  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 77879.4us
I20260709 12:26:59.075234 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:59.015146  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:59.048846  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:59.092252 10093 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:26:59.093221 10093 raft_consensus.cc:606] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:26:59.093699 10093 raft_consensus.cc:614] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:26:59.096046  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81019.7us
I20260709 12:26:59.097146  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:59.099548 16661 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:59.100232 16661 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.102435  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:59.084694 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.009s	sys 0.010s
W20260709 12:26:59.103766 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.951s	user 0.009s	sys 0.010s
W20260709 12:26:59.104314 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 951274us
I20260709 12:26:59.092451 18047 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:26:59.013603 17936 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.107353 17617 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:59.107926 10670 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:26:59.108433 17936 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.108597  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:59.109030 17936 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:59.109669 17936 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:59.108045  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.099588 18011 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:59.113369 18011 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.113755 18011 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.114820 18053 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:26:59.115276 10672 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:26:59.115700 10672 pending_rounds.cc:85] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 14
I20260709 12:26:59.115943 10672 pending_rounds.cc:107] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 12:26:59.095278 17899 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:59.116110 18077 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:59.116873 18077 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.116712 17899 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.117378 18077 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:59.067579 18019 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.118269 16661 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:59.118537 18019 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:58.956655 18013 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.116168 10672 raft_consensus.cc:2889] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:59.047715 18007 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.014s	sys 0.004s
I20260709 12:26:59.144502 10671 pending_rounds.cc:85] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 14
I20260709 12:26:59.109043 17865 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:26:59.177881  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 75738.3us
I20260709 12:26:59.179841  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:59.165004 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 59848.2us
W20260709 12:26:59.170743 18018 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.504s	user 0.014s	sys 0.000s
I20260709 12:26:59.100236 17968 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:59.063704 17793 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.105s
I20260709 12:26:59.011576 17914 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.119603 18058 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:26:59.173623 10019 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:59.107398 18002 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:59.063869 17846 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.089s
I20260709 12:26:59.174700 10671 pending_rounds.cc:107] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:26:59.227710 10671 raft_consensus.cc:2889] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:26:59.228911  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49204.3us
I20260709 12:26:59.063457 18041 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.048528 17393 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:59.166647 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:59.174520 18007 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.047s	user 0.015s	sys 0.006s
I20260709 12:26:59.162678 18009 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.010062 18068 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.173715 18016 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:26:59.216791 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 321833us
I20260709 12:26:59.261101 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:26:59.262351 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:26:59.048334 18071 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.175304 18059 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.175139 18020 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:26:59.229702  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:59.230052 10093 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:59.173382 17617 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:26:59.212927 17899 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.277285 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.272559 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:59.305281 10017 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:59.215824 18018 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.252782 18008 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:59.279981 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:59.309131 10668 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:59.281212 18071 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:26:59.283064 18084 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:26:59.272373 17919 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:59.285463 10674 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:26:59.339344 10674 pending_rounds.cc:85] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 15
I20260709 12:26:59.339704 10674 pending_rounds.cc:107] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:26:59.339936 10674 raft_consensus.cc:2889] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:26:59.287352 18016 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:26:59.287889 18002 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:26:59.288743 17838 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:26:59.289139 17936 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.303804 18076 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:26:59.209461 17865 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.226480 18053 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:26:59.278535 10676 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:26:59.306895 10063 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.001s	sys 0.016s
W20260709 12:26:59.394234 10063 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.170s	user 0.002s	sys 0.016s
I20260709 12:26:59.307322 10677 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:26:59.307725 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2912) took 1671 ms. Trace:
I20260709 12:26:59.395432 10092 rpcz_store.cc:276] 0709 12:26:57.636754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.754397 (+117643us) service_pool.cc:224] Handling call
0709 12:26:57.834844 (+ 80447us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:26:57.914513 (+ 79669us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:57.914566 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:26:57.929940 (+ 15374us) log.cc:844] Serialized 52 byte log entry
0709 12:26:58.284075 (+354135us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:26:58.284114 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:58.284185 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:58.422613 (+138428us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:26:58.828073 (+405460us) raft_consensus.cc:1685] finished
0709 12:26:58.828101 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.307554 (+479453us) spinlock_profiling.cc:243] Waited 78.3 ms on lock 0x7b540014a0d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:59.307672 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95395,"raft.queue_time_us":79235,"raft.run_cpu_time_us":18239,"raft.run_wall_time_us":512013,"spinlock_wait_cycles":191519360,"thread_start_us":317890,"threads_started":1,"wal-append.queue_time_us":552707}
I20260709 12:26:59.308085 10018 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:59.310520 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:59.313431  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 83830.3us
I20260709 12:26:59.284507 18068 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.317255 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:59.317399 17644 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:26:59.319561 18050 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.790s	user 0.000s	sys 0.011s
I20260709 12:26:59.320528 18075 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.298902 18059 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.340724 10017 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:59.338296 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:59.435433  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:59.436231 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 98010us
I20260709 12:26:59.362871  9941 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:59.364848 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:59.437599 10668 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:59.366005 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.367811 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:26:59.439013 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71286.4us
I20260709 12:26:59.440804 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:59.271678 17874 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:26:59.260371 18009 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.370662  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1713) took 1881 ms. Trace:
I20260709 12:26:59.444644  9945 rpcz_store.cc:276] 0709 12:26:57.471164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.471285 (+   121us) service_pool.cc:224] Handling call
0709 12:26:59.352212 (+1880927us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b540003e154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:59.352368 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3118205696}
I20260709 12:26:59.289578 18083 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:26:59.372572  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 262249us
I20260709 12:26:59.374059 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.270864 18041 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.374861 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.452167 10095 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:59.379560 18047 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:26:59.401628  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:59.455231  9944 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:59.389645  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.360988 18082 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:59.429701  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.370221 17919 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.427729  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.436954 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:59.314916 18008 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.476197  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:59.438836 10094 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:26:59.436453  9940 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:59.450021  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:59.378712 18076 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:59.450692 10093 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:26:59.383088 18002 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:26:59.377377 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:59.453570 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 191303us
I20260709 12:26:59.308643 17393 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.460310  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.479604 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:26:59.448813 18058 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:59.486991 16661 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:59.494961 16661 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:26:59.484036  9947 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:59.496443  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.497004 10094 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:26:59.496786  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.504577  9945 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:26:59.506464 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:26:59.508308 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:59.509848  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.511453 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.511579 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.512607 10022 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:26:59.481160 17846 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:26:59.511612 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.513837 10094 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:59.514132 10095 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:26:59.516794 16661 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.519014 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.520002 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.520975 10093 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:59.468918 17644 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.058s
I20260709 12:26:59.462981 18079 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:26:59.512955 18063 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:59.522572  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.524082 10098 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:26:59.524793 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.530191  9944 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:59.525892 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:59.527824  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:26:59.512436 18090 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 12:26:59.497177 18086 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.334s	user 0.001s	sys 0.001s
I20260709 12:26:59.538558  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.539490  9940 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:26:59.540136 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.541095 18020 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:59.480893 18100 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.491914 18071 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:26:59.492753 17617 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:26:59.522564 18079 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.494395 18058 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:26:59.553233 10021 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:59.532825 18063 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:59.488128 17914 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:59.481580 17919 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.601800  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.493462 18076 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:26:59.536036 17907 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.005s
I20260709 12:26:59.512903 17874 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:26:59.610129 17874 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.610550 17874 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:26:59.564374 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:59.457671 18083 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.566709 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:26:59.621711 10094 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:26:59.567262 18068 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:26:59.568507  9946 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:26:59.569305 17928 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:59.555836 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.575361 17617 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.585582  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:59.637611  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.586715  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.561877 18020 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:26:59.608146  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:26:59.615049 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50784us
I20260709 12:26:59.616348 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:26:59.616773 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.617165 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.617360 17619 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.618347 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.619048 17914 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:26:59.620041 17899 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:26:59.654556 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.564699 18075 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:59.625351 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.566108 18071 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.503265 17936 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.065s
W20260709 12:26:59.628500  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 229410us
I20260709 12:26:59.681726  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:59.482137 18082 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.685372 18082 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:59.579353 18058 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:26:59.634538 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:59.691008 10671 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:26:59.692442 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:59.693974 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:26:59.638968  9947 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:26:59.695777  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.697131  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.697799  9947 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:26:59.580468 18053 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.698951 18053 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.701712 18053 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.645732 10023 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:26:59.646216 10017 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:26:59.707762 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.708704 10017 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:26:59.709825 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.601150 18063 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.646440 10021 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:26:59.716374 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.717941 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.647405  9942 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:26:59.650112 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 209378us
I20260709 12:26:59.719662 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:59.721941 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.650890  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:26:59.601310 17921 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.722843 17921 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.723991 10017 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:26:59.724862 10671 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.725044 18021 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:26:59.725299 17921 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.523200 18016 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.725579 18021 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.724011 18085 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:59.727211 18085 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.727572 18085 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:26:59.727532 17907 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:26:59.625311 18068 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.729005 17907 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:26:59.729215 18068 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:26:59.730099  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48485.7us
I20260709 12:26:59.665433  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.621850 18013 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:59.733348 18021 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:59.679176 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.646890 17617 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:26:59.735733 10092 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:26:59.736835 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.738422 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:26:59.734407  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:59.742734 18016 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:26:59.747416  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.748443  9946 raft_consensus.cc:2410] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:26:59.649932 17846 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:59.748317  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:26:59.673257 18091 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:26:59.756222 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:26:59.674891 18030 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.673s	user 0.007s	sys 0.004s
I20260709 12:26:59.620684 18093 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.146s
I20260709 12:26:59.666158 18075 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:26:59.679605 18071 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:26:59.566287 18100 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.756237 17846 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:26:59.761111 18101 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:26:59.753448 17865 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.761415 18016 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:26:59.772279  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:59.775637  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:26:59.785902  9943 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:26:59.781538  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.786501 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:26:59.664412 17914 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:26:59.788851 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 155515us
I20260709 12:26:59.789654 10023 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:26:59.621245 18047 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 22.19, Last appended by leader: 19, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.767117 17708 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.665089 17619 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.766742 18075 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.800491  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:26:59.800734 18080 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:26:59.749456 18002 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:26:59.805545 18019 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.638s	user 0.004s	sys 0.004s
I20260709 12:26:59.805711 18085 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.782704 17846 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:26:59.785893 18009 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:26:59.764832 18030 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.768011 17617 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.813513 10095 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
W20260709 12:26:59.950932  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:26:59.921633 10671 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:26:59.934289 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:26:59.936342  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2751) took 2287 ms (client timeout 1679 ms). Trace:
I20260709 12:26:59.794185 17911 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:26:59.988068  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2765) took 1154 ms. Trace:
W20260709 12:26:59.989603  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 176416us
W20260709 12:26:59.986208  9941 rpcz_store.cc:269] 0709 12:26:57.648779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.649082 (+   303us) service_pool.cc:224] Handling call
0709 12:26:59.936125 (+2287043us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540042de54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:26:59.936244 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3856066304}
I20260709 12:26:59.815959 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:26:59.988925  9949 rpcz_store.cc:276] 0709 12:26:58.833559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.835033 (+  1474us) service_pool.cc:224] Handling call
0709 12:26:58.835324 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.835670 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.835700 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.836161 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.177211 (+341050us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:26:59.177269 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.177341 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:26:59.987816 (+810475us) raft_consensus.cc:1685] finished
0709 12:26:59.987839 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:26:59.987995 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":265,"raft.run_cpu_time_us":84,"raft.run_wall_time_us":83,"thread_start_us":340640,"threads_started":1,"wal-append.queue_time_us":539645}
W20260709 12:26:59.924055 18098 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:26:59.898486 17979 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.024s	user 0.010s	sys 0.005s
W20260709 12:26:59.998453 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 182564us
I20260709 12:27:00.007558 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:26:59.906445 18081 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
W20260709 12:26:59.922732 18096 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
W20260709 12:26:59.924823 18115 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:26:59.923419 18103 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
W20260709 12:26:59.824085 17418 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.832s	user 0.007s	sys 0.006s
W20260709 12:26:59.924691 18077 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.807s	user 0.011s	sys 0.002s
I20260709 12:26:59.807976 17921 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:00.005125  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:59.849910 17865 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.899780 18058 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:26:59.828987 18016 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:26:59.893205 17963 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:26:59.940366 17919 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:59.929822 17914 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.051554 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.052870 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:26:59.926122 18085 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.017028 16661 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:00.019059 10022 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:27:00.025581 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.058442 10671 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:00.059494 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.060540 10671 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:26:59.912111 17619 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:26:59.808976 18053 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.193s
I20260709 12:26:59.809865 18007 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.199s
I20260709 12:26:59.997794 18079 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.062680 17779 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.063300 17779 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.063899 18058 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.065351 18030 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.066458 17617 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.067556 18071 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.065421 17779 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.070267 18071 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.071162 17617 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.071290  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:00.071745 17617 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:00.072526 18011 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.958s	user 0.008s	sys 0.004s
I20260709 12:27:00.072949 18011 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.074883 10024 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:00.076012 10674 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:27:00.076294 17949 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:00.076783 10025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3830) took 2696 ms (client timeout 2101 ms). Trace:
W20260709 12:27:00.077193 10025 rpcz_store.cc:269] 0709 12:26:57.380819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.380989 (+   170us) service_pool.cc:224] Handling call
0709 12:27:00.076546 (+2695557us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b54001c5054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:00.076683 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3316644608}
I20260709 12:27:00.078675 10090 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db738483c1444059b752648d302dc0d9"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:00.076330 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.079383 10090 raft_consensus.cc:606] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:00.079747 10024 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:00.079809 10090 raft_consensus.cc:614] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:00.080404 10018 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 12. (term mismatch)
I20260709 12:27:00.080770 10018 pending_rounds.cc:85] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 11
I20260709 12:27:00.081028 10018 pending_rounds.cc:107] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 24.12
I20260709 12:27:00.081281 10018 raft_consensus.cc:2889] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:00.081836  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:00.081933  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:00.082764 18030 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:00.083145 18030 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.083474 18030 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.078321 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:26:59.808264 17708 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:26:59.988942 17911 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:00.012135 18083 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:26:59.914114 18112 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.265s
W20260709 12:27:00.101727 17804 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.132s	user 0.014s	sys 0.000s
I20260709 12:27:00.112066 17804 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.040321  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:26:59.983799 18019 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.040990 18016 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:00.044591 18110 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 12:27:00.045254 17963 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.041594 17899 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:00.047333 18117 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:00.092805 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.033916 17418 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.038697 17865 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.080771 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.059504 18100 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:00.101801  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94205.5us
I20260709 12:27:00.138062 17865 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.137780 17899 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.101046 18041 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.110424 18057 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.107460 17911 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.040040 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:00.168169 10673 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.041774 17979 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:00.115932  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 75655.7us
I20260709 12:27:00.169693  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:00.096628 10671 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:00.132138  9944 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:00.132318 18014 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:00.034407 18077 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.158489 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2927) took 1252 ms. Trace:
I20260709 12:27:00.180142 10091 rpcz_store.cc:276] 0709 12:26:58.905746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.098839 (+193093us) service_pool.cc:224] Handling call
0709 12:26:59.099149 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.099522 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.099552 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.100030 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.314215 (+214185us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:26:59.314288 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:26:59.314362 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.157861 (+843499us) raft_consensus.cc:1685] finished
0709 12:27:00.158263 (+   402us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.158427 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":134994,"threads_started":1,"wal-append.queue_time_us":304440}
I20260709 12:27:00.182474  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.183183  9949 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:27:00.183979  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.160882 10016 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:00.128551 18016 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:00.174481  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:00.184913 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.184929  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.185251  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:00.159053 17874 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.003s
I20260709 12:27:00.191532  9949 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:00.193063  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.195500  9941 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:00.193634  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.196250  9947 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:00.197258  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.197770  9947 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:00.199258 10670 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:00.199607 18075 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.201857 18075 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.157572 18063 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.204787 18063 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.206630 17418 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.194553  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:00.194159  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.208560  9945 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:00.209596 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.116302 17921 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.060s
I20260709 12:27:00.158739 17907 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:00.185283 18020 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.518s	user 0.010s	sys 0.000s
W20260709 12:27:00.216617 18127 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:27:00.131992 18068 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.218817 18018 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.219879 17804 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.218817 18039 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.221268 18068 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.213280 10673 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:00.215292 17921 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:00.186215 18011 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.225379 10674 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:27:00.215627 18126 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:00.215355 17907 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.216318 18020 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.214676 18104 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:27:00.222463 17921 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:00.158046 17779 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 12:27:00.207981 18041 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:00.230404 18126 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.294915 18126 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:00.226470 18117 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:27:00.280444 10024 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:00.287572 18019 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.290620  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.298521  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.299166  9941 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:00.290975  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.225011 18063 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:00.294859 18041 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:00.303587  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 96548us
I20260709 12:27:00.270128 18077 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:00.301972 18076 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.000s
I20260709 12:27:00.212256 17874 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.272187 17921 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:00.301687 18063 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.303316 18041 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:00.289737  9945 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:00.336201  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.336907  9945 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.337852  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.289238 17907 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.299191 17617 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:00.306252 18004 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.335s	user 0.001s	sys 0.013s
W20260709 12:27:00.315681  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 121191us
I20260709 12:27:00.310503  9947 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:00.300194 17914 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:00.339948  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 132381us
W20260709 12:27:00.340812  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.363410  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:00.347772  9948 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:00.323545 18063 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:00.365622 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:00.366396 10674 raft_consensus.cc:2368] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23.
I20260709 12:27:00.386417 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.349237 18085 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:00.322763 18076 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.400894 17922 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.401365 18076 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:00.401556 18132 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:27:00.402715 17257 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:00.403200 17257 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:00.403527 17874 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.404261 17257 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.406034 17257 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:00.407433 18133 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:00.408295 18022 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.385288 17914 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.398699 18093 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:27:00.412606 17914 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.414364 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:00.415571 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.416157 10024 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:27:00.399287 18085 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.417248 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:00.417974 10024 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:00.419013 18039 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:00.402634 10021 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:27:00.424113 18085 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.431370 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:00.434060 18136 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:00.435035 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:00.436406 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:00.437554 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:00.438087 10092 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.443279 18008 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.442943 17921 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.444259 17921 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:00.410341 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.445766 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.446303 10017 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:00.410293 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2933) took 1242 ms. Trace:
I20260709 12:27:00.434053 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:00.447106 10098 rpcz_store.cc:276] 0709 12:26:59.167974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.336029 (+168055us) service_pool.cc:224] Handling call
0709 12:27:00.410101 (+1074072us) spinlock_profiling.cc:243] Waited 40.7 ms on lock 0x7b54002fce54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:00.410231 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93517696}
I20260709 12:27:00.447644 10091 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.449405 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.452459 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:00.388792 18078 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.174s	user 0.007s	sys 0.013s
I20260709 12:27:00.457702 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.464148 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:00.391695 18095 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:00.475203 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:00.475807  9941 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:00.396890 18068 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:00.365639 18004 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.395102 17708 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.491138 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1473) took 1577 ms. Trace:
I20260709 12:27:00.492199 16661 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:00.494168 18075 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:00.461714 18117 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:00.491650 10019 rpcz_store.cc:276] 0709 12:26:58.913742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.913911 (+   169us) service_pool.cc:224] Handling call
0709 12:27:00.490909 (+1576998us) spinlock_profiling.cc:243] Waited 217 ms on lock 0x7b54001c5f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:00.491073 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":498455296}
I20260709 12:27:00.496518 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.497087 10019 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:00.497905 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.492837 16661 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.498492 10019 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:27:00.494334 18068 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:00.463292 18078 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.250s	user 0.007s	sys 0.014s
I20260709 12:27:00.512204 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.482278  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:00.485832  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.398000  9945 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:00.474341 17963 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.536188 10091 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:00.484290 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:00.511541 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.543746 10098 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.539724  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.544274 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:00.543856 10017 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:00.544883 10091 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:27:00.550650 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3836) took 2999 ms (client timeout 1576 ms). Trace:
I20260709 12:27:00.548003  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:00.551311 10020 rpcz_store.cc:269] 0709 12:26:57.551231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:57.553582 (+  2351us) service_pool.cc:224] Handling call
0709 12:27:00.550323 (+2996741us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54001c6954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:00.550529 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3719155456}
I20260709 12:27:00.498317 18083 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:00.498469 18112 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.478924 17899 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 12:27:00.573150 18112 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.580540 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:00.581300 10093 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:00.583947  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.584594  9940 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.543164 17911 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.586894 16661 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.588028 10021 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:27:00.590267 10021 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:27:00.592573  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.593490 18041 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.559551 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:00.482591 18030 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.541630 18008 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.566154 18057 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.569530 18136 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:00.503849 18126 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.570919  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:00.595904 17846 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.602s	user 0.006s	sys 0.003s
I20260709 12:27:00.573115 18059 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.582336 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:00.563494  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.593591 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.644277 16661 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:00.645853  9945 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:00.603484 10017 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:27:00.549500 18085 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:00.608191 18030 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:00.609028  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.609221 10024 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:00.559837 10092 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:27:00.649741 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56264.5us
I20260709 12:27:00.593781  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.613013 10098 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:00.635730 10020 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 17. Current term is 18. Ops: [17.12-17.12]
I20260709 12:27:00.571211 18068 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.644869 16661 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.613662 17865 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:00.597640 18004 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.650655  9948 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:00.569597 18063 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.662632  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.604893 10091 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:00.650719 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.656095 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:00.663719  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.667021 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.674692 10668 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:00.674708  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.634174 18112 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.676100  9948 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:00.676709 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1654) took 2095 ms. Trace:
I20260709 12:27:00.677064  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.677654 16661 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.680066  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.682919 10091 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:00.684346  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.690140 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:00.687193 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.649230 17914 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:00.697836 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.698777 10673 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:00.701392 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.703883 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:00.704934 10093 raft_consensus.cc:2368] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19.
I20260709 12:27:00.655537 18059 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:00.689465 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:00.690461  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.707458  9948 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:27:00.708508  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.709084  9948 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:00.692685 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.694707 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:00.689754 16661 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:00.711503 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.634311 18016 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.708830  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.711510 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:00.724999 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.634549 18008 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.727476 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:00.727955 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:00.696204 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:00.677256 10669 rpcz_store.cc:276] 0709 12:26:58.581518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.581974 (+   456us) service_pool.cc:224] Handling call
0709 12:26:58.601079 (+ 19105us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:58.752347 (+151268us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:58.752380 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:58.752837 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:26:59.164915 (+412078us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:26:59.165182 (+   267us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.659526 (+1494344us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400611a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:00.659576 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.659631 (+    55us) raft_consensus.cc:1685] finished
0709 12:27:00.659642 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.676634 (+ 16992us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7117,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":785,"spinlock_wait_cycles":347049216,"thread_start_us":411221,"threads_started":1,"wal-append.queue_time_us":500816,"wal-append.run_cpu_time_us":1784,"wal-append.run_wall_time_us":1032307}
I20260709 12:27:00.731076 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:00.731180 18126 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.731761 10020 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:00.725365 17874 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:00.732693 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4115) took 1735 ms. Trace:
I20260709 12:27:00.733052 10097 rpcz_store.cc:276] 0709 12:26:58.997637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.346520 (+348883us) service_pool.cc:224] Handling call
0709 12:26:59.346969 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.366943 (+ 19974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.366991 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.406771 (+ 39780us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.224284 (+817513us) spinlock_profiling.cc:243] Waited 5.78 ms on lock 0x7b180045e3e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:00.391585 (+167301us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:00.391628 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.391699 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.711102 (+319403us) raft_consensus.cc:1685] finished
0709 12:27:00.711124 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.732615 (+ 21491us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11621,"raft.queue_time_us":49388,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":9677,"spinlock_wait_cycles":13291392,"thread_start_us":167100,"threads_started":1,"wal-append.queue_time_us":1158150}
I20260709 12:27:00.735083  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.735433 10097 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:00.735675  9948 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:00.740446 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45857.4us
I20260709 12:27:00.741505  9941 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:27:00.743122 10091 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:00.744458  9948 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:00.745832  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.747144  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.688560 18030 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:00.748188  9948 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:00.651013 17846 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.739588 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.758633 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:00.760056 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.685400 17911 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.711997 18083 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:00.767802 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.768404 10668 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:00.692031 17899 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:00.688269 18011 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:00.739337 17963 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:00.758013 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.762786 16661 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:00.777532 16661 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:00.777932 16661 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:00.763432 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1706) took 1677 ms. Trace:
I20260709 12:27:00.751592 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:00.751413 18112 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:00.779449 10677 rpcz_store.cc:276] 0709 12:26:59.085973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.086375 (+   402us) service_pool.cc:224] Handling call
0709 12:26:59.086803 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.395088 (+308285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.753719 (+1358631us) spinlock_profiling.cc:243] Waited 218 ms on lock 0x7b54004c9ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:00.763352 (+  9633us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500514560}
I20260709 12:27:00.658710 18085 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.765679  9947 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:00.766794  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.769407 17921 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.767468 18058 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:00.767604 18002 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:00.745920 17922 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:00.776575 18011 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.765151  9945 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:27:00.788673  9940 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:00.788774 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.799345  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.799823 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.800377 10017 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:00.757470 18126 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:00.801390 18126 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:00.776809 17963 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.755909 18030 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.802918 17963 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:00.802979 18030 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:00.803583 18016 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.751276 18059 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:00.809509 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2639) took 1176 ms. Trace:
I20260709 12:27:00.809743 18059 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.809976 10018 rpcz_store.cc:276] 0709 12:26:59.632791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.103577 (+470786us) service_pool.cc:224] Handling call
0709 12:27:00.103933 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.114848 (+ 10915us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.114884 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.124661 (+  9777us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.608674 (+484013us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:00.608714 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.611016 (+  2302us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.809269 (+198253us) raft_consensus.cc:1685] finished
0709 12:27:00.809293 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.809446 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":79,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":12759,"thread_start_us":435547,"threads_started":1,"wal-append.queue_time_us":625702}
I20260709 12:27:00.811039 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:00.811638 10676 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:00.712935 18066 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.813279 18066 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.814203 17846 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.793926 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:00.756738 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:00.772821 18083 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:00.816659 18083 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:00.789762 17899 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:00.780236 17617 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:00.726707 17914 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:00.800947 18149 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:27:00.818392  9943 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:00.824373  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.797432 18058 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:00.819645 18011 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:00.831723 17617 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:00.798947 17921 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.802263 18126 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.842497 18058 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:00.845698 16661 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.823853 17922 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:00.849571  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.879132 10676 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:00.864053  9943 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:00.871068 10020 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:00.879659  9948 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:27:00.856261  9942 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:00.883020  9942 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:00.887981 16661 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.837846 18041 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.852036 18063 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:00.849774 18020 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.848687 18117 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:00.870184 18002 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:00.894274 16661 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:00.850742 17921 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.847687 17617 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:00.923501  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.932319 17914 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.933565 17922 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.882714 18066 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:00.896132 18154 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:27:00.955847  9948 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:00.957484 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:00.960371 18008 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.961584 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1429) took 1315 ms. Trace:
I20260709 12:27:00.961452 17963 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.966248  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:00.975481 17911 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:00.976117 17911 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.976914 17963 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:00.969677 18011 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:00.979025 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2935) took 1617 ms. Trace:
I20260709 12:27:00.978956  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.980224  9941 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:00.979504 10096 rpcz_store.cc:276] 0709 12:26:59.361719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.459868 (+ 98149us) service_pool.cc:224] Handling call
0709 12:26:59.487615 (+ 27747us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:26:59.690389 (+202774us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:26:59.888582 (+198193us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.002997 (+114415us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.003668 (+   671us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:00.003725 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:00.003776 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.978819 (+975043us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54002fd354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:00.978861 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.978971 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7076,"prepare.queue_time_us":583651,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":11799,"raft.queue_time_us":896,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":42093,"spinlock_wait_cycles":473000064,"thread_start_us":265777,"threads_started":2,"wal-append.queue_time_us":153728}
I20260709 12:27:00.980988 16661 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:00.982210 17911 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:00.982777 18112 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:00.983083  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 3042) took 1394 ms. Trace:
I20260709 12:27:00.982999  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:00.983322 18112 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.983719  9944 rpcz_store.cc:276] 0709 12:26:59.588944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.603269 (+ 14325us) service_pool.cc:224] Handling call
0709 12:27:00.982909 (+1379640us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b54001a40d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:00.983031 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116795648}
I20260709 12:27:00.984089  9949 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:00.985141 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1659) took 1692 ms. Trace:
I20260709 12:27:00.985613 18112 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:00.986114 10095 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1451) took 1376 ms (client timeout 1700 ms). Trace:
I20260709 12:27:00.986735 16661 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:00.986975 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2635) took 1360 ms. Trace:
I20260709 12:27:00.987561 16661 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:00.962118 10671 rpcz_store.cc:276] 0709 12:26:59.646098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.061111 (+415013us) service_pool.cc:224] Handling call
0709 12:27:00.061441 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:00.170798 (+109357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.961308 (+790510us) spinlock_profiling.cc:243] Waited 34.9 ms on lock 0x7b540050a5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:00.961519 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80267520}
W20260709 12:27:00.989324 10016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3843) took 2047 ms (client timeout 2718 ms). Trace:
W20260709 12:27:00.989696 10016 rpcz_store.cc:269] 0709 12:26:58.941514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.984193 (+ 42679us) service_pool.cc:224] Handling call
0709 12:27:00.989146 (+2004953us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54001c6454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:00.989274 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2703056128}
I20260709 12:27:00.994900  9949 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:01.001659 17911 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:01.002331 17911 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.021908  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:00.986495 10095 rpcz_store.cc:269] 0709 12:26:59.609643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.677720 (+ 68077us) service_pool.cc:224] Handling call
0709 12:27:00.985962 (+1308242us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b54002fc454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:00.986060 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124590464}
I20260709 12:27:01.023272  9942 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:27:00.987320 10022 rpcz_store.cc:276] 0709 12:26:59.626232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.632208 (+  5976us) service_pool.cc:224] Handling call
0709 12:27:00.986826 (+1354618us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54002161d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:00.986924 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":831099136}
I20260709 12:27:01.024742  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:01.025488  9947 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:01.030725  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:00.892079 18093 raft_consensus.cc:2764] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election decision vote started in defunct term 18: won
I20260709 12:27:01.033134  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:01.034871  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 68716.6us
I20260709 12:27:01.046904 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:01.047679  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:01.039186  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:01.028738 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54165.6us
W20260709 12:27:01.033924  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66956.5us
I20260709 12:27:01.052064 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:01.053334 17911 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:01.032523 18008 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:01.054169 18008 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.056381 18008 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:01.057824  9941 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:01.058933 18022 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:01.026465  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:00.985517 10672 rpcz_store.cc:276] 0709 12:26:59.292300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:59.331297 (+ 38997us) service_pool.cc:224] Handling call
0709 12:26:59.332099 (+   802us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:26:59.870255 (+538156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:26:59.870290 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:26:59.888465 (+ 18175us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.011002 (+122537us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:00.011042 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.984778 (+973736us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54006d01d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:00.984825 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.984869 (+    44us) raft_consensus.cc:1685] finished
0709 12:27:00.984882 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:00.985084 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4099,"raft.queue_time_us":113188,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":356,"spinlock_wait_cycles":728497408,"thread_start_us":324936,"threads_started":2,"wal-append.queue_time_us":384222}
I20260709 12:27:01.064949 10094 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:01.068194 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:01.061026  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:01.110850 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:01.111304  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50386us
I20260709 12:27:01.111806 10669 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:01.112046  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:01.074697 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.080188 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:01.112757 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:01.090884  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:01.092782 17788 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:01.097761 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:00.977474 17393 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:00.846887 17899 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:01.048306 18156 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.054s	user 0.002s	sys 0.000s
I20260709 12:27:01.099143 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.150444 10094 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:01.108439 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.087023 10096 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:01.116794  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:01.119547 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.700s (SENT) [suppressed 9 similar messages]
I20260709 12:27:01.115074 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.098645 18008 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:01.075374 18110 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:01.117087  9947 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:01.014714 18002 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.138509  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:01.167227  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:01.147121 10097 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:27:01.156378 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:01.168816  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:01.158938  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 68163.4us
I20260709 12:27:01.160059 10099 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:27:01.159871 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 401288us
I20260709 12:27:01.160601 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:01.171422  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:01.172115 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:27:01.162200 10092 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:01.156565 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:01.125262 18020 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:01.096081 18030 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:01.165108  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48410.7us
I20260709 12:27:01.127252 18019 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:01.168185  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:01.174031 10024 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:27:01.174274 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:01.188815 10022 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:01.174026 10016 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:01.179812  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:01.181034 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:01.142283 18011 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:01.143257 18018 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.143267 18126 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:01.184731  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:01.149363 17788 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:01.190083 10093 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 17.
I20260709 12:27:01.212162  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:01.164556 17968 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:01.174808 18041 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:01.165861 18110 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:01.170388 18077 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:01.227238  9990 leader_election.cc:400] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 18 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19.
I20260709 12:27:01.182372 18020 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.292698 18020 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:01.241412 18153 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 12:27:01.241570 18053 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:01.173520 18031 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:01.242480 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:01.246995 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3846) took 1357 ms. Trace:
I20260709 12:27:01.248593 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:01.216225 17846 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:01.275874 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:01.247494 18018 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.279575 18058 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.225251 18126 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.173951 17979 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:27:01.173300 18022 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.097s
I20260709 12:27:01.225308 18144 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:01.286469  9946 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 23 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:27:01.287261 18083 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.288337 17921 raft_consensus.cc:2764] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
W20260709 12:27:01.289218  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 117862us
I20260709 12:27:01.269870 18110 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:01.269611 18096 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:27:01.270224 18078 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:01.186794 18019 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.302309 10093 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22.
I20260709 12:27:01.355396 10096 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:01.224331 10094 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:01.300657 10019 rpcz_store.cc:276] 0709 12:26:59.889506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.546794 (+657288us) service_pool.cc:224] Handling call
0709 12:27:00.547233 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:00.558413 (+ 11180us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:00.591396 (+ 32983us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.773107 (+181711us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:00.783797 (+ 10690us) log.cc:844] Serialized 52 byte log entry
0709 12:27:00.783966 (+   169us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:00.784692 (+   726us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.788620 (+  3928us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.788890 (+   270us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.788956 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.848084 (+ 59128us) spinlock_profiling.cc:243] Waited 7.46 ms on lock 0x7b18003e98c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:01.246704 (+398620us) raft_consensus.cc:1685] finished
0709 12:27:01.246729 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.246911 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25219,"prepare.queue_time_us":43930,"prepare.run_cpu_time_us":1160,"prepare.run_wall_time_us":2257,"raft.queue_time_us":10383,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":31703,"spinlock_wait_cycles":22071808,"thread_start_us":180713,"threads_started":1,"wal-append.queue_time_us":286789}
I20260709 12:27:01.317420 18058 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:01.319561 18126 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:01.324772 17874 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:01.335222 17907 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.362341 17874 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:01.363001 17907 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:01.346993 17865 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:27:01.363729 17874 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:01.329921 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.252766 18030 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:01.350795  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:01.357544 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.332069 17846 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:01.369846 10094 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:01.346979 18008 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:01.387089  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.388078 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.360306 18019 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:01.393179  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:01.393764 10098 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:01.360073 18078 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:01.365310 18002 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:01.399521  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:01.398953 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:01.400635 10098 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:01.381040 10094 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:01.362900 18083 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:01.383890 17393 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:01.405673 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.400817  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:01.395102 18057 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.011s
I20260709 12:27:01.407703 18056 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:01.407419 18078 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:01.408111 18056 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.408789 18058 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.408385 18008 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.409296 18058 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.360888 17788 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.411369 17788 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:01.411311 17968 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:01.412812 10094 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:27:01.402756 17921 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:01.414021 17921 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:01.414374 17921 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:01.383878 18031 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:27:01.406270 17393 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:01.416090 17393 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:01.415397 17921 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:01.417318 18161 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:27:01.355031 18014 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.043s
I20260709 12:27:01.408486 18056 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:01.406636 17999 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:01.497865 10094 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:01.496253 17999 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:01.503104 18085 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:01.530560 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 462414us
W20260709 12:27:01.538816  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:01.539685  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.506855 17999 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:01.535601 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:01.518628 17617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.017s	sys 0.008s
I20260709 12:27:01.549103 18002 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:01.555488 18019 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:01.487514 17846 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:01.596980 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 477441us
I20260709 12:27:01.570996 18058 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:01.576116 17617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.018s	sys 0.008s
W20260709 12:27:01.598315  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.560169 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:01.611148 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:01.582302 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:01.611346 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51266.3us
W20260709 12:27:01.614583 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.616531 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:01.617445 10099 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
W20260709 12:27:01.601300 16661 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.014s	sys 0.010s
W20260709 12:27:01.622385 16661 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.015s	sys 0.010s
I20260709 12:27:01.613409 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:01.629375 10023 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:01.623531 17919 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.013s	sys 0.013s
I20260709 12:27:01.631853 10094 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22.
I20260709 12:27:01.632074 10090 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:01.675910  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3051) took 1228 ms. Trace:
I20260709 12:27:01.739185  9945 rpcz_store.cc:276] 0709 12:27:00.447412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.823100 (+375688us) service_pool.cc:224] Handling call
0709 12:27:00.837002 (+ 13902us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.837371 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.837400 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.845390 (+  7990us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.339941 (+494551us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:01.339983 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.340055 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.675637 (+335582us) raft_consensus.cc:1685] finished
0709 12:27:01.675660 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.675843 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3692,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":19066,"thread_start_us":430422,"threads_started":1,"wal-append.queue_time_us":729009}
I20260709 12:27:01.676389 18020 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.683450 18019 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:01.606092 18066 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:01.610028 17911 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:01.757390  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 350074us
I20260709 12:27:01.695930  9941 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:01.752573 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:01.699649  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:01.701072  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 428901us
I20260709 12:27:01.702373  9948 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:01.703989  9944 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:27:01.763703  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.625196 18002 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:01.634428 17617 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:01.723031 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:01.785806 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:01.786499 10674 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:27:01.748411 17922 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.007s	sys 0.016s
I20260709 12:27:01.225495  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:01.797418 10669 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:01.585469 18157 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.107s
I20260709 12:27:01.757850 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:01.804556 10673 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:01.762897  9947 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:01.724939 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.796118 16661 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:01.631307 18041 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:01.737849 17919 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.013s	sys 0.013s
W20260709 12:27:01.800369 10642 leader_election.cc:336] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.006s (SENT)
I20260709 12:27:01.807040 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:01.796597 10670 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:01.806288 16661 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.815397  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:01.816270 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2641) took 2182 ms. Trace:
I20260709 12:27:01.815397 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:01.819124 10676 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:01.820621  9947 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:01.821316  9912 leader_election.cc:400] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23.
I20260709 12:27:01.821749  9912 leader_election.cc:403] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:27:01.816677 10025 rpcz_store.cc:276] 0709 12:26:59.633750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.107266 (+473516us) service_pool.cc:224] Handling call
0709 12:27:00.492445 (+385179us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.492969 (+   524us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:00.495020 (+  2051us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.762257 (+267237us) log.cc:844] Serialized 14 byte log entry
0709 12:27:00.765666 (+  3409us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.770514 (+  4848us) log.cc:844] Serialized 26 byte log entry
0709 12:27:00.770690 (+   176us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:00.770731 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:00.770794 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:00.861305 (+ 90511us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b18003d4b00. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:01.486474 (+625169us) raft_consensus.cc:1685] finished
0709 12:27:01.486518 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.562311 (+ 75793us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b18003d4b00. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:01.816093 (+253782us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b54001c5f50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:01.816212 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3173,"prepare.queue_time_us":5085,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":63,"raft.queue_time_us":7451,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":5531,"spinlock_wait_cycles":853317120,"thread_start_us":266063,"threads_started":1,"wal-append.queue_time_us":414559}
I20260709 12:27:01.818699 17899 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:01.785993 17617 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:01.846274 18002 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:01.843850 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:01.863890 10674 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:27:01.840991 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 76356.9us
W20260709 12:27:01.851876 10091 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.019s	sys 0.027s
I20260709 12:27:01.854979 10673 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
W20260709 12:27:01.865017 10091 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.019s	sys 0.027s
I20260709 12:27:01.865701 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2954) took 1309 ms. Trace:
I20260709 12:27:01.873744 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:01.874567 10673 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:01.866101 10091 rpcz_store.cc:276] 0709 12:27:00.556656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.743945 (+187289us) service_pool.cc:224] Handling call
0709 12:27:00.744302 (+   357us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:00.760697 (+ 16395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.760739 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:00.768104 (+  7365us) log.cc:844] Serialized 78 byte log entry
0709 12:27:01.111261 (+343157us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:01.157310 (+ 46049us) log.cc:844] Serialized 14 byte log entry
0709 12:27:01.335427 (+178117us) log.cc:844] Serialized 14 byte log entry
0709 12:27:01.865396 (+529969us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:01.865471 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:01.865512 (+    41us) raft_consensus.cc:1685] finished
0709 12:27:01.865525 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:01.865651 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199770,"prepare.queue_time_us":652720,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":78,"raft.queue_time_us":244818,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":283,"thread_start_us":827668,"threads_started":2,"wal-append.queue_time_us":700817}
I20260709 12:27:01.878324 10676 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:01.789335 18066 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.882155 10669 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:01.867957 18144 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.013s	sys 0.035s
W20260709 12:27:01.876706 18095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.020s	sys 0.029s
I20260709 12:27:01.863119 17617 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:01.825143 17999 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:01.858421 17928 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:27:01.927384 18078 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.018s	sys 0.053s
W20260709 12:27:01.915706 18057 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.014s	sys 0.056s
I20260709 12:27:01.875869 17922 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.059s
W20260709 12:27:01.924369 18170 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:27:01.926964 10673 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:01.810007 17911 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:01.917347 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:01.932731 16661 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:01.934254  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:01.926301 18020 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:01.848109 18056 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:01.933866 18078 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.019s	sys 0.053s
W20260709 12:27:01.923532 18095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.020s	sys 0.030s
I20260709 12:27:01.914281 17899 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:01.918766 18144 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:01.986541 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:01.962473 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:01.931092 17928 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:01.964761 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:01.996399 10676 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 24
W20260709 12:27:01.934639 18057 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.015s	sys 0.056s
W20260709 12:27:02.012591 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 399284us
I20260709 12:27:02.001582  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:01.937829 17393 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.521s	user 0.012s	sys 0.000s
I20260709 12:27:02.018317 10097 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:27:01.976788 18184 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:02.031757 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:02.019719 18110 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.001s
I20260709 12:27:02.011181 18008 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:02.028045 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:02.036615 10669 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:02.018952 18179 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:01.936288 18041 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:01.975035 17899 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.049412 10676 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:02.050172 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:02.052012  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 230892us
I20260709 12:27:02.036868 18057 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.071273 16661 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.072837 16661 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.069473 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.052785 10674 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:02.084261 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.082445 16661 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:01.977128 18002 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.097661 18085 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:02.099439 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:02.100800 18085 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:02.101377 18085 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:02.102319 18085 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.103060 10669 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:27:02.079193  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 77668.6us
W20260709 12:27:02.076069  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 260805us
I20260709 12:27:02.052333 18180 raft_consensus.cc:2764] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:27:02.091005  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 107717us
I20260709 12:27:02.043208 17644 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:27:02.042845 17393 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.075152 18171 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:02.113389 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.106710 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.083863 18176 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:02.115272 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:02.126611 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.085737 18095 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:02.087054 18157 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:02.108034 18007 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:02.046272 18184 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.121348 10019 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:02.050005 18110 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.068559 18179 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.055069 18008 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.118736 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.132365 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:02.152830 18177 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:27:02.135226 10025 raft_consensus.cc:2368] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22.
I20260709 12:27:02.120491 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.138149 18195 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.003s
W20260709 12:27:02.138723 18175 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:27:02.140595 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.128701 18174 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:02.206565 18020 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:02.069232 18144 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.209833 18020 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.210698 17914 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:02.211088 17914 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:02.117547 18171 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:02.133549 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:02.154524 10640 leader_election.cc:336] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.539s (SENT)
I20260709 12:27:02.172870 10024 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:02.212579 10020 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:02.174481 17393 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.176252 18110 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:02.211861 18020 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:02.148721 18002 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:02.141743 18095 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.222560 18095 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:02.177250 18056 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:02.223143 17911 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:02.209122 18174 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.178732 18184 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:02.213546 18171 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:02.179335 18179 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.204849 17999 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:02.238752  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.238899 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.239915  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.150830 17846 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.245456 18184 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:02.250916 18020 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:27:02.179654 18189 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:02.182180 18008 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:02.254141 18126 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:02.254851 18126 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:02.255914 18126 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.256713 18018 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:02.183051 18183 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:27:02.257521 18126 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.069128 17928 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.183441  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3066) took 1624 ms. Trace:
W20260709 12:27:02.185367 18158 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.825s	user 0.007s	sys 0.000s
I20260709 12:27:02.259133  9943 rpcz_store.cc:276] 0709 12:27:00.559233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.892384 (+333151us) service_pool.cc:224] Handling call
0709 12:27:00.892746 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:00.893175 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:00.893210 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:00.943767 (+ 50557us) log.cc:844] Serialized 26 byte log entry
0709 12:27:01.975295 (+1031528us) spinlock_profiling.cc:243] Waited 5.9 ms on lock 0x7b1800362b60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:02.046647 (+ 71352us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:02.076508 (+ 29861us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.097999 (+ 21491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.098094 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:02.183132 (+ 85038us) raft_consensus.cc:1685] finished
0709 12:27:02.183242 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.183377 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2783,"prepare.queue_time_us":31071,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":63808,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":3601,"spinlock_wait_cycles":13579008,"thread_start_us":91833,"threads_started":2,"wal-append.queue_time_us":1232854}
I20260709 12:27:02.260310 17922 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:02.193735 18066 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.199990 17899 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:02.209095 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3860) took 1054 ms. Trace:
I20260709 12:27:02.212440 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.262434 10023 rpcz_store.cc:276] 0709 12:27:01.154997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.155153 (+   156us) service_pool.cc:224] Handling call
0709 12:27:01.155444 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.629963 (+474519us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.208876 (+578913us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54002666d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:02.209041 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1089419520}
W20260709 12:27:02.263087 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 108619us
I20260709 12:27:02.264282 17899 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:02.264297  9945 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:02.173764 18204 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:02.266016  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.110431  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:02.266908  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.266732  9947 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:27:02.267623  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 157384us
I20260709 12:27:02.267416 18008 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:02.268360  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:02.268797 18030 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:02.269246 18030 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:02.269654  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.268386  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.271229  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.272410  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:02.272670  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.273155  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:02.273895  9946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 3056) took 1825 ms (client timeout 1852 ms). Trace:
I20260709 12:27:02.273958 17968 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:02.274444  9946 rpcz_store.cc:269] 0709 12:27:00.448566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:00.570632 (+122066us) service_pool.cc:224] Handling call
0709 12:27:02.273721 (+1703089us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b540042e354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:02.273840 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490518400}
I20260709 12:27:02.274634 17968 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.276104  9946 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:27:02.277451  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:02.277969  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.278996  9947 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:02.279747 18185 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:02.267510 17999 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:02.272958  9944 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:02.283085  9942 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:27:02.250469  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:02.288240  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.289013  9942 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:02.292194  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.294402 17846 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.297470 17846 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:02.309895  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 185747us
I20260709 12:27:02.235935 17907 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:02.310380  9947 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:02.321856 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:02.315346  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:02.268308 18066 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:02.281113 17999 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.341224 17999 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.246097 18184 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.343580 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.328473 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.344833 10097 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:27:02.328747  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.329404 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:02.331045 10642 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.014s (SENT)
I20260709 12:27:02.333410 10017 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:02.345835 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:02.334966 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:02.346715 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.347100 10024 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:02.347350  9943 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:02.337783 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:02.338932 18174 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.329689  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:02.336018 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.355831  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:02.309525 18018 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:02.318982 18095 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:02.364704  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:02.371560 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.364055 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.270147 18030 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.211925 17914 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.363312  9944 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:02.372587  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:02.376622  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.345443 17649 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:02.379331 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.378911 18093 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:02.386001 17914 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.386452 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:02.387451 18196 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.401841  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:02.373055 18174 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:02.365420 17846 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:02.379320 18018 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:02.394745  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:02.418342  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 357095us
I20260709 12:27:02.329970 18056 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:27:02.405453 18196 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.403688 18093 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:02.409722 18171 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.413599 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.426903 10092 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:02.360359 17899 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:27:02.375980 17963 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 12:27:02.379655 18078 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:02.391222 18095 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:02.400589 18030 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:02.404613  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:02.422506 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 289499us
I20260709 12:27:02.423626 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:02.439262 10022 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:02.439162 10095 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:02.440374 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:02.440785 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.448705 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:02.448952 10676 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
W20260709 12:27:02.449559 10642 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.586s (SENT)
I20260709 12:27:02.433490  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.451508 10097 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:27:02.452703 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:02.454010 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.454661 10097 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:02.446322 17257 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.455577 17257 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.417922 17649 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:02.450964 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:02.420040 17968 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.462321 10643 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:02.464293 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.461351 17257 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.464989 10669 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:02.419790 18174 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:02.467973 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.469406  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64897.4us
I20260709 12:27:02.469753 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:02.427814 17846 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:02.479092 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:02.509097 10673 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:02.524964 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.526119 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.467022 17911 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:02.526808  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:02.532116  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.518455 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.532768  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:02.527676 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48660.2us
I20260709 12:27:02.534950  9947 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:27:02.535416 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:02.536550 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:02.537227 10018 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:02.537343 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.536669 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.538261 10023 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:02.469758 17919 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.541404 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:02.544864  9947 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:02.470264 18009 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:02.493510 10097 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:02.507576 18191 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:02.557490 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:02.558175 10023 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:02.507817 18013 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:02.551838  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:02.512895 18041 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:02.563083 18041 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:02.563359 17911 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:02.564725 18009 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:02.564397 18041 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.20, Last appended by leader: 20, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.567237 10677 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:02.568419 10025 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:27:02.550388 10095 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:02.477363 17619 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:02.553378 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.573379 10019 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:02.574487 10023 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:27:02.574622 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.553954 10092 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:02.554535 18057 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23.
I20260709 12:27:02.565148 18009 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:02.517586 17649 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:02.539665 18179 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:02.579979  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.479036 18058 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.091s
I20260709 12:27:02.593112 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.593529 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.552613 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.586251 10019 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:02.546802 18157 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:02.576372 18095 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.544716 18018 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.593775 10025 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:02.587900 10018 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
I20260709 12:27:02.582834 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.575853 18185 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.591629  9946 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 24 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:27:02.594300 10092 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:02.577854 18013 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:02.595721 17649 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.575681 18016 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:02.595435 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:02.572036 18056 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:27:02.612836 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1456) took 1437 ms. Trace:
W20260709 12:27:02.591462 10671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.008s	sys 0.052s
I20260709 12:27:02.614806 10096 rpcz_store.cc:276] 0709 12:27:01.175726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.331452 (+155726us) service_pool.cc:224] Handling call
0709 12:27:01.355138 (+ 23686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.360806 (+  5668us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.612626 (+1251820us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b5400305f50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:02.612775 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53694848}
W20260709 12:27:02.615715 10671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.008s	sys 0.052s
I20260709 12:27:02.552408 17919 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.602156  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:02.611845 18174 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.621099 18056 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:02.622047 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.609851 18185 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.625771 18176 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:02.611784 18212 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:27:02.593277 18184 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.636936  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:02.611632 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:02.591507 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:02.644730 10668 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:02.651300  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:02.551666 18093 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.589350 17874 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:27:02.613015 18016 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.648056 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:02.644436 18056 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:02.649705 18185 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:02.651587 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60164us
I20260709 12:27:02.604580 18214 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:02.655965 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.667088 10096 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:02.602629 17846 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.681095 10668 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:02.644251 18219 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:02.684155 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.684836 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.695791 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.683095 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:02.662253 18212 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:27:02.698817 10668 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:02.658986 18184 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:02.655237 18007 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:02.661005 18016 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:02.706728 18056 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:02.699142 18068 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.019s	sys 0.019s
I20260709 12:27:02.697763 18013 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.700176 18179 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:27:02.683032 18171 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:02.705832 18184 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.711978 10097 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:02.726817 17257 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:02.656985 17922 raft_consensus.cc:2764] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
W20260709 12:27:02.716300 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.692497 18219 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:02.723335 10092 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
W20260709 12:27:02.712855  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.651726 18009 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.727983 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:02.731895 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.735827  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 447279us
I20260709 12:27:02.735821 10025 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:02.734064 10092 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:27:02.722437 18068 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.019s	sys 0.024s
I20260709 12:27:02.723869 18196 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.729247 18184 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.726636 18007 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:02.734978 18219 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.753321 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:02.744139 17617 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.808s	user 0.012s	sys 0.000s
I20260709 12:27:02.744244 18056 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:02.758419  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:02.760867 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 296998us
W20260709 12:27:02.762135  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.768910 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:02.778235  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.779402 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:02.793924  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.797561 18213 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.815121 18213 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.815685  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.816967  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:02.817344 18213 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.817543 17919 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:02.817637  9945 raft_consensus.cc:2368] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26.
I20260709 12:27:02.819509  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.820149  9945 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:02.821720  9945 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:02.770196 18207 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:02.798460 18068 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:02.759559 17617 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.800102 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:02.822746 18222 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.074s	user 0.001s	sys 0.001s
W20260709 12:27:02.822842 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43516.4us
I20260709 12:27:02.824681 10065 leader_election.cc:400] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26.
W20260709 12:27:02.825001 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45856.7us
I20260709 12:27:02.837678  9947 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:02.846987 10065 leader_election.cc:403] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:27:02.847460 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:02.843370 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:02.851902  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.827364 18207 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:02.817636 18007 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.848974 10025 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:02.861852 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 167667us
I20260709 12:27:02.862717 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:02.857388 18207 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.864575 18207 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:02.860850 18233 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:02.857666 18068 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:27:02.867535 18233 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:02.858246 17617 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:02.858860 10090 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:02.868795 10025 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:02.859627 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:02.854835 18126 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.597s	user 0.015s	sys 0.000s
W20260709 12:27:02.868520 18224 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:27:02.894366  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 123486us
I20260709 12:27:02.895358 10025 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:27:02.860160 17914 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26.
I20260709 12:27:02.895753 10025 pending_rounds.cc:85] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 12
I20260709 12:27:02.897116 10025 pending_rounds.cc:107] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:27:02.897389 10025 raft_consensus.cc:2889] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:02.899114 16661 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:02.890952 18239 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:02.900028 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1676) took 1132 ms. Trace:
I20260709 12:27:02.900609 10092 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:02.900492 10671 rpcz_store.cc:276] 0709 12:27:01.767500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.776426 (+  8926us) service_pool.cc:224] Handling call
0709 12:27:01.819481 (+ 43055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:01.819686 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:02.047622 (+227936us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.898840 (+851218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.898899 (+    59us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:02.898946 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:02.899850 (+   904us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:02.899977 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":360611,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":98875,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":5842,"thread_start_us":1047830,"threads_started":3,"wal-append.queue_time_us":634966}
I20260709 12:27:02.896320  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:02.868122 18233 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.868214 18228 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
W20260709 12:27:02.915591 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53013.3us
I20260709 12:27:02.886777 18191 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 12:27:02.887149 17617 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:27:02.921633  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:02.919668 17921 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:02.922005 18126 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:02.940090 18239 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:02.929286 18191 consensus_queue.cc:1243] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 17.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:02.950883  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.932092 18078 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:02.949649 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79497.7us
I20260709 12:27:02.935266 18176 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:02.938784 18242 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:02.926865 18232 raft_consensus.cc:2764] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
W20260709 12:27:02.955169 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:02.962546  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:02.943250 18126 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:02.964102 18016 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.954305 18247 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:27:02.975819 18247 consensus_queue.cc:1243] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:02.968250 18233 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:02.952626 18078 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:02.969161 18184 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:02.971446 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.973847 10020 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:02.974799 18126 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:02.980255 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:02.982417 18243 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:02.982926 18243 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:02.983807 18243 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:02.984948 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.986115 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:02.988485 18053 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:02.989076 18053 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:02.992679 17928 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:02.980319 18078 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.001065 10025 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: [18.14-18.14]
I20260709 12:27:03.002053 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:03.006722 17922 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.007397 17914 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.007305 17922 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.007882 17914 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.009536 17922 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.009742 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:03.010407 18232 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.011595 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:03.012694 18221 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.285s	user 0.000s	sys 0.001s
I20260709 12:27:03.012835 18157 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:03.013006 18219 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.013403 18157 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.014535 17874 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:03.015525 18157 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.015640 17874 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:03.016110 17874 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:03.016485 17874 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:03.017139 10016 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:03.018884 17617 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:03.019274 17617 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:03.020087 17617 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.021287 17617 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:03.022385 18227 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.241s	user 0.003s	sys 0.000s
I20260709 12:27:03.009779 17914 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:03.024263 18220 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.211s	user 0.001s	sys 0.004s
I20260709 12:27:03.025679 10017 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:02.974078 18176 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.026818  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:03.037961  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.027388 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.012712 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:03.040407 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:03.033406  9947 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:03.038534  9946 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:03.044636  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.046465 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:03.045197  9946 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:03.049401  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.050968 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.016110 18157 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:03.050566 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:03.048594 17914 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:03.068651 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:03.052192 18008 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:03.070968 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:03.074241 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.075804 10668 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:02.990103 18053 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.076287 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:03.078631 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:03.079317 17914 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:03.079792 17914 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.080196 17914 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.062191 18157 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:03.103032 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 152343us
I20260709 12:27:03.072053 18216 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:03.077239 18008 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.104873 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.101495 18157 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.103269 18053 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:03.118431 18232 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:03.120669 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.110136 18219 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.160094 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.170349 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:03.182848 10668 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:03.166141 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 87702.4us
W20260709 12:27:03.171342 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 120844us
I20260709 12:27:03.061390 18239 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.170955  9946 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:03.182973 10025 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:03.186558 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:03.187197 10097 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.114455 18253 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
I20260709 12:27:03.115221 18216 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:03.188186 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.192780 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.114207 18008 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.196548 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:03.200219  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.158385 18007 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.157100 17921 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:03.207316 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.169128 18191 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.205372 10668 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:03.212070 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.213117 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.218168 10097 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:03.205245 18144 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:03.185361 18232 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.230859 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.234822 10092 raft_consensus.cc:2376] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21. Re-sending same reply.
I20260709 12:27:03.251295 18016 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.251847 17921 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:03.253067  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.256253 18176 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.259547  9942 raft_consensus.cc:2410] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:27:03.195060 17979 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 12:27:03.269858 10025 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:03.291810 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:03.292368 10025 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:03.274240 10668 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:03.277110 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:03.295812 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:03.281626 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.283020  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:03.264618 18259 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:03.275581 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:03.290318  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.300869  9949 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:03.302165  9949 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:03.302165 10018 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:03.287958 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.298311  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:03.307322 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.308643 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.321013 10025 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:03.285266 17874 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.220250 18007 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.267040 17914 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.331877 10668 raft_consensus.cc:2410] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 20 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:27:03.333274  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:03.358377  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.302495 18004 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:03.286690 18126 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.364722 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.380430 10097 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:03.382319 18201 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:03.383496 18004 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:03.383939 17914 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:03.348694 18016 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:03.277014 18144 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.387571 17874 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.391460 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.393364 10017 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:03.332279 18019 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:27:03.357167 18219 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 12:27:03.407897 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:03.408254 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:03.414443 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:03.400609 18016 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:03.426563  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.401889 18201 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:03.443908 10675 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:27:03.446245 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:03.447973 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:03.453449 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 189279us
I20260709 12:27:03.440991 10668 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
W20260709 12:27:03.458436 18261 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:03.459167 18126 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.461298 17921 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.409091 18078 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.463039 18258 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.463877 18078 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.464509 18078 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:03.465068 18078 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.465435 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:03.466028 10670 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.466144 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1491) took 1356 ms. Trace:
I20260709 12:27:03.466522 10022 rpcz_store.cc:276] 0709 12:27:02.110173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.110360 (+   187us) service_pool.cc:224] Handling call
0709 12:27:02.123766 (+ 13406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.449789 (+326023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.465920 (+1016131us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54001392d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:03.466087 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":725132544}
I20260709 12:27:03.467021 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:03.468091 18267 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:03.468333 17914 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:03.469204 17914 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:27:03.469599 18267 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:03.470162 18267 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:03.472220 18232 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.472822 18157 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.474097 18144 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.474727 18144 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.425889 17899 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:03.445842 18259 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:03.475200 18144 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.475711 18259 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:03.475310 18267 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.476138 18259 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:03.477509 18144 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.463833 18009 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.478556 18004 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.480835 18007 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.482892 10091 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 25 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:27:03.463762 18258 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:27:03.476888 18157 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.486730 10670 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:03.454532 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:03.489471 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.443872 18016 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:03.492530 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.405345 18008 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.487926 18007 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.497586 18228 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:03.501437 10668 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: [23.17-24.18]
I20260709 12:27:03.493287 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:03.515115 10675 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:03.515682  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:03.522317  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:03.524475  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.525096 10095 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:03.487814 18232 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:27:03.530301  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.536842  9942 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:27:03.538022  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.531749 17257 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:03.531680 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:03.539152 17257 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.548314 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:03.552904  9942 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:03.558612 18185 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.559167 18185 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.561370 18185 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.567801 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4134) took 1555 ms. Trace:
I20260709 12:27:03.568281 10090 rpcz_store.cc:276] 0709 12:27:02.012722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.014800 (+  2078us) service_pool.cc:224] Handling call
0709 12:27:02.035192 (+ 20392us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.868722 (+833530us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.567531 (+698809us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54002fbf54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:03.567737 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893837056}
I20260709 12:27:03.572018 18214 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:03.572450 18214 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 18 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:03.573267 18214 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.554291 18241 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:03.575035 10093 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:27:03.510896 18007 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.580034 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.592026 10092 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:03.586997 17257 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.534572 18228 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:03.554955 18232 consensus_queue.cc:1243] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:03.591002  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:03.599086 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 152914us
I20260709 12:27:03.578781 18078 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.557446 18233 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.580746 18008 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.579217 17617 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:03.602516 18022 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:03.574564 18214 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.609939 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:03.610423  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:03.612097  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:03.570518 18016 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.617918  9941 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:03.620051 18016 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.597146 17257 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.534256 18201 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.621675 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:03.622963 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:03.624786 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:03.625138 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.622714 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1460) took 2288 ms. Trace:
I20260709 12:27:03.626223 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:03.624809 10092 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:03.615131 18185 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:03.623575 17922 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:03.631034 18078 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:03.607010 18022 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:03.636226 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:03.639900 10099 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:03.661870 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:03.607316 17617 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:03.675452 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:03.670827 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.675132 10097 raft_consensus.cc:2376] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24. Re-sending same reply.
I20260709 12:27:03.677773 10097 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:03.672137 10669 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:03.679164 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:03.620818 18195 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:03.625778 10098 rpcz_store.cc:276] 0709 12:27:01.334220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:01.401231 (+ 67011us) service_pool.cc:224] Handling call
0709 12:27:01.411210 (+  9979us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:01.411666 (+   456us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:01.786115 (+374449us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.076576 (+290461us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:02.080110 (+  3534us) log.cc:844] Serialized 78 byte log entry
0709 12:27:02.080255 (+   145us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:02.106840 (+ 26585us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.114419 (+  7579us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.119340 (+  4921us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.260356 (+141016us) spinlock_profiling.cc:243] Waited 6.08 ms on lock 0x7b1800207100. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:02.683391 (+423035us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b1800207100. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:03.484714 (+801323us) spinlock_profiling.cc:243] Waited 2.19 ms on lock 0x7b1800207100. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:27:03.622225 (+137511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.622341 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.622400 (+    59us) raft_consensus.cc:1685] finished
0709 12:27:03.622419 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.622620 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":779,"prepare.queue_time_us":461070,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":4033,"raft.queue_time_us":26282,"raft.run_cpu_time_us":648,"raft.run_wall_time_us":31041,"spinlock_wait_cycles":29202944,"thread_start_us":765918,"threads_started":4,"wal-append.queue_time_us":354124,"wal-append.run_cpu_time_us":3284,"wal-append.run_wall_time_us":1003347}
W20260709 12:27:03.652534 18265 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
W20260709 12:27:03.683398 18178 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.722s	user 0.028s	sys 0.008s
W20260709 12:27:03.698553 18178 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.754s	user 0.029s	sys 0.008s
I20260709 12:27:03.684800 18232 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:03.705596 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:03.706105 18250 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:03.706636 18247 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:03.707213 18247 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:03.708326 18232 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:03.709576 18232 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.714155  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:03.714888  9940 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.708124 18247 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.682375 18233 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:03.682993 18228 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:03.721382 18228 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.723627 18228 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.682444 17922 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.728099  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:03.729509  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:03.734769  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:03.741273 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.744818 10097 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:03.680320 18022 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:03.743602  9940 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:03.756520  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:03.758160 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:03.763041 18271 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:27:03.765321 18011 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:03.768680  9947 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:03.769999  9947 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:03.771433  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:03.771795 17649 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:03.772506 17649 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:03.773002 17649 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:03.773146  9943 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1746) took 1503 ms (client timeout 1592 ms). Trace:
W20260709 12:27:03.773538  9943 rpcz_store.cc:269] 0709 12:27:02.269451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.270502 (+  1051us) service_pool.cc:224] Handling call
0709 12:27:03.772969 (+1502467us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b54001b9c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:03.773092 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":168954240}
I20260709 12:27:03.774246  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3088) took 1024 ms. Trace:
I20260709 12:27:03.773869 17649 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.774837  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:03.775084  9943 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:27:03.778471  9943 pending_rounds.cc:85] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 13
I20260709 12:27:03.778731  9943 pending_rounds.cc:107] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:27:03.778970  9943 raft_consensus.cc:2889] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:03.775218  9945 rpcz_store.cc:276] 0709 12:27:02.749501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.842939 (+ 93438us) service_pool.cc:224] Handling call
0709 12:27:03.773093 (+930154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.773259 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.773286 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.773328 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:03.773365 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.773416 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.774096 (+   680us) spinlock_profiling.cc:243] Waited 930 ms on lock 0x7b54001b9c50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:03.774192 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2138289920}
I20260709 12:27:03.780961  9945 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:03.777870 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.685854 18195 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:03.785643  9942 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 26 index: 12. (index mismatch)
I20260709 12:27:03.776290  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:03.787066 10676 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:03.792402 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:03.793653  9946 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:03.793795 16661 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:03.794054  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:03.794185 16661 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:03.794699 16661 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:03.795616  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:03.797613 10672 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:03.798015  9942 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 19 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
W20260709 12:27:03.797856 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.799688  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:03.682566 18078 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.803683  9942 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:03.806829 18078 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.807058 18278 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:03.807785 18278 consensus_queue.cc:1243] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:03.808295 18278 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:03.809305 18278 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:03.809667 18278 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:03.810789 18157 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:03.811288 18157 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:03.811808 18179 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:03.812227 18179 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:03.813171 18179 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.813580 18002 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:03.814112 17928 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:03.814527 18268 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:03.814911 17928 consensus_queue.cc:1243] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:03.815117 18268 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:03.815375  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:03.816540 18226 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:03.817296 18226 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:03.818221 18007 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.812112 18157 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.816111 18268 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.819909 18203 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:03.820113 18243 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:03.820993 18174 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:03.801558 18014 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.823457 10090 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 12. (term mismatch)
I20260709 12:27:03.823839 10090 pending_rounds.cc:85] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 11
I20260709 12:27:03.824059 10090 pending_rounds.cc:107] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 25.12
I20260709 12:27:03.824287 10090 raft_consensus.cc:2889] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:03.789847 18195 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:03.816125 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 2961) took 1407 ms. Trace:
I20260709 12:27:03.829723 10091 rpcz_store.cc:276] 0709 12:27:02.408568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.525904 (+117336us) service_pool.cc:224] Handling call
0709 12:27:03.815907 (+1290003us) spinlock_profiling.cc:243] Waited 956 ms on lock 0x7b5400192dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:03.816063 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2198470400}
I20260709 12:27:03.833036  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:03.837685 16661 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.849813 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:03.850502 10091 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:03.850509  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3085) took 1524 ms. Trace:
I20260709 12:27:03.851464  9944 rpcz_store.cc:276] 0709 12:27:02.326255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.377949 (+ 51694us) service_pool.cc:224] Handling call
0709 12:27:03.517348 (+1139399us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:27:03.517607 (+   259us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:03.518078 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.518120 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:03.523561 (+  5441us) log.cc:844] Serialized 52 byte log entry
0709 12:27:03.640658 (+117097us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:03.640697 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.640759 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:03.828546 (+187787us) raft_consensus.cc:1685] finished
0709 12:27:03.828569 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:03.850445 (+ 21876us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2076,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":5196,"spinlock_wait_cycles":69255424,"thread_start_us":83459,"threads_started":1,"wal-append.queue_time_us":257584}
I20260709 12:27:03.852723 16661 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:03.783752 18196 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:27:03.802171 18276 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.870375  9944 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:27:03.846166 17914 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:03.875797  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 60503.4us
I20260709 12:27:03.854187 18011 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:03.865723 18179 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:03.883312 10091 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:03.846922 18174 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:27:03.868333 18178 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:03.884992 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:03.810532 18278 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:03.855532 18078 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.025s
W20260709 12:27:03.918109  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.864487 18226 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:27:03.897233 18058 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:03.900790 18002 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:27:03.876030 18276 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.911626 18268 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:03.950582  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73766us
I20260709 12:27:03.880611 18019 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:27:03.938657 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.901042 18267 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:03.975342 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.975630 18267 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:03.976223  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:03.878883 17914 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.978526 18267 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:03.978935  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.980942 17914 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:03.947659 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.949726 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:03.981997 10668 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:03.982364 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:03.983336 10677 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:03.984282 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.951689 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.949009 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:03.939585 18058 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:03.931411 18282 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:27:03.942263 10094 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:03.989297 18058 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:03.990430 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:03.976166 10091 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:03.995330 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:03.996470 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:04.001752 10091 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 27 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:27:03.997210  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:03.998296 18219 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.001220  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:03.999027 17907 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.036070 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:03.903589 18095 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:27:04.036791 10676 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:04.044273 10668 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:04.058923 10098 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:04.038336  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:04.039330 10091 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:27:03.951534 18068 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.061367  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:04.069018 17257 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:04.069499 17257 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:04.070293 17257 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:04.037786  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56954.3us
I20260709 12:27:03.950987 18007 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.111s
I20260709 12:27:04.038730 18004 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:04.083446  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45265.5us
I20260709 12:27:04.085546  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:04.063694 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:04.113181 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:04.072258 18185 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:04.088498 18259 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.001s
I20260709 12:27:04.107427  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:04.086246 10676 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:04.114411 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:04.115639 18283 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
I20260709 12:27:04.147368 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:04.148100 10093 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:04.150241 10093 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:04.150717 18004 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:04.115844 18285 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:04.165045  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2816) took 1183 ms. Trace:
I20260709 12:27:04.165498  9949 rpcz_store.cc:276] 0709 12:27:02.982078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.756852 (+774774us) service_pool.cc:224] Handling call
0709 12:27:03.757182 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.757520 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.757539 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.783223 (+ 25684us) log.cc:844] Serialized 26 byte log entry
0709 12:27:03.940061 (+156838us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:03.940112 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:03.940184 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:04.164794 (+224610us) raft_consensus.cc:1685] finished
0709 12:27:04.164816 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.164978 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":359,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":23475,"thread_start_us":102044,"threads_started":1,"wal-append.queue_time_us":319461}
W20260709 12:27:04.171823  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64549.8us
I20260709 12:27:04.059158 17907 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:04.061746 17393 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.049s
W20260709 12:27:04.137470  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51992.3us
I20260709 12:27:04.187523  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:04.188710  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:04.076800 18184 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:04.176560 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.059597 18219 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:04.110283  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 222973us
I20260709 12:27:04.215378 10672 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:04.214107 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:04.004132 18268 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:04.220871 10017 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:27:04.222203 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:04.232296 10024 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:04.233494 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:04.226023 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:04.073455 18068 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.234715 10025 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:04.233407 10016 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:04.235911 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:04.237344 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:04.238067 10025 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:04.239331 10025 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:04.247525 10025 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:04.243047 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:04.257761 17922 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:04.259279 10022 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:04.263074 18219 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:04.191144 18078 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:04.196081 18016 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:04.267875 18178 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:04.268298 18178 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:04.269080 18178 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:04.269742 17257 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:04.270121 17257 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:04.270205 18178 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:04.271258 17257 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:04.272222 17257 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:04.275362 10676 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:27:04.278048  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1749) took 2007 ms. Trace:
I20260709 12:27:04.278530  9948 rpcz_store.cc:276] 0709 12:27:02.270251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.273834 (+  3583us) service_pool.cc:224] Handling call
0709 12:27:02.274210 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.276793 (+  2583us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:02.277347 (+   554us) log.cc:844] Serialized 14 byte log entry
0709 12:27:02.603673 (+326326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:02.603726 (+    53us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:02.603782 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.277812 (+1674030us) spinlock_profiling.cc:243] Waited 2.27 ms on lock 0x7b540003e154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:04.277863 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.277991 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":135,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":31,"raft.queue_time_us":4235,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":5226752,"thread_start_us":324790,"threads_started":1,"wal-append.queue_time_us":433609,"wal-append.run_cpu_time_us":2353,"wal-append.run_wall_time_us":1158693}
I20260709 12:27:04.283991 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1474) took 1090 ms. Trace:
I20260709 12:27:04.284458 10095 rpcz_store.cc:276] 0709 12:27:03.193389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.193517 (+   128us) service_pool.cc:224] Handling call
0709 12:27:03.193848 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.525890 (+332042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.283754 (+757864us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b540014afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:04.283927 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":761318272}
I20260709 12:27:04.228441 18243 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:04.287221 18243 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.290840 17393 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:04.293452 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2664) took 1074 ms. Trace:
I20260709 12:27:04.293851 10018 rpcz_store.cc:276] 0709 12:27:03.219319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.304849 (+ 85530us) service_pool.cc:224] Handling call
0709 12:27:03.346437 (+ 41588us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.426788 (+ 80351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.426843 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.426903 (+    60us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:03.426948 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.293213 (+866265us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b54002657d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:04.293265 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.293392 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":184321792}
W20260709 12:27:04.196801 18229 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:27:04.308835 16661 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:04.300911 10017 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:27:04.315477 10095 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:04.319100 18247 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:27:04.330902  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 130960us
I20260709 12:27:04.334367  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:04.335433  9946 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 19: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:04.244681 18201 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:04.351539 18273 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:04.359701 18273 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:04.194553 17899 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.125s
I20260709 12:27:04.158385 17617 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.353825 18185 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:04.348922 18078 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:04.230970 18004 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:04.358719 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:04.378149 10669 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:04.186637 18259 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.367780 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:04.381842 10672 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:04.216276 17979 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.140s
I20260709 12:27:04.382026 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:04.244445 18182 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.132s
I20260709 12:27:04.385288 18243 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:04.389983  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:04.403012 18078 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.394958 10675 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:04.396977  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:04.355502 18201 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:04.247754 18292 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.164s
I20260709 12:27:04.362951 17907 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:04.440263 10092 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:04.367228 18269 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:04.410164  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:04.424727 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.439024  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:04.363155 18276 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:04.376461 18214 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.753s	user 0.005s	sys 0.005s
W20260709 12:27:04.400279 18014 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.577s	user 0.002s	sys 0.008s
I20260709 12:27:04.377840 18004 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:04.395263 17617 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:04.395357 18041 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:04.402463 18253 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:04.462848 18269 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.464840  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:04.465413 18287 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:04.447860 18020 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:04.446135 18022 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.661s	user 0.010s	sys 0.000s
I20260709 12:27:04.467896 18259 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:04.469627 18268 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:04.447638 18226 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:27:04.470659 18273 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:04.476372 18004 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:04.477748 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1448) took 1506 ms. Trace:
I20260709 12:27:04.502833 10671 rpcz_store.cc:276] 0709 12:27:02.971397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.971542 (+   145us) service_pool.cc:224] Handling call
0709 12:27:02.975377 (+  3835us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.976678 (+  1301us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:02.976937 (+   259us) log.cc:844] Serialized 14 byte log entry
0709 12:27:03.056173 (+ 79236us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:03.056243 (+    70us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:03.056318 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.477490 (+1421172us) spinlock_profiling.cc:243] Waited 1.04 ms on lock 0x7b54007d3cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:04.477538 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.477675 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":305,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":759,"raft.queue_time_us":373,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":2389888,"thread_start_us":78271,"threads_started":1,"wal-append.queue_time_us":78589,"wal-append.run_cpu_time_us":1104,"wal-append.run_wall_time_us":1003188}
W20260709 12:27:04.459487 18195 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.000s
W20260709 12:27:04.487362  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.487483 10677 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:04.483561 18179 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.581s	user 0.009s	sys 0.001s
I20260709 12:27:04.496070 18095 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:04.506906  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:04.510061  9947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s	user 0.035s	sys 0.023s
I20260709 12:27:04.484700 18253 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:04.489449 18214 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:04.528414  9947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.035s	sys 0.023s
I20260709 12:27:04.487993 18014 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:04.538091  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:04.541131 10093 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:04.550628 18020 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:04.567924 10098 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:27:04.572507 18269 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:04.577652 16661 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.578018 18058 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:04.578250 16661 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.567412  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:04.584784 10091 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:04.613124  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:04.614333  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:04.495949 18022 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:04.623441 16661 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:04.530880 18214 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:04.624770  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 407101us
I20260709 12:27:04.628073 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:04.578882 18019 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.567930 18020 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:04.633114 10021 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 12:27:04.634366 10099 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.020s	sys 0.065s
W20260709 12:27:04.636384 10099 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.020s	sys 0.065s
I20260709 12:27:04.657991 16661 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.658921 16661 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.663929 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:04.664664 10099 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:04.665683 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:04.496486 18201 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.666329 10099 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:04.675680 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:04.676678 10019 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:27:04.669574 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:04.679559 10643 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:04.551203 18287 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 12:27:04.676285  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:04.684273 10020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1504) took 2352 ms (client timeout 1531 ms). Trace:
W20260709 12:27:04.703823 10020 rpcz_store.cc:269] 0709 12:27:02.331853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.337526 (+  5673us) service_pool.cc:224] Handling call
0709 12:27:04.684081 (+2346555us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b54001f6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:04.684202 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1388644608}
I20260709 12:27:04.631392 18214 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:04.534889 18195 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:04.484285  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74195us
I20260709 12:27:04.713451  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:04.691633 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:04.715037 10017 raft_consensus.cc:2376] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24. Re-sending same reply.
I20260709 12:27:04.709303 16661 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:04.578619 18058 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.700680 10093 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:04.663031 18020 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:04.725271 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:04.729872 10017 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:04.666249 17907 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:04.733434 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.735550 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:04.737773 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2985) took 1156 ms. Trace:
I20260709 12:27:04.683200 18247 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:04.761729  9940 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:04.742482 10090 rpcz_store.cc:276] 0709 12:27:03.581476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.581670 (+   194us) service_pool.cc:224] Handling call
0709 12:27:03.823101 (+241431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:03.824715 (+  1614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.737583 (+912868us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400192dd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:04.737721 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":544321920}
I20260709 12:27:04.760927 10016 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:04.719890 17899 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:04.662681 18019 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.545359 18253 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:04.769114 18031 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:04.777006 18031 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.779240 18031 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:04.781855  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3101) took 1403 ms. Trace:
I20260709 12:27:04.782292  9945 rpcz_store.cc:276] 0709 12:27:03.379025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.781608 (+402583us) service_pool.cc:224] Handling call
0709 12:27:03.781882 (+   274us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.786566 (+  4684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.786622 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.789649 (+  3027us) log.cc:844] Serialized 26 byte log entry
0709 12:27:04.161878 (+372229us) spinlock_profiling.cc:243] Waited 7.95 ms on lock 0x7b18001e66a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:04.485991 (+324113us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:04.487090 (+  1099us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.487181 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:04.781558 (+294377us) raft_consensus.cc:1685] finished
0709 12:27:04.781583 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.781792 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":652,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":13996,"spinlock_wait_cycles":18286976,"thread_start_us":322980,"threads_started":1,"wal-append.queue_time_us":934784}
I20260709 12:27:04.784474  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:04.785109  9945 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:27:04.785988  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:04.792936  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.737231 18016 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:04.744899 18201 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:04.806069 10017 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:04.806890 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:04.796633 10090 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:04.808360 18016 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:04.816872 18016 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:04.804085 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:04.810918  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 80108.3us
W20260709 12:27:04.812901  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.821668 10017 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:04.818434 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:04.830965 10090 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:04.833968 10090 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:27:04.836047 10090 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:04.773960 18095 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:04.841702 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.853682  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:04.763118 17907 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:04.878135 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.009s
I20260709 12:27:04.817796 18182 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:27:04.829898 17999 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 12:27:04.883047 17899 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.883893  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2828) took 1054 ms. Trace:
I20260709 12:27:04.902899 17899 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.905815 17899 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:04.906879 18022 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:04.896950  9947 rpcz_store.cc:276] 0709 12:27:03.829496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.838472 (+  8976us) service_pool.cc:224] Handling call
0709 12:27:03.856216 (+ 17744us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:03.856648 (+   432us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:03.856693 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:03.865167 (+  8474us) log.cc:844] Serialized 26 byte log entry
0709 12:27:04.531323 (+666156us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:04.531389 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:04.531470 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:04.883705 (+352235us) raft_consensus.cc:1685] finished
0709 12:27:04.883725 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:04.883840 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6440,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":4608,"thread_start_us":634315,"threads_started":1,"wal-append.queue_time_us":1002904}
I20260709 12:27:04.850185 18095 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.885301 10093 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:04.888195 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.011s	sys 0.009s
W20260709 12:27:04.879447  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59668.3us
I20260709 12:27:04.925560  9946 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 23 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:04.926841 18303 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:04.932255 10668 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.007s	sys 0.015s
W20260709 12:27:04.932706 10668 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.007s	sys 0.015s
W20260709 12:27:04.916059  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 202702us
I20260709 12:27:04.935875 18031 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:04.884014 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:04.943778 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:04.951417 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67689.4us
I20260709 12:27:04.879067 18297 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:27:04.938583 18031 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:27:04.950706 18018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.008s	sys 0.008s
W20260709 12:27:04.910683 18232 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.008s	sys 0.008s
I20260709 12:27:04.991539  9947 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:27:04.993319  9947 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:04.947181 17999 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.893050 18126 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
W20260709 12:27:04.970010  9943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.016s	sys 0.007s
W20260709 12:27:04.999590 18018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.009s	sys 0.008s
I20260709 12:27:04.983687 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:05.000209 18232 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.008s	sys 0.008s
I20260709 12:27:04.978585 17899 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:27:05.016266  9949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.004s	sys 0.023s
I20260709 12:27:04.979190 18297 consensus_queue.cc:1243] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 18.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:04.979776 17649 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.939900 18022 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:27:04.939955 18174 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
W20260709 12:27:04.965490 18216 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.011s	sys 0.011s
W20260709 12:27:05.022128  9943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.017s	sys 0.008s
I20260709 12:27:05.020578 18058 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:04.933416 18004 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:04.999959 18276 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:04.969723 18294 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.050s
W20260709 12:27:05.050585  9949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.004s	sys 0.023s
W20260709 12:27:05.048472 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.008s	sys 0.008s
I20260709 12:27:05.022243 18195 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.022307 18179 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.082790 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:05.021384 17999 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.049921 17899 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:05.078400 18216 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.012s	sys 0.011s
I20260709 12:27:05.076514 18007 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.090466  9941 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22. Ops: []
W20260709 12:27:05.086901 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.008s	sys 0.008s
W20260709 12:27:05.096016 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 732423us
W20260709 12:27:05.095602 10096 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.019s	sys 0.010s
W20260709 12:27:05.101639 10096 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.019s	sys 0.010s
I20260709 12:27:05.108114 10093 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:27:05.089468 18309 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:27:05.110474  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.110932 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.089282 17899 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:05.087486 18294 consensus_queue.cc:1243] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 19.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:05.076773 17649 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.090785 17999 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.088963 17907 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.090284 18004 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:05.129988 10092 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.010s	sys 0.005s
I20260709 12:27:05.087555 18276 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:05.150772 10092 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.010s	sys 0.005s
I20260709 12:27:05.092829 18179 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.160972 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:05.146625 18095 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.162361 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:05.164362 10676 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:27:05.107686 18007 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.172079 16661 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:05.173862 16661 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:05.185434  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:05.192638 18195 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.193253 18195 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.194351 18297 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:05.220698 18214 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.221527 18020 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.232136 18020 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.232246 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1748) took 1283 ms. Trace:
I20260709 12:27:05.232923 10020 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:05.232725 10677 rpcz_store.cc:276] 0709 12:27:03.948539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.986164 (+ 37625us) service_pool.cc:224] Handling call
0709 12:27:03.986877 (+   713us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.508650 (+521773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.231989 (+723339us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54004d49d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:05.232185 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1150825728}
W20260709 12:27:05.233930 18315 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:27:05.235654  9944 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:27:05.222165 18179 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.223423 18276 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:05.227219 18019 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.229312  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:05.238972  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:05.200860 18031 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.240319 18031 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.241715 10023 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:27:05.243026  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:05.246266  9941 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:05.247927 18068 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:05.250065 18068 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:05.250411 18068 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:05.249437 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.251309 18068 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.249718 18016 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.253893 18068 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:05.261445 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.266080 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.279582  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:05.285981  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:05.288179 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.286729  9941 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:05.290124  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:05.291553  9941 raft_consensus.cc:2368] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
I20260709 12:27:05.212940 18297 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:05.246388 18178 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.931s	user 0.002s	sys 0.007s
I20260709 12:27:05.280799 18290 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.308622 17649 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:05.309701 18323 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
W20260709 12:27:05.309967 18319 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:27:05.285179 18018 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.287559 17393 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.314648 18330 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.315361 18330 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.306016 18297 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:05.317992 18330 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.318506 18330 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.318992 18330 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.321858 18330 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.322449 18330 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.323355 18330 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.332090  9948 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:05.323134 18007 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.333833 17393 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.311735 18290 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.400899  9948 pending_rounds.cc:85] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 17
I20260709 12:27:05.404407  9948 pending_rounds.cc:107] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 24.18
I20260709 12:27:05.404711  9948 raft_consensus.cc:2889] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:05.424468 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.427882 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:05.328190 18018 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.434183 10017 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:27:05.438640 10017 pending_rounds.cc:85] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 12
I20260709 12:27:05.438941 10017 pending_rounds.cc:107] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:27:05.439186 10017 raft_consensus.cc:2889] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:05.440389 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.359647 18330 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.445566 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.446686 10018 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 26 because replica has last-logged OpId of term: 24 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 19.
I20260709 12:27:05.441939 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.442248 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.403339 18250 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:05.454109 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.497956 10017 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:27:05.498636 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.465185 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:05.419664 18196 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.540s	user 0.002s	sys 0.014s
I20260709 12:27:05.506177 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.506882 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:05.439642 18292 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.508908 10025 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 24 index: 20. Preceding OpId from leader: term: 25 index: 20. (term mismatch)
I20260709 12:27:05.510573 10025 pending_rounds.cc:85] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 19
I20260709 12:27:05.510864 10025 pending_rounds.cc:107] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 24.20
I20260709 12:27:05.511091 10025 raft_consensus.cc:2889] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:05.517573 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.519356 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:05.509423 18196 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.632s	user 0.002s	sys 0.014s
W20260709 12:27:05.518968 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 357652us
I20260709 12:27:05.504549 18278 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:05.504776 18330 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.523635 18068 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.518187 18292 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:05.515484  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 70404.8us
I20260709 12:27:05.508342 18347 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.512957 18007 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:05.546121 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3888) took 1320 ms (client timeout 1666 ms). Trace:
W20260709 12:27:05.546713 10024 rpcz_store.cc:269] 0709 12:27:04.225925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.233168 (+  7243us) service_pool.cc:224] Handling call
0709 12:27:05.543500 (+1310332us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54002652d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:05.546059 (+  2559us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1684887296}
I20260709 12:27:05.536343 18068 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.538571 18292 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.539422 10677 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:05.541695 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:05.556497 10676 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:05.535202 10668 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:05.549610 18348 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:05.551775 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:05.522922 18345 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:05.526865 18201 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.708s	user 0.007s	sys 0.007s
I20260709 12:27:05.569414  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:05.505226 18250 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.571877  9942 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
W20260709 12:27:05.565814 18339 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 12:27:05.587801 18338 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:27:05.543258 18330 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.591148 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:05.571491 18342 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:27:05.533667 18353 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.064s
I20260709 12:27:05.571309 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:05.608552 10670 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:05.601727 18201 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:05.550889 18297 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.590653 18350 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.580300 18345 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.581175 18226 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.051s
W20260709 12:27:05.613030  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43858.3us
I20260709 12:27:05.564802 18007 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.620757 18356 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.633246 18356 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.636245 18356 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.637009 18356 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:05.637640 18356 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:05.622134 18340 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 12:27:05.629009  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:05.641896 18353 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:05.623234 18290 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.647974 18068 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:05.648874 18068 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:05.621917 18018 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.616407 18297 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.653177 18184 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:05.653877 18184 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:05.654874 18290 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.655479 18290 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.656626 18007 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.657873 18290 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:05.658102 17899 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.002s
W20260709 12:27:05.659263 18344 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:27:05.662045 18341 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:27:05.662045 10670 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:05.667163 18292 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.668004 18292 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:05.669466 18250 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.671116 17899 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.674157 17899 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:05.616184 18350 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.614362 18343 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:27:05.613427 18030 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:05.620652 18345 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:05.678272 18253 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:05.679663 18058 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:05.551877 18347 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.618062 18330 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.613566 18201 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:05.688935 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 161552us
W20260709 12:27:05.689554  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.682755 18144 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:05.682268 18007 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:05.624174  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3116) took 1290 ms. Trace:
I20260709 12:27:05.698048  9949 rpcz_store.cc:276] 0709 12:27:04.333474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.333585 (+   111us) service_pool.cc:224] Handling call
0709 12:27:04.333906 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.354790 (+ 20884us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:27:05.086508 (+731718us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.192292 (+105784us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:05.195790 (+  3498us) log.cc:844] Serialized 26 byte log entry
0709 12:27:05.195971 (+   181us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:05.196019 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.196109 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.623922 (+427813us) raft_consensus.cc:1685] finished
0709 12:27:05.623952 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.624109 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":570464,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":26977,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":3032,"thread_start_us":796776,"threads_started":2,"wal-append.queue_time_us":403878}
W20260709 12:27:05.695472 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 143454us
I20260709 12:27:05.696238  9945 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:05.702978 10091 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:27:05.706965  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:05.687829 18030 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:05.688865 18058 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:05.713343  9949 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:27:05.713762  9949 pending_rounds.cc:85] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 12
I20260709 12:27:05.713968  9949 pending_rounds.cc:107] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:27:05.694766 18345 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:05.714183  9949 raft_consensus.cc:2889] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:05.712118 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 123764us
I20260709 12:27:05.718863 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:05.717995  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1763) took 1171 ms. Trace:
I20260709 12:27:05.720965  9944 rpcz_store.cc:276] 0709 12:27:04.546274 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.615205 (+ 68931us) service_pool.cc:224] Handling call
0709 12:27:04.615553 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.236021 (+620468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.717598 (+481577us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b540009b5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:05.717912 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1425606912}
I20260709 12:27:05.696571 18144 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:05.711771 18352 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:05.701606 18292 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.710517 18343 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.712136 18030 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:05.725726 18352 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:05.730156 18352 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:05.729367 18362 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:05.732735 18362 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:05.733122 18362 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:05.719383 18247 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.717859 18345 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:05.723877 18348 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:27:05.744567 18058 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.737480 18343 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.720000 18329 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:05.772770 18350 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:05.779775 18030 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:05.749716 18300 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.661s	user 0.002s	sys 0.000s
I20260709 12:27:05.800931 18030 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.704478 18184 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.802850 18184 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:05.768329 18247 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:05.797050 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69765.2us
I20260709 12:27:05.798206 18329 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.819025 18247 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:05.819438 18201 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.809370 18144 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.839227 18144 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:05.845132 18343 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.845644 18329 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:05.846508 18318 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.691s	user 0.000s	sys 0.003s
I20260709 12:27:05.849231 18157 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:05.847694 18314 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.689s	user 0.004s	sys 0.003s
I20260709 12:27:05.851054 18203 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:05.851508 18203 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:05.851624 18352 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.852346 18203 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.853565 18203 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:05.854023 18352 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.858093 18195 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.859931 18362 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.869885 18214 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:05.872466 18308 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.797s	user 0.000s	sys 0.003s
W20260709 12:27:05.873509 18326 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.748s	user 0.003s	sys 0.000s
W20260709 12:27:05.874130 18312 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.725s	user 0.002s	sys 0.000s
I20260709 12:27:05.862584 18362 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:05.879827 18356 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:05.879874 18347 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:05.819285 18345 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:05.934190  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2834) took 1570 ms. Trace:
I20260709 12:27:05.935386  9943 rpcz_store.cc:276] 0709 12:27:04.363435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.368505 (+  5070us) service_pool.cc:224] Handling call
0709 12:27:04.368813 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.376102 (+  7289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.376143 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:04.376602 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 12:27:05.077270 (+700668us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:27:05.077327 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.077413 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.873413 (+796000us) spinlock_profiling.cc:243] Waited 44.8 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:05.932778 (+ 59365us) raft_consensus.cc:1685] finished
0709 12:27:05.932892 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.933721 (+   829us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6935,"raft.queue_time_us":7673,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":14382,"spinlock_wait_cycles":103005952,"thread_start_us":615127,"threads_started":1,"wal-append.queue_time_us":706886}
I20260709 12:27:05.949986  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2817) took 2963 ms. Trace:
I20260709 12:27:05.951249  9940 rpcz_store.cc:276] 0709 12:27:02.986135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.760082 (+773947us) service_pool.cc:224] Handling call
0709 12:27:03.761782 (+  1700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.762129 (+1000347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.949154 (+1187025us) spinlock_profiling.cc:243] Waited 1e+03 ms on lock 0x7b54000b5254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:05.949699 (+   545us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2299232512}
W20260709 12:27:05.959921  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.964823  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:05.972016  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.972318 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2670) took 2363 ms. Trace:
I20260709 12:27:05.973079 10021 rpcz_store.cc:276] 0709 12:27:03.608319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.608439 (+   120us) service_pool.cc:224] Handling call
0709 12:27:03.608731 (+   292us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:04.635105 (+1026374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.971343 (+1336238us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b54001f7554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:05.972006 (+   663us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2355307776}
I20260709 12:27:05.976487 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3873) took 3427 ms. Trace:
I20260709 12:27:05.977300 10023 rpcz_store.cc:276] 0709 12:27:02.549149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.575211 (+ 26062us) service_pool.cc:224] Handling call
0709 12:27:02.575526 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.242144 (+2666618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.975526 (+733382us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:05.976194 (+   668us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6125005696}
W20260709 12:27:05.985018 10091 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4157) took 1357 ms (client timeout 1707 ms). Trace:
W20260709 12:27:05.988051 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 173467us
W20260709 12:27:05.991183 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.995496 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2988) took 1452 ms. Trace:
W20260709 12:27:05.999677 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.000023 17649 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:06.000633 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.001140 17649 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:06.004494 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.006678 17649 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:06.007388 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:05.995994 10097 rpcz_store.cc:276] 0709 12:27:04.543187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.543326 (+   139us) service_pool.cc:224] Handling call
0709 12:27:04.543736 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:04.549296 (+  5560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.549339 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:04.550104 (+   765us) log.cc:844] Serialized 52 byte log entry
0709 12:27:04.914679 (+364575us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:04.979955 (+ 65276us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.845743 (+865788us) spinlock_profiling.cc:243] Waited 40.1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:05.994149 (+148406us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b18001d13a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:27:05.994765 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:05.994923 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:05.995003 (+    80us) raft_consensus.cc:1685] finished
0709 12:27:05.995018 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:05.995255 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45398,"prepare.queue_time_us":65259,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":6220,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":31959,"spinlock_wait_cycles":108839680,"thread_start_us":288111,"threads_started":1,"wal-append.queue_time_us":514353}
I20260709 12:27:06.015190 17907 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:05.985615 10091 rpcz_store.cc:269] 0709 12:27:04.627693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.627816 (+   123us) service_pool.cc:224] Handling call
0709 12:27:05.984314 (+1356498us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540002dd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:05.984671 (+   357us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2470119296}
I20260709 12:27:06.009603  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1762) took 1471 ms. Trace:
I20260709 12:27:06.015260 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2995) took 1284 ms. Trace:
I20260709 12:27:06.026376  9942 rpcz_store.cc:276] 0709 12:27:04.537686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.537823 (+   137us) service_pool.cc:224] Handling call
0709 12:27:06.009220 (+1471397us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000b7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:06.009405 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2375532416}
W20260709 12:27:06.068048 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 77444.8us
I20260709 12:27:06.041493 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:06.081487  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.081664 17907 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.084954  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.088721  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.028467 10090 rpcz_store.cc:276] 0709 12:27:04.730174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.837210 (+107036us) service_pool.cc:224] Handling call
0709 12:27:04.837533 (+   323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:04.881367 (+ 43834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:04.881399 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:04.882039 (+   640us) log.cc:844] Serialized 52 byte log entry
0709 12:27:05.062795 (+180756us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:05.063107 (+   312us) log.cc:844] Serialized 14 byte log entry
0709 12:27:05.872777 (+809670us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:06.014012 (+141235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.014240 (+   228us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:06.014378 (+   138us) raft_consensus.cc:1685] finished
0709 12:27:06.014566 (+   188us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.014984 (+   418us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":993594,"prepare.queue_time_us":836,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":51689,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":8202,"spinlock_wait_cycles":162984704,"thread_start_us":180383,"threads_started":1,"wal-append.queue_time_us":203683}
I20260709 12:27:06.090090  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:06.023932  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.025213  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.096478  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.091662 18329 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:06.114739  9943 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:06.081507 17874 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.141057  9947 raft_consensus.cc:2410] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 24 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:27:06.133788  9944 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:06.146436 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.146554 18232 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:27:06.148945 18008 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:06.149473 17922 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:06.149889  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 420088us
I20260709 12:27:06.149969 18008 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.150261 17922 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.152971  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:06.154209 18214 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:06.144099  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:06.159377  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.160557 17914 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.161283 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4158) took 1532 ms. Trace:
I20260709 12:27:06.161373 17914 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.160583 17393 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:06.163141 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:06.163946 17922 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:06.164405 17914 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:06.165380 18329 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:06.165824 18329 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.166188 18329 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:06.162009 10099 rpcz_store.cc:276] 0709 12:27:04.629102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.674511 (+ 45409us) service_pool.cc:224] Handling call
0709 12:27:04.674899 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.675150 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:04.996257 (+321107us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.130804 (+1134547us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.145578 (+ 14774us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:06.160737 (+ 15159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.160878 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.161084 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":841279,"prepare.queue_time_us":112254,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":979436,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":177,"thread_start_us":539020,"threads_started":3,"wal-append.queue_time_us":288718}
I20260709 12:27:06.168857  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:06.169160 18008 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:06.170277  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.150285 18345 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:06.171147  9943 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:06.174948 18144 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:06.175869 18144 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.180255 17874 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.158027  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.183952  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.184178 17874 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:06.184803  9949 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:06.185220 18014 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.185822 18014 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.188562 18014 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:06.190744  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.193415 17907 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.196121 18356 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:06.196872 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.197589 10091 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:06.198483  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:06.200789 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 194359us
I20260709 12:27:06.201984 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:06.203183 18368 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:06.205091 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 110192us
I20260709 12:27:06.205279  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1774) took 1538 ms. Trace:
I20260709 12:27:06.206310 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:06.209539 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:06.215153 17922 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:06.215793 10091 pending_rounds.cc:85] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 13
I20260709 12:27:06.216253 10091 pending_rounds.cc:107] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 12:27:06.216607 10091 raft_consensus.cc:2889] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.219903 18358 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:06.220374 18358 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:06.221496 18358 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.223100  9948 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:06.229981 18356 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.211400  9942 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:06.232378  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.182981  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 182348us
I20260709 12:27:06.194151  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:06.205739  9946 rpcz_store.cc:276] 0709 12:27:04.666995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.090403 (+423408us) service_pool.cc:224] Handling call
0709 12:27:06.010835 (+920432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.011476 (+   641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.011515 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.018495 (+  6980us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:06.018562 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.023696 (+  5134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.204960 (+181264us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54000b7050. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:06.205115 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12025,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":42500,"spinlock_wait_cycles":2115731712}
I20260709 12:27:06.248433  9941 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:06.271301 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:06.271826 18356 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.271724 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:06.272293 18144 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:06.275235  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 122539us
I20260709 12:27:06.276983  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:06.279016 18204 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:06.260344 18008 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:06.237429 18214 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:06.303174 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 93876.6us
I20260709 12:27:06.314759 18030 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:06.279819 18362 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:06.283325 18204 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.315167 18004 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:06.266737 18232 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.058s
I20260709 12:27:06.320344 17874 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.338984 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2991) took 1753 ms. Trace:
I20260709 12:27:06.339538 17874 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.339576 10098 rpcz_store.cc:276] 0709 12:27:04.583227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:04.585647 (+  2420us) service_pool.cc:224] Handling call
0709 12:27:04.586008 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:04.586263 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:27:05.051182 (+464919us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.301556 (+1250374us) log.cc:844] Serialized 14 byte log entry
0709 12:27:06.327982 (+ 26426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.330261 (+  2279us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:06.333216 (+  2955us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.333377 (+   161us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.336112 (+  2735us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":934219,"prepare.queue_time_us":363541,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":969015,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":94,"thread_start_us":700408,"threads_started":3,"wal-append.queue_time_us":263375}
W20260709 12:27:06.341689  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 147747us
W20260709 12:27:06.342662 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 104898us
I20260709 12:27:06.327955 18030 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.344493 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:06.345525 18030 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:06.342947 18362 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:06.342223 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.340351 18011 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.361678 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:06.327906  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:06.349852 18250 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:06.363173  9830 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 96708.9us
I20260709 12:27:06.363380 18250 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.363858 18250 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:06.366922  9841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 214) took 88060 ms (client timeout 15000 ms). Trace:
W20260709 12:27:06.367415  9841 rpcz_store.cc:269] 0709 12:25:38.306032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:38.308733 (+  2701us) service_pool.cc:224] Handling call
0709 12:27:06.366621 (+88057888us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1722823,"reported_tablets":39}
W20260709 12:27:06.351642  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.342519 17874 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:06.362509 10673 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:06.387257 18016 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.388096 17617 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.388810 17617 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.390700 18004 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:06.328334 18144 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:06.362038 18011 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.371575 18362 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:06.392038 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.942s (SENT) [suppressed 12 similar messages]
I20260709 12:27:06.399573 10092 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:06.405479 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.405877  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:06.411500 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.412266 18031 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:06.412504 10092 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:27:06.412952 17874 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:06.413936 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.415307 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.416002 10092 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:06.416960 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:06.417574 10092 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:06.418753 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2976) took 3517 ms. Trace:
I20260709 12:27:06.419209 10096 rpcz_store.cc:276] 0709 12:27:02.878136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:02.878302 (+   166us) service_pool.cc:224] Handling call
0709 12:27:02.878740 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:02.879020 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:02.883278 (+  4258us) log.cc:844] Serialized 14 byte log entry
0709 12:27:04.476595 (+1593317us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b180033bfe0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:27:05.233444 (+756849us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:06.318830 (+1085386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:06.328593 (+  9763us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:06.371213 (+ 42620us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.371285 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.395647 (+ 24362us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":921295,"prepare.queue_time_us":3987,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":1053050,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":134,"spinlock_wait_cycles":75433600,"thread_start_us":806237,"threads_started":2,"wal-append.queue_time_us":2270792,"wal-append.run_cpu_time_us":3618,"wal-append.run_wall_time_us":1118322}
I20260709 12:27:06.421609 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:06.428153 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.433862 10098 raft_consensus.cc:2410] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 20 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:27:06.442324  9944 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
W20260709 12:27:06.444097 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:06.449230 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.408484 18362 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:06.465113 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.466977 10092 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:06.476913 10676 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:27:06.477371 10676 pending_rounds.cc:85] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 14
I20260709 12:27:06.477617 10676 pending_rounds.cc:107] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:27:06.477869 10676 raft_consensus.cc:2889] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.482836 18372 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:06.483618 18372 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.484365 18011 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.486178 18058 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:06.419211 17649 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.462878 17874 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.507562 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:06.448619 18358 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:06.509989 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:06.509233 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:06.515618  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.513027  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:06.523092 18157 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.673s	user 0.017s	sys 0.002s
I20260709 12:27:06.523167 10092 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:06.486354 18372 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:06.533183 18294 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:06.535116 10643 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:06.538776 17874 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.539511  9837 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 13 to 16, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 16 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:06.541325 17874 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:06.542537 18232 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:06.543222 18232 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:06.543452 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:06.544297 18232 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.544808 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:06.546033 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:06.546792 17649 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:06.549741  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 145900us
I20260709 12:27:06.533993 18294 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:06.551015 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.558634  9942 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:06.549590 10098 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:06.487267 17617 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:06.403316 18004 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.565392 10670 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
W20260709 12:27:06.570257  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44362.5us
I20260709 12:27:06.583396 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:06.585201 18203 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.670s	user 0.011s	sys 0.016s
I20260709 12:27:06.587054 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:06.587283 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 146809us
I20260709 12:27:06.569380 18204 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.590543 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:06.623950 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.633700 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:06.601279 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:06.599287 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:06.569135 18375 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.162s	user 0.000s	sys 0.002s
I20260709 12:27:06.622378 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:06.611305 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:06.599902 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:06.590917 10096 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:06.618379 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:06.643652 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:06.624048 18157 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:27:06.600489 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:06.598819 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:06.618873 17649 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.652726 18356 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:06.642827 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:06.653651 18356 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:06.651046 10671 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:06.645977 18345 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.649565 18369 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:06.655496 18345 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.656623 18371 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:06.656874 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.649842 10021 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:06.657773 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 67439.1us
I20260709 12:27:06.658183 18345 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.658769 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:06.658990 10020 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:06.659958 10017 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:06.660887 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:06.657819 10099 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:06.649003 10019 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:06.662766 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.603247 18294 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.641384 18363 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:06.690675 18363 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.693815 18363 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:06.652559 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 90992.4us
I20260709 12:27:06.687527 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.696815 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.697527 10097 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 18.
I20260709 12:27:06.652223 18350 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.698342 18350 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.701273 18350 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:06.654747 18356 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.702167  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:06.704274  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.667141 10094 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:06.668367 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:06.671116  9991 leader_election.cc:400] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
I20260709 12:27:06.675319 10017 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:06.705421  9991 leader_election.cc:403] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 12:27:06.677078 18365 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.217s	user 0.004s	sys 0.000s
I20260709 12:27:06.646415  9837 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:27:06.679064 18374 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.003s
W20260709 12:27:06.680917 18366 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.211s	user 0.000s	sys 0.001s
I20260709 12:27:06.684629 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.644474 18008 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:27:06.594077 18011 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:06.666587 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.711844 18216 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:06.714365 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 343133us
I20260709 12:27:06.722889 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:06.726137 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:06.709445 18369 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:06.626165 18329 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:06.730533  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 159690us
I20260709 12:27:06.716127 18371 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:06.727396 10098 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
W20260709 12:27:06.736375 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 75706.9us
I20260709 12:27:06.737174 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:06.743336 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:06.746058 18275 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:06.751049 10094 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4150) took 2781 ms (client timeout 1914 ms). Trace:
W20260709 12:27:06.751499 10094 rpcz_store.cc:269] 0709 12:27:03.969768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:03.990199 (+ 20431us) service_pool.cc:224] Handling call
0709 12:27:06.750840 (+2760641us) spinlock_profiling.cc:243] Waited 2.55 s on lock 0x7b5400162954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:06.750960 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5857180416}
I20260709 12:27:06.752514 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:06.753072 10094 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:06.752931 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.754655 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.755149 10091 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:06.756568 10096 pending_rounds.cc:85] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 11
I20260709 12:27:06.756469 10091 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:06.756942 10096 pending_rounds.cc:107] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 18.12
I20260709 12:27:06.757154 10091 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:06.757233 10096 raft_consensus.cc:2889] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:06.723361 18011 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:06.762902 10099 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 12:27:06.762980 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.764590 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:06.768689 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 120102us
I20260709 12:27:06.762382 18294 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:06.792717 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:06.797583 10097 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:06.797885 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:06.842223 10098 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:06.731725 18216 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:06.846887  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53266.8us
I20260709 12:27:06.765715 18011 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:06.849999  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 77175.6us
I20260709 12:27:06.850623 10021 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:06.775198 18363 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.865765 10091 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:06.867578 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:06.791951 18214 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.895074  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:06.895581 18329 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:06.895951  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 96146.6us
W20260709 12:27:06.918495 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 120696us
I20260709 12:27:06.882997 18031 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:06.925269  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:06.927439 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.898813 18014 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:06.899158 18008 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.901710 18214 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:06.888906 18371 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:06.945681 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:06.949746 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.952821 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:06.957484  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56620.7us
I20260709 12:27:06.958400  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:06.949069 17999 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:06.959950 17999 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:06.941020 18250 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.572s	user 0.012s	sys 0.002s
I20260709 12:27:06.964545 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:06.964751 17999 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:06.901571 17617 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:27:06.965050 10668 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:06.972440 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.917400 18171 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:06.947701 18014 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:06.981890 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62567.2us
I20260709 12:27:07.000669 10098 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:27:07.010919 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4167) took 2098 ms. Trace:
I20260709 12:27:06.998039  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:06.961866 18008 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.030508 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:07.027344 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:06.955806 18031 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.003633 18014 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:07.005275 18329 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:07.037071 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:06.971855 18362 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:07.026583 18058 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.506s	user 0.000s	sys 0.012s
I20260709 12:27:06.885295 18363 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.043241 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:07.011365 10093 rpcz_store.cc:276] 0709 12:27:04.856267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.109098 (+252831us) service_pool.cc:224] Handling call
0709 12:27:05.125671 (+ 16573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.145982 (+1020311us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.160320 (+ 14338us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.297009 (+136689us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.638387 (+341378us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:06.638638 (+   251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.719517 (+ 80879us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:06.938558 (+219041us) raft_consensus.cc:1685] finished
0709 12:27:06.938586 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.954750 (+ 16164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":884750,"raft.queue_time_us":1028008,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":190,"thread_start_us":413723,"threads_started":2,"wal-append.queue_time_us":555253}
I20260709 12:27:07.013199 17617 consensus_queue.cc:1243] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:27:07.055517 10671 raft_consensus.cc:2368] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
I20260709 12:27:07.004207 18144 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:07.005061 18171 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.069025 18031 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.069959 18232 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:07.070500 18232 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.070919 18232 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:07.066171 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.000705 18371 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.077329 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.076354 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.086192 10094 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:07.087327  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1775) took 1424 ms. Trace:
I20260709 12:27:07.082921 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:07.093791 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.095500 10090 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:27:07.096769 10090 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:27:07.103830 18008 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.078895 18203 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:07.104699 18203 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:07.105075 18203 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:07.111446  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 537954us
W20260709 12:27:07.114161  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 150042us
W20260709 12:27:07.093842 18383 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:27:07.087870  9945 rpcz_store.cc:276] 0709 12:27:05.527344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.702991 (+175647us) service_pool.cc:224] Handling call
0709 12:27:05.703314 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:05.703698 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:05.703727 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:05.713809 (+ 10082us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.146850 (+433041us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:06.146905 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.147004 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:06.676124 (+529120us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:06.923403 (+247279us) raft_consensus.cc:1685] finished
0709 12:27:06.923425 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:06.951252 (+ 27827us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1268,"raft.queue_time_us":1942,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":3640,"spinlock_wait_cycles":101336320,"thread_start_us":403010,"threads_started":1,"wal-append.queue_time_us":707938}
I20260709 12:27:07.078130 18004 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:07.139822 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.120345 10097 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
W20260709 12:27:07.153889  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.155560 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 82441.5us
I20260709 12:27:06.974478 18250 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.111896 18031 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:07.119807 10674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.014s	sys 0.010s
I20260709 12:27:07.072371 18329 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:07.164631 10674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.773s	user 0.015s	sys 0.011s
I20260709 12:27:07.150695 10669 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:07.080044 18144 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.243479 10093 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:07.247471 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.189786 18031 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:07.254748 10675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.002s	sys 0.020s
W20260709 12:27:07.255256 10675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.002s	sys 0.021s
I20260709 12:27:07.160029 18008 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.104s
I20260709 12:27:07.261317  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:07.164259 18004 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:07.250301 18031 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.264838 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:07.266024 18058 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.265873 18011 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:07.271306 18385 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.180s	user 0.000s	sys 0.000s
I20260709 12:27:07.273352 18144 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:07.275281 18397 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 12:27:07.291345 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1713) took 1016 ms. Trace:
I20260709 12:27:07.294680 18358 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:07.304013 18371 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:07.305948 18329 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.307664 18216 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:07.309528 18250 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:07.312883 10098 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 12:27:07.314388 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49420.5us
I20260709 12:27:07.321025 10676 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:07.183972 18350 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:07.327260 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.333477 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1715) took 1041 ms. Trace:
I20260709 12:27:07.333736 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:07.320753 18157 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:27:07.334647 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:07.334764  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 150611us
W20260709 12:27:07.337286 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.330544 10668 rpcz_store.cc:276] 0709 12:27:06.274420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.306864 (+ 32444us) service_pool.cc:224] Handling call
0709 12:27:06.337391 (+ 30527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.000420 (+663029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.291033 (+290613us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b5400501ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:07.291260 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":950080768}
W20260709 12:27:07.343554 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 233704us
I20260709 12:27:07.320288 18185 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:07.353718 10090 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:07.355461 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.356120 10017 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 19.
W20260709 12:27:07.351125  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 458469us
I20260709 12:27:07.364531  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:07.325273 18058 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.349365 10672 rpcz_store.cc:276] 0709 12:27:06.275344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.311420 (+ 36076us) service_pool.cc:224] Handling call
0709 12:27:06.386047 (+ 74627us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.386661 (+   614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.386705 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.387225 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.715138 (+327913us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:06.715183 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.715261 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.316060 (+600799us) raft_consensus.cc:1685] finished
0709 12:27:07.316088 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.316264 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1584,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":92,"spinlock_wait_cycles":258944,"thread_start_us":327467,"threads_started":1,"wal-append.queue_time_us":536532}
I20260709 12:27:07.346889 18214 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.396088 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.342280 18358 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.406785 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.427124 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.428329 10090 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:27:07.362804 10022 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:27:07.430017 10025 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:27:07.362160  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.431628  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 473306us
I20260709 12:27:07.429497  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:07.475695  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78406.8us
I20260709 12:27:07.479569 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.480515 10017 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:07.487089 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.488010 10093 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:07.361593 18185 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.362423 18157 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:27:07.496096 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.499919 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 2997) took 2477 ms. Trace:
I20260709 12:27:07.500379 10095 rpcz_store.cc:276] 0709 12:27:04.944292 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:05.157652 (+213360us) service_pool.cc:224] Handling call
0709 12:27:05.157988 (+   336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.237136 (+1079148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.246434 (+  9298us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.338891 (+ 92457us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.843809 (+504918us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:06.843875 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.849972 (+  6097us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.399445 (+549473us) raft_consensus.cc:1685] finished
0709 12:27:07.399464 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.421249 (+ 21785us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":858936,"raft.queue_time_us":996641,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":109,"thread_start_us":540453,"threads_started":2,"wal-append.queue_time_us":608247}
I20260709 12:27:07.485739 18214 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:07.400990 18203 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.493171 18031 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:07.397574 18348 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:07.514297 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.498088 18157 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:07.503589 18329 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:07.517201  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:07.530486 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.524542  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:07.518348 18031 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.509526 18214 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:07.513062 18144 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.520138 18348 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.554381 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.434412 18004 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:07.533506 18294 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.662s	user 0.013s	sys 0.000s
I20260709 12:27:07.505458 18014 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.554929 18031 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:07.577790 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.558507 18214 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:07.562897 18348 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:07.592625 18373 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.593364 18373 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.594777 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.595584 18373 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.594811 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1723) took 1277 ms. Trace:
I20260709 12:27:07.596098 10673 rpcz_store.cc:276] 0709 12:27:06.317884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.589947 (+272063us) service_pool.cc:224] Handling call
0709 12:27:06.590420 (+   473us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:06.731394 (+140974us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.731430 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:06.791065 (+ 59635us) log.cc:844] Serialized 52 byte log entry
0709 12:27:07.339405 (+548340us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:07.339448 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.339517 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.594451 (+254934us) raft_consensus.cc:1685] finished
0709 12:27:07.594484 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.594724 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10133,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5937,"thread_start_us":412868,"threads_started":1,"wal-append.queue_time_us":718539}
I20260709 12:27:07.587152 10090 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:07.601076  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:07.568903 18294 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
I20260709 12:27:07.607350 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1710) took 1334 ms. Trace:
W20260709 12:27:07.607527 18388 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.199s	user 0.000s	sys 0.001s
I20260709 12:27:07.608918 18363 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.630766 18363 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:07.610324 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4181) took 1062 ms. Trace:
I20260709 12:27:07.611608 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1718) took 1334 ms. Trace:
I20260709 12:27:07.612052 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.612784 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.613701 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.615974 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:07.616340 18014 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.616865 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:07.617208 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 106473us
W20260709 12:27:07.638221 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.602775 17999 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:07.631618 10099 rpcz_store.cc:276] 0709 12:27:06.548243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.769583 (+221340us) service_pool.cc:224] Handling call
0709 12:27:06.843614 (+ 74031us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:06.919232 (+ 75618us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.919261 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:06.956884 (+ 37623us) log.cc:844] Serialized 52 byte log entry
0709 12:27:07.395096 (+438212us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:07.395134 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.395196 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.610104 (+214908us) raft_consensus.cc:1685] finished
0709 12:27:07.610123 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.610270 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22206,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":29945,"thread_start_us":386730,"threads_started":1,"wal-append.queue_time_us":585392}
I20260709 12:27:07.619378 18362 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:07.621165 18379 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.433s	user 0.000s	sys 0.003s
I20260709 12:27:07.640235 10019 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:07.640403 18058 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.636659 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:07.641243  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1785) took 1557 ms. Trace:
I20260709 12:27:07.641319 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.641700 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.642907 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.641764  9943 rpcz_store.cc:276] 0709 12:27:06.084085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.172978 (+ 88893us) service_pool.cc:224] Handling call
0709 12:27:06.173377 (+   399us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.174054 (+   677us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.174082 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.238712 (+ 64630us) log.cc:844] Serialized 26 byte log entry
0709 12:27:06.809901 (+571189us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:06.810628 (+   727us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:06.810748 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.620270 (+809522us) spinlock_profiling.cc:243] Waited 1.65 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:07.640869 (+ 20599us) raft_consensus.cc:1685] finished
0709 12:27:07.640890 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.641173 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":158,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":3790080,"thread_start_us":492354,"threads_started":1,"wal-append.queue_time_us":899733}
I20260709 12:27:07.633917 10676 rpcz_store.cc:276] 0709 12:27:06.276827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.586196 (+309369us) service_pool.cc:224] Handling call
0709 12:27:06.586622 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.353606 (+766984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.611392 (+257786us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b54007f4754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:07.611552 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1545871104}
I20260709 12:27:07.617599 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.559022 18329 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.646667 18185 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.647334 10097 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:07.647902 18329 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:07.648640 18387 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:27:07.651706 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:07.624424 10091 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:07.642040 10018 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:07.626080 10674 rpcz_store.cc:276] 0709 12:27:06.273045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.298517 (+ 25472us) service_pool.cc:224] Handling call
0709 12:27:06.322763 (+ 24246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.390718 (+ 67955us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.390763 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.391394 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.165185 (+773791us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:07.165221 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.165285 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.607110 (+441825us) raft_consensus.cc:1685] finished
0709 12:27:07.607137 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.607276 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":773286,"threads_started":1,"wal-append.queue_time_us":1117763}
I20260709 12:27:07.652374 18008 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:07.658790 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.625717 18011 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.587785 18144 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.665068 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4173) took 1250 ms. Trace:
I20260709 12:27:07.639544 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.673408 10021 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:07.667577 10016 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:07.682294 10096 rpcz_store.cc:276] 0709 12:27:06.446722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.755972 (+309250us) service_pool.cc:224] Handling call
0709 12:27:06.756311 (+   339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:06.794758 (+ 38447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.794791 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:06.850725 (+ 55934us) log.cc:844] Serialized 52 byte log entry
0709 12:27:07.354862 (+504137us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:07.354906 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.354970 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.664813 (+309843us) raft_consensus.cc:1685] finished
0709 12:27:07.664834 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.664999 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2476,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":27475,"spinlock_wait_cycles":2816,"thread_start_us":424673,"threads_started":1,"wal-append.queue_time_us":667377}
I20260709 12:27:07.688109 18358 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.695294 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:07.695422 18203 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.696097 18144 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.696437 18369 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:07.703495 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.679917 18008 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:07.680979 18275 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:07.665674 18157 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.715692 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.706410 18407 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.716260 18157 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:07.727126  9915 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.048s	sys 0.087s
W20260709 12:27:07.755348  9915 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.048s	sys 0.088s
W20260709 12:27:07.755942  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:07.756579 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.713146 18008 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:07.758358 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3014) took 1368 ms. Trace:
W20260709 12:27:07.758769  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 170080us
I20260709 12:27:07.762740 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.740335  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:07.763419 10024 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:07.776551 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.776897 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:07.778151  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:07.713907 18369 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.059s
W20260709 12:27:07.778923  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 303393us
I20260709 12:27:07.763610 10092 rpcz_store.cc:276] 0709 12:27:06.390427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.536147 (+145720us) service_pool.cc:224] Handling call
0709 12:27:06.536482 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:06.536804 (+   322us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:07.117010 (+580206us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.757559 (+640549us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:07.757628 (+    69us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:07.757679 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.758129 (+   450us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:07.758281 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17309,"prepare.queue_time_us":75955,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":10360,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":11387,"thread_start_us":1054278,"threads_started":3}
I20260709 12:27:07.781464 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:07.775151 18363 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:07.792363  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:07.802994 18408 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:07.798034 18204 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:07.807090 18204 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:07.807526 18204 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:07.810173  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 298299us
I20260709 12:27:07.811137  9912 leader_election.cc:400] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
I20260709 12:27:07.811434  9912 leader_election.cc:403] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:07.796392 18363 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:07.827137 18363 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:07.827894 18411 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:07.828440 18171 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:07.821866 18185 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 24
W20260709 12:27:07.828925 18413 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:27:07.817162 18408 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:07.832360 18408 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:07.833412 18232 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.762s	user 0.014s	sys 0.001s
I20260709 12:27:07.833838 18232 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:07.831717 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56679us
I20260709 12:27:07.835799 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:07.836678 18157 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:07.836741 18416 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:07.837642 18008 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.772389 18407 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:07.839537 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:07.844367 18214 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.847852 18214 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.844578 18408 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:07.846423 18171 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:07.840672 18348 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.852274 18348 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.846760 18369 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:07.853504 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:07.846143 18204 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.854821 18232 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.840013 18008 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:07.856797 18204 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.860225 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:07.860991 10671 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:07.861966 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:07.864969 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1540) took 1196 ms. Trace:
I20260709 12:27:07.865397 10022 rpcz_store.cc:276] 0709 12:27:06.668883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.670154 (+  1271us) service_pool.cc:224] Handling call
0709 12:27:06.670461 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:07.454138 (+783677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.864700 (+410562us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b5400265cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:07.864891 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1561892608}
I20260709 12:27:07.866827 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:07.870841  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57641.9us
I20260709 12:27:07.871106 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:07.871783 10670 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:07.872655 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:07.873229 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:07.873522  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:07.846208 18407 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:07.879428 10091 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:07.879673  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:07.880079 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4170) took 1466 ms. Trace:
I20260709 12:27:07.880553 10091 rpcz_store.cc:276] 0709 12:27:06.413630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.755934 (+342304us) service_pool.cc:224] Handling call
0709 12:27:06.756255 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:07.624062 (+867807us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54001d8ba0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cba1d 00007f4e094b21a4 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419
0709 12:27:07.879849 (+255787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.880030 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332027136}
I20260709 12:27:07.880263  9948 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
W20260709 12:27:07.882534  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82659.7us
I20260709 12:27:07.862201 18031 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.883845 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:07.884177  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:07.884632 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45223.9us
I20260709 12:27:07.884471 10668 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:07.883596 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:07.886206 10676 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 21 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:27:07.888648  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:07.889097 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:07.889845  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:07.890865  9943 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:07.890758 18008 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:07.891767 18369 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:07.892203 18369 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:07.893469  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:07.893391 18369 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:07.889967 18371 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:07.892072  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:07.898089  9944 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:07.880067 18185 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
W20260709 12:27:07.902765  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 115269us
I20260709 12:27:07.909641 10671 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:27:07.912431  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:07.882781 18407 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:07.913309  9945 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:07.924639  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:07.925601  9946 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:07.902052 18058 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:07.886034 18014 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:27:07.922967 18204 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:07.942463 10674 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:07.949918 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:07.945694 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.939213 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:07.923861 18369 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:07.937788 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:07.937158 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:07.946352 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:07.933760  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49543us
I20260709 12:27:08.030956 10090 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:07.946064 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:07.982465  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 159782us
I20260709 12:27:07.933609  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:07.934314 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.031258  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:08.033356 10091 raft_consensus.cc:2410] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 15.
I20260709 12:27:07.950100 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.033840  9946 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:08.034665 10673 raft_consensus.cc:2410] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 23 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:27:07.945489 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.025465 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:07.914630 18329 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.944481  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:08.027235  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94531.1us
W20260709 12:27:08.029856  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 141330us
W20260709 12:27:08.031096 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.037178 10093 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:08.032032 10096 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:08.026114 18204 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.037923 10669 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:08.038081 10095 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:08.039441 10092 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 28 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
W20260709 12:27:08.042505  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 98159.7us
I20260709 12:27:08.043967  9946 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 21 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:08.040180 10099 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:08.046383 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:08.049458 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:08.026669 18362 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:07.931932 18407 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.045185 17617 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.046317 18204 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:08.057998  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:08.048331 18185 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:08.030032 18363 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.026643 18058 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.050104 18371 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.071269  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.045567 18329 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.080358 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:08.075378 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1711) took 1801 ms. Trace:
I20260709 12:27:08.077786 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.083290 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:08.084602 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:08.084806 10096 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:08.081291 10675 rpcz_store.cc:276] 0709 12:27:06.273504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.306936 (+ 33432us) service_pool.cc:224] Handling call
0709 12:27:06.340633 (+ 33697us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:06.395764 (+ 55131us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.395811 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:06.491565 (+ 95754us) log.cc:844] Serialized 52 byte log entry
0709 12:27:07.255682 (+764117us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:07.558011 (+302329us) log.cc:844] Serialized 14 byte log entry
0709 12:27:07.558662 (+   651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.558839 (+   177us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:07.849822 (+290983us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b18004a6680. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:08.070935 (+221113us) raft_consensus.cc:1685] finished
0709 12:27:08.075026 (+  4091us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.075270 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32883,"prepare.queue_time_us":87705,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":44214,"raft.run_cpu_time_us":377,"raft.run_wall_time_us":36609,"spinlock_wait_cycles":32359296,"thread_start_us":1012900,"threads_started":2,"wal-append.queue_time_us":850671}
I20260709 12:27:08.099126  9944 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:08.069326 18185 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.092121 10671 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:27:08.093904 18058 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:08.102969 18031 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:08.103365 18058 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:08.087839  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:08.111303  9946 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:27:08.072216 18433 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:08.115693 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:08.116534 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.058578 18362 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.122915 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:08.120636 10097 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:08.068944 17617 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.093600 18329 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:08.107137 18185 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:08.130064  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73663.4us
I20260709 12:27:08.131160  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:08.101653 18371 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:08.136138  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.117522 18433 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.137766 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:08.086333 18011 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 12:27:08.119686 18031 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:08.141819 18433 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:08.143680 17617 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:08.122344 18294 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:08.127381 18431 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.148226  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:08.150038  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47626.7us
I20260709 12:27:08.135767 18329 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:08.151657 18434 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:08.152503 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:08.167225 10024 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:08.155048 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:08.168704 10018 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:08.156028 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:08.150364 18014 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:08.171726  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:08.174382 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:08.175652 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:08.181797 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:08.183511  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:08.187397 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 70188.3us
I20260709 12:27:08.190990 10016 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:08.195672 10023 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:08.196048 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:08.179347 18420 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
I20260709 12:27:08.203552 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:08.205190 10017 raft_consensus.cc:2376] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20. Re-sending same reply.
W20260709 12:27:08.206620 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.170373 18014 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:08.209002  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:08.212677  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:08.213387  9940 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:08.218274 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:08.219364  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 69205.3us
I20260709 12:27:08.168259 18399 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:27:08.229997  9945 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:08.230274 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:08.224928 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:08.233592 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:08.235283 10023 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:08.212780 18014 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:08.239213 10017 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:08.242084  9946 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:08.228788 18031 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:08.212378 18329 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
W20260709 12:27:08.252056  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 84698.2us
I20260709 12:27:08.252279  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.186302 18431 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:08.255513 18031 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:08.257189 18362 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:08.244362 18422 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:27:08.259387  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:08.261953 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:08.264725  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60133.9us
I20260709 12:27:08.265548  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:08.267551 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.267520  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:08.268904 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:08.272475 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:08.277091  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.251143 18416 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:08.254256 18363 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:08.260409 18011 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:08.315058  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 99327.9us
I20260709 12:27:08.306581  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:08.281602 17617 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.283950 18434 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.319710  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:08.323140  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94844.8us
I20260709 12:27:08.351073  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.353892  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.354810  9947 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:27:08.351948  9945 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
W20260709 12:27:08.375604 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 103215us
I20260709 12:27:08.376585 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:08.385103  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46493.7us
I20260709 12:27:08.388574 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:08.336899 18011 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:27:08.391791  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:08.393095 10090 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:08.349717 18436 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:08.404304 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:08.408526  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.410773 10018 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: [25.20-25.20]
I20260709 12:27:08.320292 18416 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.376729 18329 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:08.339176 17617 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.343698 18434 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:08.435662 10096 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:08.437744 10024 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:08.424293 18008 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:08.417454 18447 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:08.428176 18444 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:08.428741 18428 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 12:27:08.429792 18452 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.087s
I20260709 12:27:08.439674  9948 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:08.413797 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.430884 18451 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:08.468322 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:08.411808 18454 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:08.403324 18450 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.072s
I20260709 12:27:08.476984 10018 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
W20260709 12:27:08.423148 18426 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
I20260709 12:27:08.482195  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.417026 18363 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.469640 18447 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.422856 18414 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.431098 18371 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:08.474391 18436 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.469287 18008 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.493203 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 3899) took 1229 ms. Trace:
W20260709 12:27:08.487437 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 98939.1us
I20260709 12:27:08.480755 18454 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.403052 18294 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:08.471752 18444 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.538066 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.538328 10021 rpcz_store.cc:276] 0709 12:27:07.260588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.260759 (+   171us) service_pool.cc:224] Handling call
0709 12:27:08.489297 (+1228538us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54001383d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:08.489468 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":739892608}
W20260709 12:27:08.493577  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 150027us
I20260709 12:27:08.559894  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:08.565444 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.489086 18329 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:08.536489 18379 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:27:08.554157 18385 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:08.487437 18461 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.588639 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.598009 10098 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:08.599074 18371 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:08.599162 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.600903 18455 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:08.602064 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63029us
W20260709 12:27:08.603829 18392 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.070s	user 0.006s	sys 0.001s
I20260709 12:27:08.577893 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.490128 18363 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:08.585386 18008 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:08.599778 18371 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.488276 18447 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:08.606076 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:08.573202 18329 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:08.632229 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.539477 18454 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:08.589574 18431 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:08.594727 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 193438us
I20260709 12:27:08.625991  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.645882 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3906) took 1085 ms. Trace:
I20260709 12:27:08.646744  9948 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:27:08.643906  9947 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:08.631196 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:08.645869  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85945.9us
I20260709 12:27:08.622062 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:08.622491 10094 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:08.569789 18446 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.082s
I20260709 12:27:08.551002 18436 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.651887 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:08.639233 17617 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:08.653496  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.492777 18414 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.706142  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:08.665025 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:08.641081 18329 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:08.707324 18445 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.317s	user 0.007s	sys 0.004s
I20260709 12:27:08.707518  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.708253 18343 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:08.708668 18343 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.709014 18343 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:08.708937  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 322057us
I20260709 12:27:08.666965  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.646981 10025 rpcz_store.cc:276] 0709 12:27:07.560631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.560844 (+   213us) service_pool.cc:224] Handling call
0709 12:27:07.583495 (+ 22651us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:07.644900 (+ 61405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.644938 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:07.689260 (+ 44322us) log.cc:844] Serialized 52 byte log entry
0709 12:27:07.878301 (+189041us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:07.880123 (+  1822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.880192 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.645583 (+765391us) raft_consensus.cc:1685] finished
0709 12:27:08.645613 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.645811 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9124,"raft.queue_time_us":22989,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"thread_start_us":131790,"threads_started":1,"wal-append.queue_time_us":244258}
I20260709 12:27:08.702927 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.661374 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:08.711159  9941 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:08.711738 10092 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:08.711102  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.716158  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:08.716738  9941 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:27:08.717284 18343 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.719506 18343 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:08.719924 17617 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:08.724939 18329 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.727685 18329 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:08.728875 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67521.4us
I20260709 12:27:08.640086 18216 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.650977 18444 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:08.737061 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72156.4us
I20260709 12:27:08.737610 18455 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:08.738734 18448 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:08.739013  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.779757 18448 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.651723 18452 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:27:08.754349 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.731508  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.816700  9941 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:27:08.758692  9940 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.532s	user 0.015s	sys 0.000s
I20260709 12:27:08.736228 18436 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:08.773768 10065 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:08.756727 18428 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:27:08.780004  9948 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:08.739358 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:08.629885 18461 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.784929 18414 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:08.786412 18363 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:08.787011 18216 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.787609 18371 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:08.803944 18470 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:27:08.807437  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:08.833137  9943 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:08.811443 18453 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 12:27:08.821617 18014 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.574s	user 0.011s	sys 0.000s
I20260709 12:27:08.812664 18416 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:08.764519 10098 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:27:08.818712  9940 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
W20260709 12:27:08.771958 18204 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.698s	user 0.012s	sys 0.000s
I20260709 12:27:08.781149 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:08.755272 18459 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:27:08.823176  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:08.824448 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:08.816398 18448 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:08.757601 17617 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:08.816777 18452 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:08.837608 18216 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:08.842800 10098 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:27:08.841908 18371 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:08.847249 18414 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:08.835456 18434 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:08.856534 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 75509.9us
I20260709 12:27:08.857780 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:08.849885 17617 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:08.849414 18437 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.664s	user 0.003s	sys 0.000s
I20260709 12:27:08.864316 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:08.865734  9942 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:27:08.867587 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1772) took 1013 ms. Trace:
I20260709 12:27:08.868027  9942 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:27:08.869738 10673 raft_consensus.cc:1217] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Deduplicated request from leader. Original: 16.12->[19.13-22.15]   Dedup: 22.15->[]
I20260709 12:27:08.870244 10673 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: [19.13-22.15]
W20260709 12:27:08.871716 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 98031.8us
I20260709 12:27:08.865950 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:08.875247 10094 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:27:08.876397 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:08.893548 18371 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:08.894395 18371 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.895108 18371 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:08.868000 10672 rpcz_store.cc:276] 0709 12:27:07.853912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.854029 (+   117us) service_pool.cc:224] Handling call
0709 12:27:07.854416 (+   387us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:27:07.855041 (+   625us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:07.855073 (+    32us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 12:27:07.855865 (+   792us) log.cc:844] Serialized 104 byte log entry
0709 12:27:07.946337 (+ 90472us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:08.036832 (+ 90495us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.045658 (+  8826us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.047725 (+  2067us) log.cc:844] Serialized 14 byte log entry
0709 12:27:08.052808 (+  5083us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:08.052869 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.867353 (+814484us) raft_consensus.cc:1685] finished
0709 12:27:08.867373 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:08.867526 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47224,"prepare.queue_time_us":177229,"prepare.run_cpu_time_us":171,"prepare.run_wall_time_us":9323,"raft.queue_time_us":20023,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":4303,"spinlock_wait_cycles":37028992,"thread_start_us":90029,"threads_started":1,"wal-append.queue_time_us":205729}
I20260709 12:27:08.854004 18460 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.024s
I20260709 12:27:08.841182 18014 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.878367 10674 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:08.883111 18343 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:08.882927 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:08.886930 18434 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:27:08.889103 18462 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
W20260709 12:27:08.891291 18468 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:27:08.893692 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:08.843016 18204 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:08.869073 18436 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:08.876881 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:08.893517 10676 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:08.894315 18185 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.756s	user 0.011s	sys 0.002s
I20260709 12:27:08.903471 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.941928 10098 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:08.943125 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:08.905304 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.893862 18407 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:08.912243  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:08.875998 18369 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.841s	user 0.014s	sys 0.000s
I20260709 12:27:08.869874 18232 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:08.922788 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:08.928733 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:08.932094 18456 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:08.932241 17617 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:08.932878 18398 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:08.932915 18487 raft_consensus.cc:2764] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:27:08.948802 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:08.942265 18436 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:08.926347 18343 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:08.962019 10097 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:08.953737 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:08.963500  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54813.2us
I20260709 12:27:08.950379 18414 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:08.977700 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:08.952677 18456 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:08.950896 18185 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:08.882927 18452 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.056130  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2851) took 1154 ms. Trace:
I20260709 12:27:09.054337 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1538) took 2386 ms. Trace:
I20260709 12:27:08.958319 18477 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:08.987987 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:09.084030 10093 raft_consensus.cc:2410] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 20 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 15.
I20260709 12:27:08.959853 18369 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:08.976523 10094 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
W20260709 12:27:08.970721 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 266392us
W20260709 12:27:09.082331 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 104726us
I20260709 12:27:09.080504  9944 rpcz_store.cc:276] 0709 12:27:07.902337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:07.915702 (+ 13365us) service_pool.cc:224] Handling call
0709 12:27:07.932033 (+ 16331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.099816 (+167783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.055781 (+955965us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b540042de54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:09.056044 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383729920}
I20260709 12:27:08.992748  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:09.088193 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:08.958408 18398 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:08.966166 18436 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:09.079406 18427 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:27:09.094316 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:09.098476  9946 raft_consensus.cc:2368] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22.
I20260709 12:27:09.043965 18363 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:08.978708 18216 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:09.080498 10020 rpcz_store.cc:276] 0709 12:27:06.668233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.669342 (+  1109us) service_pool.cc:224] Handling call
0709 12:27:06.669690 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:06.670181 (+   491us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:06.670214 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:06.670754 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:27:07.113302 (+442548us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:07.113361 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:07.113451 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:08.404623 (+1291172us) spinlock_profiling.cc:243] Waited 4.97 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:09.053347 (+648724us) spinlock_profiling.cc:243] Waited 3.78 ms on lock 0x7b2400404408. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a7fb0 00007f4e0911e488 00007f4e0911ebe8 00007f4e090f6398 00007f4e090f176a 00007f4e090f0bb6
0709 12:27:09.054079 (+   732us) raft_consensus.cc:1685] finished
0709 12:27:09.054102 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.054268 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":657,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"spinlock_wait_cycles":20118528,"thread_start_us":442067,"threads_started":1,"wal-append.queue_time_us":703711}
I20260709 12:27:09.101186 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:09.101442 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.107888  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.127789  9944 raft_consensus.cc:2368] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20.
I20260709 12:27:09.087788 18369 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.125276 18216 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.129961 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.094910 18452 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:09.135921 18008 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.101265 18477 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:09.142765 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:09.142926 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:09.143133 18477 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.144292 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.145313 18464 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:09.145597 18455 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:09.145781 18464 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.146304 18464 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:09.155321  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 162710us
I20260709 12:27:09.099002 17617 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:09.156510  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:09.159358 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.160332 10021 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:09.085819 18414 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.163185 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.129493 18216 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:09.164877 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.165128 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.135037 18369 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:09.130215 18479 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.259s
I20260709 12:27:09.168728 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.168826 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.134536 18343 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.169713 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.156807 18456 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.170950 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.171113 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.171878 18491 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.125929 18408 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:09.180068 10018 raft_consensus.cc:2410] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:27:09.178484 10023 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:09.169118 10020 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:09.168792 18483 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 12:27:09.102936 18416 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:27:09.128654 18363 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.186509 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:09.187189 10025 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:09.193020 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.180485 18014 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.195794 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.216703 10020 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:09.196574 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.158975 18398 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.219623 18398 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:09.219928 18461 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.221813 18373 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.198019 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.222393 18373 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.224619 18373 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.198278 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.202919 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.203297 18185 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.176322 17617 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.229405 17617 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.183950 18491 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.233412 18491 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:09.233778 18447 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.588s	user 0.008s	sys 0.002s
I20260709 12:27:09.235459 18456 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:09.210212 18440 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.981s	user 0.002s	sys 0.000s
W20260709 12:27:09.248051 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.244227 10097 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:09.249845 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.252638 10024 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:09.193751 18204 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:09.234311 18447 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.268766  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:09.270992  9948 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:09.269317  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.196360 18008 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:09.274124 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.276960 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.280067  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.280092 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.282459 10018 raft_consensus.cc:2368] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20.
I20260709 12:27:09.283387  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.286286 10668 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:09.248467 18448 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.291910  9948 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
W20260709 12:27:09.287071 18454 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.001s
I20260709 12:27:09.298070 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.298456 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:09.302356 10020 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:27:09.305032 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.306252 10024 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:09.307060 18487 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:09.308732 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:09.309602 18493 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:27:09.320597 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1472) took 1270 ms. Trace:
I20260709 12:27:09.321169 10673 rpcz_store.cc:276] 0709 12:27:08.049994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.053715 (+  3721us) service_pool.cc:224] Handling call
0709 12:27:08.869620 (+815905us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:08.870525 (+   905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.320346 (+449821us) spinlock_profiling.cc:243] Waited 807 ms on lock 0x7b540073afd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:09.320522 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1857225984}
I20260709 12:27:09.299906 18454 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.342649 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:09.343912 10024 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:09.344283 18454 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:09.346134 10017 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:27:09.311703 18371 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:09.345186 18433 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.166s	user 0.014s	sys 0.000s
I20260709 12:27:09.351027 18414 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:09.356860 18371 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:09.358273  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:09.358915  9948 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:09.363169 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.351045 18232 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:09.354660 18433 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.377010 18433 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:09.377934 10016 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:09.398381 10025 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:09.367159 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.388798  9948 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:09.410162 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.411207 10025 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:09.382151 18363 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.395598 18371 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.395386 18447 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.403393 18448 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:09.424881 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.448390 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.452136 18464 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.452579  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:09.454492 18456 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:09.455088 18456 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:09.455986 18456 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:09.456362 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.459172 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:09.460587 18232 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:09.461153 18232 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:09.460615 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:09.462041 18232 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.462992 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.463693 10024 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:09.464254 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.464437 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.464887 10065 leader_election.cc:400] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20.
W20260709 12:27:09.465197  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:09.466328 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.467023 18476 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:09.464917 10019 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:09.465199 10065 leader_election.cc:403] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:09.468612 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.468461 18494 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:09.472852 18494 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:09.475875  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2864) took 1079 ms. Trace:
I20260709 12:27:09.481882 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:09.479101  9947 rpcz_store.cc:276] 0709 12:27:08.396332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.467321 (+ 70989us) service_pool.cc:224] Handling call
0709 12:27:09.475699 (+1008378us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54003a0554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:09.475816 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":363149184}
I20260709 12:27:09.482703 10096 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:09.484104 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:09.485136 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:09.495453 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:09.497298 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:09.499694 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:09.500213 10097 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:09.494045 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:09.505487 10095 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:09.509522 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:09.512940 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:09.515341 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:09.516533 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:09.518712 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:09.519719 18373 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:09.520324 17617 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:09.521665 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:09.526382 10097 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:09.450213 18436 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.454474 18464 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:09.493304 18416 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:09.471879 18371 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.498687 18496 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:09.508119 18491 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:09.498817 18497 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20.
I20260709 12:27:09.541600 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:09.534795 10096 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:09.552948  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:09.556308 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.556970 10025 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
W20260709 12:27:09.557061 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.559907 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:09.564000 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:09.565210 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:09.552125 18373 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:09.563215 18496 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:09.544618 18416 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.606741 18416 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:09.560928 18402 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:09.547540 18446 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:27:09.614991 18446 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:09.615406 18446 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:09.616295 18446 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.616739 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:09.617710 18446 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:09.626125 10098 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:09.628935 10062 leader_election.cc:400] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20.
I20260709 12:27:09.630759 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.618474 18416 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:09.561234 18008 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:09.636508 18436 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:09.568398 18371 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.646610 18369 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.622630 18373 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:09.645862 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:09.627272 18216 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.666556 18008 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:09.695899 17617 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:09.691818 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 197839us
W20260709 12:27:09.694505 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48762.5us
I20260709 12:27:09.698228 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:09.698482 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:09.699174 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:09.699962 18491 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:09.700456 18491 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:09.695407  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:09.711870 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:09.715386  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.717330 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:09.718094 10097 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:09.724828 10095 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:09.726146 10092 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:09.727211 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.695319 18373 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:09.727833 10023 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:09.737720  9945 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:09.738445  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.739163  9944 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:09.697391 17617 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.713209 18416 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:09.721231 18402 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:09.721890 18497 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:09.758152 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:09.725854 18371 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:09.724850 18250 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:09.776598 18250 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:09.777562 18250 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.755257 18008 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:09.780716 18008 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.787107 10097 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:09.701355 18491 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.757999 18497 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.797284 18514 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:09.798023 18514 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.795874 18369 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:09.766783 18416 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:09.807238  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.775652 18402 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:09.775854 18371 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:09.810191 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.811743  9945 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
W20260709 12:27:09.813650 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.813308  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:09.800460 18514 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.808100 18497 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:09.823458  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:09.815922 10643 connection.cc:256] RPC call timeout handler was delayed by -1.0109218679999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
I20260709 12:27:09.825919 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1475) took 1707 ms. Trace:
I20260709 12:27:09.826179  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:09.813766 18371 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:09.829989 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:09.830796 10669 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:09.827042 18350 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:09.831822 18350 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.832324 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:09.822371 18362 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:09.834031 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:09.834149 18350 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:09.835120 17649 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:09.835309  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.811885 18512 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:09.836555 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:09.837304 10669 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:09.837093 18512 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.840034 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:09.815388 18517 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:09.841504 18517 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.841892 18517 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:09.843695  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.844493 18476 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:09.845085 18476 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.847627 18216 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:09.848242 18476 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.848515 18402 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.849004  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.849803  9944 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:09.828656 18456 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:09.852483  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.853675  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:09.853724 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4203) took 1443 ms. Trace:
I20260709 12:27:09.854286 10094 rpcz_store.cc:276] 0709 12:27:08.410644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.654108 (+243464us) service_pool.cc:224] Handling call
0709 12:27:09.853540 (+1199432us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b54002fce54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:09.853652 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122928512}
I20260709 12:27:09.840087 18512 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.826342 10675 rpcz_store.cc:276] 0709 12:27:08.118427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.118558 (+   131us) service_pool.cc:224] Handling call
0709 12:27:08.134458 (+ 15900us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:08.148317 (+ 13859us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:08.148339 (+    22us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:08.153318 (+  4979us) log.cc:844] Serialized 52 byte log entry
0709 12:27:09.478941 (+1325623us) spinlock_profiling.cc:243] Waited 6.4 ms on lock 0x7b18003d0de0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:09.547697 (+ 68756us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:09.818454 (+270757us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.820566 (+  2112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.820972 (+   406us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:09.821025 (+    53us) raft_consensus.cc:1685] finished
0709 12:27:09.825700 (+  4675us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.825861 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":251,"prepare.queue_time_us":89595,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":19882,"raft.queue_time_us":4627,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":9886,"spinlock_wait_cycles":14716160,"thread_start_us":332301,"threads_started":2,"wal-append.queue_time_us":1417509}
I20260709 12:27:09.827419 18008 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:09.859367  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.866201  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:09.867611  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.869240  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.854095 17649 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:09.822937 18369 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:27:09.836107 18416 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.871937 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.872174 10091 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:09.872949  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:09.873538  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.864261 18460 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:09.879746 18446 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:09.885164 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:09.888273 10673 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:09.894752 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:09.899194 18484 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.021s	user 0.004s	sys 0.000s
I20260709 12:27:09.886036 18451 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:09.893669 18456 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:09.900405 18516 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:27:09.901206 18407 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:09.901593 18456 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:09.902004 18456 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:09.902822 18456 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.904310 18456 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:09.893251 18350 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:27:09.880927 17649 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:09.896438 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.911090 18446 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:09.927969 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 153852us
I20260709 12:27:09.937799 18446 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:09.932814  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.939985 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:09.940987 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:09.941674 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.941888 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:09.942728 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.942957 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.943944 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:09.923694 18434 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:27:09.946458 18446 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:09.941021 10021 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:09.948127 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:09.949070 10091 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
W20260709 12:27:09.955116 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:09.956223 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:09.962150  9990 leader_election.cc:400] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
I20260709 12:27:09.962510  9990 leader_election.cc:403] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:09.961308 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:09.967993  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3152) took 1330 ms. Trace:
I20260709 12:27:09.972940 18451 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:09.972937 18402 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:09.974323 18494 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:09.948885 18350 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:09.971408  9943 rpcz_store.cc:276] 0709 12:27:08.637268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.848753 (+211485us) service_pool.cc:224] Handling call
0709 12:27:08.856565 (+  7812us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.863304 (+  6739us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:08.909398 (+ 46094us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.123997 (+214599us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.463787 (+339790us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18000575e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:09.802571 (+338784us) spinlock_profiling.cc:243] Waited 3.06 ms on lock 0x7b18000575e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:09.965067 (+162496us) spinlock_profiling.cc:243] Waited 5.31 ms on lock 0x7b18000575e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:27:09.967551 (+  2484us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:09.967623 (+    72us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:09.967681 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.967766 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.967916 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":830802,"prepare.queue_time_us":71704,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":91,"raft.queue_time_us":846774,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":12222,"spinlock_wait_cycles":26672512,"thread_start_us":176980,"threads_started":1,"wal-append.queue_time_us":246192}
I20260709 12:27:09.964742 18464 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:09.978713  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 822243us
I20260709 12:27:09.970892 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:09.980014 10097 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:09.980536 10643 leader_election.cc:400] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22.
I20260709 12:27:09.980892 10643 leader_election.cc:403] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:09.979907  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:09.981779 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:09.985459 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.910544 18497 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:09.985957 18362 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:09.993470  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:09.990489 10643 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.989751  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3156) took 1351 ms. Trace:
I20260709 12:27:09.977540 18451 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:09.985064 18457 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:10.006286 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.040059 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:09.997766  9942 rpcz_store.cc:276] 0709 12:27:08.638566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.918991 (+280425us) service_pool.cc:224] Handling call
0709 12:27:08.923496 (+  4505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.943445 (+ 19949us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:09.194707 (+251262us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.989372 (+794665us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:09.989431 (+    59us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:09.989491 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:09.989565 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:09.989694 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":674353,"prepare.queue_time_us":135095,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":674512,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":313403,"threads_started":2,"wal-append.queue_time_us":135637}
I20260709 12:27:10.002556 18358 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.002722 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.042474  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.003273 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:10.006505  9948 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:10.043239  9942 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.006865 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.007404  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.045675  9947 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:10.007922  9944 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:10.008953  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:10.011394  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.042531 10669 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:10.016247 18031 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:10.016975 18373 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.017779 18008 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:10.018537 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1783) took 1141 ms. Trace:
W20260709 12:27:10.022634 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 73077.7us
I20260709 12:27:10.056591 10668 rpcz_store.cc:276] 0709 12:27:08.876780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.876892 (+   112us) service_pool.cc:224] Handling call
0709 12:27:10.018338 (+1141446us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:10.018452 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861172480}
I20260709 12:27:10.023165 17617 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:10.024623 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:10.061906 10090 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:10.028506 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:10.029210 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:10.030305 10020 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:10.030956 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:10.031601 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:10.032292 18476 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:10.030963 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 74874.6us
I20260709 12:27:10.034433  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:10.036034 18363 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:09.999635 18444 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:10.014809 18477 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:10.037475 10675 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:10.044641 10673 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:27:09.996333 18407 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:10.051750  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.080451  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.081583  9948 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:10.082981  9942 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:10.084628  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.048202 18232 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:09.989748 18464 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:10.060299  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.059450  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.072865 10017 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 28: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:10.074831  9943 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:10.076716 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.020926 18517 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.085245  9945 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:09.990845 18250 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:10.075434 18363 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.075277 18516 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:10.074326 18530 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:10.053467 18350 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:27:10.059897 18008 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:10.150167 18008 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.151970 18008 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.153591 18431 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.054384 18356 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.076s
I20260709 12:27:10.074819 18444 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.054163 18443 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:10.071936 18371 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.118767 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:10.175484 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56900.7us
I20260709 12:27:10.133986 18250 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:10.131749 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 55116.4us
I20260709 12:27:10.182755 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.136525  9944 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:10.144230  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.146649 18530 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.147512 18350 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:10.173780 18443 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:10.144680 18451 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.185063 18369 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:10.203329 18443 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:10.205701 18517 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:10.203341 18369 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:10.207741 18369 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.206691 18464 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.212206  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1807) took 1892 ms. Trace:
W20260709 12:27:10.212616  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 196605us
I20260709 12:27:10.213591  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.184705 18250 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:10.219729 18461 raft_consensus.cc:2764] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:27:10.199990  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.221433  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.221678  9944 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:10.111207  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45895.9us
I20260709 12:27:10.166316 18030 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 12:27:10.197927  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.224287 18434 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:10.224732 18434 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:10.225639 18434 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.137844 17617 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:10.230919 17617 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:10.231451 17617 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.200704 18530 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:10.197273 18491 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.235360 18461 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:10.235827 18461 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:10.236979  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.237880 18363 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:10.238307  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.236186 18461 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:10.240906 18436 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:10.241003 18373 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:10.241413 18436 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:10.242664  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.243314  9947 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:10.243359  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:10.244843  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:10.245642 18030 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:10.246085  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:10.246394 18030 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:10.247519 18030 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:10.157160 10021 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:10.154333 18431 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.214480 18204 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:10.212675  9949 rpcz_store.cc:276] 0709 12:27:08.320134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:08.320395 (+   261us) service_pool.cc:224] Handling call
0709 12:27:08.341051 (+ 20656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:08.577371 (+236320us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:27:08.595698 (+ 18327us) log.cc:844] Serialized 14 byte log entry
0709 12:27:09.467410 (+871712us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b180031e3a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a
0709 12:27:10.003882 (+536472us) log.cc:844] Serialized 14 byte log entry
0709 12:27:10.016879 (+ 12997us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:10.016935 (+    56us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:10.016983 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.211957 (+194974us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540003eb54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:10.212018 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.212147 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":539811,"prepare.queue_time_us":25310,"prepare.run_cpu_time_us":127,"prepare.run_wall_time_us":3502,"raft.queue_time_us":549644,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":177,"spinlock_wait_cycles":546665984,"thread_start_us":34553,"threads_started":1,"wal-append.queue_time_us":273748}
I20260709 12:27:10.214790 10096 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:10.129824 18232 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.216756 18371 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:10.216954 18494 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.174333 18497 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:10.198347 18444 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:10.247937  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:10.237421  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.248762  9945 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:10.260812  9948 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:27:10.177117  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.160977  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.134912 18407 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
W20260709 12:27:10.256392 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 217813us
I20260709 12:27:10.141230 18399 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.111s
I20260709 12:27:10.259483  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.275704  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.276171 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.260916 18185 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:10.277498 18031 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:10.263538 18373 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:10.275966 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:10.279193 17649 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.279892 18497 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:10.280470 18497 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:10.185212 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.263738  9942 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:27:10.287045  9941 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:10.295460 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.296192 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 111061us
I20260709 12:27:10.297246 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.261452 18348 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:10.299062  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:10.299455  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.304018 10023 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:10.267392 18434 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.311045 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:10.268347 18454 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:10.278566 18373 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.313292  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.316239 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:10.323875 10097 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:10.128378 10090 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:10.317299 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:10.327816  9948 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:10.327927  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.256032 18204 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.331810 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:10.263363 18473 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 12:27:10.339525 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:10.340494 10094 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: [21.17-22.18]
I20260709 12:27:10.341509 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:10.343667  9949 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:10.303817 18350 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:10.312104 18008 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:10.346060 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:10.323033 18031 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:10.390730 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44769.5us
I20260709 12:27:10.392320 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:10.346644 18216 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:10.348620 18473 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.349385 18371 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:10.351153 18460 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:10.351742 18477 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:10.326709 18185 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:10.256139 18436 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.352705  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.352942 18171 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:10.353957 18232 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:10.355736 17649 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:10.390389 18454 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 12:27:10.358899 18204 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:10.406481 18530 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:10.359267 18399 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:10.361932 10099 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:10.324419 18479 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:10.271849 18250 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.335631 18444 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:10.323628 18521 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:10.342087 18518 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:10.376153 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 336108us
I20260709 12:27:10.394416  9947 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
W20260709 12:27:10.397953 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56527.7us
I20260709 12:27:10.523105  9942 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:10.345324 18014 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:27:10.369995 18373 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:10.440367 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:10.421056 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:10.525019  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 201774us
I20260709 12:27:10.387300 18491 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:10.421769  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.480237 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:10.481025 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:10.531890 10019 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:10.479305 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:10.512483 18436 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.395838 18362 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22.
I20260709 12:27:10.491461 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:10.365835 18497 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.357009 18431 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:10.408190 18416 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:10.408442 18408 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:10.394969 18457 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:10.406107 18371 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.555121 18371 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:10.450171 18496 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:10.509977  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.483880 18402 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
I20260709 12:27:10.492631 18031 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:10.528046  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:10.532770 10021 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:10.485271  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 132640us
W20260709 12:27:10.539389  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 220693us
I20260709 12:27:10.543289 10020 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:10.409307 18473 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.545445 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:10.479303 18460 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.555024 18451 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:10.449946 18514 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:10.517302 18521 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:10.564646 10019 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:10.565169  9943 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:10.527508 18482 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.525429 18479 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:10.584554 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.577658  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:10.618466  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:10.546895 18014 consensus_queue.cc:1243] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 24.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:10.596105  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.554332 18457 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:10.597755  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 69803.8us
I20260709 12:27:10.548264 18399 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:10.528267 18461 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.600901 10021 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
W20260709 12:27:10.603283 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.602017 10020 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
W20260709 12:27:10.607060 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61734.2us
W20260709 12:27:10.621228 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 172633us
I20260709 12:27:10.594584 18473 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:10.624851 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:10.652212 18250 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:10.617130 18521 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:10.654691 18031 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:10.655339 18004 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:10.650038 17649 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:10.615167 18030 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:10.659484 18534 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:27:10.660115 10094 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:27:10.672528  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 76532.4us
I20260709 12:27:10.673321  9941 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:10.675508 10095 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:10.686899  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.675287 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:10.677383 10090 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:10.616405 18541 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.683323 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.616747 18232 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:10.623932 18399 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:10.661213 18543 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:10.619493 18514 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:10.692166 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.700033 10675 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:10.700393 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.695029  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.695158 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:10.696583  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.687189 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:10.699007 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.703594 10669 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:10.690969 18004 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:10.672228 18477 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
W20260709 12:27:10.703773 18528 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 12:27:10.703666 18535 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:27:10.699119 18473 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:10.622381 18482 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.699604 18543 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:10.689004 17649 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.711175  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:10.728369 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:10.738001  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:10.738730  9943 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:10.713598 18497 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:10.705389 18546 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:10.733363  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.706323 18348 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:10.714102 18531 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:10.716943 18527 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:27:10.694278 18541 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.685221 18451 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:10.730754 18477 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:10.707672 18250 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:27:10.736593 18543 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:10.776108 18538 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
W20260709 12:27:10.776866 18539 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:27:10.777561 18540 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
W20260709 12:27:10.782299 18544 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.094s	user 0.002s	sys 0.001s
W20260709 12:27:10.750205 17617 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.001s
I20260709 12:27:10.786532 10016 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:10.744058 18497 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:10.815683 10094 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:10.747313 18546 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:10.791718  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:10.806380  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.785717 18473 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:10.796332  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85243.7us
I20260709 12:27:10.769466 18532 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:10.819036  9944 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:10.784322 18477 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:10.804523 18550 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:10.836102 18541 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:10.754928 18514 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.837627 18514 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:10.824288 18532 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.823215  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:10.798791  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.825454  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:10.825903 18358 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.774s	user 0.008s	sys 0.004s
I20260709 12:27:10.786403 18482 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:10.803596 18460 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:10.828061 18477 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.848675 18477 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.805028 18461 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:10.821115 18546 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:10.818300 17617 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.853736 17617 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:10.818285 18497 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:10.830029 18443 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.626s	user 0.009s	sys 0.002s
I20260709 12:27:10.749686 18399 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.832926 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:10.795773 18451 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:10.705006 18479 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.871150  9947 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:10.865535 10676 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.840250  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.821538 18473 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:10.875641  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:10.876549  9944 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
W20260709 12:27:10.879191  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 204903us
I20260709 12:27:10.874994  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2870) took 1598 ms. Trace:
I20260709 12:27:10.881435  9946 rpcz_store.cc:276] 0709 12:27:09.276303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:09.280985 (+  4682us) service_pool.cc:224] Handling call
0709 12:27:09.285047 (+  4062us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:09.985251 (+700204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:09.985280 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:09.991103 (+  5823us) log.cc:844] Serialized 26 byte log entry
0709 12:27:10.210072 (+218969us) spinlock_profiling.cc:243] Waited 5.59 ms on lock 0x7b1800459340. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:10.398944 (+188872us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:10.399002 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.400416 (+  1414us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:10.856620 (+456204us) raft_consensus.cc:1685] finished
0709 12:27:10.856644 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:10.874907 (+ 18263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":699718,"raft.queue_time_us":699855,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":2518,"spinlock_wait_cycles":12846464,"thread_start_us":188656,"threads_started":1,"wal-append.queue_time_us":608533}
I20260709 12:27:10.804755 18004 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.856251 18419 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:10.897723  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:10.864389 18358 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:10.898931 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.904003 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.905578 10672 raft_consensus.cc:2368] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
I20260709 12:27:10.907954 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.871627 18530 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:10.874763 18473 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:10.921239 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.921648 10091 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:10.920631 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:10.923117 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:10.928925 10674 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:10.929143 10099 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:10.894503 18461 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:10.902104 18514 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:10.940927 18451 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.942490 10022 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:10.943051 10022 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:10.943825 10022 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:10.940982 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.955562 10676 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
W20260709 12:27:10.932344 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 122401us
I20260709 12:27:10.899005 18419 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:10.964767 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:10.871054 18443 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:10.911809 18444 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:27:10.933323 18363 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:27:10.985392  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:10.985639 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:10.962389 18419 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:10.991648  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94043.3us
I20260709 12:27:10.992868  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:10.933979 18431 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:27:10.996945  9945 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:10.928563 18530 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.002357 18461 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:11.002863 18461 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:11.003233 18461 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:11.003847 18530 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:10.986660 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 192091us
I20260709 12:27:11.008025 18408 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:11.008625 18408 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:11.020166 17649 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.026387  9940 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:11.034739  9942 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:11.035392  9942 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:11.036571  9942 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.070869  9942 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:10.948832 18514 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:11.121886 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.083998 17649 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.135700 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 107367us
I20260709 12:27:11.138456  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:11.106302 18371 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.003s
W20260709 12:27:11.117959 18436 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.575s	user 0.010s	sys 0.002s
I20260709 12:27:11.172225 18521 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.105077 18543 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.176554 18543 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:11.139793 18416 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.000s
I20260709 12:27:11.177455 18416 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:11.127831 18431 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:11.148640  9943 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:11.223857  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2885) took 1042 ms. Trace:
I20260709 12:27:11.224298  9945 rpcz_store.cc:276] 0709 12:27:10.181020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.350390 (+169370us) service_pool.cc:224] Handling call
0709 12:27:10.350741 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:11.001607 (+650866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:11.223604 (+221997us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:11.223794 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485674240}
I20260709 12:27:11.229892 10090 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:11.198931 18521 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.232900  9947 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 18. (term mismatch)
I20260709 12:27:11.233323  9947 pending_rounds.cc:85] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 17
I20260709 12:27:11.233498 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:11.237624 18514 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.128159 18546 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.233561  9947 pending_rounds.cc:107] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 22.18
I20260709 12:27:11.241751  9947 raft_consensus.cc:2889] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:11.242820 18546 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.156548 18371 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.244011 10096 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:27:11.234798 18532 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:11.236455 18443 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.237306 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:11.186370 18431 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:11.234355 18531 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:11.247609 10673 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:11.244840 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:11.149912 17649 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.254194  9942 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 24 index: 19. (term mismatch)
I20260709 12:27:11.254583  9942 pending_rounds.cc:85] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 18
I20260709 12:27:11.254829  9942 pending_rounds.cc:107] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 23.19
I20260709 12:27:11.255044  9942 raft_consensus.cc:2889] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:11.255824 18514 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:11.256361 18514 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:11.257277 18514 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.259477 18457 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:11.263533 18363 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:11.263963 18363 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:11.264848 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:11.264734 18363 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.265220 10643 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:11.265813 18363 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:11.266746 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:11.259684 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:11.270511 10670 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:11.281556 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.283035 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.250886 18546 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:11.287361 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.255092 17649 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.295639 10091 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:11.306399 10095 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:11.331457 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:11.275024 18461 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.333369 10020 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:11.286801 18548 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:11.336772  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:11.305639 18436 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:11.330216 18460 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.309271 18358 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 12:27:11.335706 18546 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:11.387537 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 117416us
I20260709 12:27:11.361771 18454 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:11.416540 10020 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:11.364071 18408 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.361699  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:11.392602 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:11.339668 18532 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:27:11.370760 18548 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:11.360445 18456 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:27:11.336580 18431 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.374620 18446 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 12:27:11.388628 18460 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:11.451069 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 58520.2us
I20260709 12:27:11.464207 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:11.453058  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 91637.7us
W20260709 12:27:11.451722  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 90808.5us
I20260709 12:27:11.434875 18436 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:11.458400 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:11.472872 18530 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:11.475407 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:11.466470  9987 leader_election.cc:400] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
I20260709 12:27:11.480504  9987 leader_election.cc:403] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:11.487255 10090 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:27:11.460072 17649 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:11.464963 18416 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:11.495949  9948 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:11.495780  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:11.500877 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:11.462863 18521 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.504293  9948 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:11.504956  9943 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:11.468300 18454 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:11.483568 18523 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:11.511843  9945 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:11.508276  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:11.496295 18497 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.001s
I20260709 12:27:11.490235 18371 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.472918 18407 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:27:11.526724  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:11.528628  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:11.529001  9942 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
W20260709 12:27:11.509565 18569 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 12:27:11.499975 17617 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:27:11.530646 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:11.505395 18571 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22.
I20260709 12:27:11.525620 18454 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.549875 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:11.490442 18171 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:27:11.552790 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:11.559461 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.560297 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:11.482635 18546 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.562577 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:11.509614 18558 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:27:11.508877 18521 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:11.577004  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 68776us
I20260709 12:27:11.576170 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:11.553965 17649 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.594910 17617 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:11.595947 17617 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:11.583761 18419 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.509s	user 0.008s	sys 0.002s
I20260709 12:27:11.527287 18497 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.585593 18532 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:11.597782 18559 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 12:27:11.609007  9987 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.931s (SENT) [suppressed 22 similar messages]
I20260709 12:27:11.526240 18548 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.595515 18416 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:11.616560 17617 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:11.554894 18436 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:11.602967 18473 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.668s	user 0.010s	sys 0.004s
I20260709 12:27:11.625670 10094 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:11.575917 18582 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 15, Time since last communication: 0.036s
W20260709 12:27:11.611429 18553 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
W20260709 12:27:11.610558 18579 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 12:27:11.623907 18560 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
I20260709 12:27:11.597152 17649 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.605944 18419 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.631845 18408 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:11.637683 18171 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:11.650736 18416 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.658165 10672 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:11.642904 18558 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.663959  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.641122 18588 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:11.633255 18473 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.655293 18171 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:11.665788 18460 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:11.685252  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 76283.5us
W20260709 12:27:11.686077  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.693288 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:11.696290 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:11.700973 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:11.703549 10025 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:11.709398 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.711390 10025 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:11.717926 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.723693 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1739) took 1016 ms. Trace:
I20260709 12:27:11.725814 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1735) took 1046 ms. Trace:
W20260709 12:27:11.731693 18583 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:27:11.732590 18454 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.729470 10668 rpcz_store.cc:276] 0709 12:27:10.679519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.680006 (+   487us) service_pool.cc:224] Handling call
0709 12:27:10.680449 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.680960 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:10.681004 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:10.681621 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 12:27:10.902526 (+220905us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:10.904962 (+  2436us) log.cc:844] Serialized 14 byte log entry
0709 12:27:10.905193 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.906526 (+  1333us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:11.232677 (+326151us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b480002ea58. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:27:11.722274 (+489597us) raft_consensus.cc:1685] finished
0709 12:27:11.725544 (+  3270us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.725739 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47330,"prepare.queue_time_us":1790,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":26237,"raft.run_cpu_time_us":1989,"raft.run_wall_time_us":301628,"spinlock_wait_cycles":6097152,"thread_start_us":220388,"threads_started":1,"wal-append.queue_time_us":571284}
I20260709 12:27:11.735067 18600 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:11.735368 18590 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 12:27:11.736043 18591 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:27:11.668401 18558 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:11.737090 18592 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 12:27:11.738804 18558 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.735598 18600 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.744182 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.733425 18171 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.739140 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:11.739527 18574 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:11.743074 18584 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:27:11.727847 10669 rpcz_store.cc:276] 0709 12:27:10.712374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.721922 (+  9548us) service_pool.cc:224] Handling call
0709 12:27:10.740213 (+ 18291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:10.743323 (+  3110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:10.743367 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:10.743816 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 12:27:10.913915 (+170099us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:10.930889 (+ 16974us) log.cc:844] Serialized 14 byte log entry
0709 12:27:10.932108 (+  1219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:10.932182 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:11.723477 (+791295us) raft_consensus.cc:1685] finished
0709 12:27:11.723502 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:11.723634 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8374,"prepare.queue_time_us":16672,"prepare.run_cpu_time_us":90,"prepare.run_wall_time_us":10561,"raft.queue_time_us":10732,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":7093,"thread_start_us":120782,"threads_started":1,"wal-append.queue_time_us":580886}
I20260709 12:27:11.762468 10023 raft_consensus.cc:2368] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 25.
I20260709 12:27:11.762328 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.763286 10018 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:11.765138 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.765473 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.768865 10025 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:11.769265 10021 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:11.770687 10025 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:11.773588 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:11.774350 10017 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:11.780083 18601 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:11.780874 10020 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:11.787443 18497 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.788010 10016 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:11.789032 18171 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:11.789520 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:11.790144 10016 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:11.792553 18600 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.795285 18419 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.797156 18431 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:11.799019 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:11.799891 10023 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:11.760954 18521 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:11.802667 18473 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.803682  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:11.804283 18497 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.804883 18512 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:11.805065 18363 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:11.804786 18497 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.764008 17649 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:11.815049 10677 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:11.817353 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:11.798349 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:11.820670 10020 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:11.812666 18521 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.835988  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:11.839455  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:11.858757  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.806900 18512 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.823146 18595 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:27:11.844769 18363 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:11.864719 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:11.871021 18565 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.348s	user 0.000s	sys 0.003s
I20260709 12:27:11.826848 18599 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:27:11.857986 18479 raft_consensus.cc:2764] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:27:11.843500 18521 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.885568  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:11.882732 18548 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:11.898829  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:11.874884 18473 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:11.902570  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:11.916956  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:11.917611  9944 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:11.883334 18497 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:11.898536 18548 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:11.880846 18443 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:27:11.924072 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:11.926204 18548 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:11.931782 18603 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:11.932468 18603 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:11.920037 18473 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.935963  9940 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:11.937677 18473 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.940094 18548 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:11.942183 18548 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.935662 18603 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:11.946403  9940 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:11.953449 18523 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:11.955366 18497 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:11.966429 18456 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:11.966946 18456 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:11.967835 18456 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:11.975569  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.976935  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:11.977368  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:11.979614 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:11.979862 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:11.980479 10672 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:11.981720 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:11.986727  9946 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:11.997301 10090 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:11.998941 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:11.999444 18461 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:11.999704 10090 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:12.002628 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:12.004295 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:12.025574  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.032370 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.036774 10024 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:12.040230 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:12.040412 18600 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:12.041610 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1733) took 1375 ms. Trace:
I20260709 12:27:12.041978 10673 rpcz_store.cc:276] 0709 12:27:10.678178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.685576 (+  7398us) service_pool.cc:224] Handling call
0709 12:27:12.041451 (+1355875us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400490a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:12.041554 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146790272}
I20260709 12:27:12.042294 18454 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:12.043510  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3171) took 1957 ms. Trace:
I20260709 12:27:12.039077 18479 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:12.044050  9941 rpcz_store.cc:276] 0709 12:27:10.086487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.290735 (+204248us) service_pool.cc:224] Handling call
0709 12:27:10.291155 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.687017 (+395862us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.027984 (+1340967us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b540003fa54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:12.043425 (+ 15441us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":878419328}
I20260709 12:27:12.061436  9941 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:12.063841  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:12.042763 18454 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:12.072191 18454 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.078074 18014 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.078707 18014 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.080999 18014 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:12.083171 18473 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.086043 18523 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:12.086684 18523 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.088835 18523 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:12.093536 18014 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.094127 18014 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.097393 18014 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:11.962994 18497 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.099148 18497 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.103422 18482 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.113225 18482 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.117473 18482 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:12.119765 18482 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:12.120380 18482 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.123497 18482 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:12.133810 18578 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.747s	user 0.000s	sys 0.008s
I20260709 12:27:12.143896 18521 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:12.145993 18363 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:12.146437 18363 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:12.147231 18363 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.150942  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:12.150882  9834 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 18 to 22, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:12.153375  9834 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 11 to 21. New cstate: current_term: 21 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:12.154781 18521 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.156068 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1744) took 1440 ms. Trace:
I20260709 12:27:12.155471 18521 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.156487 10670 rpcz_store.cc:276] 0709 12:27:10.716168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.716354 (+   186us) service_pool.cc:224] Handling call
0709 12:27:12.155882 (+1439528us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:12.156010 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1185946496}
I20260709 12:27:12.158016 18030 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:12.156185 10675 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:12.158389 18521 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:12.158452 18030 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.159864 18030 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:12.160997 10022 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.162750 18571 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:27:12.150283  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.180429  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.074293 18454 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:12.206413 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.207115 10092 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:12.208006 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.211066 10098 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:12.201836 18571 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.215626 18571 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.219249 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:12.221956  9948 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.223093 18571 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:12.198892 10094 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:12.199841 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.224650 18580 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:12.225206  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.225963  9940 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:12.226718  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:12.227623  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.227777  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.228482  9940 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:12.228730  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.229476 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4216) took 1650 ms. Trace:
I20260709 12:27:12.230253 10096 rpcz_store.cc:276] 0709 12:27:10.579289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.592057 (+ 12768us) service_pool.cc:224] Handling call
0709 12:27:10.592423 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.248196 (+655773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.229118 (+980922us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b540022ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:12.229399 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1498018560}
I20260709 12:27:12.229542  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:12.233012  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 166020us
I20260709 12:27:12.233925  9940 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:12.229525  9942 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 27 because replica has last-logged OpId of term: 25 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 19.
I20260709 12:27:12.235528  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.235790 10095 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:12.236590  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.237370  9942 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:12.240437 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:12.111923 18456 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:12.245357 10670 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:12.225211 18580 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:12.247402 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:12.247190 18580 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.247964 10669 raft_consensus.cc:2376] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23. Re-sending same reply.
I20260709 12:27:12.248234 18580 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:12.241564 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:12.250705 18542 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:12.243413 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:12.251466 18542 consensus_queue.cc:1243] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:27:12.253624  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:12.255971 18588 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:12.256356 18588 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.256506 18356 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:12.257036 18356 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.257500 18457 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.257993 18457 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.258580 18030 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.259502 18356 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:12.260037 18457 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:12.260210 18356 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:12.260591 18457 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:12.260768 18030 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:12.261118 18457 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.186636 18399 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:12.253973 18575 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.265650  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.273237  9940 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:12.274509  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:12.275421  9940 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:12.266431  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.276631 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:12.279873 10098 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:12.278780  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:12.277261 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:12.297458 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.299499 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:12.300491 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:12.300908  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.302625  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.268126 18588 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.309129  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.301954 18407 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:12.304450 18250 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 12:27:12.261096 18356 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.301483 18571 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:12.325556  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.305667 10640 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:12.273716 18530 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:27:12.287524 18542 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:27:12.308058 18532 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:12.366937 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.367614 10091 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
W20260709 12:27:12.346374  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.369804 18457 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:12.371169 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.350862 18571 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.371846 18571 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.375679  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:12.376798 18402 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.379446  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:12.380669  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:12.381814 10062 thread.cc:641] server-negotiator [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.053s	sys 0.080s
W20260709 12:27:12.382395 10062 thread.cc:608] server-negotiator [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.054s	sys 0.080s
I20260709 12:27:12.385649 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:12.387401 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:12.388020 10672 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:12.346211 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.394237 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:12.394858 10671 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:27:12.396060 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:12.403383 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:12.393239 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:12.396883 10017 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:12.411214  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:12.313395  9942 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:12.352391 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.414271  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.414721 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:12.155503  9834 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 14 to 18. New cstate: current_term: 18 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:12.295845 18575 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.359208 18542 consensus_queue.cc:1243] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 20.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:27:12.425144  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.425849  9942 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:12.426188 10017 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:12.426781  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:12.363622 18431 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.566s	user 0.011s	sys 0.001s
I20260709 12:27:12.413304 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.429292 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:12.429708 10091 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:12.429972 10021 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:12.430616 10017 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 12:27:12.267099 10022 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:12.432013 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:12.432456 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2727) took 1988 ms. Trace:
I20260709 12:27:12.432914 10022 rpcz_store.cc:276] 0709 12:27:10.443643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.443809 (+   166us) service_pool.cc:224] Handling call
0709 12:27:10.479607 (+ 35798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.942388 (+462781us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400241e20. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:27:12.160636 (+1218248us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b58007b71d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:27:12.432154 (+271518us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.432347 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170276864}
I20260709 12:27:12.421237 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:12.429798 10024 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:27:12.385704  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:12.430665 10019 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:12.417775 10097 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:12.446518 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:12.447451 10020 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:12.419458 18456 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:12.419620 18521 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:12.497872 18575 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:12.424269 18402 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:12.499370 18402 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.499732 18402 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:12.352427 18586 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:12.501493 10673 raft_consensus.cc:2410] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 27 because replica has last-logged OpId of term: 25 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 19.
W20260709 12:27:12.504158 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 83015.1us
W20260709 12:27:12.504863  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59698.5us
I20260709 12:27:12.515190  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:12.504799  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 78098us
I20260709 12:27:12.500509 18482 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:12.518275 18597 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:12.518683 18552 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:12.519402 18552 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.519876 18552 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:12.524255 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:12.513923 10021 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:27:12.518795  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 85054.3us
I20260709 12:27:12.527220  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:12.502669 18398 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
W20260709 12:27:12.530741  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 145146us
I20260709 12:27:12.533262 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:12.536142 10672 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:12.539379 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:12.541642  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:12.503793 18407 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:12.431404 18457 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.112s
I20260709 12:27:12.535072 18548 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:27:12.425731 18572 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.117s
I20260709 12:27:12.553524 18407 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.538455 18521 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:12.536787 18617 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:12.554448 18216 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:12.551131 18030 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:12.544692 18586 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:12.570168  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.533154 18419 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:27:12.545492 10016 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:12.559069 18454 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.575461  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:12.561412 18407 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:12.571059 17649 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.009s	sys 0.006s
W20260709 12:27:12.599941 17649 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.010s	sys 0.006s
I20260709 12:27:12.567008 18431 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:12.610894 18588 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.550904 18597 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:27:12.601231 18600 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.565727 18461 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:27:12.569195 18216 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:12.602149 18030 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:12.629460 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1741) took 1915 ms. Trace:
I20260709 12:27:12.578482 10094 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:12.630019 10674 rpcz_store.cc:276] 0709 12:27:10.714116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:10.722646 (+  8530us) service_pool.cc:224] Handling call
0709 12:27:10.726135 (+  3489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:10.929340 (+203205us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.629171 (+1699831us) spinlock_profiling.cc:243] Waited 203 ms on lock 0x7b5400723654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:12.629379 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":465943936}
W20260709 12:27:12.622450  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.631893  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.634239  9943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.632s	user 0.006s	sys 0.010s
I20260709 12:27:12.630707 10095 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:12.636277  9943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.007s	sys 0.010s
I20260709 12:27:12.621698 10671 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
W20260709 12:27:12.641590  9946 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.026s	sys 0.020s
W20260709 12:27:12.642026  9946 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.026s	sys 0.020s
I20260709 12:27:12.604926 18572 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:27:12.648974 10093 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.044s	sys 0.053s
W20260709 12:27:12.649626 10093 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.044s	sys 0.053s
I20260709 12:27:12.670215 10023 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:12.574123 18586 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:12.628139 18497 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.528s	user 0.004s	sys 0.007s
I20260709 12:27:12.630291 18216 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:12.705158 18030 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.669435 18572 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:12.706640  9940 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:12.706759  9945 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:12.627629 18600 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:12.734409 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 312034us
W20260709 12:27:12.674412 18514 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.213s	user 0.010s	sys 0.012s
I20260709 12:27:12.707135 18497 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.727260 18431 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:12.742102  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 166860us
I20260709 12:27:12.727097 18572 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:12.737358 18625 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
W20260709 12:27:12.725740 18408 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.747s	user 0.027s	sys 0.049s
I20260709 12:27:12.755842  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:12.761457 10090 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:12.764932  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:12.757130 18408 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.028s	sys 0.049s
I20260709 12:27:12.848055  9945 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:12.863054 10676 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:12.762692 18399 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:12.866297 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1793) took 1383 ms. Trace:
I20260709 12:27:12.867478 10676 rpcz_store.cc:276] 0709 12:27:11.482832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.594001 (+111169us) service_pool.cc:224] Handling call
0709 12:27:11.594289 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.862914 (+1268625us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b540073ab20. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094cddc0 00007f4e095581c0 00007f4e09559231 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c
0709 12:27:12.865954 (+  3040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.866207 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":105646336}
I20260709 12:27:12.756628 18431 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:12.867991  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:12.873720  9945 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:12.738976 18514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.284s	user 0.011s	sys 0.012s
I20260709 12:27:12.878640  9943 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:12.859994 18597 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:12.861718 18582 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:12.890386  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:12.864279 18461 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:12.876012 18603 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:12.910466 18457 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:12.924743 18457 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.899011 18461 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.927026 18461 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:12.928710 18630 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:12.928876 18457 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:12.929175 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:12.930429 18628 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:12.919782  9945 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:12.934017 18457 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:12.934726 18457 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.938359 18457 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:12.938740 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:12.943362 10096 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:12.951740  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.952692  9945 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
W20260709 12:27:12.953126  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.029s	sys 0.061s
W20260709 12:27:12.953513  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.029s	sys 0.061s
I20260709 12:27:12.956597 18363 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:12.963995  9834 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 11 to 22. New cstate: current_term: 22 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:27:12.965426  9837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 313) took 78476 ms (client timeout 15000 ms). Trace:
I20260709 12:27:12.984009 18407 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.987533  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:12.889016 18588 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:12.889938 18402 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.992031 18586 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:12.993062 18497 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:12.994189 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:12.899937 18582 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:12.890491 18597 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:12.999190 18626 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.268s	user 0.000s	sys 0.001s
I20260709 12:27:12.942579 18603 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.014344 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1787) took 1540 ms. Trace:
I20260709 12:27:13.085557 10675 rpcz_store.cc:276] 0709 12:27:11.473910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.480004 (+  6094us) service_pool.cc:224] Handling call
0709 12:27:12.156007 (+676003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.158881 (+  2874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.014173 (+855292us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b54005497d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:13.014285 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1509955968}
I20260709 12:27:13.016597 18596 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:13.017930 18636 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:13.018683 18521 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:13.019579 18576 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.646s	user 0.005s	sys 0.004s
I20260709 12:27:13.032052 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:13.093833 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61874.1us
I20260709 12:27:13.033831 18640 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:13.034268  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:13.035357 18627 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.174s	user 0.003s	sys 0.000s
I20260709 12:27:13.035676 18434 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:13.037101 18408 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:13.039956 18356 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.041118 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:13.107740 10097 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:13.108145  9834 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 16 to 23, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 23 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:13.109790 18408 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:13.110903 18371 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:27:13.042393  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 1.17878e+06us
W20260709 12:27:13.068773  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.115509  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:13.117350  9912 leader_election.cc:400] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 25.
I20260709 12:27:13.117625 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:13.044199 18362 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:13.118458 18362 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.118894 18431 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.118948 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:13.119375 10022 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:13.120069 10020 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:13.121840 18461 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:13.123024 18443 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:13.123538 10096 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:13.123042 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:13.124928 10095 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:13.125823 18617 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:13.126209 18617 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:13.126574 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:13.127048 18617 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:13.129405 18621 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.205s	user 0.002s	sys 0.000s
I20260709 12:27:13.129467 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:13.128635  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:13.132258 18479 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:13.133194 18541 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:13.133984  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:13.134210 18541 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:13.134824 18399 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:13.135265 18399 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.135619 18399 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:13.136217 18434 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.138698 18640 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.139283 18479 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:13.141667 10016 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:13.150638 18617 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:13.153431  9834 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 10 to 21, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 21 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:13.165185 10021 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:13.166534 18588 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.167205 18588 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.169953 18588 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.171133 18461 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:13.171943 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:13.172565  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2899) took 1309 ms. Trace:
I20260709 12:27:13.171803  9834 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 17 to 18, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 18 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:13.173179 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:13.174945  9942 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:13.174122  9834 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 12 to 19. New cstate: current_term: 19 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:13.178350  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2904) took 1249 ms. Trace:
I20260709 12:27:13.178022  9834 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 8 to 16, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 16 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:13.178880  9944 rpcz_store.cc:276] 0709 12:27:11.928872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.929033 (+   161us) service_pool.cc:224] Handling call
0709 12:27:11.954732 (+ 25699us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.955296 (+   564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:11.955325 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:11.960506 (+  5181us) log.cc:844] Serialized 26 byte log entry
0709 12:27:12.571406 (+610900us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:12.571461 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.572475 (+  1014us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.034569 (+462094us) spinlock_profiling.cc:243] Waited 4.05 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:13.177989 (+143420us) raft_consensus.cc:1685] finished
0709 12:27:13.178019 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.178278 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1001,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":2122,"spinlock_wait_cycles":9316096,"thread_start_us":467397,"threads_started":1,"wal-append.queue_time_us":724798}
I20260709 12:27:13.180202  9834 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:13.103801 18521 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.183084 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:13.183781 10090 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:13.184281 18521 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:13.185138  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:13.185300 18586 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:13.186004 18588 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:13.185842 18586 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.186427 18588 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:13.186805 18603 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:13.186980 18588 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:13.182337 18399 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.207628 18600 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:13.080914 18571 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.705s	user 0.012s	sys 0.000s
I20260709 12:27:13.099723 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:13.106506 18464 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:27:13.310774 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 211154us
I20260709 12:27:13.311470 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:13.173048  9947 rpcz_store.cc:276] 0709 12:27:11.863538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.874750 (+ 11212us) service_pool.cc:224] Handling call
0709 12:27:11.875107 (+   357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.296608 (+421501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:12.296657 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:12.314806 (+ 18149us) log.cc:844] Serialized 26 byte log entry
0709 12:27:12.862899 (+548093us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:12.862937 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:12.863004 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.172293 (+309289us) raft_consensus.cc:1685] finished
0709 12:27:13.172321 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.172493 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22383,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":3289,"thread_start_us":872855,"threads_started":2,"wal-append.queue_time_us":720072}
I20260709 12:27:13.314328  9948 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:13.090436 18596 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:13.328626  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:13.329411  9949 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:13.333170 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:13.105118 18497 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:13.334606 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:13.333525 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 43769.1us
I20260709 12:27:13.141635 18640 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:13.344549 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:13.345204 10019 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
W20260709 12:27:13.343780 18532 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.041s	sys 0.077s
I20260709 12:27:13.309548 18464 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:13.317109 18581 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.317648 18593 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:13.335132 18552 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.815s	user 0.008s	sys 0.005s
W20260709 12:27:13.273068  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.364112  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:13.274428  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:13.282296 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:13.350567 18532 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.041s	sys 0.077s
I20260709 12:27:13.366866 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:13.367740 10016 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
W20260709 12:27:13.279039  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.308601  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:13.316007 18530 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:13.333205 18232 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:13.344122 18457 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:13.339015 18497 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.366494 10023 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
W20260709 12:27:13.434046  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.331736 18371 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:13.397707 18532 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:13.350404  9834 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 17 to 21, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:13.399073 18548 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:27:13.327028 18369 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:27:13.404907 18617 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:13.467655 18617 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.381553 18632 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.112s
I20260709 12:27:13.469192 18632 consensus_queue.cc:1243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 24.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:13.469812 18632 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:13.469920 18617 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.410444 18407 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.470269 18632 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:13.438220 18497 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:13.471473 18632 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.473479 18632 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:13.473642  9834 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 15 to 22. New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:13.431088 18637 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:27:13.413748 18512 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:13.414357 18596 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.383805 18581 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.366475 18436 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:27:13.423702  9940 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:13.427119  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:13.486510  9945 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:13.394927 18446 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:13.348670 18408 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.429136 18588 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.336560 18521 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.145s
I20260709 12:27:13.396538  9944 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:13.317950 18571 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.463598 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:13.406457 18572 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.652s	user 0.004s	sys 0.005s
W20260709 12:27:13.454928 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 144822us
I20260709 12:27:13.382993 18552 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.373785 18434 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:13.456815 18216 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.004s
I20260709 12:27:13.488166  9941 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
W20260709 12:27:13.489176  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.021s	sys 0.019s
W20260709 12:27:13.537117  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.999s	user 0.021s	sys 0.019s
W20260709 12:27:13.537635  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 999862us
I20260709 12:27:13.433722 18530 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.548765 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:13.493651 18575 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.005s	sys 0.004s
I20260709 12:27:13.496404 10023 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:13.358634 18533 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.135s
I20260709 12:27:13.384022 18593 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:13.529242 10099 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.030s	sys 0.037s
I20260709 12:27:13.533862  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:13.583781 10099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.031s	sys 0.039s
I20260709 12:27:13.584051  9945 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:13.535797 10024 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:13.496371 18446 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:13.531534  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:13.540483 18617 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.541909  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:13.542014 18443 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.545552 10097 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:13.546702 10096 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
W20260709 12:27:13.567356 18575 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.005s	sys 0.004s
I20260709 12:27:13.548099 10022 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:27:13.291669  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:13.561590  9949 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:13.492123 18461 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:13.499631 18408 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:13.455758 18371 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.500854 18597 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:13.580991 18636 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:13.588806  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57523.9us
I20260709 12:27:13.547896 10093 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:27:13.591439 18617 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.592433 18482 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:13.593070 18593 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:13.593905  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52142.6us
I20260709 12:27:13.621963  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:13.595521 10095 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:13.566398 18216 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:13.605388  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 313808us
I20260709 12:27:13.587532  9948 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:13.476310  9834 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 9 to 18. New cstate: current_term: 18 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:13.603792 18419 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:13.613710  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:13.609139 18460 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:13.502499 18647 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.103s
W20260709 12:27:13.624917 10025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.017s	sys 0.039s
W20260709 12:27:13.699175 10025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.017s	sys 0.039s
I20260709 12:27:13.500746 18572 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:13.704952  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 91336.6us
I20260709 12:27:13.602923 18456 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.606352 18371 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:13.620205  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:13.624086 18030 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.729s	user 0.020s	sys 0.038s
I20260709 12:27:13.720009  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:13.618678 18482 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:13.627265  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2892) took 2132 ms. Trace:
I20260709 12:27:13.721354  9948 rpcz_store.cc:276] 0709 12:27:11.494735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.495557 (+   822us) service_pool.cc:224] Handling call
0709 12:27:11.495838 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:12.221578 (+725740us) spinlock_profiling.cc:243] Waited 5.37 ms on lock 0x7b4c0016ebc0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07822a20 00007f4e07833a29 00007f4e077f9644 00007f4e07820f2d 00007f4e07820b93 00007f4e0955118b 00007f4e09551cb5 00007f4e0954cf30 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416
0709 12:27:13.209454 (+987876us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b5800a23ed0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:27:13.626823 (+417369us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.627181 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35510016}
I20260709 12:27:13.599432 18443 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.716449  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:13.725294  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:13.616286  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 478623us
W20260709 12:27:13.717335 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 364316us
I20260709 12:27:13.602689 18582 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.610386 18461 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:13.720350 18030 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.021s	sys 0.038s
I20260709 12:27:13.730921  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:13.724061 18645 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.136s	user 0.000s	sys 0.001s
I20260709 12:27:13.705214 18431 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:13.783165 18431 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.783540 18548 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:13.784376 18443 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.784178 18548 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.784968 18443 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:13.785356 18431 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.785458 18443 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:13.732501 10668 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:13.734264 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:13.786321 10675 raft_consensus.cc:2376] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28. Re-sending same reply.
I20260709 12:27:13.704607 18419 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:13.621699 18636 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.788091 18419 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.789140 18636 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.724448 18204 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:13.789849 18402 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:13.791296 18362 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.790573 18402 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.793469 18402 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.794134 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2747) took 1054 ms. Trace:
I20260709 12:27:13.794488 10025 rpcz_store.cc:276] 0709 12:27:12.740118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.740690 (+   572us) service_pool.cc:224] Handling call
0709 12:27:12.743843 (+  3153us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:12.751403 (+  7560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:12.751433 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.114502 (+363069us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b2c00046bc8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:13.115088 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.699671 (+584583us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:27:13.699715 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.699798 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.793830 (+ 94032us) raft_consensus.cc:1685] finished
0709 12:27:13.793850 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.794075 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2267,"raft.queue_time_us":10873,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":5033,"spinlock_wait_cycles":34882816,"thread_start_us":584194,"threads_started":1,"wal-append.queue_time_us":676997}
I20260709 12:27:13.795665 18402 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:13.740857 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 411486us
I20260709 12:27:13.797261 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:13.727756 18649 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:13.744802 18371 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.746835 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:13.747150 18644 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:27:13.748996 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:13.753765 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:13.755932 18216 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:13.756064 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:13.759116 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:13.731586 18600 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:13.704980 18446 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.735445 18647 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:13.722306 18456 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:13.062969  9837 rpcz_store.cc:269] 0709 12:25:54.485547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:54.488798 (+  3251us) service_pool.cc:224] Handling call
0709 12:27:12.962295 (+78473497us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:07.767129 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:07.807174 (+ 40045us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:07.807234 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:27:07.807262 (+    28us) tablet.cc:662] Decoding operations
0709 12:27:07.807806 (+   544us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:07.807905 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:07.807939 (+    34us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:27:07.808372 (+   433us) tablet.cc:701] Row locks acquired
0709 12:27:07.808420 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:27:07.808624 (+   204us) write_op.cc:270] Start()
0709 12:27:07.808780 (+   156us) write_op.cc:276] Timestamp: P: 1783600027808520 usec, L: 0
0709 12:27:07.808812 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:27:07.809527 (+   715us) log.cc:844] Serialized 924 byte log entry
0709 12:27:08.321770 (+512243us) op_driver.cc:464] REPLICATION: finished
0709 12:27:08.752872 (+431102us) write_op.cc:301] APPLY: starting
0709 12:27:08.768556 (+ 15684us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:08.785394 (+ 16838us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:08.785515 (+   121us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:08.835291 (+ 49776us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:08.868454 (+ 33163us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=4,mrs_lookups=0
0709 12:27:08.897163 (+ 28709us) write_op.cc:312] APPLY: finished
0709 12:27:08.964377 (+ 67214us) log.cc:844] Serialized 33 byte log entry
0709 12:27:09.008026 (+ 43649us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:09.043330 (+ 35304us) write_op.cc:454] Released schema lock
0709 12:27:09.072954 (+ 29624us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:09.134153 (+ 61199us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:09.138056 (+  3903us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2477690,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":293799,"apply.run_cpu_time_us":7444,"apply.run_wall_time_us":441021,"cfile_cache_hit":10,"cfile_cache_hit_bytes":12491,"num_ops":2,"prepare.queue_time_us":461681,"prepare.run_cpu_time_us":14118,"prepare.run_wall_time_us":209765,"raft.queue_time_us":157086,"raft.run_cpu_time_us":10063,"raft.run_wall_time_us":283515,"replication_time_us":486149,"spinlock_wait_cycles":4480,"thread_start_us":559136,"threads_started":4,"wal-append.queue_time_us":219260,"wal-append.run_cpu_time_us":18166,"wal-append.run_wall_time_us":2098196}]]}
I20260709 12:27:13.810177 18654 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:13.741731 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 400177us
I20260709 12:27:13.803158 18548 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:13.848809 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:13.854897 10669 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:13.854307 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:13.851150  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:13.818178 18600 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.860354  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:13.860949 18402 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:13.862519 18371 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:13.812784 18649 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.863932  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1836) took 1879 ms. Trace:
I20260709 12:27:13.811028 18521 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:13.863152 18575 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:13.862529 18650 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:13.865005 18030 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:13.874142 18575 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.874912 18402 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:13.875980 18402 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.876878 18575 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.867723  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:13.857857 18582 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:13.814265 18597 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:27:13.870872  9946 rpcz_store.cc:276] 0709 12:27:11.984087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:11.989068 (+  4981us) service_pool.cc:224] Handling call
0709 12:27:11.989438 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:11.989815 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:12.642460 (+652645us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.063316 (+420856us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:13.063813 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:27:13.063954 (+   141us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:13.063987 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.064058 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.863685 (+799627us) raft_consensus.cc:1685] finished
0709 12:27:13.863712 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.863863 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":625481,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":2358,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":4372,"thread_start_us":1071549,"threads_started":2,"wal-append.queue_time_us":852320}
I20260709 12:27:13.852380 18647 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:13.828474 18588 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:13.871759 18473 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:13.898995 18473 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:13.899147 18456 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.899356 18461 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:13.819595 18369 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.052s
I20260709 12:27:13.899952 18473 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.857571 18637 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:27:13.934420 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:13.857973 18548 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:13.935350 10024 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:13.891422 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:13.877656 18464 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.934231 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:13.870476 18363 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.806s	user 0.007s	sys 0.004s
I20260709 12:27:13.879683 18521 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:13.897715  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.938714 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:13.901471 18620 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:13.901679 18617 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.902417 18647 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.901067 18369 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:13.906502 18399 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.803591 18482 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.902310 18552 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.908255 18461 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:13.911391 18572 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.940989 18521 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:13.911751 18443 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.880241 18582 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:13.918311 10669 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:13.865269 18600 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:13.869560 18530 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:13.941864  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 319962us
I20260709 12:27:13.943179 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:13.907362 10673 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:27:13.915725 18030 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:13.955071 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:13.968659  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:13.936702 18497 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:13.968735 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1554) took 1557 ms. Trace:
I20260709 12:27:13.971480 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:13.958920 18562 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:13.944579 18638 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:13.949079 18581 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.991524 10020 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:13.969365 18586 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:13.988780  9946 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:13.991641  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47550.5us
I20260709 12:27:13.994917 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:13.995373  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:13.992098 10099 rpcz_store.cc:276] 0709 12:27:12.411402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:12.411511 (+   109us) service_pool.cc:224] Handling call
0709 12:27:12.411798 (+   287us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:12.425937 (+ 14139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:12.425970 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:12.949652 (+523682us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b2c00115cd8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:12.950247 (+   595us) log.cc:844] Serialized 52 byte log entry
0709 12:27:13.584444 (+634197us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:13.946352 (+361908us) log.cc:844] Serialized 14 byte log entry
0709 12:27:13.954545 (+  8193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:13.954659 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:13.954702 (+    43us) raft_consensus.cc:1685] finished
0709 12:27:13.968508 (+ 13806us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:13.968675 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16999,"raft.queue_time_us":41578,"raft.run_cpu_time_us":337,"raft.run_wall_time_us":69573,"spinlock_wait_cycles":3948928,"thread_start_us":988416,"threads_started":2,"wal-append.queue_time_us":777066}
I20260709 12:27:13.991000 18250 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:13.976446 18582 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:13.984272 18473 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:13.943459 18548 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:14.002142  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 271337us
I20260709 12:27:13.974545 18461 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:13.945438 18620 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.958983 18369 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:13.996371 18638 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:14.035141 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.031939  9944 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:14.038245 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:14.038893 10096 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:14.028086 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 214321us
I20260709 12:27:13.997911 18407 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:14.040851 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:14.043535 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.047735 10091 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:14.049809 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:14.050969 10095 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.053895 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:14.054850 10091 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:14.057076 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.059034 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.066021 10098 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:14.003346 18250 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:14.006161 18586 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:14.069788 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:14.070425 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.062336 18638 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:14.080303 10093 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:14.081384 18649 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.081543 18620 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:14.084708  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 78333.8us
I20260709 12:27:14.031571 18216 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:14.033512 18371 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.062572 18407 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:14.112562 18672 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:14.113497 18661 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:27:14.113636 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
W20260709 12:27:14.114101  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.133237 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:14.071424 18597 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.077987 18654 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:14.012446 18562 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.071s
I20260709 12:27:14.068488 18363 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:14.125247 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.013247 18530 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.078s
I20260709 12:27:14.114425 10091 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:14.115391 10095 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:14.114516 18456 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:14.109134 18667 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.073354 18663 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.100s
W20260709 12:27:14.135965 18666 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:27:14.123512 18407 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:14.165654 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 100487us
I20260709 12:27:14.126415 18620 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:14.141134 18204 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:14.167594  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 67194us
I20260709 12:27:14.167430 18674 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:27:14.079063 10099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:14.199438 10099 raft_consensus.cc:606] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:14.199834 10099 raft_consensus.cc:614] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:14.200583  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.199700  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:14.206297 18530 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:14.206588 18572 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:14.206794 18530 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:14.207231 18572 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:14.208200 18572 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.208830  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.208684 18654 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.210136  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.210706  9945 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:14.213443  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:14.213478  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:14.126000 18649 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:27:14.201990 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:14.220418  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:14.224790  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 210849us
I20260709 12:27:14.137253 18408 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.231971  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.121912 18216 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.131286 18371 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.232566  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:14.236037 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.243681  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.192833 18637 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:14.286653 10018 raft_consensus.cc:2368] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
W20260709 12:27:14.295447  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 282691us
I20260709 12:27:14.300189 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.234820 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:14.302850  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 70361us
I20260709 12:27:14.305477 10021 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:14.305368 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.314265 10018 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:14.164414 18363 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.225353 18675 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.214125 18571 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.326349 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.146788 18597 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.330268 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.246933 18497 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:14.333480  9945 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:14.334470 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.291115 18637 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:14.340519 10018 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:14.212791 18620 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.335357 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:14.217453 18204 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.138861 18672 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.340830  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.306164 18674 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:14.343930 10021 raft_consensus.cc:2368] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
I20260709 12:27:14.304221  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.345706 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:14.335001 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:14.314759 18683 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:14.346701  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.351665  9948 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:14.169276 18461 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.172569 18667 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.214519 18250 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.350419 10023 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:14.355259  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.345329 18593 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:14.362007  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.364024  9945 raft_consensus.cc:2376] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28. Re-sending same reply.
I20260709 12:27:14.364293 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.364879  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.349129 10018 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:14.365881  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.303316 18408 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.346554 18685 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.362282  9946 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:14.372025  9945 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:14.362957  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:14.374639  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.379774  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.354455 18497 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:14.389878  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.380805  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.390986  9945 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.392225  9946 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:27:14.391927 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.393728  9949 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:14.397243 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.400146  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.393013 18695 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.405460  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.412009  9991 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:14.412415 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.412642 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.413163 10021 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:14.413388 10099 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:14.414088 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.417770 18548 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.420334  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:14.421999  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:14.338717 18675 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.426034  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.433970  9945 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:14.434978  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.393623 18593 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:14.429594  9949 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:14.432638 18690 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.443691 18482 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:14.466995 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:14.467589 10671 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
W20260709 12:27:14.446200 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.390183 18685 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.470383 18216 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.471030 18216 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:14.471285 18369 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.471535 18216 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.472509 18685 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.433142 18572 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:14.473613 18216 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:14.379549 18637 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.475996 18637 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.478822 18549 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:14.356195 18683 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.479557 18690 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.422786 18446 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:14.483117 18446 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.485071 18446 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:14.419152 18695 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.487025 18593 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.488641  9943 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:14.396358 18497 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.491045 18696 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.491570 18497 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.459503  9948 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:14.493767  9948 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:27:14.497539  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.498345 18597 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.499697 18363 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:14.500133 18549 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:14.501260 18549 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.503266 18620 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:14.465989 18674 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.505280  9946 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:14.446758 18586 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.506934 18586 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.509816 18586 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:14.511958 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.512403 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.491573 18696 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:14.517392 18680 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:27:14.518939 18690 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.523253 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.524072 10672 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:14.487560 18695 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.529403 10018 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:14.530110 10023 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:14.534296 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.535414 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.535513 10671 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:27:14.536329 10676 raft_consensus.cc:2368] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:14.541455  9948 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:27:14.533411 18690 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:14.553774 18434 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:14.557771 18572 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:14.561720 18677 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
I20260709 12:27:14.562073 18689 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:27:14.562969 18684 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:27:14.564388 18204 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.482223 18582 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.573611 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.608322 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.614511 10092 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:14.615697 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.573874 18434 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.616948 18696 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.630764 18434 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:14.635193 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.635272 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.638424 10092 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:14.558116 18446 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:27:14.626185 18582 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:14.642633 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.644290 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.649267 10671 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:14.645118 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.554386 18638 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.660094 18514 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.660786 18514 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.661186 18638 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:14.663146 18514 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.663198 18250 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:14.650883 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:14.655536 18521 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.667s	user 0.011s	sys 0.000s
I20260709 12:27:14.576356 18572 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.571878 18407 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.670910 10024 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
W20260709 12:27:14.696765  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.671294 10671 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:14.670542 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:14.699026  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.699688 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:14.684593  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 1853) took 1084 ms. Trace:
I20260709 12:27:14.698333  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:14.701148  9944 rpcz_store.cc:276] 0709 12:27:13.599719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.599855 (+   136us) service_pool.cc:224] Handling call
0709 12:27:14.684181 (+1084326us) spinlock_profiling.cc:243] Waited 312 ms on lock 0x7b54000b5254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:14.684430 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":717769600}
I20260709 12:27:14.691135 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:14.693421 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.683511 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:14.697376 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.707224 10017 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:14.706910 10094 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.708221  9949 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:14.709257  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.710580 10092 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 20: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:14.713155  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.713960  9949 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:14.715219  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.716584  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:14.718962  9949 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:14.651422 18446 consensus_queue.cc:1243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 25.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:14.571969 18690 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.730955 10094 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
W20260709 12:27:14.693982 18600 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.703s	user 0.010s	sys 0.001s
I20260709 12:27:14.759209 18682 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.758242 10017 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:14.741032 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:14.711018 18664 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:14.802469 18664 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.802975 18664 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.690865 18672 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.762118 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.806910 10092 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:14.763432 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.685258 18572 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.807952 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.689215 18671 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:14.687355 18407 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:14.724555 18030 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:27:14.769482  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.777216 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.813824 10094 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
W20260709 12:27:14.752328  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:14.683030 18521 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.760426 18649 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.801173 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:14.809031 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 68128.6us
I20260709 12:27:14.815217 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.836184 10062 leader_election.cc:400] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:14.836588 10062 leader_election.cc:403] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:14.837756 10094 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:14.840749 10065 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:14.840760  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.848282 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:14.849368  9949 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:14.849313 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:14.811771 18443 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.854270 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:14.856950 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:14.840251 18649 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.865636 10062 leader_election.cc:400] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:14.819564 18682 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.855888 18696 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:14.866688 18696 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:14.866873 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:14.867205 18696 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:14.852798 18656 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:14.869344 10096 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:14.834280 18572 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.798427 18600 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.873502 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:14.834261 18593 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.851711 10017 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:14.839166 18548 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:14.872807 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 166497us
I20260709 12:27:14.860606 18673 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:14.868779 18667 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:14.880530 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.881094 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.881189 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.819077 18672 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:14.860811 18479 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.883132 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:14.859176 18699 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:14.884608 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:14.810936 18363 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:14.875810 18597 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:14.863790 18446 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.880981 18572 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.932070 18446 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.934266 18446 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.908216 10016 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:14.868875 18656 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:14.856118 18514 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:27:14.967680  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.908443  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.910430 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.968945  9948 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 19.
I20260709 12:27:14.915871 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:14.872532 18698 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:14.971860 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.862392 18443 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:14.920493 18407 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:14.909394 18216 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:27:14.938086 10676 pending_rounds.cc:85] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 17
I20260709 12:27:14.979985 10676 pending_rounds.cc:107] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 25.18
I20260709 12:27:14.980258 10676 raft_consensus.cc:2889] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:14.980325  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.942411 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:14.942329 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57835.7us
W20260709 12:27:14.943354 18706 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:27:14.944680 18521 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:14.946293 18695 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:14.946738 18691 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:27:14.948347 18682 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:14.948899 18646 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.328s	user 0.004s	sys 0.000s
I20260709 12:27:14.997375 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1594) took 1054 ms. Trace:
I20260709 12:27:14.920749 18479 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.914439 18672 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:14.952557 18408 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.950302 18664 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.953115 18697 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:27:14.953773 18715 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:14.954007  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.954805 18597 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:14.955516  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:14.955668 18620 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.936087 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.957701 18600 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:14.958900 18702 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 12:27:14.962265 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.969468  9945 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
W20260709 12:27:14.970186 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54404.7us
I20260709 12:27:14.969489 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:14.973845  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.974427  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:14.975533 10091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:15.028081 10091 raft_consensus.cc:606] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:14.904175 18030 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 12:27:14.971516  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:14.934388 18572 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.834748 18671 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.050853 18461 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:15.051291 10670 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:15.051606 18461 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:14.888096 18548 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.052249 18461 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:15.052464 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.053746 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.054042 18548 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:14.982573 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:14.981984 10021 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:15.055313 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.055398 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.998605 10097 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:27:15.010125 18672 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:15.059298 18712 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:27:15.061721 18649 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.063814 18399 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:15.064441 18399 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.064895 18399 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:15.067390  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.068323  9946 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:15.069540  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:15.070271  9946 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:14.997802 10024 rpcz_store.cc:276] 0709 12:27:13.942648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.942929 (+   281us) service_pool.cc:224] Handling call
0709 12:27:14.997206 (+1054277us) spinlock_profiling.cc:243] Waited 685 ms on lock 0x7b54002d4e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:14.997313 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574819584}
W20260709 12:27:15.092978  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 118659us
I20260709 12:27:14.978981 18689 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:14.885545 18593 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:14.997148 18675 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.017777  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:15.029676  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74267.4us
I20260709 12:27:15.113916  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:14.993673 18690 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:15.040997  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 69544.3us
I20260709 12:27:15.042025 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:15.043018 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:15.112358  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.043511 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.002233 18695 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:15.048110 18688 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:15.072254 18704 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.159s	user 0.002s	sys 0.000s
I20260709 12:27:15.035807 18620 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.074312 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.029214 18464 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:15.036286 18715 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.076393  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.137499 10091 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:15.076947 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:14.999871  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:15.078195 18371 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.079608  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.079941 18683 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.086678  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:14.978160 18575 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:15.094473 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.094300  9991 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:15.097792 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 115324us
I20260709 12:27:15.098191 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:14.981215 18363 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.102252 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.118309 10677 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 19: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:15.121230 10017 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:15.110989 18548 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:15.159597 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.160709 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.161463 10017 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:15.121956 10668 raft_consensus.cc:2368] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27.
I20260709 12:27:15.122201 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.162284 10671 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:15.117408  9942 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:15.123935  9912 leader_election.cc:400] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
I20260709 12:27:15.153795 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:15.165704  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.165951 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.166327  9948 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:15.166488 10024 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:15.168970 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.170006 10025 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:27:15.134696 18675 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:15.092941 18667 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:15.135408 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.190270 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.144807 18663 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:15.194621 18548 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:15.199821 18548 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.158795 18408 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.142798  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.206387  9947 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:27:15.165362 18571 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:15.139058 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:15.207950 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54267.6us
I20260709 12:27:15.208921 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:15.208971  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 209201us
I20260709 12:27:15.130530 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:15.212631 10018 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:15.051959 18671 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:15.180393  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 86214.6us
I20260709 12:27:15.220342 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1783) took 1026 ms. Trace:
I20260709 12:27:15.181609 18575 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:15.182042  9940 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:15.184672 10677 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:15.164258  9912 leader_election.cc:403] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:27:15.173226 18371 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:15.139205  9946 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
W20260709 12:27:15.206653 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 65866us
I20260709 12:27:15.179531 18683 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:15.220896  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:15.128839 18532 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.134243 18688 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.112289 18689 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.220214 18512 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:27:15.230074 18697 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.251390 18656 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:15.251837 18656 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:15.251778 18697 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.252728 18689 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.223873 10017 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:15.252880 18656 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.186144 18675 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.253836 10021 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:15.254094 18697 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.254190 18675 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.255534 18620 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.255865 18656 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:15.227977 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:15.228384  9946 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:15.190070 18456 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:15.234018  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:15.258874 10018 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:15.220728 10669 rpcz_store.cc:276] 0709 12:27:14.193807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.194214 (+   407us) service_pool.cc:224] Handling call
0709 12:27:14.194723 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:14.195012 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:14.195050 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:14.217432 (+ 22382us) log.cc:844] Serialized 26 byte log entry
0709 12:27:14.696279 (+478847us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:14.696314 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:14.696376 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.071478 (+375102us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:15.196502 (+125024us) raft_consensus.cc:1685] finished
0709 12:27:15.196535 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.220278 (+ 23743us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4776192,"thread_start_us":395355,"threads_started":1,"wal-append.queue_time_us":629132}
W20260709 12:27:15.240516  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116746us
I20260709 12:27:15.260316 10018 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:15.260591  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:15.204159 18638 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.185027 18654 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 12:27:15.262225 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.219918 18691 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:15.238305 18596 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:15.267105 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:15.267436  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:15.261377 18638 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.263792 10018 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:15.279521 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:15.246634 18571 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:15.259125 18617 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:15.213181 18408 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.262113 18456 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.266551 18030 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:15.309047 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:15.313653 10099 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:15.185691 18597 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.302325 18571 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:15.207252 18663 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.317325  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.277249 18479 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.277122 18562 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:15.278120 18691 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:15.277369 18620 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:15.258514 18532 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.248023 18683 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.328189 18562 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:15.309185 18436 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:15.299052 18682 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:15.299402 18671 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:15.316048 18030 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:15.351790 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72331.7us
I20260709 12:27:15.353427 18732 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:15.313066 18456 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:15.355772 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.358101 10669 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:15.356297 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:15.327189 18571 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:15.338212 18637 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.862s	user 0.020s	sys 0.005s
I20260709 12:27:15.357331 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.332474 18691 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:15.349969 18671 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:15.363379 18728 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:15.364518 18683 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.365134 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.435078 10675 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:15.365324 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:15.435730 18722 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:27:15.366163 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.436383 10098 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:15.436619 10671 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
W20260709 12:27:15.366942 18723 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:27:15.368007 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.358673 18030 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:15.437335 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:15.335050 18575 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.438016 10098 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:15.382754 18530 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.439066 18530 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:15.384891 18688 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.385995 18363 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.387131 18593 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.388574 18638 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.335189 18443 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.396876 18532 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:15.441360 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.442001 10091 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:15.361600 18436 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:15.400219  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.360718 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.453918 10099 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:15.358114  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2914) took 1150 ms. Trace:
I20260709 12:27:15.454936  9945 rpcz_store.cc:276] 0709 12:27:14.207448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.491746 (+284298us) service_pool.cc:224] Handling call
0709 12:27:14.969066 (+477320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.022578 (+ 53512us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.357932 (+335354us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b5400206f50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:15.358062 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096987904}
I20260709 12:27:15.248315 18695 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.359548 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:15.329265 18664 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.425951 10673 raft_consensus.cc:2410] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:15.338776 18562 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:15.439868  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.435299 18399 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.479182  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.442569  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.480849  9946 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:15.481138 18399 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.443976  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:15.442807 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.443647  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:15.483100 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2764) took 1243 ms. Trace:
I20260709 12:27:15.449327 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:15.483492 10024 rpcz_store.cc:276] 0709 12:27:14.239231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.177667 (+938436us) service_pool.cc:224] Handling call
0709 12:27:15.178011 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.178219 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.178244 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:15.189172 (+ 10928us) log.cc:844] Serialized 26 byte log entry
0709 12:27:15.238112 (+ 48940us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:15.238174 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.238278 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:15.482870 (+244592us) raft_consensus.cc:1685] finished
0709 12:27:15.482895 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.483040 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":48563,"threads_started":1,"wal-append.queue_time_us":162849}
I20260709 12:27:15.485183 18728 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:15.485351 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:15.485749 18728 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:15.437387 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 3957) took 1128 ms. Trace:
I20260709 12:27:15.486763 18728 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.486953 10017 rpcz_store.cc:276] 0709 12:27:14.309422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.185958 (+876536us) service_pool.cc:224] Handling call
0709 12:27:15.186243 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.253423 (+ 67180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.437148 (+183725us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b54001c5a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:15.437326 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":86097792}
I20260709 12:27:15.459726 18436 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.488005 18532 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:15.488453 18532 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:15.488816 18532 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:15.490095 18436 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.494660 10020 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
W20260709 12:27:15.490763 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 131289us
I20260709 12:27:15.491092 18593 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:15.491219 18581 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.491971 18664 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:15.449074 18732 consensus_queue.cc:1175] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" responder_term: 23 status { last_received { term: 22 index: 17 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:27:15.497589  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53685.6us
I20260709 12:27:15.496488 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:15.422821 18671 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:15.457756 18250 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.794s	user 0.017s	sys 0.008s
I20260709 12:27:15.356190 18682 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.492901 18521 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:15.545879 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.534130 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:15.590536  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:15.593199 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59201.9us
I20260709 12:27:15.593295  9945 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:15.591364 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 231482us
I20260709 12:27:15.596779 10674 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:27:15.499030 18593 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:15.598829 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.599296 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:15.600459  9942 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:15.509094 18664 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.512199 18399 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:15.611097 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.612345 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.613318 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.614286 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.598577 18711 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:15.615324 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1785) took 1153 ms. Trace:
I20260709 12:27:15.616137 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1569) took 1066 ms. Trace:
I20260709 12:27:15.658778 10673 rpcz_store.cc:276] 0709 12:27:14.462952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.463116 (+   164us) service_pool.cc:224] Handling call
0709 12:27:15.615124 (+1152008us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b5400501ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:15.615271 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2202632576}
I20260709 12:27:15.658842 10094 rpcz_store.cc:276] 0709 12:27:14.549682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.842787 (+293105us) service_pool.cc:224] Handling call
0709 12:27:14.843166 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.054788 (+211622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.054817 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.054868 (+    51us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:15.054898 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.615926 (+561028us) spinlock_profiling.cc:243] Waited 211 ms on lock 0x7b540014afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:15.615970 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.616087 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":486222080}
I20260709 12:27:15.617479 10091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:15.661478 10091 raft_consensus.cc:606] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:15.661854 10091 raft_consensus.cc:614] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:15.599767 18593 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:15.662907 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:15.655356 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 250884us
W20260709 12:27:15.606287 18696 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.738s	user 0.017s	sys 0.007s
I20260709 12:27:15.591351 18672 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.592065 18521 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:15.619513  9942 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:27:15.621996 18407 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.642s	user 0.012s	sys 0.010s
I20260709 12:27:15.622198 10019 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:15.624531 18582 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:15.627977  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:15.632025 18408 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.632745  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2912) took 1425 ms. Trace:
I20260709 12:27:15.632905  9945 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:15.678907 10017 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:15.633476 18532 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.634989  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:15.679180 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.450901 18461 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.660972 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.606325 18664 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:15.511796 18581 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.647759 18216 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:15.548336 18699 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:15.678373  9943 rpcz_store.cc:276] 0709 12:27:14.206876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.212335 (+  5459us) service_pool.cc:224] Handling call
0709 12:27:14.212621 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:14.489121 (+276500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.632470 (+1143349us) spinlock_profiling.cc:243] Waited 276 ms on lock 0x7b540042e354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:15.632676 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633757568}
W20260709 12:27:15.681439 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.661767 18711 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.687474 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.677579 18407 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:15.689890  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54955.6us
I20260709 12:27:15.753510  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:15.693104 10099 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:27:15.693874 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.695017 10093 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:27:15.697326 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.697917 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.703413 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:15.706787 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.755339 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:15.708747 10023 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:15.714066  9942 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:15.722159 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.759056 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.759919 10017 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:15.760994 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.761700 10017 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
W20260709 12:27:15.687979  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60073.9us
I20260709 12:27:15.763114  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:15.698510 18479 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:15.766757 18521 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.715422 18699 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.772317 18699 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:15.772656 18521 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:15.723382 18711 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:15.780504 18725 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.539s	user 0.002s	sys 0.000s
I20260709 12:27:15.783521 18562 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.735349 18250 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:15.736788  9943 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:15.737310 18682 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.737875 18663 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:15.807411 10019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2760) took 1569 ms (client timeout 1753 ms). Trace:
W20260709 12:27:15.807900 10019 rpcz_store.cc:269] 0709 12:27:14.238228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.326068 (+ 87840us) service_pool.cc:224] Handling call
0709 12:27:15.807211 (+1481143us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54001f7f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:15.807343 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2967275264}
I20260709 12:27:15.811467 18581 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:15.817170 18705 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.948s	user 0.003s	sys 0.000s
I20260709 12:27:15.818459 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4242) took 1084 ms. Trace:
I20260709 12:27:15.819003 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:54288 (request call id 0) took 1088 ms. Trace:
I20260709 12:27:15.818859 10093 rpcz_store.cc:276] 0709 12:27:14.734314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.950638 (+216324us) service_pool.cc:224] Handling call
0709 12:27:14.951070 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.754820 (+803750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.818229 (+ 63409us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b5400305a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:15.818402 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1710353280}
I20260709 12:27:15.819602 10092 rpcz_store.cc:276] 0709 12:27:14.730417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.865867 (+135450us) service_pool.cc:224] Handling call
0709 12:27:15.818692 (+952825us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b54002fd354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955c735 00007f4e095549fa 00007f4e0e51c4c9 00007f4e07a98c54 00007f4e07a98be2 00007f4e07a98b11 00007f4e07a98a8c 00007f4e07a97d42 00007f4e0789148c 00007f4e078908c6
0709 12:27:15.818901 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":80661248}
I20260709 12:27:15.820314 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:15.820492 18715 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.820961 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.822037  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:15.825448 18663 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:15.826202 18663 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:15.827387 18667 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:15.827438 10099 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:15.827859 18667 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:15.828270 10099 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:15.826231 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1597) took 1295 ms. Trace:
I20260709 12:27:15.829429 10022 rpcz_store.cc:276] 0709 12:27:14.530958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.207544 (+676586us) service_pool.cc:224] Handling call
0709 12:27:15.230515 (+ 22971us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.279772 (+ 49257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.279801 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:15.279853 (+    52us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:15.279883 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.825917 (+546034us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b540038ccd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:15.825980 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:15.826158 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112822656}
I20260709 12:27:15.828728 10099 raft_consensus.cc:614] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:15.833151 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:15.737349 10098 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:15.743786 18672 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.744642 18532 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.745666 18675 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.683467 18647 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:15.672873 18696 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:15.765867 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62576.3us
I20260709 12:27:15.695474 18690 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:15.766721 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.767350 18586 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:15.768046 18461 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.712754 18216 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.771389 18691 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.725818 18617 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
I20260709 12:27:15.731822 18456 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.785295 10672 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:15.785389  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:15.788926 10024 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:15.789815  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:15.792492 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4241) took 1058 ms. Trace:
W20260709 12:27:15.861574  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 71861us
I20260709 12:27:15.861294 10097 rpcz_store.cc:276] 0709 12:27:14.734003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.874444 (+140441us) service_pool.cc:224] Handling call
0709 12:27:14.998322 (+123878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.056420 (+ 58098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.792216 (+735796us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54002fce50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:15.792417 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":280953600}
I20260709 12:27:15.862486  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:15.797595 18030 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.867221 10095 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:15.867714 10093 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:15.797860 18671 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.799396 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2763) took 1560 ms. Trace:
I20260709 12:27:15.799247 18571 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.791721 18408 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:15.793040 18250 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:15.800760 18593 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.802971 10090 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 25 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:27:15.887627 10090 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:15.887898 18571 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.888784 18456 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.889521 10090 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:15.889863 10023 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:15.804323 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1794) took 1074 ms. Trace:
I20260709 12:27:15.890818 10674 rpcz_store.cc:276] 0709 12:27:14.730410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:14.730570 (+   160us) service_pool.cc:224] Handling call
0709 12:27:15.804119 (+1073549us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b54006ff754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:15.804261 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1811607808}
I20260709 12:27:15.892205 10094 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:15.806279 18562 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.836609  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:15.894497 18696 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:15.894719  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58221.1us
I20260709 12:27:15.822355 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:15.896574 18671 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.841375 10098 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:15.795203 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.897119 18671 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:15.794348 18558 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:15.897583 18715 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:15.845173  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:15.848105  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:15.848272  9945 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:27:15.850365  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.851513 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:15.900094 10097 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:27:15.900597 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49174.8us
I20260709 12:27:15.901295 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:15.901924  9947 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:15.859802 10096 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:15.892207 10025 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:15.876019 10020 rpcz_store.cc:276] 0709 12:27:14.238993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.177559 (+938566us) service_pool.cc:224] Handling call
0709 12:27:15.202929 (+ 25370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.495063 (+292134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.799144 (+304081us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54002161d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:15.799336 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":670259584}
W20260709 12:27:15.877288  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 92002.6us
I20260709 12:27:15.878235 10095 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:15.879112 10093 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:15.907431 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:15.866690 18402 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:15.880553 18675 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:15.880885 18691 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.828735 18667 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.881824 18030 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.882797 18617 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:15.894021 18250 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.838195  9940 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:27:15.854682  9943 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:27:15.843503 18582 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:15.907930 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:15.921233  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.849720 18647 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:15.907531 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:15.869050 18586 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:15.926990 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:15.935412 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2774) took 1197 ms. Trace:
I20260709 12:27:15.908032 18593 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:15.924063 18443 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:27:15.901765 18603 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:15.938187 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.907388 18575 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:15.948350 18732 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:15.940518  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:15.976068 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.927860  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.897615 18671 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:15.936121 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:15.977037 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.977352 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.952476  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:15.926327 18562 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:15.934031 18690 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:27:15.962925 18667 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:15.919795 18664 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:15.923983 18363 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:15.961237  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:15.915470 18682 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:27:15.953528 18479 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:15.999058 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:16.001320 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:16.001911 10671 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
W20260709 12:27:15.938215 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:15.974264 10672 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:15.924680 18571 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:27:15.933490 18580 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:27:15.953244 18582 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:15.929122 18663 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:27:16.015756 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:15.975008 18603 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:15.957800 10025 rpcz_store.cc:276] 0709 12:27:14.737658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.249503 (+511845us) service_pool.cc:224] Handling call
0709 12:27:15.249946 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.905964 (+656018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:15.924439 (+ 18475us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:15.935355 (+ 10916us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1476610944}
I20260709 12:27:15.937282 18683 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:27:15.964804 18250 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:15.981006 18562 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:15.999653 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:16.010393 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:16.012092  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 50979.3us
I20260709 12:27:15.971444 18575 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:16.016463 10676 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:15.979364 10673 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:27:15.949837 18030 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:27:16.001987 18216 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:16.020201 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 371456us
I20260709 12:27:16.014119 18715 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:16.047217 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:16.037793  9945 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:16.031615  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:16.043440  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 122897us
I20260709 12:27:16.029147 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:16.107446 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78389.8us
I20260709 12:27:16.060497 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:16.104863 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:16.105023 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:16.042809  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 168856us
W20260709 12:27:16.108500  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 319187us
I20260709 12:27:15.982462 18732 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.061827 10677 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:16.015412 18582 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:16.125073 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 78099.9us
I20260709 12:27:16.177565 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:16.129419 18747 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:16.047003 18575 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:16.145124 18728 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:16.152134 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.019552 18682 consensus_queue.cc:1243] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 20.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:27:16.104246 18436 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:16.028986 18663 consensus_queue.cc:1243] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 25.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:27:16.114290 18530 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:16.172350  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:16.206192 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:16.161813  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45977.6us
I20260709 12:27:16.102814 18586 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.050971 18558 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:16.188637 18461 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:27:16.211475 18682 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:16.190970 18663 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.212555 18663 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.213387 18558 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.213783  9832 maintenance_manager.cc:419] P 2121bfede85b447c9f51ef23233606cc: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033391
I20260709 12:27:16.213972 18699 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.215832 18558 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:16.171677 18739 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.206s	user 0.000s	sys 0.001s
I20260709 12:27:16.216905 18696 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:16.217377 18696 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:16.217804 18696 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:16.001304 18363 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.220304 18363 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:16.164538 18216 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:16.175026 18718 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:16.225449 18216 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:16.226948 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:16.227973 18747 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:16.175179 18402 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.017s
I20260709 12:27:16.121279 18671 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.231405 18747 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:16.232647 18747 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:16.233167 18747 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:16.234015 18747 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.235028 18747 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:16.235349 18732 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:16.237298 18408 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:16.179024 18754 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:27:16.242715 18571 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:27:16.244887 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.246116 18456 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:16.246408 18696 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.247304 18580 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:16.247658 18580 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:16.248594 18580 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.12, Last appended by leader: 12, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.250100 18647 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.190002 18728 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:16.189073 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.215889 18663 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.204699 18688 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:16.255251 18688 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:16.255654 18664 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:16.256364 18688 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.204430 18672 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
W20260709 12:27:16.258030 18741 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:27:16.180029 18603 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.196722 18436 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.265256 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.265923 10092 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:16.190166 18695 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:16.285827  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 76818.1us
I20260709 12:27:16.177958 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:16.237728 18408 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:16.264539 18699 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.296902 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:16.298738 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 121036us
I20260709 12:27:16.306659 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:16.314822  9941 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:27:16.306003  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:16.318254 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 73929.8us
I20260709 12:27:16.292065  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:16.295239 18728 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:16.333593 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:16.334705 10024 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 19. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:27:16.315769 18617 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:16.338025 10023 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:16.341780 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:16.189702 18562 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.303493 18746 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:16.318409 18695 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:16.345799 18617 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.350461 18436 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:16.292970 18558 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:27:16.320700 18571 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:16.337170 18696 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.353405 18030 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.013s
I20260709 12:27:16.367295 18617 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.355393 18672 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
W20260709 12:27:16.376338 18748 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:16.341661 18754 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:16.360592 18521 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:27:16.392374 18711 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.616s	user 0.009s	sys 0.002s
I20260709 12:27:16.388181 18754 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:16.361874 18695 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.428449 18754 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:16.434522 18695 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:16.430197 18711 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.504416 18711 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:16.507788 18667 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.510s	user 0.013s	sys 0.000s
I20260709 12:27:16.533483 18754 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.536767 18754 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:16.572643  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 237987us
W20260709 12:27:16.575229  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 382857us
I20260709 12:27:16.337975  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:16.587955  9829 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
W20260709 12:27:16.663452  9829 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 75815.6us
W20260709 12:27:16.589040  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 481477us
I20260709 12:27:16.666378 18667 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:27:16.589898 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.668493 18667 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:16.669055 18667 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:16.589955 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3136) took 1374 ms. Trace:
W20260709 12:27:16.590972 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.676802 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:16.578418  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 546746us
I20260709 12:27:16.601823  9942 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:16.603893  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:16.610759  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:16.612994  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:16.615434  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:16.617236  9943 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:16.618614  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:16.625638 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.625926 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.626318 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:16.352949 18408 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.682194  9943 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 19 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:16.630620 18436 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.426934 18521 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:16.635102 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:16.636940 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:16.637913 18548 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.649747 10093 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:16.634352 18371 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:16.701936 18521 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:16.706995 18548 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:16.708377 10062 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.753s (SENT) [suppressed 2 similar messages]
I20260709 12:27:16.673597 10091 rpcz_store.cc:276] 0709 12:27:15.212086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.670034 (+457948us) service_pool.cc:224] Handling call
0709 12:27:15.670305 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:15.670499 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:15.671427 (+   928us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:15.677433 (+  6006us) log.cc:844] Serialized 26 byte log entry
0709 12:27:16.037729 (+360296us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:16.037771 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.037862 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:16.586172 (+548310us) raft_consensus.cc:1685] finished
0709 12:27:16.586219 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.586528 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":310500,"threads_started":1,"wal-append.queue_time_us":492595}
I20260709 12:27:16.712055  9941 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 23 because replica has last-logged OpId of term: 22 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:27:16.740334 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.748189 18689 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:16.750788 10096 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:16.752050  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:16.741031 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.752868  9947 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:27:16.753360  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.753546 18402 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:16.754631 18754 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:16.755556 18754 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.754415 18402 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.758571 18402 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:16.758702  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2924) took 2054 ms. Trace:
I20260709 12:27:16.713115 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.759301  9948 rpcz_store.cc:276] 0709 12:27:14.704463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.194314 (+489851us) service_pool.cc:224] Handling call
0709 12:27:15.222923 (+ 28609us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:15.351457 (+128534us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:15.655022 (+303565us) log.cc:844] Serialized 14 byte log entry
0709 12:27:16.293812 (+638790us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:16.293885 (+    73us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:16.293932 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:16.758148 (+464216us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b54001b8354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:16.758220 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:16.758452 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17363,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":22,"raft.queue_time_us":146370,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5664,"spinlock_wait_cycles":295037184,"thread_start_us":919511,"threads_started":3,"wal-append.queue_time_us":529509}
I20260709 12:27:16.762915 18581 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:16.765137 18667 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.767724 10090 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:16.770882  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:16.754403  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.749366 18436 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.763654 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:16.733004  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 394923us
I20260709 12:27:16.693547 18371 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:16.758381 18754 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.710558 18548 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.718494 18755 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:16.793954 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.728276 18408 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:27:16.805632 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 97466.8us
I20260709 12:27:16.807559 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:16.794413 18755 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:16.809294  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:16.793452 18581 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:16.820672 18695 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.822917 18461 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:16.823722 18371 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.824486 18461 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.827126 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:16.836196 10093 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:16.837445 10023 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:16.839856 18548 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:16.841117 18767 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:16.842008 18461 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.841913 18767 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.843987 18695 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.845029 18767 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:16.845142 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 238579us
I20260709 12:27:16.849041 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:16.850081 18731 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:16.858991 10022 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:27:16.859365 10016 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:16.814872 18667 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:16.869953 18582 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.686s	user 0.014s	sys 0.007s
I20260709 12:27:16.872627 18731 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:16.873852 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:16.871830  9987 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:16.825202 18371 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:16.876704 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:16.891724 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.892462  9947 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
W20260709 12:27:16.899652  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 105266us
I20260709 12:27:16.900910  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:16.904651  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:16.911593 18682 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.912453 18682 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.913149  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:16.914930 18402 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:16.911677 18755 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.918205 18755 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:16.931303 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.931874 10098 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:27:16.933161 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.887580 18461 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.934432 18461 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.936094  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:16.936810  9945 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:16.937199 18461 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:16.937863  9944 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:16.938279  9945 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:27:16.923882  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:16.947801 18548 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:16.924014 18760 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:16.948483 18548 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.943946 10096 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:16.951133 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.951985 10098 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
W20260709 12:27:16.926731  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 55124.8us
I20260709 12:27:16.928050 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:27:16.953104 10025 raft_consensus.cc:606] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:16.953462 10025 raft_consensus.cc:614] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:16.953544  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:16.927376 18402 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:16.922324 10094 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:27:16.954730  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:16.955185 18581 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:16.955643 18581 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:16.957172 18596 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:16.923197 18436 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:16.958381 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:16.958954 18682 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:16.959789 18548 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:16.960891  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47912.5us
I20260709 12:27:16.963498  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:16.965000 18760 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:16.965822  9912 leader_election.cc:400] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27.
I20260709 12:27:16.966154  9912 leader_election.cc:403] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
W20260709 12:27:16.968719 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 119836us
W20260709 12:27:16.929757 18250 consensus_meta.cc:287] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.885s	user 0.016s	sys 0.010s
I20260709 12:27:16.969643 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.943897  9940 pending_rounds.cc:85] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 13
I20260709 12:27:16.942006 18580 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:16.951719 18216 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:16.987248  9945 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:27:16.927376  9943 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:16.989555 10098 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:16.989810  9943 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:16.990311 10098 raft_consensus.cc:606] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:16.975641 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:16.990744 10098 raft_consensus.cc:614] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:16.971442  9940 pending_rounds.cc:107] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:27:16.991914  9940 raft_consensus.cc:2889] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:16.891645 18582 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.997381 18582 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:16.975947 18581 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:16.978256 18586 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.001583 10675 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:17.002095 18586 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.004664 18586 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:17.006120 18581 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:16.983098 10674 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:17.006810 18760 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27.
I20260709 12:27:16.985819  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.011078 18762 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.012053 18682 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:17.010996  9948 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:17.013310 18682 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:17.013667 18682 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:17.014606  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:17.014921  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.016005  9945 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:16.986485  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:17.018824 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49191.1us
I20260709 12:27:17.019170  9949 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:17.019727 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:16.958992 10099 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:17.020798 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:17.021665 18436 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:16.987522 18216 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:16.986654 10093 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:17.022483 18216 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:17.022961 18436 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:17.023154 10095 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:17.023952 18436 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.023784 18216 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.011947 18762 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.025058 18436 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:17.025465 18436 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:17.026889 18762 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:17.026553 18436 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.027972 18436 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:17.028025 18216 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.019109 18682 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.031627 18682 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:17.032757  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45572.4us
I20260709 12:27:17.034039  9943 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:17.034930 18672 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:16.980367  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.035924 18548 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:17.038915 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:17.039960 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:17.040737  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:17.039716  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:17.056137  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:16.946208 18521 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:17.073671  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 105346us
I20260709 12:27:17.078869 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.082481  9943 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:27:17.082896 10092 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:17.087168  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:27:17.092958 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54135.5us
I20260709 12:27:17.072069 18580 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:17.100392  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:17.091074 18596 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:17.112495  9942 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:17.087066 18582 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.115774 10093 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:17.090229 18250 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.117841  9947 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:17.118747 10094 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:17.121131 10090 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:27:17.131815  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:17.102831 18580 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:17.133518 10673 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:27:17.134809 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 114143us
I20260709 12:27:17.128458 18656 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:17.138809  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62188.5us
I20260709 12:27:17.131446 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.134704 10668 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:17.156530 10677 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:17.129211 18593 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:27:17.159567 18690 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.160261 18593 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:27:17.160223 18690 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.160766 18593 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:17.163132 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.164084 10672 raft_consensus.cc:2410] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 16.
I20260709 12:27:17.165134 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.165843 10672 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:17.166795 18250 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:17.167472 18575 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.980s	user 0.018s	sys 0.002s
I20260709 12:27:17.167979 18575 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.169137 18250 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:17.115267 18596 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:17.134138 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.131321  9831 maintenance_manager.cc:643] P 2121bfede85b447c9f51ef23233606cc: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.916s	user 0.038s	sys 0.007s Metrics: {"bytes_written":5460,"delete_count":0,"lbm_write_time_us":2180,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":5,"reinsert_count":0,"update_count":195}
I20260709 12:27:17.161876 18593 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:17.140198 18715 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 20 current term: 20 is higher.
W20260709 12:27:17.177538 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.135411 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.178678 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:17.113821 18695 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:27:17.135654 18696 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 12:27:17.181757 18456 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:17.182309 18699 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.182878 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.183247 18521 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:17.125288 18586 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.185122 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.176704 18580 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.192287  9947 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:17.170332 18575 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:17.195617 10676 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:17.157732 18689 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:17.202630 18689 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.204588 10020 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:17.204789 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.205008 18689 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:17.205895  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.207510 18699 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.138931 18582 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.209549 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:17.209892 18699 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:17.178695 18767 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:17.210764  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:17.211198 18582 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:17.211104 18767 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.212548  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:17.213399 18767 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:17.215407 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.216219  9832 maintenance_manager.cc:419] P 2121bfede85b447c9f51ef23233606cc: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.279595
I20260709 12:27:17.216941 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.218374 18548 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.218874 18548 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.221787 18667 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:17.222234 18667 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:17.222649 18667 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.223316 18671 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:17.223760 18671 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.224124 18671 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:17.155877 18656 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.189458 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1818) took 1203 ms. Trace:
I20260709 12:27:17.180310 18596 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.227133 10675 rpcz_store.cc:276] 0709 12:27:15.976616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:15.976720 (+   104us) service_pool.cc:224] Handling call
0709 12:27:17.142315 (+1165595us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54004d35d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:17.179803 (+ 37488us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2226963840}
I20260709 12:27:17.188993 10674 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:17.226544 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.180660 18690 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:17.256223 10016 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:17.256495 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.250006 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.259573 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.263980 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.264874 10025 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.243028 18656 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:17.276441  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:17.244316 18582 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.266847 18672 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.291075 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.254109 18760 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:27:17.296828 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.294831  9946 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:17.301156 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.198751 18682 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.077s
I20260709 12:27:17.305197 18593 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:17.305918 18672 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.275189 18586 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.308102 18672 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:17.309916 18586 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:17.311574 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.312180 10673 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.305310 10023 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 19. Current term is 20. Ops: []
I20260709 12:27:17.323235 10676 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
W20260709 12:27:17.332595 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 175712us
I20260709 12:27:17.332800 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.337393 10677 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:17.359503  9949 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 12:27:17.366382 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.378949 18436 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:17.380800 18436 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:17.315452 18690 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:17.379592 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 170267us
I20260709 12:27:17.383075 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:17.387838 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.396273 18656 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.360148  9949 pending_rounds.cc:85] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 13
I20260709 12:27:17.398828  9949 pending_rounds.cc:107] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:27:17.399102  9949 raft_consensus.cc:2889] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:17.400135 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.400892 10676 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:17.393911 18596 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.402774 18754 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.403398 18754 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.403952 10092 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:17.405500 18754 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:17.406018 18763 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:17.406246 18596 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:17.407622 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:17.408541 10091 raft_consensus.cc:2368] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21.
I20260709 12:27:17.409252 18747 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:17.409713 18747 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:17.410884 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:17.410825 18747 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.412150 18747 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:17.412496 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:17.413360 10093 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:17.416867 10016 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:17.416602 18690 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.424031 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:17.453733 10673 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:17.455016 10669 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.421883  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:17.457793 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.461298 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.469177 10671 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:17.480273 10025 raft_consensus.cc:2410] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:27:17.479869 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.484422  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:17.530475 10022 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:17.470268 18682 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:17.469396 18656 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:17.533701  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 77353us
I20260709 12:27:17.484107 18586 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:17.479815 18456 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:17.546945 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:17.547116 10019 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:17.547456 18761 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:17.548118 18761 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:17.548854 10093 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:27:17.549224 10093 pending_rounds.cc:85] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 15
I20260709 12:27:17.549638 10093 pending_rounds.cc:107] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:27:17.549165 18761 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.549880 10093 raft_consensus.cc:2889] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:17.551214 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:17.551653 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:17.554021  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 485290us
I20260709 12:27:17.554308 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.555405 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.557541 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:17.558645 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.569329 18755 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.569238  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:17.570202 18596 raft_consensus.cc:2764] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:27:17.570034 18755 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.570540 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.572062 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:17.573324 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 190318us
I20260709 12:27:17.573451 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.566573 10018 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.574314 18731 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:27:17.555994 18582 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:17.480080 18672 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.567076 18767 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:17.576570  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:17.567750 10673 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:17.574548 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.566887 10676 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:27:17.582194 10022 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:17.536616 18682 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.566700 18581 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:17.536382 18575 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:17.601977 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.609968 18688 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.611268 18690 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:17.611011 18688 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:17.575834 18731 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:27:17.615288 18767 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.618119 18767 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:17.619509 10021 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:27:17.623329 10669 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:17.623409 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.637862 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:17.638698 10669 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:17.647379  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45057.2us
I20260709 12:27:17.601948 18682 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:17.649686 10642 leader_election.cc:400] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21.
I20260709 12:27:17.651286 10018 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:17.654258 10642 leader_election.cc:403] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:27:17.603029 18581 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:17.662294  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.665033  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:17.665015 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 196103us
I20260709 12:27:17.651687 18250 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.605630 18667 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.677591  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.685952  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.696636 10021 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:27:17.642671 18575 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.689035  9948 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:17.700245  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:17.675269 18596 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:27:17.637318 18672 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.705787  9944 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:17.707428 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:17.708572 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:17.711373 10096 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:17.711849 18793 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:17.713678  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.699855 18456 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.720809 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.721751 10018 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:17.725510 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:17.726554  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61625us
I20260709 12:27:17.676017 18548 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.733332 10017 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
W20260709 12:27:17.736932 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87451.9us
I20260709 12:27:17.741387 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:17.758423  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:17.762521 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.764820  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:17.769764 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:17.770063 10020 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:17.767278 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.771021 10090 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:17.775820 18575 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:17.776595  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:17.778261  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:17.782553 18760 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:17.783212 18667 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:17.784029 18667 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:17.784874 18667 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:17.786935 18548 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:17.727896 18688 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:27:17.771131 10021 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:17.807199  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:17.788884  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.821633 18558 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:17.822181 18558 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:17.822548 18558 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:17.827086 18805 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21.
I20260709 12:27:17.827955 18805 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:27:17.791565 18580 consensus_meta.cc:287] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.538s	user 0.008s	sys 0.002s
I20260709 12:27:17.720971 18581 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.819903 18575 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.008s
I20260709 12:27:17.833838 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.831715 10018 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:17.832628 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:17.838420 10019 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:17.838485 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 167162us
I20260709 12:27:17.840600 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.842115 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:17.829425 18690 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:17.847929 10021 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:17.848994 18810 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:17.849494 18810 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.833078 18715 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:17.851904 10020 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:17.204169  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:17.878029 10025 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
W20260709 12:27:17.879370 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.863271 18671 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.639s	user 0.004s	sys 0.008s
I20260709 12:27:17.837546 18580 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:17.877717 18581 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:17.884580 18371 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:17.855906 18810 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:17.889170  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:17.900027  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:17.903980  9946 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:17.896359  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:17.876678 18408 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.844798 18586 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:17.915330 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:17.911307 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 335133us
W20260709 12:27:17.954802  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 135913us
W20260709 12:27:17.949707  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 184635us
I20260709 12:27:17.954285  9947 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:17.920202 18593 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:17.957288 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:17.925158 18399 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:17.950865 18575 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:17.898240 18671 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:17.957566 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 216265us
I20260709 12:27:17.955365 18548 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:17.951864 18408 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:17.965513 18575 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:17.973762 18672 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:17.955260 18371 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:18.024874 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67108.6us
W20260709 12:27:18.014129  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 235970us
I20260709 12:27:17.969494 18760 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:18.046919 18408 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:18.048632 18671 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.052915 18580 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:18.054059 18812 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:18.056701 18371 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.057612 18731 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.058183 18731 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.059458 18809 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:18.059783 18811 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.060340 18811 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.060415 18731 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:18.050307 18755 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:17.979391 18593 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.040921 18586 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.040568 18548 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.062557 18811 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:18.066831 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.067704  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2956) took 1295 ms. Trace:
I20260709 12:27:18.089031  9949 rpcz_store.cc:276] 0709 12:27:16.772387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.020155 (+247768us) service_pool.cc:224] Handling call
0709 12:27:17.020469 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.399593 (+379124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.067410 (+667817us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540009bad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:18.067638 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":680932096}
I20260709 12:27:18.065972 18667 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:18.090308 18667 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:18.091384 18667 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.092970 18667 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:18.096185 18798 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.352s	user 0.000s	sys 0.001s
I20260709 12:27:18.098166  9946 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:18.102881  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2959) took 1233 ms. Trace:
I20260709 12:27:18.103258  9943 rpcz_store.cc:276] 0709 12:27:16.869126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.087071 (+217945us) service_pool.cc:224] Handling call
0709 12:27:17.103289 (+ 16218us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.111251 (+  7962us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:17.227718 (+116467us) log.cc:844] Serialized 14 byte log entry
0709 12:27:17.630075 (+402357us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.663042 (+ 32967us) log.cc:844] Serialized 26 byte log entry
0709 12:27:17.663289 (+   247us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:17.663343 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.663453 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.095473 (+432020us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:18.102565 (+  7092us) raft_consensus.cc:1685] finished
0709 12:27:18.102690 (+   125us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.102823 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75275,"prepare.queue_time_us":124920,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":3531,"raft.queue_time_us":826,"raft.run_cpu_time_us":376,"raft.run_wall_time_us":24153,"spinlock_wait_cycles":4385152,"thread_start_us":326511,"threads_started":1,"wal-append.queue_time_us":405899}
W20260709 12:27:18.105165  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 78309.3us
I20260709 12:27:18.099646 18682 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.108309  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:18.109169 18767 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:18.109948 18682 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.110625 18767 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:18.113207  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:18.114811 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:18.146096 10025 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:18.114979 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.116117 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:18.116545 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.117148 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.119889 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:18.146953 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.123008 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.123740 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:18.126307 18790 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.736s	user 0.005s	sys 0.001s
I20260709 12:27:18.128154  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:18.102334 18810 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.085520 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:18.153793 10094 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:18.144301 10020 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:18.147150 10019 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
W20260709 12:27:18.151379 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 107661us
I20260709 12:27:18.154330 10099 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
W20260709 12:27:18.071574 18787 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.605s	user 0.002s	sys 0.000s
W20260709 12:27:18.085651 18801 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.368s	user 0.000s	sys 0.001s
I20260709 12:27:18.171064 10023 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:18.173480 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.161844 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:18.174698 10098 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:18.163331 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:18.164191 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:18.176430 10674 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
W20260709 12:27:18.177858  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64714.3us
I20260709 12:27:18.178969  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:18.166481 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.070392 18786 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.601s	user 0.000s	sys 0.005s
W20260709 12:27:18.181981 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 232936us
W20260709 12:27:18.028712 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60083us
I20260709 12:27:18.183991 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.184536 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:18.072371 18593 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.184645 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:18.187639 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:18.187724 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:18.189162 10669 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:18.085048 18548 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.192147 10093 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:18.186395 18580 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:18.203147 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:18.183656 18747 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:18.185961 18767 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:18.232275  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:18.078467 18558 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:18.086205 18575 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:18.252774 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47930.5us
W20260709 12:27:18.253624 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:18.254143  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.207530 18760 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.258327  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3254) took 1589 ms. Trace:
I20260709 12:27:18.258809  9941 rpcz_store.cc:276] 0709 12:27:16.668910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:16.941643 (+272733us) service_pool.cc:224] Handling call
0709 12:27:16.942108 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:16.942546 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:16.942577 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:16.944749 (+  2172us) log.cc:844] Serialized 26 byte log entry
0709 12:27:17.090021 (+145272us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:17.090275 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.090350 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.070811 (+980461us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:18.258009 (+187198us) raft_consensus.cc:1685] finished
0709 12:27:18.258033 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.258260 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":247209216,"thread_start_us":144885,"threads_started":1,"wal-append.queue_time_us":331882}
I20260709 12:27:18.265375 18760 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.269135 18558 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:18.224772 18811 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:18.275130 10025 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:18.276501 18586 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:18.276860 18767 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.277751 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:18.278520 10671 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:18.279614 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:18.282200 18672 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:18.283052 18672 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:18.284237 18695 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:18.284379 18672 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.277104 18586 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.294472 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.294831 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.295396 10019 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:27:18.274379 18811 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.299404 18811 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:18.273391 18581 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:18.302976 18581 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.305274 18581 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:18.296269 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:18.306084 18656 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:18.306387 18810 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:18.306077 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:18.306922 18656 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:18.308907 18586 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:18.307847 18656 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.312341 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1816) took 1196 ms. Trace:
I20260709 12:27:18.312736 10670 rpcz_store.cc:276] 0709 12:27:17.115581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.139658 (+ 24077us) service_pool.cc:224] Handling call
0709 12:27:17.140014 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:17.332665 (+192651us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:17.660225 (+327560us) log.cc:844] Serialized 14 byte log entry
0709 12:27:17.854085 (+193860us) log.cc:844] Serialized 14 byte log entry
0709 12:27:17.961812 (+107727us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:17.961863 (+    51us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:17.961912 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.311996 (+350084us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b54007f4754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:18.312041 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.312162 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22505,"prepare.queue_time_us":348763,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":86,"raft.queue_time_us":356809,"raft.run_cpu_time_us":388,"raft.run_wall_time_us":51371,"spinlock_wait_cycles":405754112,"thread_start_us":513019,"threads_started":3,"wal-append.queue_time_us":316802}
I20260709 12:27:18.315161 10020 raft_consensus.cc:1240] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:18.295482 10024 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:18.278997 18689 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:18.322430 10017 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:27:18.337164 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.337383 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:18.341161  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 112602us
I20260709 12:27:18.345341 18830 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:27:18.342247 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:18.348394 18586 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:18.348832 18586 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.349192 18586 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:18.273917 18747 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.362895 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:18.365263 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:18.334344 18371 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.026s
W20260709 12:27:18.366173 18833 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.099s	user 0.004s	sys 0.000s
I20260709 12:27:18.364425 18581 consensus_queue.cc:1059] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:27:18.370493 18810 raft_consensus.cc:3055] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:18.371831  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:18.373121 18558 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:18.378911 18838 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:18.379248 18558 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.379688 18656 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:27:18.381379 18838 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:18.381736 18558 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.385372 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:18.374151 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:18.360253 18582 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:27:18.382428 18838 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.365087 18831 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 12:27:18.395391 18831 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:18.395762 18831 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:18.370889 18810 raft_consensus.cc:740] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:18.383490 18548 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:18.406035 10098 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
W20260709 12:27:18.418819  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47077us
I20260709 12:27:18.423408  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:18.396790 18831 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.427623 18831 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:18.432904 18747 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.437120 18811 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.439205 18810 consensus_queue.cc:260] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:18.461913  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:27:18.443548  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 232647us
I20260709 12:27:18.444401 18760 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:18.446794 18755 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.447379  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2967) took 1240 ms. Trace:
I20260709 12:27:18.447458 18671 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.412617 18548 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:18.458994  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:18.461973  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.462301  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.497678  9946 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
W20260709 12:27:18.482278 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 97013us
W20260709 12:27:18.484067  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53247.4us
I20260709 12:27:18.454063 18656 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:18.495496  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.452739 18581 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:18.406816 18586 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.501775  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.453119 18844 raft_consensus.cc:2764] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:27:18.499225 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:18.520138 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.490998  9940 rpcz_store.cc:276] 0709 12:27:17.207263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.394675 (+187412us) service_pool.cc:224] Handling call
0709 12:27:17.395141 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:17.395634 (+   493us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:17.395674 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:17.396401 (+   727us) log.cc:844] Serialized 26 byte log entry
0709 12:27:17.625054 (+228653us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:17.625120 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:17.625221 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:18.084998 (+459777us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:18.394445 (+309447us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b2400193088. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a7fb0 00007f4e0911e488 00007f4e0911ebe8 00007f4e090f6398 00007f4e090f176a 00007f4e090f0bb6
0709 12:27:18.447107 (+ 52662us) raft_consensus.cc:1685] finished
0709 12:27:18.447138 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:18.447307 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2570,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":149,"spinlock_wait_cycles":12229248,"thread_start_us":202862,"threads_started":1,"wal-append.queue_time_us":320698}
I20260709 12:27:18.507407  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:18.502677 18548 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:18.529443  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:18.527177 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 122312us
I20260709 12:27:18.505191 18581 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:18.503501 18672 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.531363 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:18.524120 18656 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:18.537930  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:18.497463 18755 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.539748  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:18.532223 18754 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:18.567652  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.568882  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.569654  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:18.570004  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.571375 18667 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:18.571785 18667 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:18.572175 18667 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:18.574100 10091 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:18.575380 18715 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:18.544910 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:18.502418 18671 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.533006 18747 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:18.547775 18832 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.290s	user 0.000s	sys 0.002s
W20260709 12:27:18.553644 18842 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:27:18.534228 18810 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:18.589840  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:18.556842 10095 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:18.557360 18755 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.535367 18690 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:18.559608 18811 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:18.560169 18586 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:18.560295 18656 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.561648 18581 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.563603 18399 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:18.564800  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:18.565419 18456 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:18.565261 18672 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.579365 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:18.580816 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:18.768313 10020 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:18.581234 18575 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:18.581946  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.602109  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:18.770432  9941 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:18.746388  9946 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:18.773175  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.606453 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:18.773882 10099 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:18.765405  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 152679us
I20260709 12:27:18.750795 18371 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:18.744714 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:18.775075 10016 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:18.778954 10098 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:18.760917 18715 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:18.765717 18811 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:18.772553 18763 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:18.777408 18682 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.809746  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:18.811487 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 280183us
W20260709 12:27:18.813725  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 249039us
I20260709 12:27:18.818429  9946 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:18.819536  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.772826 18755 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:27:18.825449  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.826731 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:18.838074 10020 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:18.842206  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:18.845741 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:18.823587 18682 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:18.851862  9945 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:18.852902 18682 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:18.854063  9945 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
W20260709 12:27:18.858139 18850 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:27:18.835906 18763 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:18.858918 18763 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:18.860234 18847 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:27:18.859360 18849 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:27:18.866782  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.871521  9941 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:18.873158 18829 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:18.873507 18820 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:18.874116 18820 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.876296 10643 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:18.877060 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:18.877460 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:18.878997 18814 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:18.879226 18831 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:18.881282 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:18.881989 18829 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:18.882489 18829 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:18.883111 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:18.883339 18829 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.884742 10095 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:18.890805 18671 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.894761 18672 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.894950 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:18.895676 18715 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.897313 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:18.898465 18586 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:18.898808 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:18.874431 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:18.896404 18831 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:18.853443 18682 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:18.906045 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:18.907366 10016 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:18.922782 18763 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:18.923314 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:18.923408 18763 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:18.923991 18763 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.927520 18820 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.825515 18811 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.933528 10095 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:27:18.929016 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:18.908455 18814 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:18.936471  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 94394.2us
I20260709 12:27:18.918220 18671 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:18.920670 18582 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:18.948092  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:18.938913 18763 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:18.943161 18805 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:18.917969 18575 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 12:27:18.946693 18747 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:18.978644 18755 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:18.937314 18548 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:18.995234  9947 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:27:18.929752 18682 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:18.983414 18399 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:19.011633  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:19.022081  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 442043us
I20260709 12:27:19.025223  9945 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:18.988071 18667 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.028374 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2781) took 1677 ms. Trace:
I20260709 12:27:18.996527 18805 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.036623 10023 rpcz_store.cc:276] 0709 12:27:17.346322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:17.346626 (+   304us) service_pool.cc:224] Handling call
0709 12:27:18.170901 (+824275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:18.182264 (+ 11363us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.023363 (+841099us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b5400216bd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:19.023506 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1647489920}
W20260709 12:27:19.047528  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 125462us
I20260709 12:27:19.027428 18848 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:19.048113 18852 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.034175 18667 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.019661 18755 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.060251 10677 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:27:19.061287 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 159673us
W20260709 12:27:19.072089  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 84841.3us
I20260709 12:27:19.064101 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.063166 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.074302 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:19.027978 18858 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:19.091997 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:19.092739 10668 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:19.094519  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:19.099504 18827 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:19.099888 18596 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:19.107712 18859 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.108428 18682 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:19.108273 18859 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.111837 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.039994 18837 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.114431 18837 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.116312 18548 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:19.083340 18747 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.039410 18575 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:19.119483 10024 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:27:19.057521 18852 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:19.124759 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50564.5us
I20260709 12:27:19.062772 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:19.141050 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 60759.9us
I20260709 12:27:19.024533 18456 consensus_queue.cc:1243] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:27:19.143283 10099 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:19.140717 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:19.141506 10095 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:19.149058 10668 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:19.149652  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:19.150228  9941 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:19.152225 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.068434 18811 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.071120 18805 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.157189  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.161518  9940 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:19.138758 18829 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.139971 18810 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:19.176759 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.165460 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:19.155445 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.171530  9941 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:19.061151 18399 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.184381 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:19.148742 18814 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:27:19.192052  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69218us
I20260709 12:27:19.159677 18827 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.118793 18575 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.198154  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.201833  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:19.246821 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.249409 18832 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:27:19.255803 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 80768.7us
I20260709 12:27:19.256722 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:19.258862 18844 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:19.259541 18844 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:19.260353 18844 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:19.260653 18844 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:19.261494 18844 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.263568 18852 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:19.264382 18250 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.264459  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.265010 18250 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.265937 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.238770 18827 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.227270 18830 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:19.242794 18831 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.271057 18831 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.271970  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:19.272258 18856 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:27:19.265220  9946 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:19.274685  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:19.240105 18689 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:19.280853 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.293901  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 92170us
I20260709 12:27:19.197571 18862 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:19.323693 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.332901 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:19.345741 18862 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:19.344048 18864 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.073s	user 0.003s	sys 0.000s
I20260709 12:27:19.332845 18586 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.381435 18586 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.384490 18586 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.344218 18844 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:19.188874 18829 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.378827 18755 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.380089 18827 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:19.333165 18399 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:19.344069 18689 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.349066 18371 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.386686 18250 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.385933 18671 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:19.366569  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 71883.8us
W20260709 12:27:19.402338 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 256585us
I20260709 12:27:19.407757 18399 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:19.423408 18869 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:27:19.439709 10672 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:19.424340 18689 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:19.427311 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 132620us
I20260709 12:27:19.385082 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.401064 18814 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:19.404083 18870 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:19.441032 10643 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:19.442867 18399 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.464401 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:19.434458 18371 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.495908 18575 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.496270 18371 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:19.497503 18371 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:19.497951 18371 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:19.506842  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.507560  9940 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:19.498318 18371 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:19.514289  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.544389  9946 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:19.509153 18810 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:19.547006 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 82770.2us
I20260709 12:27:19.546341  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 2976) took 1011 ms. Trace:
I20260709 12:27:19.551455  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.554035 18884 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:19.555063 18839 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.113s
I20260709 12:27:19.556015  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.556638  9946 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:19.557909  9946 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:19.549690  9949 rpcz_store.cc:276] 0709 12:27:18.535172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.854263 (+319091us) service_pool.cc:224] Handling call
0709 12:27:18.854539 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:18.909117 (+ 54578us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:27:18.909148 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:18.909609 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:27:19.038835 (+129226us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:27:19.038886 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:19.038959 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.546141 (+507182us) raft_consensus.cc:1685] finished
0709 12:27:19.546160 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.546285 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16763,"raft.queue_time_us":17485,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":16168,"thread_start_us":92248,"threads_started":1,"wal-append.queue_time_us":264108}
W20260709 12:27:19.558872 18843 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:27:19.460799 18882 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.092s
I20260709 12:27:19.542241 18689 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:19.554553 18884 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:19.586793 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:19.575675 18814 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.580581 18582 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:19.606983 18582 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:19.610309  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.611056  9946 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
W20260709 12:27:19.627599 18836 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.371s	user 0.008s	sys 0.002s
I20260709 12:27:19.629012  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3282) took 1490 ms. Trace:
I20260709 12:27:19.629256 18831 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.629572 18856 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:19.630061 18856 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:19.631338 18856 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.632781 18856 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:19.634829  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66310.5us
I20260709 12:27:19.588136 18689 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:19.641230  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.646960 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.561487 18810 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.626286 18829 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:19.588958 18884 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.651863 10090 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:19.630468 18819 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:27:19.652655 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 65950.8us
I20260709 12:27:19.661266  9946 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:27:19.661620 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:19.629432  9944 rpcz_store.cc:276] 0709 12:27:18.138233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:18.138359 (+   126us) service_pool.cc:224] Handling call
0709 12:27:18.138753 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:18.162917 (+ 24164us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:18.162947 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:18.194873 (+ 31926us) log.cc:844] Serialized 26 byte log entry
0709 12:27:18.256132 (+ 61259us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:18.256193 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:18.256264 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:19.626822 (+1370558us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:19.628664 (+  1842us) raft_consensus.cc:1685] finished
0709 12:27:19.628686 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:19.628952 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7268,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":25468,"spinlock_wait_cycles":35529600,"thread_start_us":29430,"threads_started":1,"wal-append.queue_time_us":51636}
W20260709 12:27:19.653218 18866 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.329s	user 0.001s	sys 0.001s
I20260709 12:27:19.658315  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.663712  9949 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
W20260709 12:27:19.664431 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:19.667076  9831 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.373s	user 0.017s	sys 0.010s
I20260709 12:27:19.679024 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:19.679458 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:19.679994 10671 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:27:19.680712 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.680922 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:19.688061 10017 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:19.689286  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.690033  9944 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:27:19.660619 18819 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:19.663597 18896 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:19.706642  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:19.708923 18888 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:27:19.734912 18819 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.735385 18399 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.736925 18810 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.740896  9842 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a reported cstate change: term changed from 18 to 22, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 22 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
W20260709 12:27:19.749125 18875 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:27:19.751662 18689 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:19.753244  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 481340us
I20260709 12:27:19.753629 18853 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:27:19.754362  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 506245us
I20260709 12:27:19.755014 18755 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:19.755790 18596 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.755075 18715 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:19.756395 18825 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.756842 18862 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.760829 18859 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.765448 18891 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:19.767478 18885 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:27:19.779320 18688 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:19.791313  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:19.791472  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:19.775262 18593 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:19.799557 18825 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.801908 18825 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.792013 18755 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.802804 18755 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:19.802538 18891 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.804947 18891 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:19.806881 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.807539 10023 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:19.807845 18688 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.809908 18688 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.810763 18682 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:19.811198 18682 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.811522 18682 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.753084 18892 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:19.792618 18596 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.814819 18892 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:19.815263 18892 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:19.816427 18596 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:19.819073 18895 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.819324 18582 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.819648 18895 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.820515 18582 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.821864 18895 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:19.822706 18715 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:19.823246 18715 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:19.824184 18715 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:19.829454 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.830133 10091 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:19.819830  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:19.839304  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:19.841410  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:19.844146 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 163286us
I20260709 12:27:19.844352  9947 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:19.844595 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:19.844991 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:19.846277  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.847108 18399 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:19.848372  9945 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
W20260709 12:27:19.850740 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:19.845919 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:19.847590 18760 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:19.877571 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:19.842759 18596 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:27:19.880000 18830 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:19.880486 18830 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:19.880811 18830 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:19.854995 18548 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:19.867938 18893 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.871858 18671 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:19.895098 10676 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 21 index: 18. (index mismatch)
I20260709 12:27:19.903118 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.904089 10020 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:19.906431 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:19.907380 10676 raft_consensus.cc:2376] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23. Re-sending same reply.
I20260709 12:27:19.912014 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:19.917950 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.918771 10025 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:19.883606 18760 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:19.922072 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:19.927033 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:19.922925 18900 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:19.924077  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 104373us
I20260709 12:27:19.934227 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:19.935791  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:19.923543 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:19.924073 18891 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:19.925361 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:19.936640 10677 raft_consensus.cc:2376] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27. Re-sending same reply.
I20260709 12:27:19.940332 10021 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:19.936767  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:19.890332 18853 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:19.941807 10668 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 18. (term mismatch)
I20260709 12:27:19.944031 10668 pending_rounds.cc:85] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 17
I20260709 12:27:19.944450 10668 pending_rounds.cc:107] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:27:19.944779 10668 raft_consensus.cc:2889] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:19.941970  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:19.934906 18760 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:19.964541  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.911702 18682 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.943951 18671 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:19.971422  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.975672  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.977787  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.925210 18371 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.964051 18896 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:19.987394 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.981068  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:19.990373  9943 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:19.978295 18814 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:19.993489 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.995920 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:19.996600 10099 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:19.993983 18814 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:19.999442 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.003680 10090 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:19.911464 18893 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.010128  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:19.992172 18856 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 17, Time since last communication: 0.027s
I20260709 12:27:20.030069 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.031103 10090 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:20.034207 18872 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:20.040170 18905 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.040412 10096 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:20.040910 18905 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.042127 18915 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:20.043923 18814 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.049211 18893 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.049624 10021 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:20.051028  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:20.052554 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.017906 18862 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:20.057806 18862 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.058252 18862 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:20.059662 18682 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:20.062922 10672 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:19.993535 18896 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:20.065347 18889 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.462s	user 0.002s	sys 0.000s
I20260709 12:27:20.067106 18837 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.068563 18895 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:20.069031 18895 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:20.013731 18884 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.072039 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.072731 10099 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:20.073252 18884 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.073594 10676 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:20.033839 18755 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:20.074815  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.007323 18580 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:27:20.087401  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:20.088315 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.088958 10093 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:20.089857  9945 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:20.086145 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.034060 18909 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:20.095683  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 153794us
I20260709 12:27:20.097822 18695 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.098088 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.098556 18695 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.087426 18903 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 12:27:20.104681 18904 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:20.106041 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:20.098822 10090 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:20.099431 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:20.107285 10099 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:20.107782  9831 maintenance_manager.cc:643] P 2121bfede85b447c9f51ef23233606cc: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 2.881s	user 0.123s	sys 0.056s Metrics: {"cfile_cache_miss":9,"cfile_cache_miss_bytes":26764,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"dirs.queue_time_us":76319,"dirs.run_cpu_time_us":508,"dirs.run_wall_time_us":1283,"lbm_read_time_us":36795,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":23,"lbm_write_time_us":43566,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":10,"peak_mem_usage":276198,"reinsert_count":0,"thread_start_us":233537,"threads_started":3,"update_count":214,"wal-append.queue_time_us":280206}
I20260709 12:27:20.099570 18870 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.110780  9832 maintenance_manager.cc:419] P 2121bfede85b447c9f51ef23233606cc: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 12:27:20.102363 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:20.101997 18905 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:20.113546  9949 raft_consensus.cc:2368] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24.
I20260709 12:27:20.099066 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.103339 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.111456 18870 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.099050 10674 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 17. (term mismatch)
I20260709 12:27:20.114993 10669 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:20.105064 18904 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:20.118301 10674 pending_rounds.cc:85] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 16
I20260709 12:27:20.120584 10674 pending_rounds.cc:107] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 12:27:20.120873 10674 raft_consensus.cc:2889] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:20.118884 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:20.119963 10099 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:20.118098 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:20.122634  9831 maintenance_manager.cc:643] P 2121bfede85b447c9f51ef23233606cc: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 12:27:20.092929 18548 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.145025 10669 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:20.096009 18909 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:20.118376 18582 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:20.165192  9944 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:20.165520  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:20.069972 18895 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.117995 18903 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
I20260709 12:27:20.200189 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.137354 18805 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:20.168478 10093 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:27:20.214988 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.224567 10669 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:20.200359 18893 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.228832 18817 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.228681 18830 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:20.229460 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 110651us
I20260709 12:27:20.229374 18817 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.158713 18909 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:20.230710 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:20.231668 18817 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.232327 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.233345 18853 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:20.234789 18656 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:20.235365 18656 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:20.236413 18656 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.228734 18893 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.223286 18805 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:20.239156  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 100663us
I20260709 12:27:20.239687 18805 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:20.240206  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:20.240520 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:20.241170 18682 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:20.243323 18830 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:20.243783 10677 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 12:27:20.244407 18892 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.245260 18814 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:20.246049 18814 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.239427 18715 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:20.246500 18814 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.239041 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:20.246853 10098 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:20.248759  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:20.246551 18892 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:20.248343 10094 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:20.250167  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:20.248776 18814 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.250857  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:20.249902 18893 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.251379  9945 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 23 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:27:20.250896 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:20.252346 10094 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:20.252491 10669 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:27:20.249668 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:20.202708 18695 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:20.255028 18893 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:20.253165 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.255296 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.255885 10094 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:20.229221 18755 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:20.256398 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.256664 18755 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:20.256764 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.257216 18814 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:20.247628 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.257253 18755 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:20.259831 10099 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:20.260479 18755 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.258630 10640 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:20.261130 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:20.262676 18250 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:20.263167 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.263264 18250 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.263366 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.263788 10670 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:20.263993 18250 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:20.264197 18870 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:20.167311 18905 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:20.265285 18905 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:20.262624 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.266185 18905 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:20.262861 18914 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.137s	user 0.000s	sys 0.001s
I20260709 12:27:20.270838 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:20.278570 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.279531 10095 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:20.229115 18895 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:20.163914 18582 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.282768 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:20.287492 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:20.285609 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.286743 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.166940 18581 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:20.288146 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:20.293489 10677 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:20.294312 10095 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:20.274075  9912 connection.cc:256] RPC call timeout handler was delayed by -3.0048420550001538s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
I20260709 12:27:20.296340 10093 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:20.298043 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:20.294983 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.159653 18548 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.316695 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.317250 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:20.317348  9912 connection.cc:256] RPC call timeout handler was delayed by -1.6605139700000109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 12:27:20.293525 18582 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:20.321542  9912 connection.cc:256] RPC call timeout handler was delayed by -1.6452139699999861s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15360000000000001s for a response
W20260709 12:27:20.325582  9912 connection.cc:256] RPC call timeout handler was delayed by -1.3491139699999621s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
W20260709 12:27:20.326049  9912 connection.cc:256] RPC call timeout handler was delayed by -1.044913969999925s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2203s for a response
I20260709 12:27:20.324556  9945 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:20.290947 18852 raft_consensus.cc:2764] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
I20260709 12:27:20.325799 18715 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.342135 10643 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:20.200165 18904 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.387346 10093 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:20.389516 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:20.301457 18760 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:20.390532 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48473.2us
I20260709 12:27:20.392127 18903 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.407629 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:20.402159 10670 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
W20260709 12:27:20.399485 18920 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:27:20.416523 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:20.414525 18903 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.420007 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.420620 10672 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:20.345077 18817 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:27:20.401683 18893 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:20.428714 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:20.428804  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94310.8us
I20260709 12:27:20.429715 10670 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:20.414805 18904 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:20.387610 18715 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.434528 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:20.430290 18814 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:20.436197 18891 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:20.446602 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:20.466723 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:20.466073 10092 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:20.467350 10643 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:20.533286 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 66016.8us
I20260709 12:27:20.533856 10096 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:20.534070 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:20.534755  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 95059.7us
I20260709 12:27:20.536307  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:20.541859 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:20.561523 10672 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
W20260709 12:27:20.542960 18888 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:27:20.566957 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.542666 18891 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:20.588848  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52649.6us
I20260709 12:27:20.589180 18760 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.589988  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:20.589896 18760 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:20.591709 18760 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:20.592401 18760 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:20.594658 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:20.562673 18755 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:20.599043 18755 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.599460 18755 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:20.602999 18919 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:20.603451 18919 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:20.604020 10098 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:20.604290 18919 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:20.635267 18921 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.201s	user 0.001s	sys 0.000s
I20260709 12:27:20.583626 18915 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:20.582314 18882 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:20.563035 18893 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:20.606941 18548 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:20.637622 18755 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:20.667352 18924 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 12:27:20.609567 18884 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:27:20.715902 18715 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.716668 18715 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:20.708897 18900 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.720929 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.743089 10097 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:20.744714 18900 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.724417 18909 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:20.726295 18862 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.668s	user 0.005s	sys 0.005s
I20260709 12:27:20.729523 18838 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:20.736297 18250 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.700913 18915 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.702201 18882 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.717290 18715 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.738878 18805 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.750324 10671 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:27:20.738147 18755 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:20.762310  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:20.777277 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:20.779163 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:20.753793 18900 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:20.819367 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:20.817986 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.797520 18904 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.798188 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:20.775267 18878 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:20.757440 18862 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.841801 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.798704 18656 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:20.816434 18915 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.779104 18548 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.780560 18894 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:20.818188 18909 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.797371 18896 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:27:20.844187 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46074.8us
I20260709 12:27:20.770987 18838 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.819913 18715 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.829075 18904 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:20.859774 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:20.847689 18895 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.550s	user 0.012s	sys 0.000s
I20260709 12:27:20.775925 18882 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.776271 18582 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:20.871618 18928 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.324s	user 0.007s	sys 0.000s
I20260709 12:27:20.874482 18581 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.875160 18581 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.877686 18581 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:20.879733 18930 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.238s	user 0.003s	sys 0.005s
I20260709 12:27:20.883030 18891 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.883646 18891 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.885931 18891 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.886974 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:20.887585 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.887705 10018 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:20.888236 10096 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:20.888407 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:20.888752 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:20.891026 10024 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:20.749630  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:20.851148 18656 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:20.907701  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 315854us
I20260709 12:27:20.850628 18870 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:20.853519 18894 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.909868 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:20.911585 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:20.911844  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:20.912199 10025 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:20.910490 10024 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:20.915908 18870 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.917857 18870 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:20.920418 10677 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:20.921690 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:20.922700  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:20.847581 18878 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.935356 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:20.939374  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:20.942942 10669 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: [24.19-24.19]
W20260709 12:27:20.944334 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 381369us
I20260709 12:27:20.916392 18656 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:20.916146 18894 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:20.861465 18548 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.921520 18682 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:20.946250 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:20.949263 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:20.953401 18747 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:20.954115 18882 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:20.959667 18838 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:20.963505 18548 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:20.964150 18548 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:20.964603 18548 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:20.965516 18548 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.944896 18715 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:20.967703 18582 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.953955 18747 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:20.968583 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1854) took 1138 ms. Trace:
I20260709 12:27:20.969053 10671 rpcz_store.cc:276] 0709 12:27:19.830302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.830451 (+   149us) service_pool.cc:224] Handling call
0709 12:27:20.968320 (+1137869us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b5400408dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:20.968510 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2080640384}
I20260709 12:27:20.969622 18747 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:20.970731 10675 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 15. (term mismatch)
I20260709 12:27:20.971138 10675 pending_rounds.cc:85] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 14
I20260709 12:27:20.971377 10675 pending_rounds.cc:107] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 23.15
I20260709 12:27:20.971616 10675 raft_consensus.cc:2889] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:20.972836 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:20.978570 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:20.947636 18763 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:20.945150 18909 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:20.990619 18250 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:20.990813 18909 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:20.991307 18250 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:20.991672 18250 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:20.992004 18250 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:20.993131 18909 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.995136 18805 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:20.997371 18671 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:20.998306 18671 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:20.998667 18671 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:20.999446 18862 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.001048 18904 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.002177 18682 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:21.002856 10096 pending_rounds.cc:85] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 14
I20260709 12:27:21.003150 10096 pending_rounds.cc:107] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:27:21.003422 10096 raft_consensus.cc:2889] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:21.004997  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:20.987015  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:21.007383 10669 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:27:21.007980 18902 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 1.208s	user 0.000s	sys 0.013s
I20260709 12:27:21.010954 18935 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:21.011158  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.011577 18935 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:21.012457 18935 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.013160 18909 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:21.013751 18909 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:21.010850 18763 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:21.015168 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:21.025556 10669 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:21.031528  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.044413 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:21.046846  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:21.049453  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:21.049235 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 194183us
I20260709 12:27:20.999444 18671 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.014734 18763 consensus_queue.cc:1243] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:27:21.054318  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.056360  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:21.049530  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 270945us
I20260709 12:27:21.056020  9945 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:21.053470 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.068868  9944 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:21.071754 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 1541) took 1150 ms. Trace:
I20260709 12:27:21.062345 18896 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:21.075941  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.077726  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.072278 10676 rpcz_store.cc:276] 0709 12:27:19.921564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.921852 (+   288us) service_pool.cc:224] Handling call
0709 12:27:21.071519 (+1149667us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b54005b95d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:21.071685 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336728960}
W20260709 12:27:21.079982 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 127539us
I20260709 12:27:21.082826  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.033126 18805 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:21.086246  9943 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:21.025301 18900 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:27:21.076959 18896 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.073660 18934 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:21.098253  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:21.085469 18882 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.064481 18755 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:27:21.087265 18582 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.098354 10643 leader_election.cc:400] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24.
I20260709 12:27:21.099023  9943 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:27:21.116528 10643 leader_election.cc:403] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:27:21.092077 18939 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:21.117293  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 106200us
W20260709 12:27:21.129892  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 102352us
I20260709 12:27:21.125830  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.079401 18936 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.052s
W20260709 12:27:21.159736  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.177243  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:21.199935  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74304.5us
I20260709 12:27:21.091712 18805 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.099018 18934 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:21.316337  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.322621 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1545) took 1399 ms. Trace:
I20260709 12:27:21.296175  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3304) took 1337 ms. Trace:
W20260709 12:27:21.331823  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 266811us
I20260709 12:27:21.358247  9943 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:21.361315  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:21.326870  9942 rpcz_store.cc:276] 0709 12:27:19.958403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.960855 (+  2452us) service_pool.cc:224] Handling call
0709 12:27:19.970948 (+ 10093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.775145 (+804197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:20.775193 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:20.775264 (+    71us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:20.775316 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.295842 (+520526us) spinlock_profiling.cc:243] Waited 804 ms on lock 0x7b54000163d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:21.295910 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:21.296094 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1848871296}
I20260709 12:27:21.334301  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:21.365641  9945 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:21.323838 10674 rpcz_store.cc:276] 0709 12:27:19.922933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.923277 (+   344us) service_pool.cc:224] Handling call
0709 12:27:19.923604 (+   327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.126276 (+202672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:21.322255 (+1195979us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b540058f2d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:21.322523 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":402790528}
I20260709 12:27:21.302867 18915 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:21.143206 18944 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.095217 18909 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:21.381326 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 286969us
I20260709 12:27:21.121341 18582 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.118724 18882 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:21.386919 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 302727us
I20260709 12:27:21.332602 18934 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:21.399724 18904 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.400381 18904 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.130393 18939 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.402562 18904 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.404296 18939 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.404866 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.404963 18941 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.405656 18941 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.408228 18941 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.409888 18941 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:21.410362 18941 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.410796 18941 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:21.426855 18938 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:21.431140  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.431934  9943 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:21.382464 18915 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:21.397929 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 277362us
I20260709 12:27:21.378862 18672 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:21.384037 18894 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.452380 18882 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.456589 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.479207  9945 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:21.480751  9943 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:21.433744 18900 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:21.493517 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:21.496037 10092 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:27:21.505729 10091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:21.506534 10091 raft_consensus.cc:606] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:21.507002 10091 raft_consensus.cc:614] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:21.457610 18915 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:21.507252  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 174087us
W20260709 12:27:21.485327 18656 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.002s
I20260709 12:27:21.475111 18949 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.561748 10090 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:21.500626 18940 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.380s	user 0.000s	sys 0.001s
I20260709 12:27:21.498450 18882 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.500277 18938 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:21.391242 18944 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.437549 18937 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.103s
I20260709 12:27:21.479449 18898 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.088s
W20260709 12:27:21.574348 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81132.9us
I20260709 12:27:21.569418 18682 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:21.589031  9987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.007s	sys 0.002s
W20260709 12:27:21.594282  9987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.007s	sys 0.002s
I20260709 12:27:21.597079 18969 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.597675 18969 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.604110 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:21.602941 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 504732us
I20260709 12:27:21.614514 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:21.503036 18934 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.575519 18938 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.574043 18956 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.564044 18656 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.568497 18949 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:21.635793 18940 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:27:21.574728 18882 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:21.658123 18951 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:27:21.638350 18938 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:21.772995 10677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1844) took 1485 ms (client timeout 1904 ms). Trace:
W20260709 12:27:21.773458 10677 rpcz_store.cc:269] 0709 12:27:20.288047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.317020 (+ 28973us) service_pool.cc:224] Handling call
0709 12:27:21.772800 (+1455780us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b540073afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:21.772935 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374794496}
I20260709 12:27:21.635855 18962 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.774976 18962 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:21.741230  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.777626 18371 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.779124 10669 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 12:27:21.781899  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.538s (SENT) [suppressed 4 similar messages]
I20260709 12:27:21.622076 18682 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:21.738510  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.001s	sys 0.014s
I20260709 12:27:21.785773 18935 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:21.787415 18967 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:21.793478 18967 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.793275 18935 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.793869 18967 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:21.795943 18935 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.796828 18682 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.797343 18962 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.785560 18805 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.800637 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:21.789908 18965 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:21.790503 18656 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:21.788059  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.001s	sys 0.017s
I20260709 12:27:21.790679  9949 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:21.792511 18941 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.804780 18988 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.809108 18988 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.810372 18941 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:21.811470 18944 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.658541 18956 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.806700 10090 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
W20260709 12:27:21.788087 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 173798us
W20260709 12:27:21.813565 18250 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.821s	user 0.004s	sys 0.004s
W20260709 12:27:21.813652  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 752703us
I20260709 12:27:21.813895 18582 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.814440  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:21.815022 18894 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:21.815392 18870 consensus_meta.cc:287] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.897s	user 0.008s	sys 0.002s
I20260709 12:27:21.815644 18894 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.815791 18870 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.816272 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:21.816118 18894 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.817550 18934 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.817833 18870 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:21.811493 18882 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.819169 18894 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:21.819805 18954 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
W20260709 12:27:21.821619 18971 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:27:21.821532 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:21.813475 18956 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:21.805667 18959 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:21.822980 18898 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.823246 18959 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.823534 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:21.823503 18898 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.825500 18959 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.825564 18898 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.826274 18898 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:21.826777 18988 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.827603 18938 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.819773 18894 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:21.831401 10018 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:21.833252 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:21.861371 10021 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:27:21.832885 10019 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:21.831780 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:21.832085 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:21.863018 10025 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:21.863754 10024 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:21.813506 10643 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.814162 18250 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:21.869874 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56450.5us
W20260709 12:27:21.872685  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:21.876296 18969 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:21.835950 18982 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.877322 18982 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.877928 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.830457 18935 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:21.879109 18960 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.279s	user 0.002s	sys 0.000s
I20260709 12:27:21.878926 18935 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.879931 18982 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.880514 18944 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:21.881059 18944 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:21.881088 18935 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:21.882156 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:21.846114 18760 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.882071 18944 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.832955 18936 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.882906 18760 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:21.883430 18936 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.885747 18936 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:21.885638 18955 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:21.887681 18973 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:21.888091 18973 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:21.888751  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:21.889176  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:21.888976 18973 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.889350  9945 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:21.889853  9942 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:21.890239 18973 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:21.891139  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:21.892828 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:21.842558 18903 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:21.893360  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:21.895641 10096 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:21.856487 18898 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:21.898988  9945 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:21.899049  9946 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:21.802376 18952 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:27:21.903782 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 320405us
I20260709 12:27:21.896445 18967 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:21.900111  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 118241us
W20260709 12:27:21.907639  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 311278us
I20260709 12:27:21.908191 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:21.873678 18884 raft_consensus.cc:2804] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:21.869801 18910 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:21.909791 10676 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22. Ops: [21.18-21.18]
I20260709 12:27:21.909981 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:21.935715 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:21.935731 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:21.922844 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:21.933079  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.914127 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:21.941552 10092 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:21.941977  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.942327 10019 raft_consensus.cc:2368] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25.
I20260709 12:27:21.942158 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:21.943313 10096 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:21.943781  9940 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:21.903563 18898 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:21.934954 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.945133 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:21.949255 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:21.949972 10021 raft_consensus.cc:2368] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25.
I20260709 12:27:21.951924 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:21.921200 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:21.871156 18894 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:21.903792 18952 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:21.954314 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:21.961735 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:21.948040 10677 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:21.911324 18936 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.912352 18656 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:21.977880 10668 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:21.979367 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:21.934983 18935 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:21.980435  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:21.996819  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:21.997880  9945 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:21.998581 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:21.978662 18817 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.325s	user 0.016s	sys 0.028s
I20260709 12:27:21.946381 18982 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.053s
I20260709 12:27:22.000552 10677 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:21.955329 18865 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:22.002147  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:22.002568  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 188196us
I20260709 12:27:22.004180  9946 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:22.005076  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.005331 10673 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:27:22.006062 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:21.954257 18884 raft_consensus.cc:697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:22.007687  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:22.008900  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.008721 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.009711 10677 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:22.006676 10019 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:21.980435 18938 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.013293 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:22.002076 18817 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.357s	user 0.017s	sys 0.028s
I20260709 12:27:22.020881 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.015889 10668 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:22.017633 10096 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:22.024479  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.025485 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.027120  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:22.000389 18956 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.027884 10091 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:22.028353  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.007867 18948 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:22.026424 10092 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:22.036823 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.038954 10023 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:22.035605 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.036737 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.040119 10668 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:22.042754 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.042943 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.043565 10668 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:22.045167 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.045926 10019 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:22.046073  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.046989  9946 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:22.049218 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:21.961611 18910 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.044941 18948 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.068217 18978 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:22.069896  9947 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:22.070060  9949 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:27:22.070290 18760 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.000618 18656 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.045480 18965 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:22.050065 18805 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:22.071830 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.052004 18989 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.071883 18968 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:22.109556 18968 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:22.110052 18915 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.000s
I20260709 12:27:22.110788 18884 consensus_queue.cc:237] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.112094 18968 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.080714 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.113157 10023 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:22.088488 10668 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:22.075436 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:22.115360 18967 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.000452 18936 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.095326 18987 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.095791 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:22.119485 10677 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:22.119586 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 70478.7us
I20260709 12:27:22.071704 18894 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.081883 18944 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:22.074873 18952 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.105455 18250 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:22.106179  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.101401 10092 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:22.108418 18989 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:22.116564  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 111570us
I20260709 12:27:22.127592  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:22.120028 10023 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 27 index: 13. (index mismatch)
I20260709 12:27:22.016714 18935 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.121675 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.091249 18948 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.132879 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.140123 10023 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:22.140309 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.137881  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.051456 18956 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.137099 10668 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:22.144731 10092 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:22.119051 18967 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.143244 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.147521  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.147543  9942 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:22.122774 18944 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.168614 18944 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.145807 18837 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:22.153676 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1654) took 1260 ms. Trace:
I20260709 12:27:22.118907 18915 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.146060 18978 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:22.186065  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:22.163774 18656 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:22.124816 18987 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.188800  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.187000 10017 rpcz_store.cc:276] 0709 12:27:20.893098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:20.893231 (+   133us) service_pool.cc:224] Handling call
0709 12:27:20.893601 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.908021 (+ 14420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:20.908053 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:20.908097 (+    44us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:20.908144 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.153423 (+1245279us) spinlock_profiling.cc:243] Waited 14.2 ms on lock 0x7b5400265cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:22.153476 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.153616 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32704768}
I20260709 12:27:22.178025 18794 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:22.197717 18894 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:22.198151 18894 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.198500 18894 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:22.194509 18900 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.692s	user 0.010s	sys 0.000s
I20260709 12:27:22.202929 18989 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.202929 18900 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24.
I20260709 12:27:22.204908 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.205034 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.205777 18987 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.206224 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.206674 10672 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:22.206576 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.208310 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.209317 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.209970 10017 raft_consensus.cc:2376] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 29. Re-sending same reply.
I20260709 12:27:22.151062 18967 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.211494 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:22.212073 10092 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.212724 18967 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.179034 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.215509 18935 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.175381 18952 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.217994 18956 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.218128 10677 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:22.219010 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.223179 10018 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:22.220290 10092 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:22.220427 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.227319 10021 raft_consensus.cc:2410] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 22 because replica has last-logged OpId of term: 21 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:27:22.226943 18898 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.222388 10019 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:22.228112 10025 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:22.229171 18936 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.194775 18910 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.230794 18672 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:22.235131  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46398us
I20260709 12:27:22.235441 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.140570 18949 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.238641 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.239498 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.243316 10677 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:22.244558  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.210434 18915 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.170470 18878 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.250761 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:22.261642 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.264437 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.266799 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:22.267594 10092 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:22.244069 18672 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:22.259363 18949 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:22.271181 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.273299 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.273880 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.275071 10018 raft_consensus.cc:3060] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:22.244089 18978 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:22.245560 18968 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 12:27:22.274129 10021 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:22.266136 18987 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:22.278200 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:22.301165 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.282439 18814 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.282717 18948 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.272703 18949 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.290795 18919 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.292457 18671 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:22.294651 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.294853 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:22.313324 10096 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:22.294967 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.295892 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.296276 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.296999 10018 raft_consensus.cc:2468] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:22.314383 10093 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:22.278895 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.287374 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1551) took 1196 ms. Trace:
I20260709 12:27:22.315681 10676 rpcz_store.cc:276] 0709 12:27:21.091420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.093876 (+  2456us) service_pool.cc:224] Handling call
0709 12:27:21.094209 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:21.937198 (+842989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.287088 (+349890us) spinlock_profiling.cc:243] Waited 812 ms on lock 0x7b54006d01d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:22.287308 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1867806592}
I20260709 12:27:22.308233 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.310892 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.266326 18952 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:27:22.276495 18944 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.318198 10017 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:22.337596 18968 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:22.338018 18968 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.338922 18968 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.340003 18968 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:22.337622  9948 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 25 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:27:22.343139 18915 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.343585 18915 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.299649 10025 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:22.319494 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.322814 18948 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.303591 18814 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.346753 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.347528 10018 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:22.348296 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.349318 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.349984 10018 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:22.311985 18755 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.350764 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:22.350880 18755 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.352319 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.352694 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:22.307936 18949 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:22.353482 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.353830 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.354532 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.355818 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.356357 10016 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:22.356892  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.309563  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:22.326012 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.329990 18935 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:22.359052 10096 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:27:22.360291  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.361045  9942 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:22.361536 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.366832 18898 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.363255  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.352898 10021 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.344337  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.364205 18715 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.370891 18715 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.373509  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.373742 18715 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.365803 18935 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:27:22.374917  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.375501 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.376080 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.376679 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:22.364104  9942 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:22.365803 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1544) took 2443 ms. Trace:
I20260709 12:27:22.377331 10675 rpcz_store.cc:276] 0709 12:27:19.922647 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:19.922865 (+   218us) service_pool.cc:224] Handling call
0709 12:27:20.970377 (+1047512us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:20.984086 (+ 13709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.365593 (+1381507us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400408dd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:22.365723 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2408362368}
I20260709 12:27:22.353107 18755 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.364563 18814 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.379002 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:22.382278 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:22.378750  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.387142 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.387775 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:22.390437 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.391057 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:22.391760 18878 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:22.386443 18934 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:22.394928 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.395532 10675 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:22.396879 10021 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:22.379815 18894 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.385509  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.384972 18891 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:22.408828 10016 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
W20260709 12:27:22.408058  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 98605.9us
I20260709 12:27:22.407861  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.385474 18837 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.386349 18581 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:22.414489 10668 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:22.386571 18884 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:22.419377 10096 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:22.420014 10096 raft_consensus.cc:606] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:22.420408 10096 raft_consensus.cc:614] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:22.416672 18814 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:22.390086 18755 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:22.423061  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:22.390203 18937 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:22.421617  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.433741 18946 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:22.434865 18946 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:22.436597 18894 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.427000 18688 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.012s
I20260709 12:27:22.438115 18688 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:22.438467 18688 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.409690 18915 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.439399 18688 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.444687 18672 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.427255 18671 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:22.447589 18671 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.448000 18671 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:22.419026 18898 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:22.425441 18884 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.451464  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2999) took 1399 ms. Trace:
I20260709 12:27:22.451856 18884 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.451943  9948 rpcz_store.cc:276] 0709 12:27:21.051880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.053897 (+  2017us) service_pool.cc:224] Handling call
0709 12:27:22.451240 (+1397343us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540018a6d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:22.451383 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2944231168}
I20260709 12:27:22.424790 18837 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.453373  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.429546 18937 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.455152 18937 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:22.455420 18819 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:22.456897  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.459527 18943 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:22.461123 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.461777 10675 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:22.461853  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:22.464601  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.465085 18985 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:22.465255  9946 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:22.465816 18985 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:22.451498 18898 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.467501 18898 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.428220 18755 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.469050 18755 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:22.471378  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:22.472587  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.473173 18949 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.473662  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.475005 18944 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.475425 18936 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:22.435143 18860 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:22.475881 18936 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.430374  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.476342 18936 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:22.476862  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46535.6us
I20260709 12:27:22.477730 10016 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:22.478358 18915 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:22.478770 18915 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.479146 18915 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:22.476143 18860 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.481009 18860 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.482201 18860 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:22.476650 18688 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:22.454530  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:22.455168 18837 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.482998 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.483103 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.511865 10021 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:22.482537  9942 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:22.488365  9948 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:22.490944  9945 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:22.498381 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.498499  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:22.517480  9842 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 14 to 19, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 19 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:22.518730 10025 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:22.522534 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4316) took 1045 ms. Trace:
I20260709 12:27:22.525799 10019 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:22.496505  9941 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:22.522972 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.528668 10022 raft_consensus.cc:2368] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28.
I20260709 12:27:22.519536  9842 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a reported cstate change: term changed from 13 to 17, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 17 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:22.529642 18817 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.536554 10016 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
W20260709 12:27:22.537382  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 129608us
I20260709 12:27:22.534147  9842 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 17 to 18, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 18 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:27:22.485909 18998 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.323s	user 0.002s	sys 0.000s
I20260709 12:27:22.535162 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.526072 10098 rpcz_store.cc:276] 0709 12:27:21.477218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.477837 (+   619us) service_pool.cc:224] Handling call
0709 12:27:21.478252 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:21.478534 (+   282us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:21.538427 (+ 59893us) log.cc:844] Serialized 14 byte log entry
0709 12:27:22.011485 (+473058us) log.cc:844] Serialized 14 byte log entry
0709 12:27:22.171320 (+159835us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b18001b9e80. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:22.521998 (+350678us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b18001b9e80. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:27:22.522164 (+   166us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:22.522234 (+    70us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:22.522287 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.522354 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:22.522475 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15123,"prepare.queue_time_us":24412,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":4063,"raft.queue_time_us":522530,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":7379,"spinlock_wait_cycles":10049152,"thread_start_us":464427,"threads_started":1,"wal-append.queue_time_us":562266}
I20260709 12:27:22.529871 18763 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.541447 18763 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.541215  9842 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 14 to 16, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 16 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:22.543169  9842 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a reported cstate change: term changed from 17 to 19, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 19 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:22.545456 18763 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.545220  9842 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:22.546720 18941 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.547358 18941 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.549618 18941 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.550230 18837 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:22.538667 18548 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.552006  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.511884 18715 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:22.554735 10018 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:22.574877 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.563082 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.576602 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.575538 10017 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:22.577373 10096 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:22.584661 10096 raft_consensus.cc:606] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:22.585162 10096 raft_consensus.cc:614] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:22.587631 18581 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.588699 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.591174 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:22.585642 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.564195 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.582448 10025 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:22.582989  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.577946 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.554167 18763 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:22.596465 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.597379 10675 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:22.598484 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.574568 18935 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:22.535463 18817 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.605875 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.608919 10017 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:22.522730 18865 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:22.607757 18763 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.626468 18755 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.607072 18905 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.627579 18763 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.575618 18982 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 12:27:22.627868 18905 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.628587 18755 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.629120 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.629722 18949 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.629913 18905 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.605129 18935 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.631137 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.631853 10025 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:22.632560 18970 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.633194 18672 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.633069 18970 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.630580 18935 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:22.636054 18970 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:22.637036 18817 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:22.637555 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 102478us
I20260709 12:27:22.638247 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.639477 18973 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.640277 18973 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:22.640681  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 2995) took 1595 ms (client timeout 1624 ms). Trace:
W20260709 12:27:22.641074  9941 rpcz_store.cc:269] 0709 12:27:21.045611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.046527 (+   916us) service_pool.cc:224] Handling call
0709 12:27:22.640499 (+1593972us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540003e154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:22.640617 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3260351360}
I20260709 12:27:22.641499 18941 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:22.642031 18941 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:22.642635 18975 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:22.642879 18941 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.644548 18975 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:22.644918 18975 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:22.645640 10024 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:22.647579  9943 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 15. (term mismatch)
I20260709 12:27:22.643028  9944 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:27:22.647966  9943 pending_rounds.cc:85] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 14
I20260709 12:27:22.650571  9943 pending_rounds.cc:107] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 21.15
I20260709 12:27:22.651010  9943 raft_consensus.cc:2889] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:22.651105  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:22.655421 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.656956 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:22.657281  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74402.4us
I20260709 12:27:22.659152 10676 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:22.645735 18975 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.577677 18548 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.625000  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:22.663158  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46287.3us
I20260709 12:27:22.553553 18956 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.074s
I20260709 12:27:22.658277  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:22.670821 10675 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:22.670821 10677 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:22.674410 18548 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:22.665961 18857 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.675366 10672 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:22.676393 18978 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:22.676810 18978 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.677062  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3003) took 1596 ms. Trace:
I20260709 12:27:22.675410 18865 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.677470  9944 rpcz_store.cc:276] 0709 12:27:21.080737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:21.081551 (+   814us) service_pool.cc:224] Handling call
0709 12:27:22.642699 (+1561148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.648208 (+  5509us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.676856 (+ 28648us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b540003e150. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:22.677005 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3589634432}
I20260709 12:27:22.679172 18937 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.681557 18937 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:22.684417 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 70088.1us
I20260709 12:27:22.614297 18878 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:22.673897 18884 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.711531 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:27:22.710471  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52291.7us
I20260709 12:27:22.712981  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.713431 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:27:22.723438  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 133487us
I20260709 12:27:22.710983  9912 leader_election.cc:400] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25.
I20260709 12:27:22.731600  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:22.733078  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:22.676069 18949 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:22.662530 18975 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:22.744971 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.745932 10019 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:22.746071 18558 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:22.747008 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.747918 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.737725  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:22.739512  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.740231 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.734824 18548 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.006s
I20260709 12:27:22.740984 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.741636 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.764346 10024 raft_consensus.cc:2368] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25.
I20260709 12:27:22.742800 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.750866 10668 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:22.766568 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.767477 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.768618 10019 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:22.774430 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.776119  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:22.781445  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:22.782675 18939 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:22.738047 18755 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:22.785094 18755 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:22.785480 18755 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:22.785851 18755 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:22.783305 18937 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:22.738302 18955 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:22.787322 18955 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:22.786222 18939 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:22.787747 18955 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.730468 18982 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:27:22.788841 18939 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:22.730482 18672 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.076s
I20260709 12:27:22.677672 18978 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.764521 18905 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 12:27:22.789952 18894 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:22.791245 18905 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:22.791659 18905 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:22.791549 18558 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:22.792631 18905 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.793129 18558 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:22.793691 18558 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.793908 18905 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.794718 10016 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:27:22.795317 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.800665 18935 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.742601 18747 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:22.749429 18848 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.054s
I20260709 12:27:22.803004 18747 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.803407 18747 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:22.742774 18958 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:27:22.804924 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:22.806945 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.789362 18939 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.808820 18939 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.809355 18915 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.810135 18939 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.812523 18671 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.815572  9842 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 15 to 19, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 19 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:22.774067 18852 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:27:22.735436 18857 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:22.822139 18857 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.822530 18857 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:22.823145  9944 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:22.824201  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:22.821655  9945 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:22.738618 18949 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:22.826886 18949 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:22.818213  9842 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a reported cstate change: term changed from 11 to 20, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 20 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:22.835042 18937 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:22.836333 18992 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:22.836728 18992 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.837663 18992 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.839130 10023 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:27:22.839232 18992 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:22.842151 10091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:22.842885 10091 raft_consensus.cc:606] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:22.844700 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.845920  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:22.846560  9942 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:22.853087 18968 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:22.853477 10091 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:22.854568 18978 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:22.854408 18968 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:22.855449 18909 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:22.855628 18819 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:22.855923 18909 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:22.856050 18819 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.856343 18909 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:22.857035 18819 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.857542 18755 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:22.746127 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:22.863286  9943 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:22.880090 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:22.880914 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.886672 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:22.898852 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:22.900679 10670 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:22.889932  9942 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:22.903137 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:22.905268  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 129226us
I20260709 12:27:22.905970  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:22.907780 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:22.858623 18819 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:22.902313  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:22.855491 18968 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:22.797600 18936 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:22.803395 18935 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:22.916307 10670 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:22.898425 18852 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:22.924865 10062 leader_election.cc:400] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28.
I20260709 12:27:22.925305 10062 leader_election.cc:403] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:22.927922 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:22.929710 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:22.932452 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:22.936362 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 286031us
I20260709 12:27:22.910163 18986 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:22.937305 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:22.937762 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:22.939388 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:22.909250 18715 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:22.912216 18855 raft_consensus.cc:2764] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:27:22.974982 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:22.905197 18817 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:27:22.979043 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:22.943406 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:22.983614 10094 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:22.992676 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55036.3us
I20260709 12:27:23.002874 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:23.004119 10090 raft_consensus.cc:2410] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:27:22.918561 18968 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:22.904289 18973 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:23.008003  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:22.794760 18558 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.008104 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:23.012311 10096 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "db738483c1444059b752648d302dc0d9"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:22.942173 18904 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:22.975522 18878 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:22.926182 18848 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.087s
I20260709 12:27:23.014374 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:22.975291 18581 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:22.936092 18852 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.010661 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:22.927217 18898 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:23.081415  9946 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:22.979873 18910 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:23.062382  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.061877  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.183579 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 173046us
I20260709 12:27:23.186693 10022 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
W20260709 12:27:23.205715 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 191558us
W20260709 12:27:23.211380  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.007316 18938 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:23.049942 18958 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.059341 18915 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:23.047539 18942 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.348s
I20260709 12:27:23.284444 18942 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:23.274816 19025 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:23.284870 18942 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:23.285769 18942 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.288774 10093 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
W20260709 12:27:23.048630 19000 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:27:23.018718 18956 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:23.023406 18671 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.182844 18878 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.056598 18936 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.180312 18967 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:22.921216 18955 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:23.276526 18948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.002s	sys 0.031s
I20260709 12:27:23.060861 18884 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:23.066323 18755 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:23.210767 18910 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:23.330475 18910 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:23.330935 18852 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:23.333323 18939 consensus_meta.cc:287] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.000s
I20260709 12:27:23.311721 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:23.316058 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.321341 10099 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:23.322642  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:23.337066  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:23.339365 18986 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:23.340003 18910 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.341070 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:23.342485 18910 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.343557 18805 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.340096 18986 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.016477 18656 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.289s
W20260709 12:27:23.318780 18948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.003s	sys 0.031s
I20260709 12:27:23.306990 18992 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:27:23.351128 18986 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:23.351565 18986 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:23.352483 18986 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.060256 18949 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.178877 18909 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.325757 10021 raft_consensus.cc:1275] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:23.316164 18942 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:23.358397  9912 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.030s	sys 0.114s
W20260709 12:27:23.431399  9912 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.030s	sys 0.114s
W20260709 12:27:23.432008  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 694356us
I20260709 12:27:23.432708  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:23.434082  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:23.313719 18878 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:23.188400 18968 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.210978 18973 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.301062 18857 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.444916  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.021195 18715 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.319089 18956 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:23.461632  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.323577 18967 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:23.454757  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.008s	sys 0.005s
W20260709 12:27:23.464892  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.708s	user 0.009s	sys 0.006s
W20260709 12:27:23.456758  9949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.004s	sys 0.009s
I20260709 12:27:23.455655 18910 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:23.457999 18817 consensus_queue.cc:1048] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:23.430718 18992 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.430405 18936 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:23.452280 18819 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.528s	user 0.010s	sys 0.000s
I20260709 12:27:23.459460 18915 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:23.460448 18955 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:23.460600 19024 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.154s	user 0.003s	sys 0.000s
I20260709 12:27:23.456425 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:23.464039  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:23.446327 18671 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:23.465248  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 708696us
I20260709 12:27:23.450965 18948 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:27:23.465252 18865 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.032s	sys 0.061s
I20260709 12:27:23.450136 18656 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:23.468657 18910 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:23.466675  9949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.005s	sys 0.009s
I20260709 12:27:23.485136  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.484172 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:23.490657  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 1945) took 1117 ms. Trace:
W20260709 12:27:23.485857 18865 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.802s	user 0.032s	sys 0.061s
I20260709 12:27:23.482715 18956 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 19.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.493940 10095 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:27:23.494699 18910 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.495709 19021 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:23.496498 18967 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.488018 18852 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:23.488162 19022 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:23.491909  9949 rpcz_store.cc:276] 0709 12:27:22.373403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.374418 (+  1015us) service_pool.cc:224] Handling call
0709 12:27:22.374754 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:22.374946 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:22.845205 (+470259us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b18004b3c40. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf
0709 12:27:22.845353 (+   148us) log.cc:844] Serialized 14 byte log entry
0709 12:27:23.490297 (+644944us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:23.490349 (+    52us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:23.490397 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.490461 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.490581 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":180462,"prepare.run_cpu_time_us":180,"prepare.run_wall_time_us":6061,"raft.queue_time_us":87,"spinlock_wait_cycles":74982784,"thread_start_us":814506,"threads_started":2}
I20260709 12:27:23.499507 19022 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:23.500471 19022 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.503098 18955 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:23.503235 18852 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:23.503573 18955 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.504040 18955 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:23.504271 18852 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.509117 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.510248 10099 raft_consensus.cc:2410] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 21 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 17.
I20260709 12:27:23.510725 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.511039 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:23.511833 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:23.511986 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:23.512904 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.513514 10090 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.513638 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.509181 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:23.514703 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:23.515347 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:23.516748 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.517023 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:23.517194 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:23.517608 10099 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 18. (term mismatch)
I20260709 12:27:23.517437 10020 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:23.517828 10018 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 30 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:27:23.518224 10099 pending_rounds.cc:85] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 17
I20260709 12:27:23.496985  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:23.520818 10023 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:23.521399 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.522195 10099 pending_rounds.cc:107] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 27.18
I20260709 12:27:23.478826 18671 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:23.483472 18948 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.452013 18755 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:27:23.521668  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.536666 18671 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:23.537616 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.534471 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:23.532346 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.531670 10099 raft_consensus.cc:2889] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:23.550541 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.538787 18948 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:23.468865 18936 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.525900 19021 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:23.549319  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.466010 19020 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.561841 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:23.568126 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.569895  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:23.468752 18992 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.571188 18884 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:23.572381 18904 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.545s	user 0.009s	sys 0.001s
I20260709 12:27:23.572893 18904 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28.
I20260709 12:27:23.573927 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:23.574692 10671 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:27:23.579731 18909 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:23.581509 18857 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.582866 19030 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:23.583371 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.584226  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:23.564955 18805 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:23.584425 19020 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:23.585651  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.585952 19020 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:23.586774 18915 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:23.587278 18915 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:23.586851 19020 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.587641 18915 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:23.589732 18755 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:23.538831 18910 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:23.567446 19021 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:23.591092 19021 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:23.556129 18763 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:23.493013 18656 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.594748 18656 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:23.602888 19034 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
W20260709 12:27:23.604678 19032 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:27:23.610569 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:23.611336 10669 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:27:23.610116  9948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.690s	user 0.019s	sys 0.028s
W20260709 12:27:23.612250  9948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.019s	sys 0.028s
I20260709 12:27:23.618307  9912 leader_election.cc:400] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25.
I20260709 12:27:23.620147  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:23.626231 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 58179.6us
I20260709 12:27:23.627760 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:23.531637  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51152.6us
I20260709 12:27:23.643388  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:23.651700  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.610447 18747 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.807s	user 0.014s	sys 0.000s
I20260709 12:27:23.606945 18865 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:23.590385 18910 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:23.664698  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44644us
I20260709 12:27:23.624861 18992 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.629129 18817 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:23.654974 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.653832 18656 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.660495 18904 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:23.546192 10090 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:23.704636 19037 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:23.705150 19037 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:23.705534 19037 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:23.705682 18936 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:23.700839 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45986.9us
I20260709 12:27:23.667145 19030 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:23.741641 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.666323  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.702073 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4328) took 1290 ms. Trace:
I20260709 12:27:23.702937  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:23.743294  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:23.746838  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44040.2us
I20260709 12:27:23.747611 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.748728 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.706954 18865 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:23.663187 18747 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.708063 18949 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.742621 10097 rpcz_store.cc:276] 0709 12:27:22.386131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.386253 (+   122us) service_pool.cc:224] Handling call
0709 12:27:22.408368 (+ 22115us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:22.637694 (+229326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:22.637727 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:22.638176 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 12:27:22.907671 (+269495us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:22.907715 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:22.907782 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.460025 (+552243us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:23.662819 (+202794us) raft_consensus.cc:1685] finished
0709 12:27:23.662844 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.676289 (+ 13445us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":228880,"raft.queue_time_us":232544,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"spinlock_wait_cycles":94804736,"thread_start_us":260849,"threads_started":1,"wal-append.queue_time_us":492074}
W20260709 12:27:23.710511 18581 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.002s
W20260709 12:27:23.757153  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 90900.4us
I20260709 12:27:23.707408 18898 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:23.771190 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.753736 18967 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.756172 19020 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:23.754639 18884 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:27:23.771773 10016 raft_consensus.cc:2376] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 29. Re-sending same reply.
I20260709 12:27:23.703253 18817 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.784300 18817 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:23.760339 10062 leader_election.cc:400] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:23.786491 10062 leader_election.cc:403] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:27:23.787482 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:23.788683 10096 raft_consensus.cc:606] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:23.788852 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:23.755278 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.791040 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.792706 18878 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.791217 18915 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.798286 18747 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.761166 18805 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:23.799998 18955 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.800812 18898 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:23.802065 18955 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:23.802810 18975 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:23.803550 18975 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:23.803702 19033 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.334s	user 0.001s	sys 0.000s
I20260709 12:27:23.805126 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1868) took 1140 ms. Trace:
I20260709 12:27:23.806639 18865 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:23.789986 19021 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.772518 18581 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:23.810134 19021 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.811122 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.811755 10097 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:23.815732 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.815237 18915 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:23.813505 18975 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:23.817885 18910 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:23.775681 19036 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:27:23.819319 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.820022 10016 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:23.789922 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.805485 10676 rpcz_store.cc:276] 0709 12:27:22.664749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.664905 (+   156us) service_pool.cc:224] Handling call
0709 12:27:22.665207 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:22.665562 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:22.665592 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:22.837925 (+172333us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b2c000d07e8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:22.838316 (+   391us) log.cc:844] Serialized 26 byte log entry
0709 12:27:23.183295 (+344979us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:23.183337 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:23.183419 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:23.803093 (+619674us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:23.804882 (+  1789us) raft_consensus.cc:1685] finished
0709 12:27:23.804906 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:23.805052 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5877,"raft.queue_time_us":89,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":25548,"spinlock_wait_cycles":38627328,"thread_start_us":344572,"threads_started":1,"wal-append.queue_time_us":605159}
I20260709 12:27:23.824911 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.825796 18878 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:23.827348 10016 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:23.792212 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:23.794309 19023 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.478s	user 0.000s	sys 0.001s
I20260709 12:27:23.843565 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:23.844936 10099 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:23.831985 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.846171 10017 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:23.834419 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.835887  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 3338) took 1372 ms. Trace:
I20260709 12:27:23.836850  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:23.836886 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.847569  9947 rpcz_store.cc:276] 0709 12:27:22.463832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.474520 (+ 10688us) service_pool.cc:224] Handling call
0709 12:27:23.835697 (+1361177us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540003f054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:23.835830 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2833924480}
I20260709 12:27:23.849179 18972 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:23.849601 18972 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:23.849937 18972 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:23.851357 18942 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.851878 18942 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.854075 18942 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.857043 18950 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:23.857621 18950 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.858414 18972 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.825995 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:23.761132 19030 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:23.871632 10097 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:24.223431 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:24.147395 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 4026) took 1384 ms. Trace:
I20260709 12:27:23.911615  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.124073 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3243) took 1735 ms. Trace:
I20260709 12:27:24.227125  9949 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:23.877797 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.877192 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:23.913671  9944 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:23.913924  9942 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
W20260709 12:27:24.091982 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 150280us
I20260709 12:27:24.232570 10677 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:27:24.232807  9943 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:24.233068 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 484379us
I20260709 12:27:24.233948  9947 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:24.226374 10095 rpcz_store.cc:276] 0709 12:27:22.389020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.393143 (+  4123us) service_pool.cc:224] Handling call
0709 12:27:22.412183 (+ 19040us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:23.494313 (+1082130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:24.123756 (+629443us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400162954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:24.123993 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487246592}
W20260709 12:27:23.936967  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72266.4us
I20260709 12:27:24.234611 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:24.236136 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:24.236316 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:24.224409 10022 rpcz_store.cc:276] 0709 12:27:22.738807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:22.746738 (+  7931us) service_pool.cc:224] Handling call
0709 12:27:24.122984 (+1376246us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54002d5354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:24.123144 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":885513472}
I20260709 12:27:23.979171  9842 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 15 to 18, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 18 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:24.233356 18865 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:24.245087 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:24.245955 10677 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:24.252732 10016 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:24.249915  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:24.259202  9942 raft_consensus.cc:2368] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25.
W20260709 12:27:24.259563 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.261229  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.262080  9944 raft_consensus.cc:2410] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 30 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
W20260709 12:27:24.122669 18956 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.022s	sys 0.045s
I20260709 12:27:24.267536  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.268925  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:23.874390 18972 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:23.934954 19022 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:23.910157 18915 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:24.271533 18956 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.022s	sys 0.051s
W20260709 12:27:24.275172  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:23.992700 18581 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:24.233603 18894 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.279656  9942 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:24.269251 18949 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.278578 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:24.305594 10675 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:24.282761  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.232807 18967 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:24.292933  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:24.277315 18986 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:24.258970 18878 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:24.300900  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.315729  9943 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:24.293958 18755 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.883221 18671 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.276443  9842 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a reported cstate change: term changed from 14 to 16, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 16 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:24.309069  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:24.121554 18973 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.160s
I20260709 12:27:24.269718  9947 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:24.312520 10021 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:24.278800 18857 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:23.890095 18948 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:23.847512 19037 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.317493  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:24.291983 18941 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:24.310909 18915 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.312237 18894 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.340327 18941 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.341952 18915 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.342239 18894 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.339643  9945 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:24.318315  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:24.273020 18672 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:27:24.292730 18817 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:24.265516 18865 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.327173  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.305332 18950 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.331688  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.333513  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.338802  9842 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 15 to 18, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 18 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:24.303915 18937 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.022s
I20260709 12:27:24.347024  9946 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:24.351135 18948 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.323915 18755 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.353646 18688 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:24.381563 10675 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:24.354365 18972 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.385073 18972 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.386577 19050 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:24.387163 18972 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.317493 18949 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.317762 18967 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.310793 18581 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:24.363138  9834 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 333) took 43660 ms (client timeout 15000 ms). Trace:
I20260709 12:27:24.327579 18909 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:24.343523 18857 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.298095 19022 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.371866  9842 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a reported cstate change: term changed from 13 to 21, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 21 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:24.370898  9943 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:24.344115 18794 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:24.342455 18941 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.380903 10022 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:27:24.384419 19037 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.403203 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:24.370822 18817 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.407341 18671 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.408134 18878 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:24.387176 19050 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.408792 18884 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:24.417838 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:24.382030 18688 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.434291 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:24.436538  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:24.444578  9842 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 16 to 20, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 20 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:24.451040 10673 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:24.459530 10016 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: [27.13-27.13]
I20260709 12:27:24.461021 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.461939 10017 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:24.453723 10676 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:24.480400 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:24.475193 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:24.483418 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:24.485023 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:24.485095  9842 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 19 to 20, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 20 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:24.486040 10675 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:24.466207 18688 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:24.426034 18909 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.492340  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:24.494047  9943 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:24.495841 10673 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:24.470276 18950 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.498670 19039 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:24.499150 19039 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.499607 19039 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:24.499155 18950 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.504845 18950 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.506196 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:24.507138 10670 raft_consensus.cc:2368] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:24.507608 18755 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.508986 18581 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.511561 18581 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:24.510828 19039 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.513803 18865 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:24.517735  9842 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a reported cstate change: term changed from 21 to 22, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 22 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:24.519153 10097 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:24.514204  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.519379  9842 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a reported cstate change: term changed from 16 to 18, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 18 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:24.515352  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.521008  9946 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:24.521335 18895 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:24.521785 18895 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:24.523607 18949 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.525439 19050 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:24.526458 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:24.527292 10668 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:24.491570 18937 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:27:24.527876 18968 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:24.528366 18937 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:24.526705 18857 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:24.529448 19038 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:27:24.528728 18937 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:24.516078 19022 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.533119  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.533946  9946 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:24.483166 18972 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.539789 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:24.540421 10093 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:24.470474 18884 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.543653 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:24.543695 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:24.506280 18817 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.538194 19021 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.563782 19021 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.564723 18558 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:24.566160 19021 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.572623  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.548080 18865 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.575346 18865 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.577955 18865 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:24.579301 18805 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:24.579732 18805 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:24.580622 18805 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.582058 18805 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:24.533654 19039 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.546016 10093 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:24.547286 10670 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:24.545667 10097 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:24.553278  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:24.543869 18972 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.557137 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:24.606055 10093 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:24.610520 10096 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
W20260709 12:27:24.420461  9834 rpcz_store.cc:269] 0709 12:26:40.706690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:40.746031 (+ 39341us) service_pool.cc:224] Handling call
0709 12:26:42.404914 (+1658883us) spinlock_profiling.cc:243] Waited 12.3 ms on lock 0x7b5400002490. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e064320b2 00007f4e0f3ab759 00007f4e0f343a76 00007f4e0f314d58 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc
0709 12:27:24.362875 (+41957961us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:14.353947 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:14.417091 (+ 63144us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:14.417130 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:27:14.417146 (+    16us) tablet.cc:662] Decoding operations
0709 12:27:14.418241 (+  1095us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:14.418300 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:14.418325 (+    25us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:27:14.431745 (+ 13420us) tablet.cc:701] Row locks acquired
0709 12:27:14.431761 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:27:14.431926 (+   165us) write_op.cc:270] Start()
0709 12:27:14.432048 (+   122us) write_op.cc:276] Timestamp: P: 1783600034431873 usec, L: 0
0709 12:27:14.432066 (+    18us) op_driver.cc:348] REPLICATION: starting
0709 12:27:14.432783 (+   717us) log.cc:844] Serialized 5211 byte log entry
0709 12:27:15.815768 (+1382985us) spinlock_profiling.cc:243] Waited 2.78 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:16.343244 (+527476us) op_driver.cc:464] REPLICATION: finished
0709 12:27:16.476392 (+133148us) write_op.cc:301] APPLY: starting
0709 12:27:16.476515 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:16.527901 (+ 51386us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:16.527943 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:16.968685 (+440742us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b5c0026fe0c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f748331 00007f4e0f748221 00007f4e0f742f6c 00007f4e0cea330b 00007f4e0cea2fd3 00007f4e0cfa1e34 00007f4e0cf8d97c 00007f4e0cf91d61 00007f4e0cf91d19 00007f4e0cf91ca9 00007f4e0cf91c71
0709 12:27:16.969905 (+  1220us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:16.989396 (+ 19491us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=52,mrs_lookups=0
0709 12:27:16.989517 (+   121us) write_op.cc:312] APPLY: finished
0709 12:27:16.990281 (+   764us) log.cc:844] Serialized 121 byte log entry
0709 12:27:16.995756 (+  5475us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:16.998405 (+  2649us) write_op.cc:454] Released schema lock
0709 12:27:16.998479 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:16.999177 (+   698us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:16.999205 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1012916,"reported_tablets":47,"spinlock_wait_cycles":28209920,"child_traces":[["op",{"apply.queue_time_us":130314,"apply.run_cpu_time_us":31917,"apply.run_wall_time_us":522679,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58493,"num_ops":13,"prepare.queue_time_us":341728,"prepare.run_cpu_time_us":48446,"prepare.run_wall_time_us":423074,"raft.queue_time_us":469493,"raft.run_cpu_time_us":10662,"raft.run_wall_time_us":203190,"replication_time_us":1887012,"spinlock_wait_cycles":86668160,"thread_start_us":977892,"threads_started":4,"wal-append.queue_time_us":401549,"wal-append.run_cpu_time_us":43138,"wal-append.run_wall_time_us":3157235}]]}
I20260709 12:27:24.559530 18884 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:24.559522 10668 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:24.559631 18937 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.561474 18967 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:24.626415 19050 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:24.628147 18688 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.628621 19050 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.594198 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:24.631237 10674 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:24.597282 10019 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
W20260709 12:27:24.606956  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53820us
I20260709 12:27:24.636065 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.636750 10017 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:24.637775 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.639444 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.640267 10019 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:24.642292 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:24.621650 18966 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:24.643831 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:24.644690 18967 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:24.644945 18966 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:24.645057 18967 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:24.645533 18967 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:24.648008 18972 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.609242 10022 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:24.587131  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.626024 18865 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.627090 18937 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:24.597304  9987 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:24.658102  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 60869.6us
I20260709 12:27:24.659186  9987 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:24.652094 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:24.651929  9946 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:24.660622 10090 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:24.660801  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.630394 18688 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:24.659651 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 339635us
I20260709 12:27:24.586483 18909 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.622042 18856 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:24.663208 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:24.664690  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:24.666415 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:24.664175 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:24.672698 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:24.673023 10090 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:24.673558 10677 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:24.675287  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:24.675503 18581 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:24.667400 18972 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:24.675179 10674 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:24.677309 18581 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:24.666188 18856 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.686874 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.662135 18865 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.673233 18942 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:24.683604 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:24.589346 18895 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.712008 10674 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:24.676304 18688 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:24.712809 18895 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:24.713090 18688 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:24.714066  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:24.691718 10677 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:24.682158 10022 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:27:24.715476 10022 pending_rounds.cc:85] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 14
I20260709 12:27:24.715728 10022 pending_rounds.cc:107] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:27:24.715966 10022 raft_consensus.cc:2889] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:24.694236 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.717059 10017 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:24.713975 18688 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.673019 18968 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:24.718071 18968 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.718442 18968 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:24.715080 18992 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:24.678807 18986 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:24.679895 18581 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:24.726740 18992 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:24.727747 18992 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.729286 18865 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:24.691620 18856 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:24.679827 18972 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:24.732259 18942 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:24.724608 18986 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:24.734833 18967 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.739835 18548 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:24.677611 18852 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.046s
I20260709 12:27:24.731545 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:24.753463  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:24.754482 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 424946us
I20260709 12:27:24.765050 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:24.766583 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:24.783309 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 547060us
I20260709 12:27:24.785210 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:24.788159 19063 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.790581 10024 raft_consensus.cc:2368] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:27:24.791776 10017 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:24.792438 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:24.793227 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:24.795120  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:24.751235 18986 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:24.791477  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:24.799244 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:24.797757 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:24.801796 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 70345.9us
I20260709 12:27:24.802536 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:24.805836 10062 leader_election.cc:400] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24.
I20260709 12:27:24.808682 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.806883  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:24.809571 10021 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:24.806869 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.815021 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:24.789937 19063 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.796360 18884 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.003s
I20260709 12:27:24.797991 18852 consensus_queue.cc:1243] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 27.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:27:24.824693  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:24.779034 19059 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:24.833811  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:24.836525  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.837235  9943 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:24.839757  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.839978 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:24.841570 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:24.845057  9943 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:24.852672 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:24.803323 19071 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:24.845914  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:24.856050  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:24.856492 10095 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:24.863642  9949 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:24.809710 18941 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:24.879238 10021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:27:24.892786 10021 raft_consensus.cc:606] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:24.893189 10021 raft_consensus.cc:614] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:24.889015 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:24.890270 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:24.890990 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:24.813665 18747 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:24.856503 19037 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:24.858108 10093 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:24.901081 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:24.901722  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67994.9us
I20260709 12:27:24.902992  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:24.805290 19073 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:24.904927 18975 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:24.905764 18975 consensus_queue.cc:1243] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:24.846055 18898 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.061s
I20260709 12:27:24.831882 18949 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:24.906930 18982 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:24.898918 19052 log.cc:927] Time spent T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:27:24.907451 18982 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:24.841782 18972 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.908443 18982 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.905257 19060 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:24.906446 18755 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:24.899142  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:24.912635  9940 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:24.857421 18857 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:24.849790 18950 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:24.940742 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 122827us
I20260709 12:27:24.960100 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:24.910048 18581 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.899047 18941 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:24.907274 18949 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:24.841117 19022 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:27:24.906960 18747 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:24.873749 18852 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.092s
I20260709 12:27:24.891554 18948 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 27.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 12:27:24.955967 18857 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:24.996838 10677 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:24.980953 18941 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:24.845598 18942 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:24.996623  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 85447.9us
I20260709 12:27:24.917538 19060 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:24.979183 19037 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:25.011509 10022 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:24.985335 19076 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:25.002889 18857 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:24.968294 18950 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:25.042495 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 240386us
I20260709 12:27:25.058454 19071 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:25.082162  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:25.085423  9948 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:25.089409  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 175516us
I20260709 12:27:25.041045 18715 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.025758 18755 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:27:25.023633 19076 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.097282 18672 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:25.096889 19071 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.084780 18656 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:25.100680 19063 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:25.041853 18968 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.105729 19050 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:25.107059 19037 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.124070  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:25.190376  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66401.3us
I20260709 12:27:25.191366  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:25.193934  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:25.125738  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 245642us
I20260709 12:27:25.044410 18967 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:25.150257 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 190271us
I20260709 12:27:25.041623 18948 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.076s
I20260709 12:27:25.153944 18950 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:25.187788 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1682) took 1355 ms. Trace:
W20260709 12:27:25.154610 19083 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.067s	user 0.002s	sys 0.001s
I20260709 12:27:25.149865 18672 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:25.186147 19065 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
I20260709 12:27:25.152199 18860 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:27:25.207798 10097 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:25.207959  9912 leader_election.cc:400] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:27:25.146328 18715 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.210043  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:25.215612 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:25.212303 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 404207us
I20260709 12:27:25.154867 18656 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:25.198374 19098 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:25.206583 10018 rpcz_store.cc:276] 0709 12:27:23.832035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.832173 (+   138us) service_pool.cc:224] Handling call
0709 12:27:23.832438 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:23.832631 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:23.832911 (+   280us) log.cc:844] Serialized 14 byte log entry
0709 12:27:24.939078 (+1106167us) log.cc:844] Serialized 14 byte log entry
0709 12:27:25.158807 (+219729us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:25.158869 (+    62us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:25.187534 (+ 28665us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.187605 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.187733 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":225350,"prepare.queue_time_us":150943,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":244939,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":15063,"thread_start_us":1095074,"threads_started":3,"wal-append.queue_time_us":546022}
I20260709 12:27:25.083050 18949 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.212422  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:25.227847  9912 leader_election.cc:403] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:27:25.233239 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:25.235023 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:25.237723 18656 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:25.238672  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:25.220345 18972 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.240586 10095 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:27:25.230633 19064 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.243714 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:25.088104 18747 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.206084 19094 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:25.250566  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 51514.9us
I20260709 12:27:25.251660  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:25.239763 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 429732us
I20260709 12:27:25.253679 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:25.273772 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:25.229661 18860 consensus_queue.cc:1243] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 21.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:25.274065 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:25.294500  9949 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
W20260709 12:27:25.246577 18688 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.013s	sys 0.017s
W20260709 12:27:25.252573 18856 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.522s	user 0.002s	sys 0.008s
I20260709 12:27:25.322157  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:25.246469 19100 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:25.276104 18558 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.711s	user 0.016s	sys 0.000s
I20260709 12:27:25.253492 19021 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
W20260709 12:27:25.383126 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 109157us
W20260709 12:27:25.385510 18937 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.723s	user 0.010s	sys 0.002s
I20260709 12:27:25.333184  9940 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:25.226027 18852 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.099s
I20260709 12:27:25.381510 18968 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:25.273931 18581 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:25.252084 18671 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.106s
W20260709 12:27:25.355389 18688 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.013s	sys 0.017s
I20260709 12:27:25.430434 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:25.321151 19094 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:25.434334 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4036) took 1557 ms. Trace:
I20260709 12:27:25.440059 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:25.355707 18915 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:25.449085 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:25.437678 10025 rpcz_store.cc:276] 0709 12:27:23.876533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.876688 (+   155us) service_pool.cc:224] Handling call
0709 12:27:24.223584 (+346896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:24.265663 (+ 42079us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:24.265709 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:24.272454 (+  6745us) log.cc:844] Serialized 26 byte log entry
0709 12:27:24.770444 (+497990us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:25.124729 (+354285us) log.cc:844] Serialized 14 byte log entry
0709 12:27:25.241499 (+116770us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.241593 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:25.434109 (+192516us) raft_consensus.cc:1685] finished
0709 12:27:25.434132 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:25.434269 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":150612,"prepare.queue_time_us":138590,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":160535,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":9848,"thread_start_us":778315,"threads_started":2,"wal-append.queue_time_us":474080}
I20260709 12:27:25.382763 18856 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.405413 18760 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:25.322553 18972 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.430864 18955 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:25.276391 18672 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.439119 18973 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:25.471911 18973 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.473112 18856 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:25.474617 10024 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:25.475983 10673 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:25.476084 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:25.476373 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:25.477165 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:25.477291 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:25.478255 10675 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:25.478405 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:25.479888 19095 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.259s	user 0.000s	sys 0.001s
W20260709 12:27:25.481490 19093 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
W20260709 12:27:25.483285 19091 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:27:25.425457 18910 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:27:25.319176 19064 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:25.467203 19085 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 12:27:25.491329 18955 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:25.443871 19094 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:25.501293 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:25.529429 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:25.497511 18966 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.852s	user 0.001s	sys 0.012s
I20260709 12:27:25.503553 19059 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:25.600199 10023 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
W20260709 12:27:25.504292 19103 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
I20260709 12:27:25.502560 19106 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.506665 18955 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:25.636742 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:25.637405 10095 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:25.428457 19100 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.624529 19076 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:25.645925 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:25.624996 19059 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:25.571617 19065 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
W20260709 12:27:25.643741 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 247414us
I20260709 12:27:25.645035 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:25.642395 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:25.678726 10094 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:27:25.679029 10092 raft_consensus.cc:2410] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 23 because replica has last-logged OpId of term: 21 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 17.
I20260709 12:27:25.645501 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:25.678253 10095 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:25.378060 19090 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.213s
I20260709 12:27:25.643481 19106 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.646127 18978 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:25.643218 18688 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:25.662062 10097 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:27:25.664683 18857 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.621s	user 0.008s	sys 0.000s
I20260709 12:27:25.666208 18972 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:25.549767 18672 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:25.645216 18856 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:25.668813 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:25.707913 10096 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 27 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 12.
I20260709 12:27:25.658499 19102 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:25.674981 18715 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:25.625808 18794 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:25.644694 18895 consensus_meta.cc:287] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.931s	user 0.012s	sys 0.000s
I20260709 12:27:25.689380 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:25.690551 10090 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:25.634634 18966 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:25.694506 10670 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:25.719163  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 397075us
I20260709 12:27:25.708176 10025 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:25.713274 10094 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:25.652683 19084 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:27:25.724085 10065 leader_election.cc:400] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25.
I20260709 12:27:25.724997 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:25.698573 19111 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:25.708763 19050 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:25.734330 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:25.715029 19071 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:25.697633 18910 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 12:27:25.692976 18978 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.723778 19097 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.726248 18852 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:25.747958 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:25.749639 10024 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 19 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:27:25.739101 10670 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:25.696121 18688 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.698637 18857 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.742225 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:25.743422 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 298041us
I20260709 12:27:25.715161 18856 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.755679  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:25.766175 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:25.739929 18942 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:25.715909 19094 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.720769 18794 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.718142 19102 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.754302 18910 consensus_queue.cc:1243] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:25.746623 19076 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:25.883298 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 173484us
I20260709 12:27:25.770661 19090 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:25.885224 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 119141us
I20260709 12:27:25.723037 19059 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:25.890756 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:25.766777 19030 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:25.770946 18972 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:25.897724  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 142162us
I20260709 12:27:25.908766 18852 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:25.896675 10669 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:25.910730  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:25.898510 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 151679us
I20260709 12:27:25.882201 18548 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:25.785449 18581 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:25.746151 19071 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.916491  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:25.733879 19111 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:25.895412 18910 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.987389  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:25.888175 18966 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:25.742136 19050 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:25.755045 19097 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:25.991648 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 225861us
I20260709 12:27:25.894583 19090 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:25.903584 18942 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.007419  9943 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: [24.19-24.19]
I20260709 12:27:26.015502  9945 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:26.016433  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.016911  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.007422  9948 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:27:25.989125 18986 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.139s	user 0.015s	sys 0.000s
I20260709 12:27:26.008595  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:26.009161  9942 raft_consensus.cc:1217] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Deduplicated request from leader. Original: 19.14->[20.15-27.17]   Dedup: 20.15->[22.16-27.17]
I20260709 12:27:26.028350  9942 pending_rounds.cc:85] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 15
I20260709 12:27:26.028714  9942 pending_rounds.cc:107] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 21.16
I20260709 12:27:26.029022  9942 raft_consensus.cc:2889] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:25.906363 18941 consensus_meta.cc:287] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.887s	user 0.009s	sys 0.008s
I20260709 12:27:25.894042 18909 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:25.904692 19037 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:26.022006  9943 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
W20260709 12:27:26.007871  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 97279.3us
I20260709 12:27:26.030452  9947 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:26.029273  9949 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:27:26.043314  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:25.916157 18898 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:25.910228 19098 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:25.897368 19021 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.038540  9948 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:26.068946  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.062294  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:25.991097 19064 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.033694  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.026490 19090 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:26.074999  9949 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:25.916392 18715 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.139s
I20260709 12:27:26.078963  9947 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:26.110772  9947 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:26.084736 19107 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:25.919930 18972 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.091533 19125 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:26.092952 19129 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 12:27:26.093540 19132 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:26.094820 19117 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 12:27:26.041836 18986 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.099764 18857 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.020838 18910 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.042972 18942 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:26.101853 19128 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:27:26.049777 18941 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.111819  9947 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.109050 19064 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:26.067407 18898 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.118374 19101 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:26.048802 18856 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.118498 19020 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:26.166692 19120 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.020s
I20260709 12:27:26.186318 19020 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.188657 19020 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.145181  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.014061 18966 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.138090 19106 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:26.119052 19098 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:25.911872 19030 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.044760 18581 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.176962  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.137449  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.122200 19126 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:26.218433  9943 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:26.162671 19076 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.064517 19037 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.058617 18909 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.189960 10016 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:26.135495 19118 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:26.199486  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 283050us
I20260709 12:27:26.200058 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:26.215090 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 302642us
W20260709 12:27:26.223502 19117 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:27:26.185508 18978 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.109709 19021 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.218330 18857 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:26.231308 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:26.215466  9945 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:26.199702 18856 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:26.138890 19107 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.320394 19100 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.321193 18857 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.322104 19107 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.323529 19071 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.323697 18986 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.324232 18581 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.325065 18966 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.325423 18898 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.232851 19118 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.326021 18898 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:26.326622 19021 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.303082 19094 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.191205 19125 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.171810 19101 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:26.328697 18856 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.328420 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1657) took 1094 ms. Trace:
I20260709 12:27:26.228180 19076 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:26.327425 18656 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.049s	user 0.009s	sys 0.003s
I20260709 12:27:26.348944  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:26.316908 19131 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:27:26.319818 19050 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.329448 19101 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.210486 19064 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:26.331624 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 131673us
I20260709 12:27:26.314440 19030 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.330233 10099 rpcz_store.cc:276] 0709 12:27:25.233709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.254130 (+ 20421us) service_pool.cc:224] Handling call
0709 12:27:25.273335 (+ 19205us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:25.273735 (+   400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:25.273764 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:25.274217 (+   453us) log.cc:844] Serialized 26 byte log entry
0709 12:27:25.743103 (+468886us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:25.766334 (+ 23231us) log.cc:844] Serialized 14 byte log entry
0709 12:27:25.766969 (+   635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:25.767105 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:26.328194 (+561089us) raft_consensus.cc:1685] finished
0709 12:27:26.328216 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:26.328359 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30025,"prepare.queue_time_us":22870,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":64641,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":66256,"thread_start_us":468558,"threads_started":1,"wal-append.queue_time_us":614583}
I20260709 12:27:26.328472 19144 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.159446 19132 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:26.392123 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 405021us
I20260709 12:27:26.360268  9945 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:26.400031  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:26.403483  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 359330us
I20260709 12:27:26.327549 18972 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.302892 18978 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.411546  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:26.413889 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:26.415386  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 123270us
I20260709 12:27:26.327520 19118 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.411065  9948 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:26.423229  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.423931  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.429194  9945 raft_consensus.cc:2393] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:26.434015  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:26.428699  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:26.394811 19102 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.436084  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:26.436861 18949 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:26.451257 19149 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:26.427935 18672 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.719s	user 0.010s	sys 0.001s
I20260709 12:27:26.452715 19101 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:26.453281 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53428.3us
I20260709 12:27:26.454012 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:26.407539 19064 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.455190 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:26.407701 19143 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.456107 19143 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.456621 19064 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:26.457705 19087 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.415s	user 0.000s	sys 0.016s
I20260709 12:27:26.458523 19050 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:26.460093 19050 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:26.460299 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1873) took 1020 ms. Trace:
I20260709 12:27:26.460551 10091 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:27:26.460757 19050 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:26.461490 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:26.473690  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:26.464277 19097 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.464403 19076 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:26.226328 19126 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.466123 19134 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:26.468102 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:26.468490 18909 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.392908 18656 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.469732 18910 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:26.427656 18972 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:26.511317  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.512099  9940 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
W20260709 12:27:26.514533 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 283304us
I20260709 12:27:26.472357 19118 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:26.461437 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:26.472509 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.473961 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:26.475682 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.492883 18865 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.416764 19144 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.460690 10669 rpcz_store.cc:276] 0709 12:27:25.439619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.439765 (+   146us) service_pool.cc:224] Handling call
0709 12:27:26.460130 (+1020365us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b5400723654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:26.460241 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664119040}
I20260709 12:27:26.463388 19090 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.481191  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.482890  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:26.518276 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:26.523340  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:26.523909 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:26.524294  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 99662.4us
I20260709 12:27:26.483886 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:26.525730 10095 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:26.458370 19143 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:26.430766 18978 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.466351 18904 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:26.497390  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.498872 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.542115  9943 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:26.427935 18856 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.508204 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.508376 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:26.431031 19071 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.462302 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:26.462291 19050 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.556959 19050 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.560153 19142 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:26.560731 19142 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:26.560827 19050 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.561220 19142 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:26.563311 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:26.564780 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:26.567293 19125 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.546021 18909 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:26.568773 18909 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:26.572108 18656 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.572566 19144 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.516125 18972 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.519102 10640 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:26.494169 19020 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:26.532447 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:26.535319  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 106674us
I20260709 12:27:26.503293 19111 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.514365 19102 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.030s
I20260709 12:27:26.542435 10671 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:26.550776 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.537647 19098 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:26.545187 18904 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:26.552402 10096 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:26.553551 19090 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.554881 19071 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.555686 19126 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.573586 10675 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:26.493192 18949 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:26.557200 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:26.583526  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.624922  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.625694  9940 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:26.584450  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.585508  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:26.587318 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 68345.4us
I20260709 12:27:26.553582 18978 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.502985 18852 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:26.503232 19064 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:27:26.589442  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:26.595069 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:26.595372 10673 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:26.630326  9943 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:26.519094 19076 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 12:27:26.569814 18909 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.579404 18910 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:27:26.610628 18904 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.614084 19165 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.604548 19107 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:26.615121 19132 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.615947 19039 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.589706 18656 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:27:26.616570 18942 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.598775 19144 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:26.617190 19037 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.538506 18865 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.648514 19132 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.648885 19022 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:26.649348 19022 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:26.649045 19132 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.579162 19152 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:26.650243 19022 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.643859 19021 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.651861 19021 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.653956 19021 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.654663 19073 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:26.651245 10094 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:26.655162 19073 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:26.656265 19073 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.658123 19064 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:26.658756 18898 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.659061 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.659240 18898 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.542747 19094 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:27:26.627310  9942 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:26.626394  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.640672 19107 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:26.665076  9940 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:26.666158  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.666968  9940 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:26.667932  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.627455 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:26.588093 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:26.632031 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 74903.1us
I20260709 12:27:26.668949 10669 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:26.670832 19144 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:26.671864 19144 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.676216 19107 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.677696 19107 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:26.634577 18852 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.680043 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1651) took 1448 ms. Trace:
W20260709 12:27:26.680371  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.640787 19098 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:26.681896 18852 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.682691 18805 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:27:26.682873 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55507.1us
W20260709 12:27:26.682698 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.629655 19102 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:27:26.684221 19102 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:26.684569 19102 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:26.684768 18805 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:26.685396 19102 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.686394 18805 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.687132  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.682889 10097 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:26.688330  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.619805 18671 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:26.689085  9942 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:26.689167 18671 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.616452 19140 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:26.651280 19022 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:26.643697 18904 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:26.618240 18972 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.637038 19126 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:26.680516 10091 rpcz_store.cc:276] 0709 12:27:25.231831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.231966 (+   135us) service_pool.cc:224] Handling call
0709 12:27:26.679791 (+1447825us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003ae3d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:26.679954 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2605368576}
I20260709 12:27:26.686928 18805 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.689944  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.681808 19098 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.693111  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:26.694720 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.622718 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:26.717165 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 94513us
I20260709 12:27:26.651219 19132 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.660673 10095 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:26.696408  9940 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:26.691573 18671 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.721902  9943 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:26.725085  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.669724 18865 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.693233 19102 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:26.727823  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.669989 19064 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.735195 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 19 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.735739  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:26.711294 19126 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:26.645660 19039 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.740358  9942 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:26.744932 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.746218 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:26.747057 10025 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:26.747160 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.747856 10018 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:26.749756 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.751106 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.752480 19126 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:26.753453  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:26.732847 18656 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:26.823828  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.802223 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.778067  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:26.735796 18715 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.045s
I20260709 12:27:26.829713 10672 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:27:26.830734 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.733690 18972 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:26.806308 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.761884 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:26.751638 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.811853 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.838181 10017 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:26.838399 10671 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:26.803114 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.838330  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:26.811156  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:26.839210 10022 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:26.839288  9943 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:26.822569 10025 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:26.825490 10018 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 20.
I20260709 12:27:26.735507 18817 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:26.803855  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 52946.2us
I20260709 12:27:26.840859 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:26.841852 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.644392 19165 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.727411 19118 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:26.750276 10020 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:26.869167  9943 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:26.869606 10021 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
W20260709 12:27:26.825150 18955 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.147s	user 0.012s	sys 0.003s
I20260709 12:27:26.869999 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.859886 10018 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 31 because replica has last-logged OpId of term: 30 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 18.
I20260709 12:27:26.872100 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.738708 19160 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.740861 19141 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.738164 19164 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.840704 18715 consensus_queue.cc:1243] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 30.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:27:26.863626 18817 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.824936 18656 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.865911 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.831130 18581 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.916745 18581 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.917430 18973 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.803440 19132 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.919039 10022 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:27:26.919735 19039 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.920297 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.920985 19059 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:26.921499 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.923341 10676 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:26.911531 10019 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:27:26.924435 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:26.833644 19060 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.869300 19118 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:26.915390 19155 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:26.919376 19132 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.873507 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:26.947167 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:26.835358 19171 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:26.915479 18898 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.947824 10672 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 20.
I20260709 12:27:26.924993 18656 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:26.752249 19037 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.190s
I20260709 12:27:26.919091 18581 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.945597 19118 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:26.870500 18955 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:26.965939 18817 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.966349 19165 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.933197 19160 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.953784 19171 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.967325 19064 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.968966 18805 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:26.976094 19132 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:26.977176 19167 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:27:26.977669 18941 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:26.983045 19150 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:27:26.986445 18794 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.988497 19105 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:26.861595 18972 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.989101 19105 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:27.003578 19169 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.076s	user 0.003s	sys 0.000s
I20260709 12:27:27.014858 18972 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:26.926112 19141 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.952677 19155 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:26.936453 19164 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.073068  9949 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
W20260709 12:27:27.079710  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:26.889515 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:27.088804 10020 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
W20260709 12:27:27.088877  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 277681us
I20260709 12:27:27.091821 10018 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:27.096985 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:27.099473 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.126412  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:27.129573 18955 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:27.129849 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.130966 10091 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:27:27.131582 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.135247 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.136313 10099 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:27.142796 19090 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.147641 19090 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.147415 19157 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:27:27.153012 19090 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:27.157841  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.163655 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.165095 19073 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:27.169265  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 3373) took 1036 ms. Trace:
I20260709 12:27:27.122814 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:27.176901 10672 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:27.177174 19037 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:27.177815 19037 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:27.180152 19030 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.180361 19141 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:27.181546 19123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.003s	sys 0.009s
I20260709 12:27:27.181088 19030 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:27.182046 19123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.003s	sys 0.009s
I20260709 12:27:27.178220 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:27.183296  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:27.184216  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67253.2us
I20260709 12:27:27.184801 19059 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:27.185377 19059 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:27.185846 19059 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:27.186271  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 63060.9us
W20260709 12:27:27.177614  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87141.1us
I20260709 12:27:27.190315  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:27.196228 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.197383 10092 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:27.188027  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:27.206875 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.040340 19160 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:27.208032 10676 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:27.209225 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:27.209960 19171 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:27.211730 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 77133.3us
I20260709 12:27:27.173497  9949 rpcz_store.cc:276] 0709 12:27:26.132928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.138490 (+  5562us) service_pool.cc:224] Handling call
0709 12:27:27.168895 (+1030405us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54001a40d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:27.169134 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2027583872}
I20260709 12:27:27.218039 19158 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:27:27.221930 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.224136  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:27.226248 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.227190 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.228369 10099 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 30: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:27.229586 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.212559 19030 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:27.195302 19141 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:27.192128 19105 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:27.227231 10091 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:27:27.230962 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:27.236985  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 49369us
W20260709 12:27:27.239013  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53002.4us
I20260709 12:27:27.238334 10017 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:27.231468 10023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2853) took 3697 ms (client timeout 2005 ms). Trace:
W20260709 12:27:27.232210 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.976s (SENT) [suppressed 15 similar messages]
I20260709 12:27:27.243626 19030 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:27.241992 10023 rpcz_store.cc:269] 0709 12:27:23.533906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:23.534098 (+   192us) service_pool.cc:224] Handling call
0709 12:27:27.231043 (+3696945us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54001f5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:27.231212 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4268936064}
I20260709 12:27:27.250238 19176 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.007s
I20260709 12:27:27.250910  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:27.253572  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:27.254176 19141 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:27.254741 19141 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:27.255198 19141 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:27.255071 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.257628 10091 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:27.259038 19158 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:27.259809  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.261934 19060 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:27.258303 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1586) took 1365 ms. Trace:
I20260709 12:27:27.267066 19164 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:27.267918 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1895) took 2460 ms. Trace:
I20260709 12:27:27.268560 10677 rpcz_store.cc:276] 0709 12:27:24.807808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.813515 (+  5707us) service_pool.cc:224] Handling call
0709 12:27:24.825957 (+ 12442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:25.002282 (+176325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.267186 (+2264904us) spinlock_profiling.cc:243] Waited 169 ms on lock 0x7b5400690d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:27.267603 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":389434240}
I20260709 12:27:27.273434  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:27.274783 19060 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.277351 19060 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.260573 19158 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:27.280504 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.280308 19158 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:27.282270 10024 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:27.283633 19105 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:27.285509 18941 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:27.286366 18941 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.287336 10098 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
W20260709 12:27:27.292955 19142 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.731s	user 0.016s	sys 0.009s
I20260709 12:27:27.293080 18955 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:27.294936 18955 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.298190 18955 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.299754 19176 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.300077  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:27.301426  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.305869 18972 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.305764 19176 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.306819 18972 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.307806 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1659) took 1399 ms. Trace:
I20260709 12:27:27.308247 10096 rpcz_store.cc:276] 0709 12:27:25.907936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.908165 (+   229us) service_pool.cc:224] Handling call
0709 12:27:25.908512 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.600973 (+692461us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.304563 (+703590us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b540014afd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:27.307600 (+  3037us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480423680}
I20260709 12:27:27.306110  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:27.244257 19030 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:27.263362  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:27.293633 19142 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:27.324946  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.340293 18972 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.340502 19171 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:27.341009 19171 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.341430  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.341728 19111 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.325500 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.329125  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.342422 19111 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.264608 10668 rpcz_store.cc:276] 0709 12:27:25.892907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.893019 (+   112us) service_pool.cc:224] Handling call
0709 12:27:25.893308 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:26.041901 (+148593us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:26.041927 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:26.042386 (+   459us) log.cc:844] Serialized 26 byte log entry
0709 12:27:27.257195 (+1214809us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b1800316f60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1
0709 12:27:27.257542 (+   347us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:27.257634 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.257738 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:27.257800 (+    62us) raft_consensus.cc:1685] finished
0709 12:27:27.257812 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:27.258066 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55412,"raft.queue_time_us":283685,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":20249,"spinlock_wait_cycles":13199744,"thread_start_us":227450,"threads_started":2,"wal-append.queue_time_us":325838}
I20260709 12:27:27.342561  9945 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:27.317536 19030 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:27.342809 19071 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:27.357147 19124 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.304s	user 0.002s	sys 0.002s
I20260709 12:27:27.334537 19126 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.337733 18857 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:27.331837 18904 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.618s	user 0.015s	sys 0.005s
W20260709 12:27:27.360057  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 60299.7us
I20260709 12:27:27.360802 18941 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.361070 18558 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:27:27.361914 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.364806 19059 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:27.366319 19176 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.367805 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:27.368310 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:27.369386 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:27.371212 18656 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:27.346068  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 98935us
I20260709 12:27:27.343053  9949 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:27.341442 19171 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:27.391522 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 159513us
I20260709 12:27:27.345332 19111 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.400749  9944 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:27.402859 10022 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:27.409063 10092 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:27.412465  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:27.401963 18558 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:27.414971 10677 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
W20260709 12:27:27.423200  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 166494us
I20260709 12:27:27.423478 18915 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:27.424258  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:27.424808 10669 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:27.425482 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1584) took 1533 ms. Trace:
I20260709 12:27:27.426842 19060 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:27.439608 10095 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: [24.15-24.15]
I20260709 12:27:27.440714 19144 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:27.441917 19144 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:27.442619 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:27.448447 10090 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1654) took 2215 ms (client timeout 2201 ms). Trace:
W20260709 12:27:27.449084 10090 rpcz_store.cc:269] 0709 12:27:25.232792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.232918 (+   126us) service_pool.cc:224] Handling call
0709 12:27:27.447888 (+2214970us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b54002fd854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:27.448125 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1016313728}
I20260709 12:27:27.426844 19098 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:27:27.450784 19098 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:27.451217 19098 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:27.453912 10093 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:27.459059 18909 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:27.460127 18857 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:27.428246 10668 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:27.471222  9947 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:27.473905 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2866) took 2784 ms. Trace:
I20260709 12:27:27.430003 19060 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:27.475972 19060 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:27.481031 19188 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:27:27.491523  9948 raft_consensus.cc:1217] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Deduplicated request from leader. Original: 21.17->[23.18-25.19]   Dedup: 23.18->[25.19-25.19]
I20260709 12:27:27.492238  9948 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: [23.18-25.19]
I20260709 12:27:27.492074 18558 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:27.493484  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.497771 18986 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:27.499312 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.500204 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.500828 19030 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:27.500228 18837 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:27.502802 10090 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:27.501595 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.499185 18986 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:27.507249 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:27.508128  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.508595 18656 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:27.426054 10670 rpcz_store.cc:276] 0709 12:27:25.892211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.892441 (+   230us) service_pool.cc:224] Handling call
0709 12:27:25.892732 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.060062 (+167330us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:26.108127 (+ 48065us) log.cc:844] Serialized 14 byte log entry
0709 12:27:26.598623 (+490496us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:26.598815 (+   192us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:26.598864 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.424828 (+825964us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54007d3cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:27.424921 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:27.425204 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":153765,"prepare.queue_time_us":57775,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":173881,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":92,"spinlock_wait_cycles":384081792,"thread_start_us":350652,"threads_started":2,"wal-append.queue_time_us":673726}
I20260709 12:27:27.534962 18837 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:27.545606 18837 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.546111 18949 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.547363 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.547612 18949 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:27.549027 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:27.544895 19098 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.547389  9948 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:27.518841  9940 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:27.519070 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87832.5us
I20260709 12:27:27.523722  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:27.524010 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.407477 19141 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:27.527067 19189 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:27:27.442332 19144 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:27.528018 19152 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:27.530184 19142 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:27.530468 19157 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:27:27.532408  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 125007us
I20260709 12:27:27.498471  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:27.474366 10016 rpcz_store.cc:276] 0709 12:27:24.689761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.689869 (+   108us) service_pool.cc:224] Handling call
0709 12:27:24.690131 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.228225 (+1538094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.473340 (+1245115us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400021054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:27.473702 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3448902528}
I20260709 12:27:27.511024 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.541705  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:27.542052 10098 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:27.516382 19059 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:27.554719 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.542462 10096 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:27.550485 18949 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:27.563436 10095 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:27.579814  9949 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:27.581481 19094 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:27.582185 19094 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:27.582866 18837 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:27.586669  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.589828  9941 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:27.590137 10099 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:27.558650 19152 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:27.576052  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.591091 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:27.592689  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 94340.8us
I20260709 12:27:27.589254  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.596153 10095 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:27.599433 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.601359 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.595768  9944 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:27.596385 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.604957 10098 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 22: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:27.556008 19158 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:27:27.611123 19134 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:27.612622 19155 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.612813  9946 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:27.604192 19098 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.604206 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.635715 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:27.642179 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.637748 10093 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 17. (term mismatch)
I20260709 12:27:27.648777 10093 pending_rounds.cc:85] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 16
I20260709 12:27:27.649096 10093 pending_rounds.cc:107] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 12:27:27.649398 10093 raft_consensus.cc:2889] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:27.649778 10093 pending_rounds.cc:107] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:27:27.650051 10093 raft_consensus.cc:2889] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:27.649586 19152 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.650981 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3267) took 3109 ms. Trace:
I20260709 12:27:27.651321 10093 rpcz_store.cc:276] 0709 12:27:24.541927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.543073 (+  1146us) service_pool.cc:224] Handling call
0709 12:27:24.543396 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:24.711329 (+167933us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b4c0028b8c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07822a20 00007f4e07833a29 00007f4e077f9644 00007f4e07820f2d 00007f4e07820b93 00007f4e0955118b 00007f4e09551cb5 00007f4e0954cf30 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416
0709 12:27:24.857755 (+146426us) spinlock_profiling.cc:243] Waited 121 ms on lock 0x7b540009b120. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094d406f 00007f4e0954cf60 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62
0709 12:27:27.452396 (+2594641us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5800606ed0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:27:27.650618 (+198222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:27.650901 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1167998208}
I20260709 12:27:27.659005 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:27.659318 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:27.575445 19157 consensus_queue.cc:1243] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:27.660462 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:27.661079 19134 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:27.662016 19118 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.664s	user 0.013s	sys 0.001s
I20260709 12:27:27.660463 10675 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:27:27.675458 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.679307 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.681943 10096 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:27.638355 19176 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:27.683022 19176 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:27.685058 10095 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:27.691248 19102 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.949s	user 0.010s	sys 0.005s
I20260709 12:27:27.662457 19118 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.694248 19118 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:27.696796  9941 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:27.696360 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:27.713663 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.756826 10090 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:27.757176  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.601419 19136 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.089s
I20260709 12:27:27.765064 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.683440 19176 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:27.771101  9947 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 21. (term mismatch)
I20260709 12:27:27.774142  9947 pending_rounds.cc:85] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 20
W20260709 12:27:27.774529  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:27.787566  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.788545  9941 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:27.789453  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.607295 18656 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.152s
I20260709 12:27:27.663064 18857 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.212s
I20260709 12:27:27.808473  9944 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:28.057844  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.059186  9941 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:28.013082  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:27.983976 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.071894 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.072396 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.088579  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:27.852393  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.128573  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:27.795439 19107 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.117s	user 0.005s	sys 0.016s
I20260709 12:27:28.138854 10099 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:28.089989 18558 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.009s	sys 0.028s
I20260709 12:27:28.143594  9944 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:28.054785 18904 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.146278  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:27.791517 19134 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:28.165587  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.166038 10668 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:27.766801 18715 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.011535 19157 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:27.834681  9947 pending_rounds.cc:107] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 29.21
I20260709 12:27:28.172370 10096 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:28.126232 19030 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:28.163128  9943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.002s	sys 0.009s
I20260709 12:27:28.165210 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:27.767369  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:28.153805 18558 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.011s	sys 0.030s
I20260709 12:27:28.174345  9947 raft_consensus.cc:2889] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:28.177068  9943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.002s	sys 0.010s
I20260709 12:27:28.170118 10094 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:28.171101  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.179541 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:28.185119 10096 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:27:28.179821  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:28.180246 19140 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 1.176s	user 0.006s	sys 0.011s
I20260709 12:27:28.177560 10095 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:28.187549 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:28.188227 10099 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 17.
I20260709 12:27:28.191578  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.164669 19110 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:28.176185 19171 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.757s	user 0.012s	sys 0.000s
I20260709 12:27:28.186551  9991 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:28.196765  9941 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:28.186291 19152 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.172365 19102 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:28.199527 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:28.173175 18857 consensus_queue.cc:1243] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:27:28.208482  9944 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:28.211912 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:28.212016 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:28.213083 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:28.021525 19144 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:27.899547 19060 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.214809 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:28.218662 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.201471 18937 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:28.220733  9948 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:28.210848 18986 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:28.222820  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3372) took 2141 ms. Trace:
I20260709 12:27:28.167054 18904 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.201700 19102 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:28.230999 18558 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:28.261538 18558 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.261969 18558 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:28.263972 18688 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.197389 19157 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.267437 19102 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.268643 19102 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:28.270179 19157 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.231834 19144 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:28.232218 19147 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:28.279239 18837 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.696s	user 0.011s	sys 0.000s
I20260709 12:27:28.278937 19147 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.281164 19118 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:28.205509 19171 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.283972 19171 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.284276 10023 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: [22.17-22.17]
W20260709 12:27:28.284636 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.233469 19059 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:27:28.229588 18986 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:28.287315 18986 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:28.274014 10095 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:28.233733 10668 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:28.212460 18656 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:28.208356 19140 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:28.232537 19099 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.237973 18938 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.209085 19110 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.238423 19120 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.239977 19060 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.241657 10094 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:27:28.243115 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.243875 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.228951 19176 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:28.244935 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.245071  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 582126us
I20260709 12:27:28.247419 10093 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:27:28.248793 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.250793 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:28.298779 10096 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:28.303318 10018 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:28.250990 19084 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:28.214866 19030 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.253692 19063 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:28.219657 19163 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:28.255681 10025 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:28.256628  9945 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:28.259021 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.260012 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:28.260739 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.261047 18904 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:28.187693 18715 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.229835  9947 rpcz_store.cc:276] 0709 12:27:26.078250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.078454 (+   204us) service_pool.cc:224] Handling call
0709 12:27:26.078831 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.470053 (+1391222us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b58007b7dd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e094b20aa 00007f4e09534447 00007f4e0954cfe7 00007f4e0954e7de 00007f4e09558b49 00007f4e09559416 00007f4e09555d8f 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4
0709 12:27:28.207650 (+737597us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.219234 (+ 11584us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":546238848}
I20260709 12:27:28.276036 10099 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:28.286176 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:28.300813 10016 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:28.311573 19122 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.002s
I20260709 12:27:28.311944 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:28.312695 10025 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:27:28.312852 19122 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:28.313269 19122 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:28.313467 10018 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:28.313865  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:28.305125 19084 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:28.314816 19163 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.315866  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:28.316013 19110 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:28.316154 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1901) took 1503 ms. Trace:
I20260709 12:27:28.300174 19099 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.316838 18895 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:28.317070 19163 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:28.317458 18895 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:28.317729 19030 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.311960 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:28.303570 18938 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.319542 18992 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:28.319989 18992 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.320132  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.320432 18992 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:28.298151 18656 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.320163 10021 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:28.318770 19099 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:28.318536 18895 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.327970 19126 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:28.308779 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:28.339118 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1585) took 2446 ms. Trace:
I20260709 12:27:28.335361  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:28.316597 10673 rpcz_store.cc:276] 0709 12:27:26.812763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.812898 (+   135us) service_pool.cc:224] Handling call
0709 12:27:26.813200 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.124766 (+311566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:27.124840 (+    74us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:27.124936 (+    96us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:27.125026 (+    90us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.315759 (+1190733us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54006a0254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:28.315801 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.315997 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":711324032}
I20260709 12:27:28.344511 10096 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:28.346122 18760 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.306654 19063 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:28.348822  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.321260 18938 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.350057  9949 raft_consensus.cc:1217] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Deduplicated request from leader. Original: 21.17->[23.18-25.19]   Dedup: 23.18->[25.19-25.19]
I20260709 12:27:28.320000 19030 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:28.337170 19105 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:28.354699 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:28.356685  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:28.359100  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.350180 18760 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.335662 19176 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.353091  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.362401 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:28.354457 19202 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:27:28.347373 19141 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.344508 10674 rpcz_store.cc:276] 0709 12:27:25.892569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:25.892886 (+   317us) service_pool.cc:224] Handling call
0709 12:27:25.893123 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:26.226131 (+333008us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:26.281185 (+ 55054us) log.cc:844] Serialized 14 byte log entry
0709 12:27:26.587065 (+305880us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:26.587169 (+   104us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:26.587247 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.338688 (+1751441us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54004d35d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:28.338765 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.338944 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75162,"prepare.queue_time_us":8359,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":82534,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":9675,"spinlock_wait_cycles":765331712,"thread_start_us":223692,"threads_started":1,"wal-append.queue_time_us":456372,"wal-append.run_cpu_time_us":2222,"wal-append.run_wall_time_us":1271323}
I20260709 12:27:28.355607  9949 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: [23.18-25.19]
I20260709 12:27:28.372273 10022 raft_consensus.cc:2368] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25.
I20260709 12:27:28.335065 19084 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.306670 19120 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.382246 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.383481  9941 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:28.383911 10021 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:28.388098 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.365222 19171 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:28.391703 10017 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:28.393990  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49260.3us
I20260709 12:27:28.377126 19105 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:28.397011  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.381644 18837 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:28.385147 19126 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:28.405334 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:28.389056 19084 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:28.379323 19063 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.361333 19099 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
W20260709 12:27:28.416234 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 110415us
W20260709 12:27:28.423089 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 68537.3us
I20260709 12:27:28.359704 19122 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.422948 18978 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:28.429982  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:28.434878  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.438037 19063 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:28.438385 19105 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.448261 19171 raft_consensus.cc:2764] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:27:28.452792 18938 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.454557 18938 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.454932 19122 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:28.456758 18938 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:28.456780 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:28.457793 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.458623 10670 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 21.
I20260709 12:27:28.463330  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:28.463475 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:28.464349 19171 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.381179 18558 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.448217 18956 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:28.477123 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.476383 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.444768 18978 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:28.489665 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.452154 18656 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.453277 19176 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:28.489665 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:28.494452  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 76337.4us
I20260709 12:27:28.497179 10673 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:28.497527 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:28.499948  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.498922 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:28.507607 10020 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:27:28.523554 10021 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:27:28.524446  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:28.523490 10671 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 20.
I20260709 12:27:28.493088 18956 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:28.524183 10021 pending_rounds.cc:85] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 16
I20260709 12:27:28.540308 10021 pending_rounds.cc:107] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:27:28.540683 10021 raft_consensus.cc:2889] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.499495 18715 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.540865 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.489881 18986 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.521136 18656 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.549553 10016 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:27:28.552271  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 313386us
I20260709 12:27:28.554567 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:28.566929 10668 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:28.568303 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.572897 10017 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:28.574126 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.549037 19176 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:28.575805  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 69763.7us
I20260709 12:27:28.577284 10023 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:28.575505 10670 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
W20260709 12:27:28.584442  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 352796us
I20260709 12:27:28.555397 18656 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.551805 19164 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.033s
I20260709 12:27:28.591367 10671 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:28.575351 18938 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.590124  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.592147  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:28.598482 18956 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.599766 18986 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:28.582463 18978 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.610823 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:28.611515 18941 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:28.611999 18941 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:28.612933 18941 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.618741 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:28.620862 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:28.621762 19110 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:27:28.623553 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.624315 18992 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.626889 18992 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:28.629622 10676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1900) took 1868 ms (client timeout 1782 ms). Trace:
W20260709 12:27:28.630028 10676 rpcz_store.cc:269] 0709 12:27:26.761429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.761619 (+   190us) service_pool.cc:224] Handling call
0709 12:27:28.629390 (+1867771us) spinlock_profiling.cc:243] Waited 85.4 ms on lock 0x7b5400490a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:28.629541 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":196344576}
I20260709 12:27:28.633906 18558 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.634167 19037 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.635301 19037 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.637082 19120 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:28.638331 10097 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1662) took 2178 ms (client timeout 2205 ms). Trace:
W20260709 12:27:28.638799 10097 rpcz_store.cc:269] 0709 12:27:26.459520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:26.462036 (+  2516us) service_pool.cc:224] Handling call
0709 12:27:28.637826 (+2175790us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b54003aded4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:28.638153 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":309447936}
I20260709 12:27:28.639459 19171 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:28.644443 10017 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:27:28.647362 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.663132  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:28.664706  9991 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:28.670962  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.673645 10096 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:28.679171  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:28.679844 18558 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.652717 19102 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:28.686791 19106 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:28.687301 19106 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:28.687691 19106 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:28.688654 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.690129 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.677366 19141 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.690809 10022 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:28.691114 19141 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.691967 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.692759 10018 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:28.693392 19141 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.639596 18938 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.695753 18938 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.695895 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:28.697242 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:28.698511  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108514us
W20260709 12:27:28.699076  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 104780us
I20260709 12:27:28.652446 19084 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:28.701345  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.702833 18992 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:28.703439 18992 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:28.703938 18992 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:28.704027 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:28.705109 19157 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:28.706933 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.708269 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.677359 19171 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:28.679359 10021 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
W20260709 12:27:28.702672 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 357093us
I20260709 12:27:28.710036 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:28.716439 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:28.727198  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 135150us
I20260709 12:27:28.727972  9945 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 12:27:28.716172  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:28.646961 19206 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:28.728466  9945 pending_rounds.cc:85] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 16
I20260709 12:27:28.753803  9945 pending_rounds.cc:107] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 12:27:28.754225  9945 raft_consensus.cc:2889] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:28.735023  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:28.755312  9949 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:28.758190 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.758906 10668 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
W20260709 12:27:28.760015 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53155.3us
I20260709 12:27:28.767254 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.718132 19171 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.759756 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:28.730887 19140 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:27:28.768144 10671 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:28.733162 19120 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:28.777913  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:28.781318  9944 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:28.781656 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:28.779510  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79536.6us
I20260709 12:27:28.784497  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:28.734302 19144 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:28.778204 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.790825  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.791028  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.775038 18978 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:28.791657  9941 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
W20260709 12:27:28.796185  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 80123.8us
I20260709 12:27:28.775849 19171 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:28.799779 18656 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:28.800661 18978 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:28.778786 19140 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:28.802709 19155 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:28.803128 19140 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:28.780732 19152 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:28.807049 19152 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.807539 19152 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.808243  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:28.810505 18895 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:28.811028 18895 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:28.811403  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.769318 10096 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:28.812259  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:28.816793 18837 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:28.811499 18895 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:28.814985 19176 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:28.786141 19120 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.819110 19120 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:28.816478 19171 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.821581  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:28.824230 19084 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:28.825181 19176 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:27:28.833972 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.835357 19126 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:28.835831  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:28.834537 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.839803  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.840718  9941 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:28.842250 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:28.812996 10096 raft_consensus.cc:606] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.847563  9944 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
W20260709 12:27:28.847877 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53394.1us
I20260709 12:27:28.849128  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:28.852622  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64881.4us
I20260709 12:27:28.853377  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.853976 19094 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:28.854512 19094 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:28.855047 19118 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:28.855468 19118 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:28.855952 19099 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:28.856073 19118 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:28.857394 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.759399 19063 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 12:27:28.858457 19207 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.859117 19063 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:28.859018 19207 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.859547 19063 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:28.860035 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:28.860598 19063 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.861102 19157 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:28.861645 19157 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:28.852635  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:28.855362 19094 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:28.789459  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:28.865712  9949 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:28.864305  9941 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:28.861168 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:28.862012 19157 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:28.868353  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:28.872025 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 174920us
I20260709 12:27:28.872828 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:28.889037 19118 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:28.892273 19030 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.892588 19118 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.892908 19030 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:28.895172 19030 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:28.861912 19063 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:28.849997 18558 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:28.906153  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 121781us
I20260709 12:27:28.786996 19037 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 12:27:28.911275  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.861229 19207 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.865969 19094 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:28.916275  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3387) took 1367 ms. Trace:
I20260709 12:27:28.876025 18656 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:28.925944 10677 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:28.921526  9943 rpcz_store.cc:276] 0709 12:27:27.543316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:27.587783 (+ 44467us) service_pool.cc:224] Handling call
0709 12:27:27.588096 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:27.588434 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:27.588798 (+   364us) log.cc:844] Serialized 14 byte log entry
0709 12:27:28.618226 (+1029428us) spinlock_profiling.cc:243] Waited 2.2 ms on lock 0x7b180031f240. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:28.687057 (+ 68831us) spinlock_profiling.cc:243] Waited 4.61 ms on lock 0x7b180031f240. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:27:28.909746 (+222689us) log.cc:844] Serialized 14 byte log entry
0709 12:27:28.910000 (+   254us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:28.910069 (+    69us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:28.910133 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.910230 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:28.910376 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28819,"prepare.queue_time_us":3440,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":62,"raft.queue_time_us":12969,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":15041,"spinlock_wait_cycles":15655040,"thread_start_us":779427,"threads_started":2,"wal-append.queue_time_us":619688}
I20260709 12:27:28.917277 19118 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:27:28.919627 18978 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:28.871114 19141 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
W20260709 12:27:28.943439  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94426.8us
I20260709 12:27:28.944212  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:28.916051 19205 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:28.925568 18656 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:28.945606 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72926.7us
I20260709 12:27:28.942719 10669 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:27:28.930727 19207 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:28.933135 19171 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:28.941362 19135 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:28.967517  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 214157us
I20260709 12:27:28.943252 19155 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:28.971954 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.972602 10016 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:28.951447 18837 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:28.922912 18904 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 12:27:28.949865 18656 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:28.978147  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:28.982447 10018 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:27:28.952023 19207 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:28.979367 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.984102 10017 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:28.980989 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:28.984874 10024 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:28.985992  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 75085.8us
I20260709 12:27:28.945144 19107 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:27:28.963104 19135 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:28.998740 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.007381 10023 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:28.999030  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:29.001293 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:28.998965 10062 leader_election.cc:400] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25.
I20260709 12:27:29.008651 10062 leader_election.cc:403] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
W20260709 12:27:29.003304  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.009435 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:28.976917 18837 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:28.986843  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:29.019516  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 108339us
I20260709 12:27:29.024806 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:29.024827  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:28.987318 19207 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:29.010726 19135 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:29.028416  9912 leader_election.cc:400] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:27:29.029495  9912 leader_election.cc:403] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:27:29.026340 18904 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:28.977087 19155 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.034945 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:29.036610 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66782.2us
I20260709 12:27:29.044735 10669 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:29.019076 18837 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:29.050078  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 51115.6us
I20260709 12:27:29.052534  9949 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:29.053375 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:29.053025 10024 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:28.983367 19106 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.027616 19147 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.078889  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:29.081462 18558 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:29.062783 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:29.120462 10670 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:27:29.121779 19141 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:27:29.026400 19171 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:29.122812 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.123687 10016 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:29.065439 19037 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.124650 19037 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:29.065567 19212 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
I20260709 12:27:29.066169 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:29.070472  9946 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:29.127573 10671 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 27 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 19.
I20260709 12:27:29.072396 19106 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.076166 18992 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.127820 18895 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.128400 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:29.129036 10671 raft_consensus.cc:2376] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25. Re-sending same reply.
I20260709 12:27:29.129897 18992 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:29.130123 18895 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.130281 10671 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 12:27:29.092319 19202 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:29.096944 10677 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:29.097608  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 110871us
I20260709 12:27:29.062719 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:29.114619  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 87272.3us
I20260709 12:27:29.036024 19059 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:29.084581 18978 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:29.063689 19158 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:29.130792 10021 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:29.077087 19220 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.126952 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:28.982690 19094 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.137485 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.147300 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:29.137804  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:29.141029 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:29.143069  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:29.122779 19171 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:29.151080 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.146270  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:29.152709 10016 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 21. Current term is 22.
I20260709 12:27:29.133766 18558 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:29.154209 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:29.154937 10671 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:29.160388  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:29.151396 19217 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.166s	user 0.003s	sys 0.000s
I20260709 12:27:29.205750 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.206856 10016 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:27:29.204469  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67332.5us
I20260709 12:27:29.208213  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:29.201531  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:29.211592 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 238022us
I20260709 12:27:29.213192  9943 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:29.214300 19158 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:29.214754 19158 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.214833 10672 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:29.215169 19158 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:29.218780 19030 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:29.219205 19030 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.219544 19030 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:29.153353 19059 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:29.235812 19059 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.159431 18558 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:29.244791 18904 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25.
I20260709 12:27:29.245767 19107 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:29.248713 18837 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.252547 18941 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:29.257189 19094 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:29.263724 19141 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:29.265631 19227 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:29.266911 19227 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:29.269894 19221 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:27:29.154318 19220 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.277745 19220 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:29.280294  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:29.281217  9946 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:29.246446 19107 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.288617 19107 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:29.154728 19171 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:29.300213  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:29.311275  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:29.319245 19155 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:29.320329 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.323763 10670 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:29.139437 19120 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:29.326828 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.336974  9942 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:29.348505 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:29.349467 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 201013us
I20260709 12:27:29.353399 10669 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:29.203352 18992 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:29.143093 19152 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:29.361937 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 149400us
I20260709 12:27:29.152074 18656 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.294924 19140 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.357283 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.380753 19098 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:29.382287 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4380) took 1205 ms. Trace:
I20260709 12:27:29.347151 19220 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.386004 10016 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 12:27:29.358206 18992 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:29.395524  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 239143us
I20260709 12:27:29.365623 18904 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:29.384404 19176 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:29.403417 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.386366 19098 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:29.384267 10098 rpcz_store.cc:276] 0709 12:27:28.176466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.270615 (+ 94149us) service_pool.cc:224] Handling call
0709 12:27:28.638125 (+367510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:28.638478 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:28.638799 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 12:27:28.880310 (+241511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:28.880374 (+    64us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:28.880443 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.880513 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.382094 (+501581us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54003aded0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:29.382225 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1416,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":129,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":844355456,"thread_start_us":240923,"threads_started":1,"wal-append.queue_time_us":312057}
I20260709 12:27:29.404006  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:29.400055 18992 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:29.406949  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:29.421856 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:29.437674  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 291727us
I20260709 12:27:29.451771  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:29.405738 19176 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.463078  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:29.420013 19037 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:29.399492 19220 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:29.466228  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 255518us
I20260709 12:27:29.436841 19155 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:29.492023  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:29.453387 19063 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 12:27:29.465502 18837 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:29.497023 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.485831 18895 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:29.488109 19098 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.508771 19098 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:29.513851 19084 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:29.485423 19120 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:29.519682  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 80280.4us
W20260709 12:27:29.520959  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 69264.3us
I20260709 12:27:29.521944  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:29.495003 19037 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:29.527539 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 139525us
I20260709 12:27:29.528972 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:29.529008 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.465286 19176 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:29.529829 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:29.527062 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:29.534979 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:29.535818 10669 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:27:29.537933  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.538676 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:29.539760 10670 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:27:29.541661 10669 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:29.518043 19135 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.543534 19135 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.544107 19063 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:29.545288 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:29.546183 10674 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:27:29.546029  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:29.547794 18978 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:29.549537 18978 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:29.549705 19171 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.533754 19155 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:29.553253 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.554343 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.555496 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:29.556293 10020 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 12:27:29.557276 10020 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:29.559172 10023 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:29.560644 10017 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:29.561312 18904 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:29.561777 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.524240 19235 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:29.571143 10025 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:29.571583 19106 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.005s
I20260709 12:27:29.592510 19106 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:29.592942 19106 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:29.571866 19157 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.655s	user 0.010s	sys 0.000s
I20260709 12:27:29.594872 19157 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.595305 19099 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.596015 19099 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.596985 19157 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:29.597317 18558 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.598542 19099 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:29.571767 18760 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.572614 10021 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:29.576160 18941 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:29.577929 19107 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:29.600857 19107 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.578188 19084 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.578426 18904 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:29.578869 19206 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Stepping down as leader of term 20
W20260709 12:27:29.579320 19207 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.000s
I20260709 12:27:29.580049 19227 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:29.580926 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.581485  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:29.583748 19220 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:29.584281 19235 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:29.555521 19140 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.588778 18656 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:29.580336 18938 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:29.594048 19106 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.609072  9940 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:29.610242  9914 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:29.606117 19206 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 20 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:29.607089 19158 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:29.617923 19206 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.619261 19158 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.619481 19206 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:29.565090  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:29.619901 19218 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.600s	user 0.006s	sys 0.001s
I20260709 12:27:29.623481 19235 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.627631 19171 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.567081 19233 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.086s
I20260709 12:27:29.616142 19152 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:29.630343 19163 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:29.613837 18904 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:29.631685  9941 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:29.605883 19107 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:29.630990 19163 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:29.654170 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:29.654310  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44200.6us
I20260709 12:27:29.635030  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:29.659938  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 2017) took 1491 ms. Trace:
I20260709 12:27:29.659340 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.661249  9940 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:29.661118  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:29.660363  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 112935us
I20260709 12:27:29.663686  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:29.664580  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 123439us
I20260709 12:27:29.664793  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:29.667251 10021 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:29.668404  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:29.614686 18938 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:29.664316 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.667161 19105 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:29.677254 10098 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:29.609325 19207 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.673884 19163 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.673333 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.675252 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:29.660979  9948 rpcz_store.cc:276] 0709 12:27:28.168737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:28.224268 (+ 55531us) service_pool.cc:224] Handling call
0709 12:27:28.245697 (+ 21429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:28.571028 (+325331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:28.571060 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:28.589799 (+ 18739us) log.cc:844] Serialized 26 byte log entry
0709 12:27:28.846812 (+257013us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:28.846853 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:28.846926 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:29.619180 (+772254us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:29.632600 (+ 13420us) raft_consensus.cc:1685] finished
0709 12:27:29.632632 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:29.659875 (+ 27243us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31039,"raft.queue_time_us":37620,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":124,"spinlock_wait_cycles":6454016,"thread_start_us":469363,"threads_started":2,"wal-append.queue_time_us":379018}
I20260709 12:27:29.672066  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:29.672791 19233 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:29.609481 19084 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.677614  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:29.673300 19094 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:29.679325 19171 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:29.658922 19158 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:29.681085 10022 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
W20260709 12:27:29.727721 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 197990us
I20260709 12:27:29.613201 18760 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.055s
I20260709 12:27:29.687261 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.729151 10024 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:29.730149 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:29.730352 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:29.731208 10024 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:29.731385 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:29.716895 10020 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:29.671113 19141 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:27:29.655586 18978 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:27:29.715133  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:29.714167 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.687362 18956 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.675107 19164 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:29.675742 19205 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:29.670711 19155 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:29.726541 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:29.727245 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 200256us
I20260709 12:27:29.727751 10668 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:29.677532 19105 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:29.655289 18904 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:29.749393 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.723351 19227 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:29.779023 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:29.735867 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:29.816979 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81192.7us
I20260709 12:27:29.777647 18558 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:29.712692 19147 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.091s
I20260709 12:27:29.821826 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.827363 10020 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:27:29.827719 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.828887 10017 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 12:27:29.830312 10017 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:29.836073 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:29.837204 19220 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:29.837754 19220 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:29.838255 19220 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:29.736755 18656 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:29.730237 19158 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.708281 19030 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:29.852334  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 137287us
I20260709 12:27:29.853202  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:29.856166 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:29.858902 19155 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:29.860153 19155 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.863018 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.739440 19233 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:29.802424 19141 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:29.809293 19227 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:29.880472 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.736220 19102 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:29.882546 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.841333 19120 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:29.726542 19234 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.138s
I20260709 12:27:29.858484 19205 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:29.864573 19158 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:29.827029 18956 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.914018  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:29.916193  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:29.918644  9941 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:29.918843 18956 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:29.891245 19102 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.921121 19102 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:29.894145 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.840685 19238 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:29.926939 19238 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:29.901371 10020 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:29.927670 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:29.894004 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 160681us
I20260709 12:27:29.802073 19037 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:27:29.937790  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:29.943339  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 140530us
W20260709 12:27:29.951861 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:29.891991 19227 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:29.974473 10024 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:29.977257 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.981410  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:29.982822  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:29.986276 10021 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:29.850739 19164 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:29.927409 19238 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:29.908288 19239 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:29.900401 19120 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:29.900044 19152 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.086s
I20260709 12:27:29.991737 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:29.989351 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:29.983685  9942 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:29.999071 10020 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:29.909732 19256 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.088s
I20260709 12:27:29.823961 18938 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:30.001559  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 87882.2us
I20260709 12:27:29.907060 18671 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.118s
I20260709 12:27:29.902510 19244 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.111s
I20260709 12:27:30.001005 19120 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:30.010236 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 154268us
I20260709 12:27:30.010579  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:29.997777 19205 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:30.018745  9943 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
W20260709 12:27:30.034721 19257 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.089s	user 0.000s	sys 0.005s
I20260709 12:27:29.990222 19105 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.035837 19105 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:30.038774 19253 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.060s
I20260709 12:27:30.042969 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:30.044550 19255 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.055s
W20260709 12:27:30.046361 19246 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:27:30.048276 19084 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.049185 19164 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:30.050426 19030 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.049180 18956 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:30.050709 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:30.051309 18956 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.052944 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:30.054303 10669 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 20. Current term is 21. Ops: []
I20260709 12:27:30.057004  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:30.058005  9940 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:30.058519 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.059271 10023 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:30.013482 19233 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:30.062690 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.063190 19263 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.063865 19263 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.027196 19207 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.032780 19037 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:30.067927 19037 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.069024 19037 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.070416 19037 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:30.074971 19252 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.075815 19252 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:30.091187 19237 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.362s	user 0.001s	sys 0.000s
I20260709 12:27:30.093051 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1903) took 1041 ms. Trace:
I20260709 12:27:30.093585 10676 rpcz_store.cc:276] 0709 12:27:29.051818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.051925 (+   107us) service_pool.cc:224] Handling call
0709 12:27:29.117716 (+ 65791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:29.138665 (+ 20949us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:29.138696 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:29.151848 (+ 13152us) log.cc:844] Serialized 26 byte log entry
0709 12:27:29.534105 (+382257us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:29.534162 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:29.534265 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:30.092732 (+558467us) raft_consensus.cc:1685] finished
0709 12:27:30.092765 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:30.092967 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7280,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5310,"thread_start_us":175926,"threads_started":1,"wal-append.queue_time_us":454466}
I20260709 12:27:30.006108 19141 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.094925 19247 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:30.095244 19248 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:27:30.116961 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.096763 19263 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.096129 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:30.119907 19247 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:30.121480 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:30.122468 19247 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:27:30.123373 18895 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:27:30.123937 18895 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:30.123991 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:30.125284 19247 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:30.125177 18895 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.127341 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:30.128630 19247 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:30.129031 10019 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:30.094528 19158 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.131448 10096 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:27:30.132753 19158 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.133859 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.141850 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.142114 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.142342 10095 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:30.142856 10099 raft_consensus.cc:2410] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 26 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:27:30.142869 10094 raft_consensus.cc:2410] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 27 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 19.
I20260709 12:27:30.141870 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:30.162346  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.138015 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:30.165998 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:30.161449 10097 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:30.167960 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.169027 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.167071  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:30.169656 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:30.172096 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.173261 10092 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
W20260709 12:27:30.097867 18992 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.643s	user 0.010s	sys 0.001s
I20260709 12:27:30.180644 19259 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:30.129087 19247 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.200095 19247 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:30.201030 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.201673 10021 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:30.201756 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.202385 10025 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:30.199339 10020 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:30.185665 18895 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:30.181440 19084 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:30.186582  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:30.187155 19255 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:30.187904 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.193625 19118 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:30.194861 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.210336 10023 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:30.195940 19258 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:27:30.196969 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.204046 19246 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:27:30.179035 19171 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:30.199045 19106 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:30.185566 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.219063 10097 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:30.173717 19152 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:30.209080  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:30.197156 19176 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.623s	user 0.007s	sys 0.007s
W20260709 12:27:30.199156 19107 consensus_meta.cc:287] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
I20260709 12:27:30.113278 19220 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:30.218937 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 49378.3us
I20260709 12:27:30.213913 19030 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:30.225417  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:30.226680 10095 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:30.226975 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.206250 19252 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.204931 10021 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:30.212145 19118 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.231117 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.237705 10094 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:30.231266 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.214509 19258 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.184461 18992 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.239773 10099 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:30.241277 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.223220 19255 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.244699 10090 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 21: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:30.248369 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.222923 19152 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:30.252313 10095 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:30.252451  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:30.231428 19205 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:30.254524 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.214665 18904 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.022s
I20260709 12:27:30.261819  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:30.267962 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.256039  9949 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:30.229905 18956 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.223670 19171 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.280274 10023 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:30.272927 10018 raft_consensus.cc:2368] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23.
I20260709 12:27:30.280165 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.271239 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.281257 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:30.282827  9949 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:27:30.284725 19059 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.009s	sys 0.009s
W20260709 12:27:30.285091 19059 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.009s	sys 0.009s
I20260709 12:27:30.286870 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.287560 10093 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:30.287295 19255 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.288476 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.289601 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.239794 19030 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.292781 18904 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.295801 19244 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.295719 18904 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.232707 19147 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.038s
I20260709 12:27:30.293979 19171 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.231964 19107 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.266253 19227 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:30.301635 19268 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:27:30.302356 19107 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:30.240242 19135 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.050s
I20260709 12:27:30.302930 19227 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.303165 19147 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:30.303623 19147 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.303671  9941 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:30.304088 19147 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:30.306654 10098 raft_consensus.cc:606] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:30.274888 19120 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.307080 10098 raft_consensus.cc:614] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:30.308248 10091 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:30.308749 19120 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:30.318324 19242 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.459s	user 0.001s	sys 0.000s
I20260709 12:27:30.319847 10097 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:30.226502 19176 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.300431 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.311390 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:30.252336 19263 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:30.313323 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.325531 10017 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:30.313361 19220 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.313642 19118 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.315292 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.316100  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:30.316793 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:30.335373 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:30.317401 19249 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 12:27:30.335182 19258 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.380319 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.320104 10092 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:30.380781 19258 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:30.380944 10091 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:30.381898 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.296398 19244 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.387224 10017 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:30.388334 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.388643 19244 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.389097 10017 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:30.390103 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.391031 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.391758 10092 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:30.281865 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:30.320990  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:30.322049 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:30.295444 19030 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.258399 19152 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:30.298784  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:30.401930 10017 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 30: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:30.256480 19205 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.317377 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.404593 10020 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:30.338495  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:30.348577 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.360138  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.367125  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:30.367709 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.411535 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.412241 10021 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:30.413324 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.414194 10021 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:30.416599 19164 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:30.417174  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:30.418051  9945 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:30.419528 19235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.004s	sys 0.025s
W20260709 12:27:30.420077 19235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.004s	sys 0.025s
I20260709 12:27:30.367928 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:30.368522  9943 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:30.368532 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.370230 19176 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.371119 10016 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:30.371071 19238 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.371649 19118 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.343592 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:30.243824 19140 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:27:30.339015 19105 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:30.455274 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.455982 10099 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:30.374059 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:30.374426 18992 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:30.379122 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81529us
I20260709 12:27:30.402458 10025 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
W20260709 12:27:30.403278  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 104599us
I20260709 12:27:30.457724  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:30.403643  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 82730us
I20260709 12:27:30.404661 10018 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:30.424759  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 86362.3us
I20260709 12:27:30.463807  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:30.447695 10023 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:27:30.453719 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:30.454348 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:30.454947 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 111444us
I20260709 12:27:30.465869 10669 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.271370 19102 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.457180 10675 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:30.425496 19205 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:30.468850  9944 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:30.458688 19239 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.448868 19227 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:30.456001 19107 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.470562 19105 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.463918 19220 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:30.482694  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:30.471381 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.466158 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:30.450753 19244 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:30.484766 19248 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:30.485162 19248 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.483946 10021 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 12:27:30.474021 10090 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:30.485642 19248 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:30.446748 19234 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:30.475706  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:30.476974 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:30.424980 19230 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.487282 19230 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.424504 19263 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.472577 19279 consensus_queue.cc:1059] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 12:27:30.489768 10669 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:30.489078 19220 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.496800 19220 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.423763 19120 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:27:30.503525 19233 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:30.504192 19233 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:30.455827 18895 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.038s
I20260709 12:27:30.473261 19102 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.516525 10097 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:30.533357 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.483772 19084 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:30.517274  9945 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:30.515516 19233 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:30.519935 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60112.9us
I20260709 12:27:30.539050 10097 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:27:30.540017 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:30.520495 18986 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:30.520543 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.519621 19256 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:30.523458 19238 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.489133 19107 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.541585 10091 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:30.544955 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.549865  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:30.550927 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:30.561439 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:30.562728 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 77803.3us
I20260709 12:27:30.566308 18895 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:30.566972 18895 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:30.566807 19263 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.567893 18895 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.570932 18992 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:30.571498 18992 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.572163 18904 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:30.572746 18904 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:30.573697 18992 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.577836 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:30.579579 19254 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.720s	user 0.000s	sys 0.003s
I20260709 12:27:30.580631 19220 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.583467 19105 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.578488 10095 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:30.490129 19239 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.547060 18986 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:30.591060 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.599581 10091 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:30.595789 19107 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:30.546944 19256 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.551725 19140 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:27:30.606647 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57222.2us
I20260709 12:27:30.607497 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:30.615222 19163 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:30.623773  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 160071us
I20260709 12:27:30.624531  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:30.623050  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 160132us
I20260709 12:27:30.598755 19280 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:30.602139 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 125271us
I20260709 12:27:30.630636 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:30.633920 19158 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:30.634837 19158 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:30.635316 19234 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.635155 19147 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.636291 19234 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.13, Last appended by leader: 13, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.557137 19247 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.645869 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.665887 10091 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 25 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 12:27:30.648972 10017 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:27:30.653419 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:30.604764 19220 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:30.668144  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:30.675084 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.676242 10017 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 31: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:30.628268 19264 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:30.680580 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.680729 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.682251 10023 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:27:30.678673 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.683879 10022 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.679704 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:30.593879 19205 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.635191 19158 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:30.687338 10096 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
I20260709 12:27:30.663429 19107 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:30.690706  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59269.9us
I20260709 12:27:30.672001 18760 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:30.681671 19207 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:27:30.692857 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85438.8us
I20260709 12:27:30.756259 10640 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:30.699661  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:30.696267 19287 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:30.747918 19276 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:30.748790 19248 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.685611 19264 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:30.750916 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.685452 19256 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.803000 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:30.803596 10021 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.752971 19147 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:30.753101 19205 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:30.757869  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59255.7us
W20260709 12:27:30.736711 19106 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.514s	user 0.008s	sys 0.003s
I20260709 12:27:30.785429 10022 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:30.695591 19158 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:30.692592 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:30.801899  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.796559 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:30.850615 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54108.8us
I20260709 12:27:30.851065 19147 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:30.852141 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:30.853670 19297 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:30.690770 19141 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.854909 19141 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.815385 10022 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:30.857218 19141 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.819903 10099 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:30.858628 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.862353 19147 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:30.862977 19147 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.864873 19230 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.865136 19147 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.867336 18938 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:30.872604  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.874714 19267 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:30.874958  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:30.875463 19106 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.846983 10023 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:30.848753 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.877179 10091 raft_consensus.cc:2368] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25.
I20260709 12:27:30.795557 19289 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:30.878986 10092 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:30.888271 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:30.883136  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.888459 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.884727 10099 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:30.890728 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:30.891758 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.894371 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:30.800511 19207 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:30.926717 10672 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:30.897051 19287 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:30.897246 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 204739us
I20260709 12:27:30.774117 18760 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.909785 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:30.913450 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:30.914994  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:30.915669 10673 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:30.925261 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.928687 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:30.904714 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:30.935676 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:30.950260 10091 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:30.875990 19106 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:30.943753 19207 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:30.898528 19059 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.887126 19244 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:30.988958 19244 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:30.953334 10099 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:30.990761 19248 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.991425 19248 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:30.964514 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.947796 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:30.992865 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:30.978739 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50152us
I20260709 12:27:30.995098 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:30.995491 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.995790 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:30.996054 10096 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:30.996238 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:30.996570 10099 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:30.997418 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:30.997581  9842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 315) took 48141 ms (client timeout 15000 ms). Trace:
I20260709 12:27:30.997702 19293 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:27:30.926429 19285 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:30.997980 10099 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:30.904688 18956 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:30.998987 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:30.999259 18956 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.000038 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.000672 10099 raft_consensus.cc:2368] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26.
I20260709 12:27:31.001415 18956 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.003686 19244 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.004962 19244 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:30.997979  9842 rpcz_store.cc:269] 0709 12:26:42.853283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:45.125595 (+2272312us) service_pool.cc:224] Handling call
0709 12:27:30.994530 (+45868935us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:24.905151 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:25.005823 (+100672us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:25.020020 (+ 14197us) write_op.cc:435] Acquired schema lock
0709 12:27:25.022378 (+  2358us) tablet.cc:662] Decoding operations
0709 12:27:25.472094 (+449716us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:25.472219 (+   125us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:25.472249 (+    30us) tablet.cc:685] Acquiring locks for 17 operations
0709 12:27:25.473629 (+  1380us) tablet.cc:701] Row locks acquired
0709 12:27:25.473686 (+    57us) write_op.cc:260] PREPARE: finished
0709 12:27:25.473848 (+   162us) write_op.cc:270] Start()
0709 12:27:25.473996 (+   148us) write_op.cc:276] Timestamp: P: 1783600045473785 usec, L: 0
0709 12:27:25.474019 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:27:25.620456 (+146437us) log.cc:844] Serialized 6758 byte log entry
0709 12:27:26.947739 (+1327283us) op_driver.cc:464] REPLICATION: finished
0709 12:27:27.311141 (+363402us) write_op.cc:301] APPLY: starting
0709 12:27:27.311303 (+   162us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:27.756148 (+444845us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:27.756293 (+   145us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:28.437832 (+681539us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:28.542790 (+104958us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=34,key_file_lookups=34,delta_file_lookups=0,mrs_lookups=0
0709 12:27:28.552074 (+  9284us) write_op.cc:312] APPLY: finished
0709 12:27:28.643113 (+ 91039us) log.cc:844] Serialized 156 byte log entry
0709 12:27:28.904853 (+261740us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:29.009521 (+104668us) write_op.cc:454] Released schema lock
0709 12:27:29.024579 (+ 15058us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:29.070762 (+ 46183us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:29.070794 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4791214,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":355617,"apply.run_cpu_time_us":90317,"apply.run_wall_time_us":1759689,"cfile_cache_hit":70,"cfile_cache_hit_bytes":75221,"cfile_init":1,"lbm_read_time_us":667,"lbm_reads_lt_1ms":4,"num_ops":17,"prepare.queue_time_us":189889,"prepare.run_cpu_time_us":11483,"prepare.run_wall_time_us":869117,"raft.queue_time_us":311730,"raft.run_cpu_time_us":7430,"raft.run_wall_time_us":315247,"replication_time_us":1404080,"thread_start_us":632544,"threads_started":5,"wal-append.queue_time_us":800168,"wal-append.run_cpu_time_us":36504,"wal-append.run_wall_time_us":3015815}]]}
I20260709 12:27:31.005657 19276 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:31.006448 19059 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.008694 19059 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.009265 10677 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:31.011693 10669 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:30.971545 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:30.973058 19106 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:30.978653 19239 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:30.980180 19247 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.001385 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.018065 10021 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:30.981647 18760 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:30.982364 19233 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:30.985684 19118 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.000s
I20260709 12:27:30.898025 18978 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.014185  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.019385 10091 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:27:31.023964 10095 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:27:31.034538 10669 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:31.035859 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:31.037276 10096 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
W20260709 12:27:30.980243 19294 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:27:31.031291  9948 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:31.041463 10095 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:31.044020 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:31.044628 10099 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:30.998126 19230 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:31.047540 10095 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: [28.16-30.17]
I20260709 12:27:31.046083 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:31.048992 10095 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:31.050074 10095 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:31.049999 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:31.051909 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:31.059805 10672 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:31.043387 19234 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:31.047638 19118 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:31.069332 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:31.058749 10095 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:31.053598 19163 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:27:31.080622 10677 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:31.054967 19106 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:31.061564 19152 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.609s	user 0.010s	sys 0.000s
I20260709 12:27:31.082697 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.085563 10099 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:31.057351 19230 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:31.041009 18978 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.047717 19279 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 12:27:31.097002 19280 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:31.098904 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:31.099529 10676 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:31.081478 19147 raft_consensus.cc:2764] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:27:30.971418 19264 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.105432 19264 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:31.106823 19107 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:31.107409 19107 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:31.108345 19107 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.077616 19118 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.111515 18558 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:27:31.111965 18558 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:31.112838 19118 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.112747 18558 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.113817 18558 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:27:31.117827 10094 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:31.072832 18938 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:27:31.060760 19158 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.051642 19239 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 12:27:31.088243 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.113201 19267 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:31.118438 10095 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:31.092123 19152 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.120453 18956 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:31.127780 19279 consensus_queue.cc:1243] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 21.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:31.134112 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1713) took 1586 ms. Trace:
I20260709 12:27:31.134227 10024 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:27:31.120620 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.148002 10016 rpcz_store.cc:276] 0709 12:27:29.547510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.553030 (+  5520us) service_pool.cc:224] Handling call
0709 12:27:31.133802 (+1580772us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b54001f6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.134013 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1878685056}
I20260709 12:27:31.150117 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.148033 19147 raft_consensus.cc:3055] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:31.151690 19147 raft_consensus.cc:740] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:31.152007 19233 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.150996 10099 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:31.153574 10670 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
W20260709 12:27:31.133085 19300 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:27:31.125015 19255 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.155385 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.163208 10093 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:31.156201 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87107us
I20260709 12:27:31.161842 10676 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:31.164232 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.161850 18938 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:31.167330 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.165771 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.168265 10095 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:31.168741 10099 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:31.173741 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:31.174876 10670 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:31.175374 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.176976 10099 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:31.177685 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:31.179298 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:31.171479 18956 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:31.182111 10670 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 17. Current term is 18. Ops: []
I20260709 12:27:31.170290 18978 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.152644 19147 consensus_queue.cc:260] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.198307 19256 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:31.199610 18986 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.200845 18986 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:31.191869 18956 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:31.201865 18956 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:31.200989 19152 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.182335 19274 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:31.195637 10670 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:31.205646 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:31.208626 10668 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:31.180963 19305 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:31.190366 19147 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:31.172258 19255 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:31.210640 19306 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:27:31.212049 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:33250 (request call id 23) took 1546 ms. Trace:
I20260709 12:27:31.217499 10098 rpcz_store.cc:276] 0709 12:27:29.665883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:29.666050 (+   167us) service_pool.cc:224] Handling call
0709 12:27:31.211833 (+1545783us) spinlock_profiling.cc:243] Waited 629 ms on lock 0x7b5400161f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09550853 00007f4e0e51e60a 00007f4e07aa47d4 00007f4e07aa4762 00007f4e07aa4691 00007f4e07aa460c 00007f4e07aa38c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.211957 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1446097664}
I20260709 12:27:31.194393 19279 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:31.214327  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.221499 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.224082 19287 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:31.225072 10092 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
W20260709 12:27:31.235184  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.240744 10094 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:31.241809 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.242059 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.242457 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:31.242890 10019 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:31.243330 19305 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:27:31.243906 19305 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:31.244801 19305 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.250885 19235 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:31.251197 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:31.251513 19235 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.253630 19235 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.214118 19274 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:31.257131 10094 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:27:31.257800 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:31.255930 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:31.261868 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:31.262513  9941 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:31.260999 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:31.266428 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:31.280931  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:31.283667  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:31.221879 19279 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:31.289323  9943 raft_consensus.cc:2410] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 31 because replica has last-logged OpId of term: 29 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 17.
I20260709 12:27:31.295742  9942 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:31.296072 10672 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:27:31.268326 19220 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:27:31.306284  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.279492 19250 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:31.279450 19152 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:31.297475 19279 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:31.322773  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57697.3us
I20260709 12:27:31.325901 10670 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:31.285840 19287 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:27:31.326079  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45280.4us
I20260709 12:27:31.336539 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:31.266392 19233 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 12:27:31.343312 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.346933 10672 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:31.347294 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.304250 19118 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:31.352125 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.274703 19155 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:27:31.349552  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 2038) took 1086 ms. Trace:
I20260709 12:27:31.353157  9948 rpcz_store.cc:276] 0709 12:27:30.257435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.257552 (+   117us) service_pool.cc:224] Handling call
0709 12:27:31.340881 (+1083329us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b540009c4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.341033 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1768030592}
I20260709 12:27:31.324389 19220 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:31.323855 19255 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.308712 19158 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.313397 19250 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:31.366204 10099 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:31.365434 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:31.340036 19152 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:31.378458 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1690) took 1104 ms. Trace:
I20260709 12:27:31.350399 19287 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:31.297046 19274 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.351400 19163 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:27:31.379930 10093 rpcz_store.cc:276] 0709 12:27:30.259104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.289360 (+ 30256us) service_pool.cc:224] Handling call
0709 12:27:31.363413 (+1074053us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b5400305554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.363552 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998251264}
I20260709 12:27:31.378702 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 3299) took 1138 ms. Trace:
I20260709 12:27:31.365242 19250 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:31.400224 10090 rpcz_store.cc:276] 0709 12:27:30.229122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.253932 (+ 24810us) service_pool.cc:224] Handling call
0709 12:27:31.366836 (+1112904us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400305f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.367020 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":442667392}
I20260709 12:27:31.419489 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.381145 18938 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:31.421267  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 84702.8us
I20260709 12:27:31.421506 18938 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:31.385169 19305 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.362798 19118 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.421674  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 2037) took 1175 ms. Trace:
I20260709 12:27:31.443085 18956 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:31.448273  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.427747  9914 leader_election.cc:400] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 21 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22.
W20260709 12:27:31.457461  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:31.457798  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.458087  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.428313 19287 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.377544 19220 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.452723  9940 rpcz_store.cc:276] 0709 12:27:30.246239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.247222 (+   983us) service_pool.cc:224] Handling call
0709 12:27:31.421471 (+1174249us) spinlock_profiling.cc:243] Waited 636 ms on lock 0x7b54001b9254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.421611 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1463300864}
I20260709 12:27:31.456717 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:31.471014 10016 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:31.459790  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:31.471809  9946 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:31.467847 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:31.468792 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.472819 10025 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:31.481462  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:31.489650  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:31.492053  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:31.492727  9947 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:31.493347  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:31.494886  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:31.471478 19312 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.052s	user 0.004s	sys 0.000s
I20260709 12:27:31.507272 10090 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:31.466521 19297 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:31.500568  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:31.517584  9942 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:31.509152 10020 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:27:31.513046 10093 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:31.466018 19255 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.466400 19314 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.477044 19158 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:31.521643 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.523435  9948 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 18. (term mismatch)
I20260709 12:27:31.523545 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:31.529817  9948 pending_rounds.cc:85] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 17
I20260709 12:27:31.530877  9948 pending_rounds.cc:107] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 29.18
I20260709 12:27:31.531118  9948 raft_consensus.cc:2889] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:31.423079 18938 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.493359 18715 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.535117 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:31.538906  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 178116us
I20260709 12:27:31.539316 10017 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
W20260709 12:27:31.537231 10097 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 3310) took 1242 ms (client timeout 1534 ms). Trace:
W20260709 12:27:31.540172 10097 rpcz_store.cc:269] 0709 12:27:30.294547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.335041 (+ 40494us) service_pool.cc:224] Handling call
0709 12:27:31.537050 (+1202009us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b54002fdd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.537166 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":386406144}
I20260709 12:27:31.541232 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.542214 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:31.507680 19267 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.489935 19235 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:31.538079 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.538237 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.537356 10096 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:31.557816 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.465725 19305 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.522404 19147 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:31.552955 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:31.558452 10020 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:31.553877 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:31.554699 10022 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 19. (term mismatch)
I20260709 12:27:31.570248 10022 pending_rounds.cc:85] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 18
I20260709 12:27:31.570504 10022 pending_rounds.cc:107] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 27.19
I20260709 12:27:31.570781 10022 raft_consensus.cc:2889] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:31.552613 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.574359 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.575182 10090 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:31.576215 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.555070 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.490483 19176 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:27:31.580174 10094 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:31.580641 10017 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:27:31.583423 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.583873 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:31.567482  9947 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:31.591447 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.552129 19322 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:31.592196 10023 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:31.586716 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:31.588230  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:31.595113 10090 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:31.585160 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:31.595688 10090 raft_consensus.cc:606] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:31.595844 10094 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:31.596088 10090 raft_consensus.cc:614] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:31.597882 10090 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
W20260709 12:27:31.593468  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.587401 10020 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:31.599516 10090 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:31.603883 10090 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:31.604964 10090 raft_consensus.cc:606] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:31.605446 10090 raft_consensus.cc:614] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:31.598198 10094 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:31.603816  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:31.608848 10017 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:31.572036 19319 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.572144 19147 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:31.607375  9948 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:31.529762 19152 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.592549 19248 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:31.536777 19314 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.560529 19267 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.584950 19324 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:31.538774 19255 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.552644 18715 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.571229 19235 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.616066 19147 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:31.639999 10020 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:27:31.621357 19248 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.646335  9948 raft_consensus.cc:1240] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:31.652839  9946 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:27:31.644727 18558 consensus_meta.cc:287] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.002s
I20260709 12:27:31.649935 19248 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:31.667939 19307 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.538s	user 0.004s	sys 0.002s
I20260709 12:27:31.659773 19118 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.666260 19256 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:31.675978 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 133902us
W20260709 12:27:31.674912  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 86750.4us
I20260709 12:27:31.672909 18956 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.690393  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:31.678804 19311 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
W20260709 12:27:31.679114 19323 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:27:31.697460 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:31.689699 19315 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.199s	user 0.001s	sys 0.000s
I20260709 12:27:31.689169 19118 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:31.716311 10016 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:31.607765 19322 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.635690 19324 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:31.721119 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.705009 19284 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:31.552063 19316 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.165s
I20260709 12:27:31.690758 19256 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:31.731622 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 263831us
I20260709 12:27:31.707141 19118 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:31.615988 19319 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.714432 19305 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.744105  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:31.746687  9943 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:31.752185 19337 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:31.753232 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4401) took 1430 ms. Trace:
W20260709 12:27:31.754819  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.739434 19235 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.741467 19118 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:31.753615 10092 rpcz_store.cc:276] 0709 12:27:30.323261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.992592 (+669331us) service_pool.cc:224] Handling call
0709 12:27:31.753076 (+760484us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b5400162e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.753176 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":532612864}
W20260709 12:27:31.772943  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78348.3us
I20260709 12:27:31.752686 19337 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.780764 19325 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.757772 19284 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:31.793079 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:31.774936 19267 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.806344 19329 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:31.807008 19329 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:31.807471 19329 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:31.776744 18715 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.791853 19337 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:31.785030 19331 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:27:31.785308 19135 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:27:31.785756 19327 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 12:27:31.787582 19120 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:31.777858 19335 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:27:31.819960 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3315) took 1524 ms. Trace:
I20260709 12:27:31.814860 19267 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:31.822419 19235 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:31.797071 19325 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.826941 19255 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.811291 19345 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:31.782668 19347 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:27:31.831957 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:31.800493 19284 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.856568 19284 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:31.832937 10095 rpcz_store.cc:276] 0709 12:27:30.295895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.172674 (+876779us) service_pool.cc:224] Handling call
0709 12:27:31.182058 (+  9384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.182803 (+   745us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:31.214997 (+ 32194us) log.cc:844] Serialized 14 byte log entry
0709 12:27:31.303959 (+ 88962us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:31.304427 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:27:31.304556 (+   129us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:31.313172 (+  8616us) log.cc:844] Serialized 14 byte log entry
0709 12:27:31.313578 (+   406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.313662 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:31.819752 (+506090us) raft_consensus.cc:1685] finished
0709 12:27:31.819771 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:31.819901 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":106086,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":82,"raft.queue_time_us":347,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6639,"spinlock_wait_cycles":42659840,"thread_start_us":89021,"threads_started":2,"wal-append.queue_time_us":156588}
I20260709 12:27:31.844417 19158 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:31.846724 19325 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.849068 19322 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.885869 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.851394 19337 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.852231 19345 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:31.853402 19279 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.503s	user 0.008s	sys 0.002s
I20260709 12:27:31.819607 19135 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:31.864748 19324 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.841159 19235 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.867928 19147 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.871636 10022 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:31.874014 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.876013 19256 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.840684 19267 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.885408  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:31.900239 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.892179 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:31.912344 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:31.904925 10017 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 12. Preceding OpId from leader: term: 29 index: 14. (index mismatch)
I20260709 12:27:31.908020 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.913873 19344 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:31.907998 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.914793 10022 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:31.908725 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.915452 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.912469 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1942) took 1022 ms. Trace:
W20260709 12:27:31.914109 18986 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.713s	user 0.014s	sys 0.000s
I20260709 12:27:31.916416 19147 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:31.900592 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.917219 10019 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:31.914106 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.916139 10669 rpcz_store.cc:276] 0709 12:27:30.890448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.947459 (+ 57011us) service_pool.cc:224] Handling call
0709 12:27:31.912216 (+964757us) spinlock_profiling.cc:243] Waited 19.1 ms on lock 0x7b54007d7b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.912317 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43956992}
I20260709 12:27:31.919327 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.920178 10017 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:31.901566 19118 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.907941 19279 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.919888 19345 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.921087 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:31.911059  9948 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:31.924787 10018 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:31.925510  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:31.936474 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3320) took 1623 ms. Trace:
I20260709 12:27:31.935918 10096 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1691) took 1676 ms. Trace:
I20260709 12:27:31.924994  9947 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:31.946023 10096 rpcz_store.cc:276] 0709 12:27:30.259530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.042742 (+783212us) service_pool.cc:224] Handling call
0709 12:27:31.537183 (+494441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.556334 (+ 19151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.935695 (+379361us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54002fdd50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.935851 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1136486784}
I20260709 12:27:31.934568 10671 raft_consensus.cc:1240] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:31.936980  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:31.939543 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:31.944541  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:31.951754 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:31.952656 10669 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
W20260709 12:27:31.927372 19338 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
I20260709 12:27:31.955866 10017 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:27:31.945035 10099 rpcz_store.cc:276] 0709 12:27:30.312863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.190369 (+877506us) service_pool.cc:224] Handling call
0709 12:27:31.365947 (+175578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:31.366752 (+   805us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.936289 (+569537us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b5400305550. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:31.936417 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":362544384}
I20260709 12:27:31.939109  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:31.926468 19235 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:31.925108 19250 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
I20260709 12:27:31.968425 19250 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.970896 10668 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:31.970965 10674 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 21.
I20260709 12:27:31.943053 19335 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:27:31.930074 19339 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:27:31.974644 19158 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:31.964476 18986 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:31.977679 18986 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:31.978700 18986 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.982419  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:31.937150 19248 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:31.929965 19314 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.935669 19279 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:32.015686 19250 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:32.017881 19256 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:31.926115 19118 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:32.023057 19337 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:31.939941 19345 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:32.037989 19339 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:31.931684 19267 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:32.038483 19339 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:32.039373 19339 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.041754 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2907) took 1658 ms. Trace:
I20260709 12:27:32.042392 19339 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:32.042290 10024 rpcz_store.cc:276] 0709 12:27:30.382906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:30.383059 (+   153us) service_pool.cc:224] Handling call
0709 12:27:31.133964 (+750905us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.148143 (+ 14179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.041403 (+893260us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54001f6b50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:32.041625 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1726206720}
W20260709 12:27:32.061555  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.062387 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 122922us
I20260709 12:27:32.062997 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:32.066536  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 127493us
I20260709 12:27:32.068114  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:32.076010  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:32.076611 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:32.077072 19349 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:32.077497 19349 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.077858 19349 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:32.012653 19152 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:32.071877 19318 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.582s	user 0.004s	sys 0.000s
I20260709 12:27:32.083721 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4399) took 1761 ms. Trace:
I20260709 12:27:32.084067 10098 rpcz_store.cc:276] 0709 12:27:30.322008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.218232 (+896224us) service_pool.cc:224] Handling call
0709 12:27:31.218594 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:31.218958 (+   364us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:31.218979 (+    21us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:31.239984 (+ 21005us) log.cc:844] Serialized 26 byte log entry
0709 12:27:31.348381 (+108397us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:31.348420 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:31.348485 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.046015 (+697530us) spinlock_profiling.cc:243] Waited 6.03 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:32.083441 (+ 37426us) raft_consensus.cc:1685] finished
0709 12:27:32.083463 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.083665 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1173,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":10037,"spinlock_wait_cycles":13875968,"thread_start_us":51380,"threads_started":1,"wal-append.queue_time_us":115767}
I20260709 12:27:32.087302 19319 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:32.088085 19345 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.088614 19345 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.091156 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:32.092363 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:32.095211 19345 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:32.095628 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.096247 10096 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
W20260709 12:27:32.109206  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.115556 10096 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:27:32.115377 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.116541 10090 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:32.117342 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.117880 10090 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:32.118393 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.119027 10096 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:32.122002 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:32.122637  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46713.1us
I20260709 12:27:32.123706 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:32.126336 19339 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:32.131047 19333 consensus_queue.cc:1059] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:32.132150 19339 raft_consensus.cc:3055] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:32.132516 19339 raft_consensus.cc:740] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:32.133343 10099 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:32.133311 19339 consensus_queue.cc:260] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.134339  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:32.134666 19339 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:32.135290  9941 raft_consensus.cc:2410] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:27:32.135411 10092 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:32.134428 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.136329 10094 raft_consensus.cc:2393] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:32.137696 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:32.138782 19140 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:32.139429  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:32.139487 19147 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:32.140444 19147 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:32.140770 19147 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:32.141408  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:32.141618 19147 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.138459 10095 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:32.144336  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.145099  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:32.144191 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:32.150204 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.150363 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:32.151261 10096 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:32.157413 10093 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:32.157673 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:32.125312 19250 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:32.158282 10671 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:32.165840  9945 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:32.165871  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:32.165172  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:32.122514 19164 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:32.166931 19314 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:32.170300 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:27:32.170950 10025 raft_consensus.cc:606] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:32.171296 10025 raft_consensus.cc:614] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:32.171751 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:32.175087 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:32.162088  9949 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:32.176700 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:32.179035 18760 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:32.179718 19164 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:32.180117 19164 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.180481 19164 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:32.181047 19140 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:32.181697 19140 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:32.182767 19140 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.122706 19279 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.063s
I20260709 12:27:32.186126 19305 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:32.189263 19305 consensus_queue.cc:1243] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 21.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:32.194016 19289 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:32.196007 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:32.196825 19289 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:32.198648 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:32.199744 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55636.9us
I20260709 12:27:32.206139  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:32.199841 19305 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.207953 19248 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:32.149940 19244 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 12:27:32.208843 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:32.203552 19344 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:32.209550 19244 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:32.209681 19344 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:32.210026 19244 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:32.204030 19247 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:32.207705 19305 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.210533 19247 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.210119 19344 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:32.210940 19244 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.143283 19163 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.085s
I20260709 12:27:32.212994 19163 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:32.213336 19163 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:32.213996  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:32.210978 19247 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:32.217000 19305 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:32.219887  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:32.220650  9945 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:32.227005 19147 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:32.227103  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:32.228075  9941 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:32.230764 19155 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:32.201269 19164 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:32.254783 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56218.9us
I20260709 12:27:32.179162  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:32.259364 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:32.260550 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.261266 10093 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:32.266021  9945 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:32.260536 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:32.280905  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 101821us
I20260709 12:27:32.281769  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:32.282647  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:32.231221 19155 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:32.128588 19345 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.131s
I20260709 12:27:32.164899 19250 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.284917 19337 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:32.286423 10098 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:27:32.286705  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:32.289170  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:32.287531 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55508.2us
I20260709 12:27:32.293589  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:32.306792  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:32.310952 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:32.288394 19287 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:32.285610 19305 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.314777 10091 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:27:32.315538 19118 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:32.283282 19280 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:32.316073 19118 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:32.316406 19280 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:32.317129 19118 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:32.317569 19280 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.310653 18837 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.009s
I20260709 12:27:32.258755 19284 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.320609 18837 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:32.321094 18837 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:32.320700 19284 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.288959 19234 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:32.294636 19247 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.295758 19344 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:32.214166 19163 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.312472 10094 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:32.300289 19250 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:32.315181 19305 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.267235 19289 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:32.324623  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:32.308382 18895 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:32.292505 19164 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:32.292733 19321 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:32.287869  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 217397us
I20260709 12:27:32.352025  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:32.353127 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.353634  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:32.333897 10098 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
mode: GRACEFUL
new_leader_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:33250
I20260709 12:27:32.354972 10098 raft_consensus.cc:606] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Received request to transfer leadership to TS cd983415389444e292b2330ef630850a
I20260709 12:27:32.321976 18837 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:32.337567  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:32.356993  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:32.358151 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.323020 19333 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:32.361871  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:32.372318  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:32.378764  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:27:32.380630  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:32.381632  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:27:32.349967 19329 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.007s
I20260709 12:27:32.388437 19329 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.390120 10671 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:32.333853 19339 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.059s
W20260709 12:27:32.398092 19354 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 12:27:32.400399 19353 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:27:32.404948 19270 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:32.405567 19270 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.406026 19270 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:32.411743 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 2929) took 1167 ms. Trace:
I20260709 12:27:32.412307 10018 rpcz_store.cc:276] 0709 12:27:31.244286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.250929 (+  6643us) service_pool.cc:224] Handling call
0709 12:27:32.411547 (+1160618us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b54001388d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:32.411669 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529311488}
I20260709 12:27:32.418802 19284 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:32.331967 19256 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.091s
I20260709 12:27:32.291659 19155 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.425427 19155 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:32.429096 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.430761 10094 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
W20260709 12:27:32.432525  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 60339.7us
I20260709 12:27:32.440333 18837 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:32.443607 19359 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:27:32.445832 19247 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:32.439113  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 85581.6us
I20260709 12:27:32.366900 19314 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:32.448577  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:32.366317 19317 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:32.356848 19158 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:32.370101 19234 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:32.463205  9990 leader_election.cc:400] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25.
I20260709 12:27:32.463753  9990 leader_election.cc:403] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:27:32.468890  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:32.394272 19135 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.462407 19319 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:32.475564  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:32.465214 19234 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.493607 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.480494 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:32.497922 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:32.498525 10097 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:32.504665  9914 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:32.470322 19314 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25.
I20260709 12:27:32.508033 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.512735 10092 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:32.510377 10095 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:32.474690 18895 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:32.464927 19256 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.034s
I20260709 12:27:32.499081 19234 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:32.539209  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.523849  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:32.527455 19339 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:32.526950 19135 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:32.550001 19329 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:32.536556 19360 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:27:32.534691 19314 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:32.564050  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.565929 10097 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:32.566792 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:32.566663 19120 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:32.581787 10673 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:32.583889 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:32.584708 10674 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:32.588274 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:32.580845 19120 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:32.599447 19120 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:32.611645 19289 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:32.615208 10642 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:32.616074 19289 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:32.616621 19235 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:32.616896 19289 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:32.616137 19247 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.617617 19247 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.620530 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:32.620842 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:32.621774 10097 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:32.623253 10675 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:32.617033 19235 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:32.639232  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.641515 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.642248 10097 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 31.
I20260709 12:27:32.593762 19344 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:32.595510 19305 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:32.617962 19289 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.646828 10675 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:32.647215 10676 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:32.648288 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:32.655174 19345 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.005s
W20260709 12:27:32.666054 19357 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:27:32.666102 19163 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:32.684384 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 69477.5us
I20260709 12:27:32.681061 19337 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:32.681571 19328 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.710932 19342 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.711544 19342 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.714062 19314 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:32.714671 19314 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.714977 19250 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.715380 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:32.716521 19344 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:32.717002 19314 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:32.717245 10668 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:32.717355 19344 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:32.717597 19314 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.746202 19207 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:32.746903 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:32.745116 19314 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.718041 19344 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:32.718294 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:32.718497 19152 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:32.750512 19344 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.754755 19152 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:32.727449 19321 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:32.755712 19152 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.756012 19321 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.718808 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:32.718995 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:32.758279  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:32.722937 19244 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:32.723423 19355 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.443s	user 0.002s	sys 0.000s
I20260709 12:27:32.727543 19247 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:32.729187 19328 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:32.735013 19349 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.657s	user 0.004s	sys 0.008s
I20260709 12:27:32.737092 19270 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:32.738065 10674 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:32.681056 19345 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
I20260709 12:27:32.683385 19235 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.741209 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1688) took 2506 ms. Trace:
I20260709 12:27:32.785308  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:32.743345 10020 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:27:32.783854  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 260079us
I20260709 12:27:32.741782 19234 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:32.759059  9947 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:32.759768 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 57322.8us
I20260709 12:27:32.760509 10675 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 16. (term mismatch)
I20260709 12:27:32.788774 10675 pending_rounds.cc:85] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 15
I20260709 12:27:32.789129 10675 pending_rounds.cc:107] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 24.16
I20260709 12:27:32.789412 10675 raft_consensus.cc:2889] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:32.758512 10019 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:32.785274 10091 rpcz_store.cc:276] 0709 12:27:30.234993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.040564 (+805571us) service_pool.cc:224] Handling call
0709 12:27:31.367107 (+326543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.315219 (+948112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.739885 (+424666us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b5400305f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:32.740982 (+  1097us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400305f50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:32.741134 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2929101952}
I20260709 12:27:32.765511  9946 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:27:32.750384 19339 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:32.701967 18895 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:32.781513  9945 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:32.717052 19250 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:32.759361 19342 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:32.794855  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:32.802359  9943 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:32.793772 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:32.813136  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:32.814077  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:32.817498 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:32.782545 19152 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:32.827226  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 351742us
I20260709 12:27:32.824800  9941 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:32.825562  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3434) took 1270 ms. Trace:
I20260709 12:27:32.818246 10096 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:32.831207  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:32.836562  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:32.837276 10097 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:27:32.831036 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:32.838271  9990 leader_election.cc:400] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26.
I20260709 12:27:32.838908  9990 leader_election.cc:403] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:32.839107 18760 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:32.839862 18760 consensus_queue.cc:1243] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 12:27:32.841871  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.843786 10097 pending_rounds.cc:85] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 15
I20260709 12:27:32.844192 10097 pending_rounds.cc:107] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:27:32.844497 10097 raft_consensus.cc:2889] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:32.851279  9944 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:32.848179 10643 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:32.829172  9942 rpcz_store.cc:276] 0709 12:27:31.555558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.569067 (+ 13509us) service_pool.cc:224] Handling call
0709 12:27:32.419616 (+850549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.420039 (+   423us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.420080 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.420708 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 12:27:32.680155 (+259447us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:32.680228 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.680352 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:32.762466 (+ 82114us) raft_consensus.cc:1685] finished
0709 12:27:32.762483 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:32.798880 (+ 36397us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b54000b6b50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:32.825477 (+ 26597us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3214,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":119,"spinlock_wait_cycles":1955015936,"thread_start_us":258980,"threads_started":1,"wal-append.queue_time_us":270277}
I20260709 12:27:32.833137 19319 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:32.923939 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 92969.3us
W20260709 12:27:32.934872 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.826606 19333 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:27:32.943779  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105653us
I20260709 12:27:32.979848  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:32.826328 19158 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:32.944155 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 96061.6us
W20260709 12:27:32.983174  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.826735 19147 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:32.880952 10019 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:27:32.782895 19349 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:32.856067 18760 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.922857 18986 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:32.955749 19247 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:32.949991 19321 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:32.946311 19305 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:32.833451 19332 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.126s
I20260709 12:27:32.945848 19367 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.955160 19374 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26.
I20260709 12:27:32.826742 19337 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.167s
I20260709 12:27:32.926360 19250 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.080s
I20260709 12:27:32.991820 19158 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:33.025360  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:32.852629 19339 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.158s
I20260709 12:27:33.009245 19342 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:33.005971 19239 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:32.991496 19314 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:33.003197 19333 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:33.013420 18837 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:32.995508 19147 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:33.023481 19247 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:33.042362 19158 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:33.068512  9837 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 24, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 24 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:27:33.071483 19366 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:27:33.072199 19333 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.075726 19374 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:33.060284 19376 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:33.013876 18760 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.053558 19314 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:33.055174 19147 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:33.012940 18895 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:33.093469  9837 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:33.050952 19239 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:33.142187  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 253366us
I20260709 12:27:33.100044 19276 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.137975  9837 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 17 to 22, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:33.053551 19342 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.093127 19376 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:33.041402 19367 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.098004 19270 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:33.102013 19120 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:33.167683 19135 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.611s	user 0.005s	sys 0.005s
I20260709 12:27:33.168222 19120 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:33.168990  9837 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 19 to 22, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 22 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:33.170625  9837 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 13 to 26, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 26 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:27:33.174541 19382 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:27:33.177138 19369 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.465s	user 0.000s	sys 0.003s
I20260709 12:27:33.183393 19349 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:33.185837 19235 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:33.192425 19164 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:33.193076 19164 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.060767 19319 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:33.194517 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.200920  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:33.201761  9941 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:33.210726  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:33.212718  9943 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:33.214659 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:33.214879  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:33.219112  9837 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 16 to 22, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:33.221035  9837 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 15 to 19, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 19 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:33.148989 19328 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:33.206794 19164 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:33.099598 19374 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.206434 19247 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:33.227034  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 65120.2us
W20260709 12:27:33.234932 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.269905  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:33.163139 19276 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.230808 19317 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:33.249471 19239 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:33.251030 19270 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.283152 19270 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.285521 19270 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:33.230963 18837 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:33.254140 10670 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:33.234580  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.272267 19164 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.253562 19328 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:27:33.244252  9837 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 16 to 17, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 17 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:33.275079 19314 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:33.287375 19317 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:33.332285  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 352534us
I20260709 12:27:33.333125 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:33.333702 10096 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:33.287576 19345 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 12:27:33.330839  9837 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 20, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 20 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.296025 18760 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:33.356854 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:27:33.371546 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:33.372622 10094 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:27:33.344031 19155 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.004s	sys 0.009s
I20260709 12:27:33.327811 19317 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:33.375919  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:33.335721 19332 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.293431 18837 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:33.374382 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:33.383960 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:33.271996 19376 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:33.391489 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 173060us
I20260709 12:27:33.397500 10096 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:33.420120 10025 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:33.389041  9837 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 23 to 24, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 24 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:33.300626 19164 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.428803 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:33.426370 10096 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:27:33.430274  9837 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 25, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 25 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:27:33.384510 19155 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.005s	sys 0.010s
I20260709 12:27:33.444037 10096 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:27:33.388630 19207 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:33.429136 19339 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.431998 18941 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:33.426857 19163 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:33.437856 19250 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:33.474032 19371 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.626s	user 0.004s	sys 0.000s
I20260709 12:27:33.497213  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:33.498139  9942 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:33.510514  9837 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 19, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 19 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:33.426043 19332 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.515223 10018 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:33.520473  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:33.521144  9944 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:33.525089 10091 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:33.498687 19163 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.475193 19396 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.070s
I20260709 12:27:33.534660  9837 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 21 to 22, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 22 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:33.532656 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:33.536640 10019 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:33.501585 19207 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:33.513326 19256 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.484983 19339 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:33.542373 10017 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:27:33.474993 19398 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.102s
I20260709 12:27:33.475157 19394 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.073s
I20260709 12:27:33.491984 18941 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:33.531635 19402 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:33.552564  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:33.564477  9947 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:33.543604 19391 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:27:33.522580 19367 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:33.666334  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:33.532079 19383 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 12:27:33.511036 19250 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.596971  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:33.537817 19244 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.770s	user 0.013s	sys 0.000s
I20260709 12:27:33.638394 10091 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:27:33.574719 19373 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:33.586923 19384 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.300s	user 0.001s	sys 0.001s
I20260709 12:27:33.647086 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:33.586752  9837 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 17 to 20, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 20 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:33.640465 19397 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:33.579327 19319 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.524636 19239 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.059s
I20260709 12:27:33.676064  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:33.788033  9940 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:33.696381  9948 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
W20260709 12:27:33.698340 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:33.704682 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1930) took 1138 ms. Trace:
I20260709 12:27:33.697716 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:33.731858  9949 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:33.697685 10096 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:33.789315 10670 rpcz_store.cc:276] 0709 12:27:32.567297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.567417 (+   120us) service_pool.cc:224] Handling call
0709 12:27:33.704484 (+1137067us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54005267d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:33.704608 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276249984}
I20260709 12:27:33.598426 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:33.708169  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:33.543798 19374 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:33.574131 19256 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.759310  9943 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:33.774487 10017 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:33.640614 19402 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:33.671679 19414 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:27:33.672443 19392 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.337s	user 0.001s	sys 0.001s
W20260709 12:27:33.668978 19381 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.530s	user 0.002s	sys 0.000s
I20260709 12:27:33.688158 19220 raft_consensus.cc:993] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: : Instructing follower cd983415389444e292b2330ef630850a to start an election
I20260709 12:27:33.780555 10019 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:33.663054  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:33.688004 18941 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:33.654315 19284 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:33.809123  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 101046us
I20260709 12:27:33.794319 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1717) took 1104 ms. Trace:
I20260709 12:27:33.809132  9837 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 19 to 22, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 22 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:33.812566 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:33.968248 10021 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:33.725942 19342 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:33.974552 10023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.011s	sys 0.019s
W20260709 12:27:33.974958 10023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.011s	sys 0.019s
W20260709 12:27:33.663268 19417 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:27:33.676170 19147 consensus_meta.cc:287] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.000s
I20260709 12:27:33.815047 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:33.690654 19135 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:33.659174 18837 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:33.912945  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:33.686697 19398 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.148s
W20260709 12:27:33.977181 19391 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.167s	user 0.000s	sys 0.000s
W20260709 12:27:33.929649  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.004s	sys 0.018s
W20260709 12:27:34.112067  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.004s	sys 0.018s
I20260709 12:27:33.811589 19239 consensus_queue.cc:1243] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 24.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
W20260709 12:27:33.966424  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48242.7us
I20260709 12:27:33.966408 10090 rpcz_store.cc:276] 0709 12:27:32.689527 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.689759 (+   232us) service_pool.cc:224] Handling call
0709 12:27:32.690186 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.789588 (+ 99402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.789637 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.818374 (+ 28737us) log.cc:844] Serialized 26 byte log entry
0709 12:27:33.046782 (+228408us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:33.046895 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.768437 (+721542us) spinlock_profiling.cc:243] Waited 99 ms on lock 0x7b540034add4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:33.768484 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.768535 (+    51us) raft_consensus.cc:1685] finished
0709 12:27:33.768548 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.794235 (+ 25687us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6903,"raft.queue_time_us":4879,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":19917,"spinlock_wait_cycles":227605248,"thread_start_us":100410,"threads_started":1,"wal-append.queue_time_us":324445}
I20260709 12:27:33.808616 19367 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:33.979669 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4144) took 1209 ms. Trace:
I20260709 12:27:33.650838 19344 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:33.988235 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1636) took 1329 ms. Trace:
I20260709 12:27:34.122200 10676 rpcz_store.cc:276] 0709 12:27:32.658756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.666921 (+  8165us) service_pool.cc:224] Handling call
0709 12:27:32.667300 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.688636 (+ 21336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.688670 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.698086 (+  9416us) log.cc:844] Serialized 26 byte log entry
0709 12:27:32.986842 (+288756us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:32.986887 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.986952 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.987958 (+1001006us) raft_consensus.cc:1685] finished
0709 12:27:33.987982 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.988147 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9721,"raft.queue_time_us":18115,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9318,"thread_start_us":255657,"threads_started":1,"wal-append.queue_time_us":446263}
I20260709 12:27:33.918696 19372 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:34.005720 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 190783us
I20260709 12:27:33.787340 19328 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.181s
I20260709 12:27:33.926134 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:34.055634 18760 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:27:34.015049 10017 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:33.929886 18895 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:33.991317 19152 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.157s	user 0.010s	sys 0.003s
I20260709 12:27:33.778017 19373 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:33.793509 19397 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:33.992954 19235 consensus_meta.cc:287] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.807s	user 0.013s	sys 0.000s
I20260709 12:27:34.055585 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4146) took 1275 ms. Trace:
I20260709 12:27:34.070086 19339 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:33.810065 19319 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:34.056757 10022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.682s	user 0.008s	sys 0.012s
W20260709 12:27:34.018615 19158 consensus_meta.cc:287] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.926s	user 0.010s	sys 0.003s
I20260709 12:27:34.061273  9837 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.005249 18715 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:33.910336 19411 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.270s
I20260709 12:27:34.006151 19135 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:33.904572 19421 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.281s
I20260709 12:27:34.202009 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1641) took 1004 ms. Trace:
I20260709 12:27:34.012321 19147 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.202562 19373 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:34.181581 19399 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:27:34.202435 10677 rpcz_store.cc:276] 0709 12:27:33.197483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.215986 (+ 18503us) service_pool.cc:224] Handling call
0709 12:27:33.229087 (+ 13101us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:33.292846 (+ 63759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.292877 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:33.293325 (+   448us) log.cc:844] Serialized 26 byte log entry
0709 12:27:33.450226 (+156901us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:33.450268 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.703250 (+252982us) spinlock_profiling.cc:243] Waited 63.4 ms on lock 0x7b5400611a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:33.703313 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:34.201711 (+498398us) raft_consensus.cc:1685] finished
0709 12:27:34.201733 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.201941 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":822,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":4622,"spinlock_wait_cycles":145836288,"thread_start_us":152544,"threads_started":1,"wal-append.queue_time_us":771423}
W20260709 12:27:34.094208 19404 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.543s	user 0.002s	sys 0.002s
I20260709 12:27:34.204684 10021 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:34.205466 19429 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:34.206210 19429 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.096323 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4429) took 1615 ms. Trace:
I20260709 12:27:34.185257 19158 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.206776 19429 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.206911 10093 rpcz_store.cc:276] 0709 12:27:32.480748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.481742 (+   994us) service_pool.cc:224] Handling call
0709 12:27:32.503288 (+ 21546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.503732 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.503774 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.579515 (+ 75741us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b2c00115cd8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:32.594888 (+ 15373us) log.cc:844] Serialized 26 byte log entry
0709 12:27:32.839007 (+244119us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:32.839064 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.839154 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:33.668288 (+829134us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:34.095998 (+427710us) raft_consensus.cc:1685] finished
0709 12:27:34.096034 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.096245 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":95,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":15800,"spinlock_wait_cycles":28513920,"thread_start_us":195219,"threads_started":1,"wal-append.queue_time_us":449801}
I20260709 12:27:34.208545 19147 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:34.106789 19234 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:34.056440 19287 raft_consensus.cc:1081] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Signalling peer cd983415389444e292b2330ef630850a to start an election
W20260709 12:27:34.102108 19317 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.676s	user 0.008s	sys 0.001s
I20260709 12:27:34.112210 10019 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:34.209210 19429 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.214442 19425 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.005s
I20260709 12:27:34.220343 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4148) took 1332 ms. Trace:
I20260709 12:27:34.115728 10023 rpcz_store.cc:276] 0709 12:27:32.770247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.770390 (+   143us) service_pool.cc:224] Handling call
0709 12:27:32.787955 (+ 17565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:32.788215 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:33.252635 (+464420us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.979292 (+726657us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:33.979355 (+    63us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:33.979409 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.979484 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:33.979613 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3508,"prepare.queue_time_us":352422,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":69,"raft.queue_time_us":29259,"raft.run_wall_time_us":98376,"thread_start_us":1155906,"threads_started":2,"wal-append.queue_time_us":815037}
I20260709 12:27:33.988926 19284 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.133179 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:34.155608 10024 rpcz_store.cc:276] 0709 12:27:32.780247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.780386 (+   139us) service_pool.cc:224] Handling call
0709 12:27:32.780731 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.781083 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.781150 (+    67us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.781705 (+   555us) log.cc:844] Serialized 26 byte log entry
0709 12:27:33.093817 (+312112us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:33.093878 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.093976 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:34.055352 (+961376us) raft_consensus.cc:1685] finished
0709 12:27:34.055372 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.055517 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14752,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":19217,"thread_start_us":257423,"threads_started":1,"wal-append.queue_time_us":469875}
I20260709 12:27:33.768028 19207 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.097605 19256 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:34.180634 10022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.009s	sys 0.013s
W20260709 12:27:34.186884 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 395374us
I20260709 12:27:34.131814 18941 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:34.190773 10018 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:27:34.228547 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.178140 18895 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.195003  9837 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 20 to 22, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 22 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:34.196097 19250 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:34.198102 19319 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.138234 19367 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.199164 19411 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:34.136685 19344 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:34.199537 19426 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 12:27:34.189329 19152 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:34.069495 19402 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.141048 19372 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.200616 19135 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.208536 19158 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:34.112464  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 749925us
I20260709 12:27:34.258565  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:34.118783  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 147354us
I20260709 12:27:34.223270  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:34.220763 10020 rpcz_store.cc:276] 0709 12:27:32.888101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.888601 (+   500us) service_pool.cc:224] Handling call
0709 12:27:32.889023 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.948128 (+ 59105us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.948175 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.948757 (+   582us) log.cc:844] Serialized 26 byte log entry
0709 12:27:33.277801 (+329044us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:33.277842 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.277912 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:34.220153 (+942241us) raft_consensus.cc:1685] finished
0709 12:27:34.220174 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:34.220288 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58606,"raft.queue_time_us":134970,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3667,"thread_start_us":258357,"threads_started":1,"wal-append.queue_time_us":1210235}
I20260709 12:27:34.227173 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:34.259681  9940 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
W20260709 12:27:34.227468 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 94399.1us
I20260709 12:27:34.176375 19332 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:34.230237 19247 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.013s
I20260709 12:27:34.244297 19256 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.230796  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:34.232378  9837 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 14 to 20, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 20 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:34.215705 19239 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.249890  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:34.249995 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.266067 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:34.253876  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 417951us
I20260709 12:27:34.217435 19397 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.269760  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:34.272120  9948 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:34.268282 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:34.264163 19329 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:34.277261 10643 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:34.273864 19423 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:34.277664 19239 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.267966 19250 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.011s
I20260709 12:27:34.275709 19328 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:34.225072 19317 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.276932 19329 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.279848 19239 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:34.279913 19396 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:34.280543 19396 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.280977 19396 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:34.281584 19329 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:34.285640 19434 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:34.253386 19439 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:34.265383 19247 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.004s	sys 0.013s
I20260709 12:27:34.282220 19329 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:34.243884 18941 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.303324 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:34.305423 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.307926 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.279925 19328 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.309974 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:34.311585 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.316190 10096 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 22.
I20260709 12:27:34.256939 19152 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.312664 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:34.313791 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:34.302632  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.196:36979 timed out after 1.596s (SENT) [suppressed 18 similar messages]
I20260709 12:27:34.309451 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.318118 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.272670 19287 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:34.291091 19319 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:34.330274 10094 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:34.330886 10676 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:34.331471 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.332228 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.332957 10096 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:34.334091 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:34.334800 10096 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:34.254612 19411 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.303694 19373 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:34.349056 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:34.342506 10642 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:34.301860 19440 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:34.345700 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.350446 10091 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:34.348109 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:34.349858 10092 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:34.351526 10099 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:34.301433 19434 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.272490 19256 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:34.355612  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.320425 19328 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:34.291301 19443 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.076s
I20260709 12:27:34.358561 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:34.363193 10677 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:34.361837  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:34.365829  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:34.342849 19287 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:34.353144 19440 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.352722 19319 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:34.395817 19317 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.402079 10099 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:34.409420  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:34.303016 19439 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.346531 19435 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:27:34.365361 19421 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:34.367744 19373 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:34.384516 19287 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:34.340463 19155 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:27:34.383562 19270 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:34.395823 19427 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:27:34.383832 19339 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:34.458931 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 187037us
W20260709 12:27:34.438886  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 147723us
I20260709 12:27:34.465329  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:34.446769 10096 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:34.455760 10676 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:34.385831 19440 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:34.400480 19441 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:34.436674 10092 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 32.
W20260709 12:27:34.467585 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 123746us
W20260709 12:27:34.468276 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 160382us
W20260709 12:27:34.486524  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 183915us
W20260709 12:27:34.511452  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46234us
I20260709 12:27:34.460907 18895 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.459051 19270 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.466953 19339 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:34.459944 19158 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:34.513916 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:34.514883  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:34.517076  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:34.521520  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:34.463883 19349 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.512949  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:34.544391  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:34.548594  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:34.512962 19443 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.471091 19373 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.517601 19367 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:34.561429 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 96762.3us
I20260709 12:27:34.557158  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:34.573464 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59624.9us
I20260709 12:27:34.436661 19421 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.534121 19339 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:34.523859 19239 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:34.576516 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:34.585918 10676 raft_consensus.cc:2410] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 18.
I20260709 12:27:34.535449 19270 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:34.581458  9947 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
W20260709 12:27:34.592087  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44699.7us
I20260709 12:27:34.587146 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:34.599329  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:34.559244 19455 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:34.564721 19349 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.602638 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:34.601934 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:34.611495 10673 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:34.578768 19317 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:34.593647 19367 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:34.608903 18941 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:34.614058 19455 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.534790 19319 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.607509 19425 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:34.568076 19443 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.626463 19367 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:34.635114 19450 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:34.655535 19466 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.656113 19450 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:34.656610 19450 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:34.656463 19466 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.658157 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.646854 19411 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.644016 19434 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:34.645617 19367 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:34.599943 19239 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.663895 19239 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.656545 19455 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:34.668169 19451 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:27:34.668915 19446 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:34.669539 19446 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:34.670492 19446 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.672503 19443 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.674561 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.675308 10099 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:34.676707  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:34.679698 19373 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.681409 19466 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:34.635386 19435 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:34.682371 19435 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:34.682904 19435 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:34.684085 19421 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:34.655534 19256 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.640357 19464 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:34.684846 19421 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:34.667037 19448 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:34.685896 19448 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.642825 19425 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.686153 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.689888 19425 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:34.691988 19448 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:34.685379 19421 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:34.693457 19421 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.695305 19439 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:34.696273 19424 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:34.699414 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.631929 19158 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:34.754142 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.755265 10099 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
W20260709 12:27:34.767858  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 168626us
I20260709 12:27:34.772289 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:34.773154 10096 raft_consensus.cc:2368] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 31. Current term is 32.
I20260709 12:27:34.773417 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.774341 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.776181  9940 raft_consensus.cc:1217] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Deduplicated request from leader. Original: 21.17->[23.18-27.20]   Dedup: 23.18->[25.19-27.20]
I20260709 12:27:34.776193 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.776806  9940 pending_rounds.cc:85] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 18
I20260709 12:27:34.777508  9940 pending_rounds.cc:107] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 24.19
I20260709 12:27:34.777827  9940 raft_consensus.cc:2889] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:34.758868 19447 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.355s	user 0.000s	sys 0.001s
W20260709 12:27:34.943826  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 103557us
I20260709 12:27:34.821647 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:34.813788  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:34.828377 19339 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.954291  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:34.830876 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:34.782322 19449 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.359s	user 0.001s	sys 0.000s
W20260709 12:27:34.821336 19474 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:27:34.684856 19256 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:34.935083  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:34.957144  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 408606us
I20260709 12:27:34.813807 19454 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:27:34.832397 19367 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:34.893752  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:34.991053  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:34.815526 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:34.991889 10094 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
W20260709 12:27:34.812474 19456 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 12:27:34.656698 19349 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:34.996245  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:34.791926 10096 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:34.834339 19158 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.758040 19328 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:34.968739  9946 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:34.858415 19314 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:34.943540  9947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132"
dest_uuid: "cd983415389444e292b2330ef630850a"
 from {username='slave'} at 127.0.0.1:41126
I20260709 12:27:35.016660  9947 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:35.017063  9947 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:34.790797 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:35.004244  9943 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:34.959211 19443 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:27:35.005017 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 183455us
I20260709 12:27:35.006999 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:34.967504 19239 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:34.989537 19256 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.020017 10668 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:35.028991  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:35.042515 10094 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:35.051127  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:35.047859 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:35.026000 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:34.957913 19435 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.058270 10677 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:35.027042  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:35.028391 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:34.858883 19450 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.058913 10670 raft_consensus.cc:2376] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 23. Re-sending same reply.
I20260709 12:27:35.049038 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.061283  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:35.062132 10673 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:27:35.011656 19329 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:35.114009 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1645) took 1324 ms. Trace:
I20260709 12:27:35.078161  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:35.031639 19448 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:35.125630  9947 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.031906 19256 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:35.130051 10021 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:27:35.114439 10675 rpcz_store.cc:276] 0709 12:27:33.789887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.790645 (+   758us) service_pool.cc:224] Handling call
0709 12:27:33.790985 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.058887 (+267902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.058919 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.063214 (+  4295us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.225515 (+162301us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:34.226395 (+   880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.226525 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.113770 (+887245us) raft_consensus.cc:1685] finished
0709 12:27:35.113789 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.113951 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":270085,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":375158,"threads_started":2,"wal-append.queue_time_us":278111}
I20260709 12:27:35.123135 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:35.108711 19396 consensus_meta.cc:287] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.827s	user 0.006s	sys 0.008s
I20260709 12:27:35.134392 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.136315 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:35.141110 10022 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:35.136874  9947 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.144032 19367 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.144449  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:35.146713 19337 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:35.147327 19337 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.147493  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:35.020869 19314 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.113217 19471 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.138731  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:35.154008 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:35.157275 10098 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.009s	sys 0.035s
W20260709 12:27:35.157894 10098 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.010s	sys 0.036s
I20260709 12:27:35.157446 10024 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 31.
I20260709 12:27:35.043184 19239 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.138038 10025 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:27:35.138943 19440 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.622s	user 0.010s	sys 0.000s
I20260709 12:27:35.164140  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:35.126083 19454 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.166365 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:35.179889 10021 raft_consensus.cc:2410] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:35.175976 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.149623 19337 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.150584 19339 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.185240 10022 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:35.155547 19396 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.188817 10675 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:27:35.190021 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.191454 10018 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:27:35.165587 19118 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:35.166031 19425 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:35.204546 10025 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:35.186553 19421 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:35.222927 10669 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:35.230221 10022 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:27:35.210289 19118 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:35.232411 19396 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:35.233403 19495 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:35.237170 19421 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:35.239677 19477 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:27:35.259375  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:35.312655  9947 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:35.216361 19270 consensus_meta.cc:287] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.608s	user 0.007s	sys 0.003s
I20260709 12:27:35.156463 19448 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.189810 19445 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:35.231163 19475 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.140s	user 0.000s	sys 0.001s
I20260709 12:27:35.187690 19484 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:35.183190 19454 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.309238 19118 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:35.183374 19440 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.323768 19445 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:35.327443 19379 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 2.054s	user 0.005s	sys 0.002s
I20260709 12:27:35.328246 19445 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.328918 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1931) took 2761 ms. Trace:
W20260709 12:27:35.316287 19287 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.848s	user 0.012s	sys 0.000s
I20260709 12:27:35.162396 19471 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.324187 19396 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.330694 19287 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.331275 19396 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.332046 19163 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.332263 19471 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.332940 19287 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.332984 10093 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:35.329385 19440 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.329370 10671 rpcz_store.cc:276] 0709 12:27:32.567722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.568551 (+   829us) service_pool.cc:224] Handling call
0709 12:27:32.568977 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:32.569258 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.569296 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:32.569840 (+   544us) log.cc:844] Serialized 26 byte log entry
0709 12:27:32.770967 (+201127us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:32.771018 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:32.771108 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.325555 (+2554447us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:35.328581 (+  3026us) raft_consensus.cc:1685] finished
0709 12:27:35.328607 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.328837 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27044224,"thread_start_us":176375,"threads_started":1,"wal-append.queue_time_us":625810}
I20260709 12:27:35.311911 19495 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.341890 19495 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.342831 19287 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:35.334100  9990 leader_election.cc:400] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 31. Current term is 32.
I20260709 12:27:35.344689  9990 leader_election.cc:403] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Cancelling election due to peer responding with higher term
I20260709 12:27:35.353502  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:35.354661  9947 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:35.356899  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:35.343474 19287 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.378859  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:35.333374 19396 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.382339  9947 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:35.320663 19270 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.428876 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:35.381562 19435 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.428212 19486 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:35.383616 19446 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:35.430671 19497 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.474372 10643 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:35.478317 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.479014 10022 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:35.479635 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.479857  9947 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:35.385040 19319 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:35.474758 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.482370 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:35.483855 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:35.476325 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.477000 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.479961 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.472348  9949 raft_consensus.cc:2368] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26.
I20260709 12:27:35.345197 19484 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.500859 19484 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.502241 19342 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:35.481812 10024 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:35.486287 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:35.488366 19164 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.489532 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:35.507890 10093 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:35.493392 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.498214 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:35.506760 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 118536us
I20260709 12:27:35.483354 10018 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:35.508980 10023 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:35.475610 19314 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.511595 10020 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:35.429634 19499 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.100s
I20260709 12:27:35.480758 19495 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.514254 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:35.516050 10019 raft_consensus.cc:3060] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:35.509881 10640 connection.cc:256] RPC call timeout handler was delayed by -1.3471243449998838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
I20260709 12:27:35.483829 19163 raft_consensus.cc:2804] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:35.516916 19440 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:35.516956 19163 raft_consensus.cc:697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:35.517611 19440 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.518644 19163 consensus_queue.cc:237] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.520285 19440 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:35.480490 19239 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.524030 10093 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:35.525535  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:35.526036 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:35.432359 19421 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.528293 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:35.530901 10093 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: []
W20260709 12:27:35.531199 19496 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.293s	user 0.002s	sys 0.000s
I20260709 12:27:35.538887 19270 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.539867 19499 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:35.505204 19454 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.540257 19499 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:35.511382 19342 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.541172 19499 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.542922 10019 raft_consensus.cc:2468] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:35.542659 19495 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.543339 19448 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.505566 19287 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:35.544674 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:35.542348 19499 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:35.478000 19486 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.537506 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.543099 19342 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:35.549546 10640 connection.cc:256] RPC call timeout handler was delayed by -1.0180538879999403s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2026s for a response
I20260709 12:27:35.511798 19396 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.551889 10020 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 27 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
W20260709 12:27:35.606079 19490 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.370s	user 0.001s	sys 0.000s
I20260709 12:27:35.606360 19396 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:35.608547 19493 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.416s	user 0.000s	sys 0.001s
I20260709 12:27:35.566833 19479 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.633525 19479 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.635972 19479 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:35.568747 19118 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:35.638106 19412 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.648s	user 0.011s	sys 0.023s
I20260709 12:27:35.567001 19446 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election lost for term 31. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 31. Current term is 32.
I20260709 12:27:35.549669 19454 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:35.547289 19314 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.020s
I20260709 12:27:35.575393 19152 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:35.576803 19491 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.341s	user 0.000s	sys 0.001s
I20260709 12:27:35.578256 19440 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:35.582428 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:35.583135 19495 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.585958  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:35.648878  9940 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:35.587105  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:35.587955  9946 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:35.588842 19483 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:27:35.649096 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66764.3us
I20260709 12:27:35.483239 19497 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.567516 19486 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:35.617400 19462 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.382s	user 0.000s	sys 0.001s
I20260709 12:27:35.618805 19396 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.619315 19376 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:35.627094 19450 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:35.631275 19344 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.687420 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:35.646255 18986 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.711681  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:35.712268 19453 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.715824 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4436) took 2873 ms. Trace:
I20260709 12:27:35.723822 10017 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 32.
I20260709 12:27:35.716490 19284 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:35.716667 19423 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:35.724189 10097 rpcz_store.cc:276] 0709 12:27:32.842905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:32.843089 (+   184us) service_pool.cc:224] Handling call
0709 12:27:32.843522 (+   433us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:27:32.981883 (+138361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:32.981951 (+    68us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:27:32.982817 (+   866us) log.cc:844] Serialized 78 byte log entry
0709 12:27:33.383318 (+400501us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:33.435513 (+ 52195us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.916460 (+480947us) log.cc:844] Serialized 14 byte log entry
0709 12:27:33.916705 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:33.916817 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.637302 (+1720485us) spinlock_profiling.cc:243] Waited 8.74 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:35.715561 (+ 78259us) raft_consensus.cc:1685] finished
0709 12:27:35.715587 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.715762 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116843,"prepare.queue_time_us":127315,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":16172,"raft.queue_time_us":314876,"raft.run_cpu_time_us":484,"raft.run_wall_time_us":58015,"spinlock_wait_cycles":20103552,"thread_start_us":744206,"threads_started":2,"wal-append.queue_time_us":746918}
I20260709 12:27:35.734546 10093 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:35.706763 19152 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.723359  9946 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:35.738039 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:35.725761 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.707317 19454 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:35.726035 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.739084 10671 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:35.727764 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.739580 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.740080 10674 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:35.582396 19256 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.730037 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.731302 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.743100 10025 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:35.736389 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.644197 19118 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.709808 19479 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:35.721486 19495 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:35.749471 19284 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:35.761974  9948 raft_consensus.cc:2368] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:35.733860 18986 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.750238 19479 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:35.748874 19152 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.758121 19495 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.762030 19284 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:35.734043 19453 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.780673 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:35.781903 10025 raft_consensus.cc:2410] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:27:35.770409 19479 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:35.774991 19498 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 12:27:35.785768 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1728) took 1481 ms. Trace:
I20260709 12:27:35.807467 10090 rpcz_store.cc:276] 0709 12:27:34.304034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.304616 (+   582us) service_pool.cc:224] Handling call
0709 12:27:34.305043 (+   427us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.305503 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:34.340579 (+ 35076us) log.cc:844] Serialized 14 byte log entry
0709 12:27:34.463859 (+123280us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.468778 (+  4919us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.468974 (+   196us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:34.469586 (+   612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.470012 (+   426us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.616711 (+1146699us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:35.785390 (+168679us) raft_consensus.cc:1685] finished
0709 12:27:35.785505 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.785666 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":36022,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":9046,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":22241,"spinlock_wait_cycles":271198592,"thread_start_us":96312,"threads_started":1,"wal-append.queue_time_us":849358}
I20260709 12:27:35.803426 19423 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:35.803426 19118 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:35.845767 19504 log.cc:927] Time spent T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.280s	user 0.001s	sys 0.000s
I20260709 12:27:35.828233 19450 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:35.841506 19344 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:35.773646 19454 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.849807 19276 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.887480 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.888563 10674 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:35.875658 19284 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.889997 19284 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:35.887475 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.893461 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:35.894282 10674 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:35.898926 10094 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3339) took 1105 ms. Trace:
I20260709 12:27:35.906577 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.907590 10671 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:35.909014 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.899516 10094 rpcz_store.cc:276] 0709 12:27:34.793248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.060977 (+267729us) service_pool.cc:224] Handling call
0709 12:27:35.061633 (+   656us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:35.062385 (+   752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.062419 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.072062 (+  9643us) log.cc:844] Serialized 26 byte log entry
0709 12:27:35.187982 (+115920us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:35.220254 (+ 32272us) log.cc:844] Serialized 14 byte log entry
0709 12:27:35.234083 (+ 13829us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.234167 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.898657 (+664490us) raft_consensus.cc:1685] finished
0709 12:27:35.898686 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.898853 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27203,"prepare.queue_time_us":26992,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":16302,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":4597,"thread_start_us":60037,"threads_started":1,"wal-append.queue_time_us":278717}
I20260709 12:27:35.866464 19423 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.912042 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.918860 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:35.914666 10675 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:35.891144 19314 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.922772  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:35.923328  9940 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:35.924650  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:35.919670 10671 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:35.930783 19421 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:35.932098 19446 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:35.933000  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:35.987586 19345 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:35.988154  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55222.5us
I20260709 12:27:35.897735 19276 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.988277 19345 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:35.989207 19505 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.450s	user 0.003s	sys 0.003s
I20260709 12:27:35.989004 19276 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:35.991353 19478 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.757s	user 0.013s	sys 0.000s
I20260709 12:27:35.991855 19276 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.934897 19471 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:35.992697  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 2080) took 1519 ms. Trace:
I20260709 12:27:35.992576 19471 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.934315 19419 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.935981 19244 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:35.993165  9944 rpcz_store.cc:276] 0709 12:27:34.473607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.473762 (+   155us) service_pool.cc:224] Handling call
0709 12:27:34.510823 (+ 37061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.583904 (+ 73081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.583929 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.600818 (+ 16889us) log.cc:844] Serialized 26 byte log entry
0709 12:27:34.958743 (+357925us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:34.958793 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:34.958889 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.988988 (+1030099us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:35.992389 (+  3401us) raft_consensus.cc:1685] finished
0709 12:27:35.992415 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.992635 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1995,"raft.queue_time_us":87450,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":22464,"spinlock_wait_cycles":273191424,"thread_start_us":398717,"threads_started":2,"wal-append.queue_time_us":619594}
I20260709 12:27:35.995810 19163 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:35.995680 19419 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.996354 19163 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.997081  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 3148) took 1104 ms. Trace:
I20260709 12:27:35.997485  9943 rpcz_store.cc:276] 0709 12:27:34.892776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.892913 (+   137us) service_pool.cc:224] Handling call
0709 12:27:35.996883 (+1103970us) spinlock_profiling.cc:243] Waited 14.3 ms on lock 0x7b5400206f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:35.997023 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32862976}
I20260709 12:27:35.997928 19419 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:35.937515 19511 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:27:35.998652 19163 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.916319 19373 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:35.935406 18760 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:35.999429 19373 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.999894 18760 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.947376 19453 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.876981 19344 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.003005 19158 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:27:36.004058 19494 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:36.004350 19344 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.873507 19450 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.948441  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:36.005640  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57314.1us
I20260709 12:27:36.006385  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:36.006471 19450 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.007295 19435 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:36.008040  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:36.008278 19328 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:35.987754 19446 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.009796 19453 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:36.010252 19453 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:36.010656 19453 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:35.998483 19419 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:36.011112 19333 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:36.002512 18760 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:36.013324 19489 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.689s	user 0.000s	sys 0.001s
I20260709 12:27:36.014786 18895 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:36.014932  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:36.020514 19453 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.023535 19453 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.024569 18837 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:36.026901 19314 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:35.921648 19510 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 12:27:35.954042 19501 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.620s	user 0.003s	sys 0.000s
I20260709 12:27:35.955896  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.034487  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.036044  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:36.037496  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:36.042178  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.047401 18986 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:35.921373 19256 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:35.965914 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4444) took 1671 ms. Trace:
I20260709 12:27:35.968761 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:35.977123 19135 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:35.978772 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1644) took 2189 ms. Trace:
I20260709 12:27:35.980378  9946 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:35.984565 19423 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:35.868774 19118 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:35.987651 19421 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:35.994866 19244 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.004700 19494 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:36.023396  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.072891 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:36.141351 10671 raft_consensus.cc:2393] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:27:36.126565 10095 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.015s	sys 0.012s
I20260709 12:27:36.050218 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:36.143358 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 93236.1us
I20260709 12:27:36.144404 10065 leader_election.cc:400] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:36.144683 10065 leader_election.cc:403] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:35.952767  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:35.997324  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:35.956439 19509 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.110s	user 0.001s	sys 0.001s
I20260709 12:27:36.053270  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.047950 19446 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:36.054250 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:36.069103  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:36.079162 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 110655us
I20260709 12:27:36.116351 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:36.117528 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:36.059612 10098 rpcz_store.cc:276] 0709 12:27:34.294624 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.294800 (+   176us) service_pool.cc:224] Handling call
0709 12:27:34.300586 (+  5786us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:34.356779 (+ 56193us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:34.524512 (+167733us) log.cc:844] Serialized 14 byte log entry
0709 12:27:35.159786 (+635274us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:35.159844 (+    58us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:35.159892 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.965572 (+805680us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b5400192dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:35.965641 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.965822 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4258,"prepare.queue_time_us":15717,"prepare.run_cpu_time_us":880,"prepare.run_wall_time_us":1037,"raft.queue_time_us":7093,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":2531,"spinlock_wait_cycles":128747264,"thread_start_us":796017,"threads_started":3,"wal-append.queue_time_us":261905}
I20260709 12:27:36.141698  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.049012 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:35.950387 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1940) took 1638 ms. Trace:
W20260709 12:27:36.058991 19402 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.389s	user 0.052s	sys 0.118s
I20260709 12:27:36.120229 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:36.207206 10091 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:27:36.207090 10669 rpcz_store.cc:276] 0709 12:27:34.311773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.312397 (+   624us) service_pool.cc:224] Handling call
0709 12:27:35.950106 (+1637709us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54007f4754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:35.950303 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2051477248}
W20260709 12:27:36.208281 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 154152us
I20260709 12:27:36.208528 10674 raft_consensus.cc:2393] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:36.222333  9948 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:36.207613 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:36.146359 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:36.223057  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.224084  9943 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:27:36.120676 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.116434 10672 rpcz_store.cc:276] 0709 12:27:33.789515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.791801 (+  2286us) service_pool.cc:224] Handling call
0709 12:27:33.916658 (+124857us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:34.278821 (+362163us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.278878 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:34.279561 (+   683us) log.cc:844] Serialized 52 byte log entry
0709 12:27:34.524857 (+245296us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:34.763391 (+238534us) log.cc:844] Serialized 14 byte log entry
0709 12:27:34.942320 (+178929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:35.978218 (+1035898us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b5400408dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:35.978285 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:35.978353 (+    68us) raft_consensus.cc:1685] finished
0709 12:27:35.978470 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:35.978695 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":110971,"prepare.queue_time_us":100400,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":203129,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":11757,"spinlock_wait_cycles":788023168,"thread_start_us":415671,"threads_started":2,"wal-append.queue_time_us":281780}
I20260709 12:27:36.226572 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:36.227169 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82905.9us
I20260709 12:27:36.227429 10091 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.223579 10096 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:35.954522 19497 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.261720  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.263260  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.263850  9943 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:36.264719  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.265230  9943 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:36.217378  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.226614  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.270798  9946 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:27:36.271778  9946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "062c203e13aa4527a094ec24339001e5"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:27:36.272140 19373 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:36.229188 10062 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.012s	sys 0.017s
I20260709 12:27:36.272541  9946 raft_consensus.cc:606] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:36.272975 10062 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.012s	sys 0.017s
W20260709 12:27:36.273413 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 674319us
W20260709 12:27:36.217295 10095 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.016s	sys 0.014s
I20260709 12:27:36.230425 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:36.275126 10673 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:36.234606 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.058507 19440 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.273080  9946 raft_consensus.cc:614] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:36.234429  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.115247 19328 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:36.303753 10098 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:36.280155 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:36.220352 19402 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.550s	user 0.057s	sys 0.125s
I20260709 12:27:36.318095 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:36.307725 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:36.146415 19163 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:36.328358 19163 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:36.328804 19163 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:36.329068 19495 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:36.068770 18986 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.208s
I20260709 12:27:36.147991 19207 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:36.228586 19140 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:36.119138 19152 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:36.317142 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:36.216285 19415 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:36.324775 10091 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:36.327093 10096 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:36.278957 18941 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:36.308176 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:36.362859 10674 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:36.327868 19452 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:27:36.338475 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.311765 10093 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:36.316207 19497 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.343921 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:36.345187 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:36.375625 19470 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.144s	user 0.001s	sys 0.003s
I20260709 12:27:36.306253  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.364694 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:36.377244 18941 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.378134 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.378381  9943 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:36.378683 19376 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.649s	user 0.011s	sys 0.000s
I20260709 12:27:36.229713 19329 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.121s
I20260709 12:27:36.366307 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:36.381458  9946 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:36.382889 10091 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:36.367167 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.370126 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.342087 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:36.384258 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:36.384459  9949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 3149) took 1491 ms (client timeout 1661 ms). Trace:
I20260709 12:27:36.370942 19415 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:36.379670  9943 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:36.401513  9943 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:36.380268 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.335599 19440 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:36.384879  9949 rpcz_store.cc:269] 0709 12:27:34.893266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.893424 (+   158us) service_pool.cc:224] Handling call
0709 12:27:36.384235 (+1490811us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b540003f554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:36.384399 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088113024}
I20260709 12:27:36.371241 19118 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.387027 19284 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:36.403899 19284 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:36.404541 19440 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:36.405076 19440 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:36.404866 19284 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.405570 19440 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.387681 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:36.409256 10093 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:36.410226 19415 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.388093 19472 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.334667 19328 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.398658 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:36.399694 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.417567 10093 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:36.418057 10091 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:36.420498 19479 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.597s	user 0.005s	sys 0.005s
I20260709 12:27:36.365166 19450 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:36.427928 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.416399  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:36.429762  9914 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:36.437804 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.438638 10672 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:36.384485  9946 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:36.439422 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.437311 19469 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:36.440964 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.441226 19469 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:36.441828 10673 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:36.442145  9914 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:36.443320 19333 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:36.443907 19333 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:36.444906 19333 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.374878 19497 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.448326 19497 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.427155 19472 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.450572 10098 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:36.450794 19120 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:36.451174  9941 raft_consensus.cc:2410] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 26 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:27:36.451472 19120 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:36.452047 19345 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.452430 19120 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.442162 19469 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.455204 19421 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:36.433840 19479 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.456641 19244 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:36.343137 19207 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.421864 10675 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:36.461226 10669 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:36.458280 19479 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:36.426410 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:36.427402 19496 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:27:36.475659 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.476496 10093 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:36.479765 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.483664 10093 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:36.486573 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.441872 19328 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:36.511106 10669 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:36.473889 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.428229 19376 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:27:36.516007 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:36.520960 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:36.522701 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.529031 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:36.554852 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:36.499270 18986 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:36.557137  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:36.571449 19328 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:36.573729 19328 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:36.580907 10093 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:27:36.592056 10093 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: [29.17-30.18]
I20260709 12:27:36.595415 19440 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:36.559302 19155 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:36.615931 10093 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
W20260709 12:27:36.603233 19429 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.008s	sys 0.017s
W20260709 12:27:36.606109 19454 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.016s	sys 0.011s
I20260709 12:27:36.559687 19423 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.047s
W20260709 12:27:36.662375 19349 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.007s	sys 0.028s
W20260709 12:27:36.664337 18895 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.649s	user 0.007s	sys 0.004s
W20260709 12:27:36.652200 19506 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.012s	sys 0.013s
W20260709 12:27:36.762933  9991 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.009s	sys 0.018s
W20260709 12:27:36.734838 19429 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.008s	sys 0.017s
I20260709 12:27:36.664770 19118 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:36.743685 19454 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.017s	sys 0.011s
I20260709 12:27:36.593932 19495 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.150s
W20260709 12:27:36.767411  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.009s	sys 0.018s
W20260709 12:27:36.747994 19349 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.791s	user 0.008s	sys 0.029s
W20260709 12:27:36.767813 19506 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.012s	sys 0.013s
I20260709 12:27:36.606400 19484 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.161s
I20260709 12:27:36.765451 18895 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:36.588029 18986 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:36.762777 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 86310.7us
I20260709 12:27:36.737224 19155 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.747668 19377 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:27:36.781019 19429 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.809444 19429 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.810850 19329 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:36.811566 19329 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:36.812155 19329 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:36.813078 19446 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:36.813481 19446 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:36.814328 19446 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:36.822358 19467 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.508s	user 0.003s	sys 0.000s
I20260709 12:27:36.824404 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1735) took 2505 ms. Trace:
I20260709 12:27:36.824939 19328 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:36.825695 19328 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:36.824896 10095 rpcz_store.cc:276] 0709 12:27:34.319117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:34.331749 (+ 12632us) service_pool.cc:224] Handling call
0709 12:27:34.355309 (+ 23560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:34.367332 (+ 12023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:34.367383 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:34.367923 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:27:35.593561 (+1225638us) spinlock_profiling.cc:243] Waited 895 us on lock 0x7b1800455800. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:36.274226 (+680665us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:36.274285 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.274376 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:36.824039 (+549663us) raft_consensus.cc:1685] finished
0709 12:27:36.824061 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:36.824313 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11622,"raft.queue_time_us":76329,"raft.run_cpu_time_us":197,"raft.run_wall_time_us":33720,"spinlock_wait_cycles":2059008,"thread_start_us":680439,"threads_started":1,"wal-append.queue_time_us":2331161}
I20260709 12:27:36.776882 19118 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.842361 19118 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.842440 18941 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.808681 19452 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
W20260709 12:27:36.838466 19163 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.003s
I20260709 12:27:36.852509 19450 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:36.904942 10096 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:36.907034 10095 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
W20260709 12:27:36.909258 10097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.012s	sys 0.019s
W20260709 12:27:36.909816 10097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.012s	sys 0.019s
W20260709 12:27:36.887576 19419 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.023s	sys 0.015s
I20260709 12:27:36.819926 19494 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:36.911100 10670 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.009s	sys 0.022s
I20260709 12:27:36.883920 19328 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:36.919166 10670 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.009s	sys 0.022s
W20260709 12:27:36.911911 10641 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.003s	sys 0.028s
W20260709 12:27:36.921489 10641 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.004s	sys 0.028s
W20260709 12:27:36.911832 19419 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.900s	user 0.023s	sys 0.017s
I20260709 12:27:36.923327 19419 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:36.924571 19419 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:36.916867  9987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.900s	user 0.019s	sys 0.036s
W20260709 12:27:36.931326  9987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.019s	sys 0.036s
W20260709 12:27:36.931857  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 915536us
W20260709 12:27:36.917218  9948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.011s	sys 0.019s
I20260709 12:27:36.864696 19163 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:36.933444  9948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.011s	sys 0.020s
W20260709 12:27:36.947316 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.012s	sys 0.021s
I20260709 12:27:36.932724  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:36.947916 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.571s	user 0.012s	sys 0.021s
W20260709 12:27:36.949113 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 572211us
I20260709 12:27:36.950790  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:36.949352 10677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.008s	sys 0.026s
W20260709 12:27:36.951339 10016 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.010s	sys 0.008s
W20260709 12:27:36.954954 10016 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.719s	user 0.010s	sys 0.009s
W20260709 12:27:36.954607 10677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.008s	sys 0.026s
I20260709 12:27:36.972744 19376 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:36.974176 19376 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:36.974730 10065 leader_election.cc:400] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26.
I20260709 12:27:36.975082 10065 leader_election.cc:403] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:36.974874 19244 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:36.976388 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:36.977614 19423 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26.
I20260709 12:27:36.977793 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:36.946374 19494 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:36.979328 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:36.980448 10018 raft_consensus.cc:2410] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:27:36.981710 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:36.982733 10022 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:36.983788 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:36.984674 10024 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:36.986852 19155 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:36.986986 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:36.972776 19495 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:36.976311 19376 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:36.990186  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.976768 19394 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:36.992139  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.990892  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.992511  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.987726 19495 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:36.992076  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:36.986521 10023 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:37.007297 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.008196 10023 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
W20260709 12:27:36.999393 19518 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.193s	user 0.001s	sys 0.000s
I20260709 12:27:37.014847 10676 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:37.032639 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1961) took 1306 ms. Trace:
I20260709 12:27:37.033299 10676 rpcz_store.cc:276] 0709 12:27:35.726260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.726447 (+   187us) service_pool.cc:224] Handling call
0709 12:27:37.014337 (+1287890us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:27:37.014606 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.031060 (+ 16454us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.032567 (+  1507us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11809152}
I20260709 12:27:37.036144 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.045472 10023 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
W20260709 12:27:37.013059 19152 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.011s	sys 0.010s
W20260709 12:27:37.061944 10668 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.943s	user 0.014s	sys 0.015s
W20260709 12:27:37.062363 10668 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.947s	user 0.015s	sys 0.015s
W20260709 12:27:37.035073 19402 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.002s	sys 0.027s
W20260709 12:27:37.013695 19140 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.016s	sys 0.011s
I20260709 12:27:37.063611 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 1960) took 1337 ms. Trace:
W20260709 12:27:37.063666 10643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.015s	sys 0.018s
W20260709 12:27:37.065913 10643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.015s	sys 0.018s
I20260709 12:27:37.064198 10668 rpcz_store.cc:276] 0709 12:27:35.725969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.726489 (+   520us) service_pool.cc:224] Handling call
0709 12:27:35.726779 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:35.726978 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:36.114936 (+387958us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.063208 (+948272us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.063265 (+    57us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:37.063309 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.063378 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.063501 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":926305,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"thread_start_us":1287465,"threads_started":2}
W20260709 12:27:37.066658  9940 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.014s	sys 0.038s
W20260709 12:27:37.034835 19514 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 12:27:37.004091 19495 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:27:37.096041 19345 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.012s	sys 0.016s
W20260709 12:27:37.061335 19152 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.012s	sys 0.010s
I20260709 12:27:37.081223 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:37.065408 19402 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.002s	sys 0.027s
W20260709 12:27:37.066224 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 722105us
W20260709 12:27:37.066947 19140 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.016s	sys 0.011s
W20260709 12:27:37.083297 19441 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.036s	sys 0.036s
I20260709 12:27:37.063012 19376 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.091048 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1780) took 1355 ms. Trace:
W20260709 12:27:37.068982 19509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.735s	user 0.002s	sys 0.032s
W20260709 12:27:37.083381 19517 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.245s	user 0.001s	sys 0.000s
W20260709 12:27:37.083879 19515 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 12:27:37.065670 19396 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:27:37.091418 19415 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.012s	sys 0.025s
W20260709 12:27:37.163515 19415 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.012s	sys 0.025s
W20260709 12:27:37.112591  9940 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.014s	sys 0.038s
W20260709 12:27:37.079365 10090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.029s	sys 0.025s
W20260709 12:27:37.096263 19497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.009s	sys 0.019s
I20260709 12:27:37.137384  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.138403 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:37.129845 19345 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.013s	sys 0.016s
W20260709 12:27:37.150519 19441 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.036s	sys 0.036s
I20260709 12:27:37.151865  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:37.144196 19140 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:37.153247 19376 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.151912 10016 rpcz_store.cc:276] 0709 12:27:35.735781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.739033 (+  3252us) service_pool.cc:224] Handling call
0709 12:27:35.745545 (+  6512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:35.745882 (+   337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.745910 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:35.752236 (+  6326us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.231614 (+479378us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:36.956711 (+725097us) log.cc:844] Serialized 14 byte log entry
0709 12:27:36.975412 (+ 18701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.975494 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.090822 (+115328us) raft_consensus.cc:1685] finished
0709 12:27:37.090843 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.090989 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21618,"raft.queue_time_us":18876,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":4692,"thread_start_us":1186305,"threads_started":2,"wal-append.queue_time_us":979134}
W20260709 12:27:37.183952 10640 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.960s	user 0.024s	sys 0.023s
I20260709 12:27:37.105225 18986 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:37.184551 10640 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.961s	user 0.024s	sys 0.023s
W20260709 12:27:37.154971 10672 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.011s	sys 0.022s
I20260709 12:27:37.155612 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:37.184091 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 45788.3us
I20260709 12:27:37.152388 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:37.185220 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 977637us
W20260709 12:27:37.154747 19509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.002s	sys 0.032s
W20260709 12:27:37.165340 10090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.030s	sys 0.025s
I20260709 12:27:37.166636 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:37.171461  9912 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.115s	user 0.013s	sys 0.039s
W20260709 12:27:37.187207  9912 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.131s	user 0.013s	sys 0.039s
W20260709 12:27:37.158768 19519 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 12:27:37.176436  9944 raft_consensus.cc:2368] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
W20260709 12:27:37.171442 19497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.718s	user 0.010s	sys 0.021s
I20260709 12:27:37.157323  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:37.172279  9949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0df211fae593420e88f685abc8d618ec"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
W20260709 12:27:37.185918 10672 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.011s	sys 0.022s
I20260709 12:27:37.187151 10676 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:37.207834  9949 raft_consensus.cc:606] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:37.208190  9949 raft_consensus.cc:614] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:27:37.187893  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 1.13197e+06us
I20260709 12:27:37.189316 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.187044 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:37.188331 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:37.185338 10023 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:37.163450 19163 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:37.187872 19415 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:37.207530 19135 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.010s	sys 0.022s
W20260709 12:27:37.204488 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.003s	sys 0.044s
W20260709 12:27:37.211236 10671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.017s	sys 0.015s
W20260709 12:27:37.212605 10019 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.977s	user 0.013s	sys 0.019s
W20260709 12:27:37.212992 10099 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.013s	sys 0.026s
W20260709 12:27:37.309445 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.004s	sys 0.044s
W20260709 12:27:37.309788 10099 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.013s	sys 0.026s
W20260709 12:27:37.310297 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 781292us
W20260709 12:27:37.309499 10671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.017s	sys 0.015s
I20260709 12:27:37.201956  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3136) took 3702 ms. Trace:
I20260709 12:27:37.311082 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:37.219832 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:37.222621 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:37.313004 10023 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:37.314997 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:37.315793 10016 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:37.309624 10019 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.074s	user 0.013s	sys 0.019s
I20260709 12:27:37.317121 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:37.317960 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1779) took 1585 ms. Trace:
I20260709 12:27:37.311371  9942 rpcz_store.cc:276] 0709 12:27:33.499285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.500138 (+   853us) service_pool.cc:224] Handling call
0709 12:27:33.500473 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:33.545086 (+ 44613us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:33.545158 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:33.580492 (+ 35334us) log.cc:844] Serialized 26 byte log entry
0709 12:27:35.719804 (+2139312us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b18002bf7e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:36.142628 (+422824us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:36.142687 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.142779 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.201579 (+1058800us) raft_consensus.cc:1685] finished
0709 12:27:37.201609 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.201880 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50694,"raft.queue_time_us":80932,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":2165,"spinlock_wait_cycles":262767360,"thread_start_us":422560,"threads_started":1,"wal-append.queue_time_us":3152087}
W20260709 12:27:37.325398  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:37.325917  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.318472 10019 rpcz_store.cc:276] 0709 12:27:35.732366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.732562 (+   196us) service_pool.cc:224] Handling call
0709 12:27:35.732984 (+   422us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:35.733458 (+   474us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:35.733500 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:35.734178 (+   678us) log.cc:844] Serialized 52 byte log entry
0709 12:27:36.218962 (+484784us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:37.316944 (+1097982us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.317526 (+   582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.317640 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.317689 (+    49us) raft_consensus.cc:1685] finished
0709 12:27:37.317707 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.317886 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":245,"raft.queue_time_us":2570,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":1542659,"threads_started":2,"wal-append.queue_time_us":1030556}
I20260709 12:27:37.341820 10023 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:37.343408 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.344784 10023 raft_consensus.cc:2410] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 24 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 18.
I20260709 12:27:37.213536 19441 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:37.355166 19372 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:37.359007 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:37.362195 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:37.379750 10673 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 24: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:37.363240 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.228188 19140 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:37.339869 19521 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 12:27:37.363267 19163 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:37.383601 19163 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:37.384045 19163 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:37.381774 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:37.384927 10023 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:37.220554 19345 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:37.368654  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.388373  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.389211 19520 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:37.389858  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.368877 19376 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:37.393534 10093 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.014s	sys 0.038s
I20260709 12:27:37.373888 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:37.395892 10093 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.015s	sys 0.038s
I20260709 12:27:37.380862 10019 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:37.381778  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:37.371279 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:37.398895 10668 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
W20260709 12:27:37.317173 19135 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.010s	sys 0.022s
I20260709 12:27:37.401894 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:37.419781 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:37.420611 10091 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:37.422143 10091 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:37.376354 19525 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:37.397711 19329 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.000s	sys 0.056s
W20260709 12:27:37.433971 19349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.011s	sys 0.007s
W20260709 12:27:37.487576 10670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.006s	sys 0.010s
W20260709 12:27:37.704842 10670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.006s	sys 0.010s
I20260709 12:27:37.705577 19332 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:37.392418  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.707233 19332 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.369212  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.709455  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.711474  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.711920 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4464) took 1492 ms. Trace:
I20260709 12:27:37.713492  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.368891  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3164) took 1756 ms. Trace:
W20260709 12:27:37.715034 10677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.757s	user 0.010s	sys 0.006s
W20260709 12:27:37.715602 10677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.010s	sys 0.006s
I20260709 12:27:37.715485 19376 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:37.717413 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1652) took 1645 ms. Trace:
I20260709 12:27:37.716944 19376 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:37.528627 19469 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.071s	user 0.024s	sys 0.062s
W20260709 12:27:37.718900 19469 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.262s	user 0.025s	sys 0.062s
W20260709 12:27:37.628719  9946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.005s	sys 0.005s
W20260709 12:27:37.719740  9946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.006s	sys 0.005s
I20260709 12:27:37.370929  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.629433  9940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3162) took 2019 ms. Trace:
I20260709 12:27:37.401906  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:37.662966 10020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.003s	sys 0.010s
I20260709 12:27:37.663661 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:37.665341 10022 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.667s	user 0.005s	sys 0.006s
W20260709 12:27:37.667050 10025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.000s	sys 0.014s
I20260709 12:27:37.669709 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3351) took 1682 ms. Trace:
I20260709 12:27:37.433086 10093 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:37.673725 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 1959) took 1266 ms. Trace:
W20260709 12:27:37.674990 10097 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.000s	sys 0.014s
W20260709 12:27:37.457715 19329 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.000s	sys 0.057s
W20260709 12:27:37.484445 19349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.012s	sys 0.009s
I20260709 12:27:37.380151 19441 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.511222 19471 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:37.568317 19494 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.589s	user 0.005s	sys 0.006s
I20260709 12:27:37.389876 19520 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.791904  9943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:27:37.848165  9943 raft_consensus.cc:606] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:37.848783  9943 raft_consensus.cc:614] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:37.850323 19520 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:37.851655 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:37.852250 19520 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:37.852957 10093 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
W20260709 12:27:37.854701 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 573263us
I20260709 12:27:37.855365 10017 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 31 because replica has last-logged OpId of term: 30 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 17.
I20260709 12:27:37.857149 10019 raft_consensus.cc:2410] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 25 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 18.
I20260709 12:27:37.511209 19543 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:37.510015 19550 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:37.503543 19551 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:37.859126 19551 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:37.859771 19551 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.860886 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:37.570909 19495 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.862296 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.744762 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1651) took 1635 ms. Trace:
I20260709 12:27:37.732220 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1657) took 1582 ms. Trace:
I20260709 12:27:37.722532  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:37.771040  9942 raft_consensus.cc:2393] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:27:37.771801 10020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.003s	sys 0.016s
W20260709 12:27:37.771968 10025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.001s	sys 0.016s
W20260709 12:27:37.771793 10022 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.774s	user 0.008s	sys 0.008s
W20260709 12:27:37.771510 10097 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.861s	user 0.000s	sys 0.018s
W20260709 12:27:37.731168 19419 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.802s	user 0.008s	sys 0.016s
I20260709 12:27:37.863549 18941 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:37.875725 19529 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:27:37.864794  9946 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:37.877249  9946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:27:37.733587 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:37.864337 10024 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:37.863875 10671 rpcz_store.cc:276] 0709 12:27:36.149648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.231019 (+ 81371us) service_pool.cc:224] Handling call
0709 12:27:36.231311 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:37.310998 (+1079687us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:37.311028 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.311073 (+    45us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:37.311116 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.731594 (+420478us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54006dfe54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:37.731682 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.731984 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":234914,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":796439424,"thread_start_us":732970,"threads_started":1}
I20260709 12:27:37.712512 10092 rpcz_store.cc:276] 0709 12:27:36.219048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.221177 (+  2129us) service_pool.cc:224] Handling call
0709 12:27:36.305126 (+ 83949us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.442434 (+137308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.442468 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:36.442513 (+    45us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:36.453772 (+ 11259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.711299 (+1257527us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b540028fd54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:37.711379 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.711708 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315304192}
I20260709 12:27:37.874899 19372 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:37.731752 19276 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:37.713238 18986 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.875726 18941 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:37.883527 18941 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:37.883312 18986 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.883883 19376 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:37.731688 19524 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.865065 19495 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:37.867880 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.887132 19524 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.866982 19550 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:37.889096 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.722680 19328 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:37.931246 10020 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:37.931357 19328 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.932025 19276 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:37.932585 19276 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:37.934267 19328 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:37.935448 10091 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:27:37.789223 18837 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.717932 10677 rpcz_store.cc:276] 0709 12:27:36.072037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.072161 (+   124us) service_pool.cc:224] Handling call
0709 12:27:36.218333 (+146172us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.218581 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:36.957388 (+738807us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.716618 (+759230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.716773 (+   155us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:37.716946 (+   173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.717077 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.717345 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42933,"prepare.queue_time_us":566678,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":1647,"raft.queue_time_us":19783,"thread_start_us":1359790,"threads_started":2,"wal-append.queue_time_us":617669}
I20260709 12:27:37.762444 10672 rpcz_store.cc:276] 0709 12:27:36.407448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.478061 (+ 70613us) service_pool.cc:224] Handling call
0709 12:27:36.501061 (+ 23000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.501351 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:37.207942 (+706591us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.672680 (+464738us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.672811 (+   131us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:37.672961 (+   150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.673112 (+   151us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.673385 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":336650,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":1571,"raft.queue_time_us":23424,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":435,"thread_start_us":1139834,"threads_started":2,"wal-append.queue_time_us":365647}
I20260709 12:27:37.715044  9948 rpcz_store.cc:276] 0709 12:27:35.612047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.360088 (+748041us) service_pool.cc:224] Handling call
0709 12:27:36.368160 (+  8072us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.368369 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.368391 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.369107 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 12:27:36.935983 (+566876us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:36.936040 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:36.936132 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.368679 (+432547us) raft_consensus.cc:1685] finished
0709 12:27:37.368701 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.368820 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":563924,"threads_started":1,"wal-append.queue_time_us":994826}
I20260709 12:27:37.731065 19523 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:37.813319 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 502183us
W20260709 12:27:37.813174  9944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.006s	sys 0.008s
I20260709 12:27:37.760829 10090 rpcz_store.cc:276] 0709 12:27:35.987267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:35.990641 (+  3374us) service_pool.cc:224] Handling call
0709 12:27:35.990960 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.050974 (+ 60014us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.051019 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.068628 (+ 17609us) log.cc:844] Serialized 26 byte log entry
0709 12:27:37.186384 (+1117756us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:37.668155 (+481771us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.668813 (+   658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.668974 (+   161us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.669069 (+    95us) raft_consensus.cc:1685] finished
0709 12:27:37.669081 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.669401 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37259,"prepare.queue_time_us":373282,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":30543,"raft.run_cpu_time_us":574,"raft.run_wall_time_us":84193,"thread_start_us":1407262,"threads_started":2,"wal-append.queue_time_us":1388771}
W20260709 12:27:37.823844 10018 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.790s	user 0.012s	sys 0.031s
I20260709 12:27:37.827656  9949 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:37.793706 10016 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:37.814291 19332 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:37.838893  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.844591 19494 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.462702 19525 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:37.974184  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 572187us
I20260709 12:27:37.975471  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:37.976064 19525 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:37.977444  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:37.977185 19494 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.979391 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:37.980082 19494 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:37.875001 19419 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.011s	sys 0.019s
W20260709 12:27:37.878069  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.981510 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:37.870469 19440 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:37.982144 19440 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:37.983336  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:37.889278 19550 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.987908 19429 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:37.988646 10099 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.014s	sys 0.033s
I20260709 12:27:37.732096 19435 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:37.989441 10643 leader_election.cc:400] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:27:37.989841 10643 leader_election.cc:403] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:27:37.990015 19435 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:37.991080 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:37.993014 19435 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:37.989882 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4469) took 1672 ms. Trace:
I20260709 12:27:37.994470 10099 rpcz_store.cc:276] 0709 12:27:36.317267 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.317433 (+   166us) service_pool.cc:224] Handling call
0709 12:27:36.317802 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.318042 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:37.310221 (+992179us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.989378 (+679157us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.989484 (+   106us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:37.989534 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.989636 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.989811 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":35264,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":2520,"raft.queue_time_us":941,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"thread_start_us":1660668,"threads_started":2,"wal-append.queue_time_us":566139}
I20260709 12:27:37.997143 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:37.998189 10099 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:37.881614 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.999262 10024 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:37.862545 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:37.863727 10670 rpcz_store.cc:276] 0709 12:27:36.071289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.071677 (+   388us) service_pool.cc:224] Handling call
0709 12:27:36.147341 (+ 75664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.147603 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:27:36.919909 (+772306us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.705811 (+785902us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.705940 (+   129us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:37.706102 (+   162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.706270 (+   168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.706513 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":618905,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":8765,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":225,"thread_start_us":1444108,"threads_started":2,"wal-append.queue_time_us":630189}
I20260709 12:27:37.881572 19551 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.863941 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.895433 10092 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:37.871862 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4463) took 1724 ms. Trace:
W20260709 12:27:37.899196 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1739) took 1543 ms (client timeout 2025 ms). Trace:
I20260709 12:27:37.899606 10674 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:37.899273 19446 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:37.830379 19441 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:37.900338 19270 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:38.006839 19542 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:27:38.007315 19270 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:38.007448 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.007652 19499 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:38.008983 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.008752 19270 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:38.010048 19270 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:38.010193 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:37.903267 19314 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:37.909477 19575 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:37.913968  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:37.921415 10023 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.524s	user 0.014s	sys 0.004s
I20260709 12:27:37.925858 19483 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:37.927711 10093 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:37.928838 10096 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1747) took 1545 ms (client timeout 1619 ms). Trace:
W20260709 12:27:37.929466 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 706778us
W20260709 12:27:37.930073  9912 connection.cc:256] RPC call timeout handler was delayed by -1.0526036610003757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2077s for a response
W20260709 12:27:37.930135 10669 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1953) took 1612 ms (client timeout 1948 ms). Trace:
I20260709 12:27:37.932482 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.761111  9940 rpcz_store.cc:276] 0709 12:27:35.609919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.059835 (+449916us) service_pool.cc:224] Handling call
0709 12:27:36.060180 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:36.060399 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:36.060423 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:36.069030 (+  8607us) log.cc:844] Serialized 26 byte log entry
0709 12:27:37.164875 (+1095845us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:37.164925 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.164997 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:37.628534 (+463537us) raft_consensus.cc:1685] finished
0709 12:27:37.628613 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.629067 (+   454us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":904557,"threads_started":1,"wal-append.queue_time_us":1352714}
I20260709 12:27:37.732865 19472 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:38.001195 10098 rpcz_store.cc:269] 0709 12:27:36.355050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.364492 (+  9442us) service_pool.cc:224] Handling call
0709 12:27:37.898673 (+1534181us) spinlock_profiling.cc:243] Waited 67 ms on lock 0x7b5400340b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:37.898912 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":154106112}
I20260709 12:27:37.936517 18837 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.973409 19207 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:37.980120 19374 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:37.983497 19440 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:37.868361 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.890097 19524 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:37.960570 10018 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.013s	sys 0.032s
W20260709 12:27:37.960255  9944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.008s	sys 0.008s
I20260709 12:27:37.902876 18986 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:38.010762 19421 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:38.018848 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 156544us
I20260709 12:27:38.030473 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.031515 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:38.041042 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:37.981118 19523 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:38.023103 10669 rpcz_store.cc:269] 0709 12:27:36.317714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.317859 (+   145us) service_pool.cc:224] Handling call
0709 12:27:37.929659 (+1611800us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54006ff754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:37.929869 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":327071616}
I20260709 12:27:38.033576 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.033804 10092 raft_consensus.cc:2410] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
W20260709 12:27:38.022238 10096 rpcz_store.cc:269] 0709 12:27:36.383472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.383883 (+   411us) service_pool.cc:224] Handling call
0709 12:27:37.928325 (+1544442us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b5400163854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:37.928605 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195838848}
I20260709 12:27:38.018484 19551 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:38.036015 10023 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:38.045375  9943 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:38.052279 10021 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:38.052435 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61565.1us
I20260709 12:27:38.033404 19429 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:38.052121 19441 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:38.047786 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.067251 19577 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:38.068037 10094 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:38.068241 18837 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:38.042728 19207 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:27:38.070169 19396 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:27:38.072057 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 90716.5us
I20260709 12:27:38.072660 19276 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:38.060367  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 89046.4us
I20260709 12:27:38.072938 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:38.046934 19483 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.001091 10097 rpcz_store.cc:276] 0709 12:27:36.147684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.147954 (+   270us) service_pool.cc:224] Handling call
0709 12:27:36.148272 (+   318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.148520 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:36.259996 (+111476us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.870914 (+1610918us) log.cc:844] Serialized 14 byte log entry
0709 12:27:37.871272 (+   358us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:37.871372 (+   100us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:37.871455 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:37.871571 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:37.871798 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":741986,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":17804,"raft.queue_time_us":1190,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":8421,"thread_start_us":1609965,"threads_started":2,"wal-append.queue_time_us":1103350}
I20260709 12:27:38.079804 10098 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:38.080796 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.081154 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.083101 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:38.005664 19419 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.048s
I20260709 12:27:38.087163 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 4168) took 1083 ms. Trace:
W20260709 12:27:38.082823 19422 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.703s	user 0.007s	sys 0.016s
I20260709 12:27:38.075426 19276 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:38.096969 19422 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.767s	user 0.007s	sys 0.017s
I20260709 12:27:38.080433 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:38.098109 19419 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:38.097919 19421 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.099058 19494 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:38.099831 19494 consensus_queue.cc:1243] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 28.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:38.100852 19421 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:38.101174 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.101888 10021 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 18.
I20260709 12:27:38.098614 19419 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:38.103641 19419 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:38.105106 19419 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:38.107815 19483 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:38.110844 10090 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:27:38.114913 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.116386 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.117617 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.123764 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4159) took 1127 ms. Trace:
I20260709 12:27:38.141269 10022 rpcz_store.cc:276] 0709 12:27:36.996738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.996845 (+   107us) service_pool.cc:224] Handling call
0709 12:27:36.997136 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.997340 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:37.868834 (+871494us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.046045 (+177211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.059945 (+ 13900us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:38.123273 (+ 63328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.123414 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.123662 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4757,"prepare.queue_time_us":368295,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":7413,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":20508,"thread_start_us":1026434,"threads_started":2}
I20260709 12:27:38.143635 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.145376 10022 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:38.146148 19523 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:38.146420 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:38.146795 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.147466 10022 raft_consensus.cc:2376] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24. Re-sending same reply.
I20260709 12:27:38.149781 19440 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:38.150370 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.151814 10022 raft_consensus.cc:2410] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 32 because replica has last-logged OpId of term: 30 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
I20260709 12:27:38.153050 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.153558  9946 raft_consensus.cc:606] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:38.154105 10022 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 12:27:38.156232 10021 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:38.157123 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:38.157871 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:38.158562 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 71577.8us
I20260709 12:27:38.158962 10020 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:38.159646 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:38.160517 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4173) took 1099 ms. Trace:
I20260709 12:27:38.160888 10020 rpcz_store.cc:276] 0709 12:27:37.061561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.159630 (+1098069us) service_pool.cc:224] Handling call
0709 12:27:38.159976 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.160170 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.160196 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.160235 (+    39us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:38.160270 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.160318 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.160436 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.162073 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:38.147928  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94319.1us
I20260709 12:27:38.095320 10023 rpcz_store.cc:276] 0709 12:27:37.003695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.380844 (+377149us) service_pool.cc:224] Handling call
0709 12:27:38.064420 (+683576us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.080886  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:38.149163 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.169631 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 4167) took 1166 ms. Trace:
I20260709 12:27:38.171456 10022 rpcz_store.cc:276] 0709 12:27:37.003160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.152595 (+1149435us) service_pool.cc:224] Handling call
0709 12:27:38.169565 (+ 16970us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.162465 10020 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 18. (term mismatch)
W20260709 12:27:38.173667 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100859us
I20260709 12:27:38.175549 10020 pending_rounds.cc:85] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 17
I20260709 12:27:38.175680 10674 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:38.156739 10021 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:38.148627 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 4151) took 1172 ms. Trace:
I20260709 12:27:38.063303 19441 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.147588 10025 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:38.177032 10017 rpcz_store.cc:276] 0709 12:27:36.975977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.976098 (+   121us) service_pool.cc:224] Handling call
0709 12:27:38.148393 (+1172295us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b54001c6454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:38.148555 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1988938624}
I20260709 12:27:38.181335 19452 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:38.182314 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:38.184478 19319 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:38.184901 19319 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:38.184996 10094 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4468) took 1878 ms (client timeout 1952 ms). Trace:
W20260709 12:27:38.185421 10094 rpcz_store.cc:269] 0709 12:27:36.307341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.307485 (+   144us) service_pool.cc:224] Handling call
0709 12:27:38.184792 (+1877307us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54003ac0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:38.184941 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3689357696}
W20260709 12:27:38.187219  9941 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 2065) took 4634 ms (client timeout 2468 ms). Trace:
I20260709 12:27:38.187997  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:38.189078 19441 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:38.189646 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:38.190140 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 109868us
W20260709 12:27:38.187773  9941 rpcz_store.cc:269] 0709 12:27:33.552219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:33.552328 (+   109us) service_pool.cc:224] Handling call
0709 12:27:38.186661 (+4634333us) spinlock_profiling.cc:243] Waited 2.85 s on lock 0x7b54001b9254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:38.186964 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6563635584}
I20260709 12:27:38.179167  9944 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "840d8356270745f492c7d2c7099794c1"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:27:38.193044  9944 raft_consensus.cc:606] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:38.180498 10674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 1956) took 1825 ms (client timeout 1952 ms). Trace:
I20260709 12:27:38.193426  9944 raft_consensus.cc:614] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:38.182510 19402 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:38.085601 19575 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.187583  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:38.194072 10674 rpcz_store.cc:269] 0709 12:27:36.354957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.363505 (+  8548us) service_pool.cc:224] Handling call
0709 12:27:37.733123 (+1369618us) spinlock_profiling.cc:243] Waited 1.73 ms on lock 0x7b5800115b98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:27:38.180443 (+447320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3968256}
I20260709 12:27:38.185997 19319 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:38.217710  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 136886us
I20260709 12:27:38.213586 10020 pending_rounds.cc:107] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 30.18
I20260709 12:27:38.222473 10020 raft_consensus.cc:2889] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:38.226769 10016 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:38.227991  9991 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:38.224097  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:38.230772  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:27:38.231065 10021 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:38.232213 10021 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:38.236786  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.237183  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:38.237694 19421 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:38.238662  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:38.238543 19421 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.240969 19421 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:38.244508 19163 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.860s	user 0.014s	sys 0.008s
I20260709 12:27:38.245987 19422 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:27:38.250010 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 67877.6us
I20260709 12:27:38.222724 19396 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:27:38.254948 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:38.230777 10017 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:38.222354 19542 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:38.224146 19494 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:38.275470 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4179) took 1094 ms. Trace:
W20260709 12:27:38.279474  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 183475us
I20260709 12:27:38.280030 19523 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:38.283516 19329 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:38.288353 10017 rpcz_store.cc:276] 0709 12:27:37.175002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.198107 (+1023105us) service_pool.cc:224] Handling call
0709 12:27:38.224584 (+ 26477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.267438 (+ 42854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.268930 (+  1492us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.270115 19542 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.288702 19494 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:38.322479 10022 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:38.324977 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4175) took 1228 ms. Trace:
I20260709 12:27:38.327802 10020 rpcz_store.cc:276] 0709 12:27:37.086785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.161781 (+1074996us) service_pool.cc:224] Handling call
0709 12:27:38.162119 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.314797 (+152678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.315068 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:27:38.326929 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4176) took 1231 ms. Trace:
I20260709 12:27:38.330963 10022 rpcz_store.cc:276] 0709 12:27:37.095771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.180657 (+1084886us) service_pool.cc:224] Handling call
0709 12:27:38.267172 (+ 86515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.324331 (+ 57159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.326847 (+  2516us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:38.325901  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 210856us
I20260709 12:27:38.316018 19542 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:38.246802 19163 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.326975 19494 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:38.357703 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.364907 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.329461 19402 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:38.313824 18941 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.373675 19120 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:27:38.374302 19319 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:38.375818 18941 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:38.368925  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:38.383100 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.369491 19402 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:38.402101 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.413251  9945 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:38.400951 19402 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:38.421043 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 218174us
W20260709 12:27:38.423293 19588 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:27:38.421192 10098 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:38.434782 19471 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:38.418533 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48270.5us
I20260709 12:27:38.452332 18941 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.453198 18941 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:38.454824 19328 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:38.455749 19328 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.458083 19328 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:38.465319 19376 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:38.480441 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46514.4us
I20260709 12:27:38.461345 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.464485  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:38.464287 10016 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:27:38.485530 10016 pending_rounds.cc:85] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 15
I20260709 12:27:38.485878 10016 pending_rounds.cc:107] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:27:38.486171 10016 raft_consensus.cc:2889] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:38.442307 10024 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:27:38.499282 10094 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:38.500174 19525 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:38.501919  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:38.503775  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:38.504725 19207 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:38.505236 19207 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:38.505681 19207 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:38.505851 19577 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:38.506837 19577 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.516510 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.517540 10674 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:27:38.518731 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.533131 19551 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.534071  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3480) took 1150 ms. Trace:
I20260709 12:27:38.534098 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.533849 19551 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.534490  9945 rpcz_store.cc:276] 0709 12:27:37.383514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:37.901333 (+517819us) service_pool.cc:224] Handling call
0709 12:27:38.147161 (+245828us) spinlock_profiling.cc:243] Waited 36.1 ms on lock 0x7b5800000f98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:27:38.147435 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.413924 (+266489us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.533832 (+119908us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b5400096354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:38.534015 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644995712}
I20260709 12:27:38.540812 10672 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:38.541208 10669 raft_consensus.cc:2410] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 27 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:27:38.540830 19551 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:38.542013 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.541550 19441 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.543856  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:38.545758 10021 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 16. (term mismatch)
I20260709 12:27:38.546085 10677 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:38.549252 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:38.556166 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.558565 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.559192 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:38.559566 10671 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:38.560909 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.562068 10668 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:38.563282 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.562372 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.564589 10673 raft_consensus.cc:2410] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:27:38.566123 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.546615 10021 pending_rounds.cc:85] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 15
I20260709 12:27:38.560089 10675 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:38.569089 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4160) took 1571 ms. Trace:
W20260709 12:27:38.580348  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 211538us
I20260709 12:27:38.587101 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.519553 19120 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.536728 19421 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:38.593030  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:38.601747 10021 pending_rounds.cc:107] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:27:38.602241 10021 raft_consensus.cc:2889] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:38.603686 10670 raft_consensus.cc:2368] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
I20260709 12:27:38.598276  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:38.568295 19435 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:38.571311 19575 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:38.651883  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59499.6us
I20260709 12:27:38.645762 10668 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:38.654445  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:38.694058  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 89519us
I20260709 12:27:38.695755 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.605829 19446 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:38.705754 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4171) took 1670 ms. Trace:
I20260709 12:27:38.574052 10018 rpcz_store.cc:276] 0709 12:27:36.998052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:36.998173 (+   121us) service_pool.cc:224] Handling call
0709 12:27:36.998457 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:36.998676 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:38.050998 (+1052322us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.406447 (+355449us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:38.411181 (+  4734us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:38.417885 (+  6704us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.490673 (+ 72788us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:38.569027 (+ 78354us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8091,"prepare.queue_time_us":367169,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":8324,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":4624,"thread_start_us":1341410,"threads_started":2,"wal-append.queue_time_us":384424}
W20260709 12:27:38.713107  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 72176.9us
I20260709 12:27:38.710426 10021 rpcz_store.cc:276] 0709 12:27:37.035366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.155334 (+1119968us) service_pool.cc:224] Handling call
0709 12:27:38.155925 (+   591us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:38.705407 (+549482us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.705670 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:27:38.716714  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:38.678653 19270 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:38.658099 19163 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:38.602058 19499 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.112s
I20260709 12:27:38.736222 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:38.736588  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 248297us
I20260709 12:27:38.736091 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:38.741655 19581 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:27:38.742213 19594 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:27:38.742519 19333 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:38.746723 19256 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.140s
I20260709 12:27:38.759410 18941 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:38.786511 19441 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:38.792415 10677 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:38.791723 10024 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:38.731395 10671 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:38.647456 19376 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:38.799188  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 65343.1us
W20260709 12:27:38.804177 19419 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.698s	user 0.013s	sys 0.003s
W20260709 12:27:38.832962 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 338414us
I20260709 12:27:38.730756 19270 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:38.801143 19581 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:38.859146  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:38.689286 19435 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:38.851724 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 512173us
W20260709 12:27:38.910205 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 448727us
I20260709 12:27:38.868937 19270 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:38.914366  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 290846us
W20260709 12:27:38.914944  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 91279.4us
I20260709 12:27:38.842171 19374 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:38.916823 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:38.918011 10094 raft_consensus.cc:2376] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 27. Re-sending same reply.
I20260709 12:27:38.789942 19120 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:38.924510 10675 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:38.877547 19581 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:27:38.928550 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:38.936928  9944 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.013s	sys 0.046s
W20260709 12:27:38.964335  9944 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.014s	sys 0.046s
W20260709 12:27:38.913743 19319 consensus_meta.cc:287] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.539s	user 0.009s	sys 0.003s
I20260709 12:27:38.926865 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.975757 19377 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:38.976099 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.976501 19377 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:38.982075 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:38.982870 10024 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:38.984021 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:38.985416 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:38.986035 10024 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:38.986936 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:38.987668 10016 raft_consensus.cc:2368] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:27:38.988482 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:38.988644 19435 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:38.989759 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:38.990126 19598 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:27:38.992290 19402 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:38.993237 19542 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.617s	user 0.008s	sys 0.000s
W20260709 12:27:38.994329 19494 consensus_meta.cc:287] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.611s	user 0.011s	sys 0.001s
I20260709 12:27:38.995121  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:38.995689 19349 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.176s
I20260709 12:27:38.995136 10090 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:27:38.919265 19419 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:39.045410  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53245.3us
I20260709 12:27:39.052857 10099 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:39.053432 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:38.975837 19374 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.070905 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:39.072121 10677 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:39.046154 19494 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.073400 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:39.075815  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:39.076862  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:39.078867 19256 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:39.079039 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2952) took 1694 ms. Trace:
I20260709 12:27:39.079433 19256 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:39.079658 19333 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:39.080171 19333 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:39.080451 19256 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.081139 19333 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.079445 10023 rpcz_store.cc:276] 0709 12:27:37.384596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.247269 (+862673us) service_pool.cc:224] Handling call
0709 12:27:38.247609 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.248135 (+   526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.248162 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.248620 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:27:38.550478 (+301858us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:38.550521 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.550583 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.078813 (+528230us) raft_consensus.cc:1685] finished
0709 12:27:39.078843 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.078982 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":87,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"thread_start_us":301422,"threads_started":1,"wal-append.queue_time_us":828266}
I20260709 12:27:39.084403 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:39.084597 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.084982 10093 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:39.085762 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.087205 19471 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:39.088905 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:39.090381 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:39.091019 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:39.092084 10092 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:27:39.093637 19374 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:39.086395 10023 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:39.095223 19377 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.090263 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:39.091096 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:39.100652 10097 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:39.074368 19494 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:39.111143 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:39.112919 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.133580 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:39.123775 19376 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:39.126112 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 114402us
I20260709 12:27:39.101893 19402 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:39.067453 19207 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.561s	user 0.014s	sys 0.000s
I20260709 12:27:39.127015 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.042912 19542 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.128765 10016 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:27:39.135357 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.097447 19374 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.136866 10677 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:39.144382 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.146036 10023 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:39.145026 10677 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
W20260709 12:27:39.144786 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54652.5us
I20260709 12:27:39.131139 19120 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:39.153429 10674 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:39.139549 19328 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:39.156876 10023 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 14. (term mismatch)
I20260709 12:27:39.157397 10023 pending_rounds.cc:85] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 13
I20260709 12:27:39.157672 10023 pending_rounds.cc:107] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 23.14
I20260709 12:27:39.157955 10023 raft_consensus.cc:2889] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:39.165562 10022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.014s	sys 0.020s
W20260709 12:27:39.165984 10022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.014s	sys 0.020s
I20260709 12:27:39.145287 19207 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.165290 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2969) took 1530 ms. Trace:
I20260709 12:27:39.193104 10093 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:39.196623 10098 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:39.197265 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:39.193603 19328 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.180531 19163 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:39.230623 10640 leader_election.cc:400] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:27:39.229166 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:39.231642 10094 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
W20260709 12:27:39.233834 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58825.8us
I20260709 12:27:39.239355 10095 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:39.242100 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:39.246448 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.252964 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.255640 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.256517 10677 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:39.259238 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:39.264324 10020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.693s	user 0.008s	sys 0.029s
W20260709 12:27:39.264775 10020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.008s	sys 0.029s
W20260709 12:27:39.266247 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43883.2us
I20260709 12:27:39.182524 10017 rpcz_store.cc:276] 0709 12:27:37.635392 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.326294 (+690902us) service_pool.cc:224] Handling call
0709 12:27:38.354142 (+ 27848us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.378458 (+ 24316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.378493 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.396861 (+ 18368us) log.cc:844] Serialized 26 byte log entry
0709 12:27:38.876918 (+480057us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:38.911206 (+ 34288us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.164040 (+252834us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.164125 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.164168 (+    43us) raft_consensus.cc:1685] finished
0709 12:27:39.164180 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.165205 (+  1025us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4562,"prepare.queue_time_us":31590,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":13895,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":20849,"spinlock_wait_cycles":55334656,"thread_start_us":702939,"threads_started":2,"wal-append.queue_time_us":695178}
W20260709 12:27:39.274679 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 99592.9us
I20260709 12:27:39.282395 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:39.250032 19163 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.250279 19328 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.286041 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:39.287161 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.287810 10677 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
W20260709 12:27:39.292578  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78917us
I20260709 12:27:39.296237 10676 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:39.288852 19441 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:39.268029 19270 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.288216 19550 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:39.306103 19163 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.294456 19536 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:39.314054 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2974) took 1676 ms. Trace:
I20260709 12:27:39.335125 10673 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:39.324232 10020 rpcz_store.cc:276] 0709 12:27:37.637952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.329012 (+691060us) service_pool.cc:224] Handling call
0709 12:27:38.369933 (+ 40921us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.398574 (+ 28641us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.398657 (+    83us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.415227 (+ 16570us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.265154 (+849927us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:39.284136 (+ 18982us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.285280 (+  1144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.285356 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.313600 (+ 28244us) raft_consensus.cc:1685] finished
0709 12:27:39.313679 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.313938 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5562,"prepare.queue_time_us":17826,"prepare.run_cpu_time_us":1001,"prepare.run_wall_time_us":3699,"raft.queue_time_us":8623,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":21271,"spinlock_wait_cycles":4866944,"thread_start_us":694192,"threads_started":1,"wal-append.queue_time_us":759877}
I20260709 12:27:39.341552 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:39.342204 10673 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:39.320001 19536 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.313284 19207 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:39.345829  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 503263us
I20260709 12:27:39.332486 19163 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.016s
I20260709 12:27:39.348831 19270 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:39.349221 19542 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.349880 19542 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:39.350226 19542 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:39.351047 19542 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.351750 19581 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:39.352785 19615 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:27:39.354125 19536 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:39.354583 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2976) took 1715 ms. Trace:
I20260709 12:27:39.355147 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.355273 19543 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:27:39.356312 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.356978 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
I20260709 12:27:39.357793 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.354022 10016 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 32. Current term is 33. Ops: []
W20260709 12:27:39.364562 19600 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.627s	user 0.005s	sys 0.000s
W20260709 12:27:39.366557 19597 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.650s	user 0.003s	sys 0.005s
W20260709 12:27:39.367038 19601 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.583s	user 0.003s	sys 0.003s
I20260709 12:27:39.367805 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 2949) took 2045 ms. Trace:
I20260709 12:27:39.368191 10019 rpcz_store.cc:276] 0709 12:27:37.322311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.183115 (+860804us) service_pool.cc:224] Handling call
0709 12:27:38.183729 (+   614us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.184330 (+   601us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.184386 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.185101 (+   715us) log.cc:844] Serialized 26 byte log entry
0709 12:27:38.383821 (+198720us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:38.383878 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.383948 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.365967 (+982019us) spinlock_profiling.cc:243] Waited 41.9 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:39.367517 (+  1550us) raft_consensus.cc:1685] finished
0709 12:27:39.367538 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.367744 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19763,"raft.queue_time_us":8306,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5314,"spinlock_wait_cycles":96292608,"thread_start_us":198183,"threads_started":1,"wal-append.queue_time_us":410266}
I20260709 12:27:39.369890 10671 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:39.370872 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4178) took 2214 ms. Trace:
I20260709 12:27:39.371291 10025 rpcz_store.cc:276] 0709 12:27:37.156411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.198273 (+1041862us) service_pool.cc:224] Handling call
0709 12:27:38.257744 (+ 59471us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:38.280640 (+ 22896us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:38.281012 (+   372us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.577090 (+296078us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:38.682946 (+105856us) log.cc:844] Serialized 52 byte log entry
0709 12:27:38.683411 (+   465us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:38.734744 (+ 51333us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.815370 (+ 80626us) log.cc:844] Serialized 14 byte log entry
0709 12:27:38.902414 (+ 87044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.902498 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.348994 (+446496us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b18002ecb40. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9
0709 12:27:39.350578 (+  1584us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b24003dd438. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e062ef9c4 00007f4e062edacb 00007f4e064a8dc9 00007f4e064a8a55 00007f4e064a809d 00007f4e090da10e 00007f4e090f188c 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221
0709 12:27:39.370656 (+ 20078us) raft_consensus.cc:1685] finished
0709 12:27:39.370680 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.370810 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76789,"prepare.queue_time_us":2072,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":8493,"raft.queue_time_us":102540,"raft.run_cpu_time_us":475,"raft.run_wall_time_us":26671,"spinlock_wait_cycles":193306880,"thread_start_us":287787,"threads_started":1,"wal-append.queue_time_us":502069}
I20260709 12:27:39.373729  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3191) took 1120 ms. Trace:
I20260709 12:27:39.374141  9941 rpcz_store.cc:276] 0709 12:27:38.253346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.257839 (+  4493us) service_pool.cc:224] Handling call
0709 12:27:38.266016 (+  8177us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.273970 (+  7954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.274014 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.274780 (+   766us) log.cc:844] Serialized 26 byte log entry
0709 12:27:38.588910 (+314130us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:38.589218 (+   308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:38.589676 (+   458us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.363917 (+774241us) spinlock_profiling.cc:243] Waited 947 us on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:39.373413 (+  9496us) raft_consensus.cc:1685] finished
0709 12:27:39.373437 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.373670 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2178816,"thread_start_us":312960,"threads_started":1,"wal-append.queue_time_us":461748}
I20260709 12:27:39.376768  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:39.378826 19523 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:39.381232 10674 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:39.378347  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:39.387418 19525 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:39.388859 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.389875 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.390522 10019 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:39.392501  9991 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:39.393038 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 21 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.394089 19523 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:39.394493 19523 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.394965 19523 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:39.395308 19622 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:27:39.395816 19577 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:39.397244 19593 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.710s	user 0.006s	sys 0.003s
I20260709 12:27:39.390618 19328 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:39.395849 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 108112us
I20260709 12:27:39.401188 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.401827 10673 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
W20260709 12:27:39.402696  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 92990.7us
I20260709 12:27:39.354986 10022 rpcz_store.cc:276] 0709 12:27:37.638855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.331947 (+693092us) service_pool.cc:224] Handling call
0709 12:27:38.465569 (+133622us) spinlock_profiling.cc:243] Waited 8.73 ms on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:27:38.465844 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.466415 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:38.466459 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:38.467261 (+   802us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.166335 (+699074us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:39.166991 (+   656us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.167203 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.167277 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.354304 (+187027us) raft_consensus.cc:1685] finished
0709 12:27:39.354402 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.354529 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7362,"prepare.queue_time_us":183,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":38,"raft.queue_time_us":15860,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":29954,"spinlock_wait_cycles":20069248,"thread_start_us":698528,"threads_started":1,"wal-append.queue_time_us":795874}
I20260709 12:27:39.400166 10024 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
W20260709 12:27:39.407824 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 106667us
I20260709 12:27:39.413487 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.414250 10025 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:39.421380 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:39.423105 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4488) took 1089 ms. Trace:
W20260709 12:27:39.412458  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.663s (SENT) [suppressed 25 similar messages]
I20260709 12:27:39.437633 19584 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:39.438292 19140 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:27:39.439304 19536 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.439766 19576 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.008s
I20260709 12:27:39.440526 10677 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: [24.18-24.18]
I20260709 12:27:39.440610 19576 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:27:39.441064 19576 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:39.441607 19536 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:39.423704 10091 rpcz_store.cc:276] 0709 12:27:38.333852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.334347 (+   495us) service_pool.cc:224] Handling call
0709 12:27:38.334764 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:38.416598 (+ 81834us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:38.909664 (+493066us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.283679 (+374015us) log.cc:844] Serialized 14 byte log entry
0709 12:27:39.283959 (+   280us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.284657 (+   698us) log.cc:844] Serialized 26 byte log entry
0709 12:27:39.284859 (+   202us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:39.284905 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:39.284991 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:39.360626 (+ 75635us) spinlock_profiling.cc:243] Waited 1.24 ms on lock 0x7b18000b1100. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:39.413437 (+ 52811us) raft_consensus.cc:1685] finished
0709 12:27:39.413467 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:39.423031 (+  9564us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":185658,"prepare.run_cpu_time_us":1034,"prepare.run_wall_time_us":2977,"raft.queue_time_us":5467,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":16151,"spinlock_wait_cycles":12648192,"thread_start_us":607313,"threads_started":2,"wal-append.queue_time_us":385581}
W20260709 12:27:39.444279  9912 connection.cc:256] RPC call timeout handler was delayed by -1.6858116069997777s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
I20260709 12:27:39.445274 10017 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:39.455539  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:39.455902 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48870.9us
I20260709 12:27:39.456794 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:39.457489  9912 connection.cc:256] RPC call timeout handler was delayed by -1.4039086790003239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
I20260709 12:27:39.453059  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.426203 19577 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.431985 19582 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:39.466866  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:39.467403 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.442070 19576 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.20, Last appended by leader: 20, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.475409 19140 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.475821 19617 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.476009 19140 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.476418 19617 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.469076  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.471693 19374 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:39.479570 19374 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.481703 19374 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:39.463577 19581 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:39.483096 19617 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:39.482784 19577 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:39.484215  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.485970 19577 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.486069  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.484009 19576 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:39.486745 19582 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:39.464214 19452 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:39.504383  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.505905  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:39.506013  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.506840  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.507426  9948 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:39.500124 19140 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.517084  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:39.513578  9941 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 31: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:39.514025  9945 raft_consensus.cc:1275] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:27:39.515218  9948 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:27:39.518954 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:39.520489  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:39.516680  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.524317 19582 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.527096 19446 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:39.527760 19446 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:39.527927  9942 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:39.533645  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:39.530608 19617 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:39.604048 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.707293  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:39.710701  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 177080us
I20260709 12:27:39.712080  9991 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:39.713299  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 240108us
I20260709 12:27:39.720448 19118 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.721167 19118 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:39.536559 19581 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:39.727622  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:39.732666 19270 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:39.734184 19270 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:39.735517 19616 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:39.735992 19616 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:39.737097 19616 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.740347 19616 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:39.743983  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:39.745321 19376 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:39.745690 19376 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:39.746065 19376 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:39.747068 19446 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.748863 19446 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:39.718873 19374 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:39.751371 19374 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:39.751856 19374 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:39.761951 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 257948us
W20260709 12:27:39.763793  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49598.3us
I20260709 12:27:39.714056 19452 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:39.708453 19163 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.713707 19435 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:39.787374  9947 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:39.794853  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:39.795593  9942 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:39.809891  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.811800 19499 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:39.812415 19499 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:39.827026 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:39.828855 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:39.830978 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.831673 10017 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:39.834162 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.834724 19435 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:39.835327 19435 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:39.836084 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.836823 10022 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:39.837298 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.837865 19499 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.838068 10017 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:39.838156 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.838896 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:39.839447 10023 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:39.837671 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.841702 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.843245 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.843815 10021 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:39.847007  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:39.837478 19435 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.850637 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.853627 10024 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:39.849248 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.852401  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.856925 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:39.858246  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.859108  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:39.862567 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.771032 19617 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.070s
I20260709 12:27:39.865509 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:39.866699 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:39.867950  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 70681.6us
I20260709 12:27:39.869019 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:39.867573 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:39.866977  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.867486  9944 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:39.878388 10669 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 33 because replica has last-logged OpId of term: 32 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 19.
I20260709 12:27:39.880380 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:39.855652 19584 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:39.858963 18941 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:39.881165 10677 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 21.
I20260709 12:27:39.802448 19376 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.796412 19163 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:39.875914 19256 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:39.907927  9943 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:39.908934 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:39.911245 10024 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 21.
I20260709 12:27:39.920481  9941 raft_consensus.cc:1240] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:39.926219 10673 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:27:39.928156 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:39.928799 10025 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:39.910961 19256 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:39.941926 10025 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:39.949652 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:39.900251 19584 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:39.954555 19639 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:27:39.959398 10674 pending_rounds.cc:85] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 18
I20260709 12:27:39.959740 10674 pending_rounds.cc:107] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 25.19
I20260709 12:27:39.959911 10670 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1974) took 1448 ms. Trace:
I20260709 12:27:39.959985 10674 raft_consensus.cc:2889] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:39.960103 19163 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:39.960364 10670 rpcz_store.cc:276] 0709 12:27:38.511637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.519170 (+  7533us) service_pool.cc:224] Handling call
0709 12:27:39.959714 (+1440544us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b54006d01d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:39.959858 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90425728}
I20260709 12:27:39.961985 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:39.963069 18941 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:39.963510 18941 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:39.963902 18941 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:39.967028 19118 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:39.967654 19118 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:27:39.969233 19640 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:27:39.969863 19423 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:39.971465 19422 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:39.972488 18941 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:39.979846  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:39.995976 19376 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:39.997635 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:39.998703  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:39.999028 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:40.020336 10025 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
W20260709 12:27:40.020488  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 169877us
I20260709 12:27:40.000512 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:40.025955  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:40.027848  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:40.019351 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52476.6us
I20260709 12:27:40.030330  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:39.944141 19256 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:40.051142  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.060235  9944 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:40.081552 19446 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.082154 19446 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.082530 19446 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.083279 19374 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.030331 19140 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:27:40.088032  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57795.8us
I20260709 12:27:40.088398  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.088873  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:40.014631 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61052.5us
I20260709 12:27:40.021414 10019 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:40.039857 19632 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:27:40.090378 19618 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:40.091895  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:40.092554  9944 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.093641 10024 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:27:40.047385 19617 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.035s
I20260709 12:27:39.969458 19452 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:40.030432 19319 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:40.096364  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:40.096800  9942 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:40.024704 10018 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:27:40.098380 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:57396 (request call id 1754) took 1011 ms. Trace:
I20260709 12:27:40.013228 10017 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:40.115203  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:39.969255 19118 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.109s
W20260709 12:27:40.111047 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 108963us
I20260709 12:27:40.043936 18941 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:40.044179 19423 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:40.113435 10098 rpcz_store.cc:276] 0709 12:27:39.087144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.092086 (+  4942us) service_pool.cc:224] Handling call
0709 12:27:39.193954 (+101868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:39.212128 (+ 18174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.098112 (+885984us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54002fc950. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:40.098295 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233599744}
I20260709 12:27:40.120224 10023 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:40.001101 19120 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.077s
W20260709 12:27:40.097987 19523 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.703s	user 0.013s	sys 0.000s
I20260709 12:27:40.123602  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:40.126564  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:40.129091  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 160871us
I20260709 12:27:40.109467 19140 consensus_queue.cc:1048] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.027s
I20260709 12:27:40.131009 19499 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:40.137924 19140 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:40.121073 19270 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:27:40.114203 19452 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.141620 19374 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:40.141191 19423 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.142838 19452 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:40.143167 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4497) took 1289 ms. Trace:
I20260709 12:27:40.143522 10099 rpcz_store.cc:276] 0709 12:27:38.853318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:38.865329 (+ 12011us) service_pool.cc:224] Handling call
0709 12:27:38.908390 (+ 43061us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:39.060513 (+152123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.142938 (+1082425us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54001928d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:40.143115 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331748096}
I20260709 12:27:40.111682 19319 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:40.148313  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:40.132328 19618 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:40.153745 19617 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:40.133875 19435 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:40.167601  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:40.145529 19542 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.183524  9944 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:40.155958 19319 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:40.185860 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.178843 19374 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.179908 19163 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:40.145318 19523 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.199144 19163 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:40.206951 19617 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.025s
I20260709 12:27:40.212880 19582 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:40.233477 10016 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:40.236552 19582 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.237764 19581 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:40.238198 19581 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:40.239248 19582 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:40.229817 19284 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:40.265834  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 154192us
I20260709 12:27:40.240509 19523 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:40.257788  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:40.271217  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.273110  9949 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:40.269065 19284 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.276250 19163 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.290087 19374 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:40.295498 19584 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:40.302361 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 178772us
W20260709 12:27:40.320276 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 199170us
W20260709 12:27:40.322381 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 203052us
I20260709 12:27:40.326910 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.321878 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:40.325959 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 211381us
I20260709 12:27:40.283286 19581 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.337438 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:40.342403  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:40.343393 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 437067us
I20260709 12:27:40.347028  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.347061  9940 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
W20260709 12:27:40.348143  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 148564us
I20260709 12:27:40.350325  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.353299  9990 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:40.350828  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:40.366947 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.324110 19582 consensus_queue.cc:1059] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.022s
I20260709 12:27:40.366658 19435 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:40.374496 19616 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.005s
I20260709 12:27:40.388531 19256 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.389515 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:40.399248 10099 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:27:40.392045 19642 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:27:40.393086 19446 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.394929 19284 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:40.373073 19328 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.396337 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:40.416142 10098 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:27:40.417687  9946 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29. Ops: []
W20260709 12:27:40.400143  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 143734us
W20260709 12:27:40.400867 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.384145 19584 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:40.401170 19550 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.417682 19652 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:40.432575 19374 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.465879 19550 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.466398 19374 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.466897 19374 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.468232 19550 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:40.433131 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:40.410122 19435 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:40.433838 19651 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:40.469871 19651 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.470292 19651 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:40.470858 19435 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:40.471227 19163 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:40.435341 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:40.415673 19328 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.474499 19328 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.438196 19446 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:40.440585 19641 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:27:40.423554 19525 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:40.446074 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:40.458400 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:40.483891 10097 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:40.460264 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1674) took 1147 ms. Trace:
I20260709 12:27:40.461082 19140 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:40.484552 10674 rpcz_store.cc:276] 0709 12:27:39.313041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.381961 (+ 68920us) service_pool.cc:224] Handling call
0709 12:27:39.959182 (+577221us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:39.960753 (+  1571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:39.960795 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:39.961246 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.172168 (+210922us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:40.184807 (+ 12639us) log.cc:844] Serialized 14 byte log entry
0709 12:27:40.185036 (+   229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.188579 (+  3543us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:40.395769 (+207190us) raft_consensus.cc:1685] finished
0709 12:27:40.395798 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:40.460079 (+ 64281us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54006dfe50. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:40.460200 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12863,"prepare.run_wall_time_us":2905,"raft.queue_time_us":4733,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8120,"spinlock_wait_cycles":1331610752,"thread_start_us":210508,"threads_started":1,"wal-append.queue_time_us":375667}
I20260709 12:27:40.486927 10672 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:40.451968  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:40.444702 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:40.489705 10090 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.484731 19319 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:40.487536 19647 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:27:40.432598 19256 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:40.477607 19584 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:40.507009  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.508189  9945 raft_consensus.cc:2410] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
W20260709 12:27:40.503470  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 138723us
I20260709 12:27:40.526451 10098 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:27:40.527793  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 75939.1us
I20260709 12:27:40.528941 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:40.536721 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 199361us
I20260709 12:27:40.537621 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.538245 19256 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:40.540288 19650 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:40.511596 19376 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:40.532303  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:40.546324  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.541756 10095 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:27:40.562004 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 196412us
I20260709 12:27:40.534423  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:40.570071 19550 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:40.571509 19542 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:40.571974 19542 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:40.572898 19542 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.563546 19376 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:40.575271 19523 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:40.574517 19542 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:40.575348 19376 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:40.575810 19523 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.576313 19523 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:40.578205 19655 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:40.578724 19655 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.579082 19655 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:40.578102 19584 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.580173 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:40.597996 10090 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
W20260709 12:27:40.588387 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.604318  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.599398  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:40.606096  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 72074.6us
W20260709 12:27:40.655663  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56385.8us
I20260709 12:27:40.581733 19422 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:40.655939  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:40.607299 19667 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:40.607935 19256 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:27:40.610948 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 73412.9us
I20260709 12:27:40.615051 19376 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.633752 19499 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.644884 10097 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:40.658532 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.663798  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:40.667095  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:40.668962 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:40.671584 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:40.673033  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:40.705879  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:40.709159 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 382412us
I20260709 12:27:40.711642 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.709261 19582 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.715659 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:40.736323 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:40.657883 19422 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 22 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:40.657828 19667 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:40.663231 19618 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:40.738166  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 73071.2us
W20260709 12:27:40.740268  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.742504 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:40.746867 10677 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 31: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:40.747285 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:40.747773  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 396998us
I20260709 12:27:40.707338 19333 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:40.716460 19644 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:40.713299 19664 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.737211 19435 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:40.754729 10676 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:27:40.759305  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.747390 19207 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:40.743903 19270 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.742642 19667 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.705175 19499 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:40.766017 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54451.6us
I20260709 12:27:40.768790 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:40.749868 19663 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:40.723489 19650 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:27:40.753582 19319 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:40.781090 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:40.784961 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 126859us
I20260709 12:27:40.766489 19270 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.773046 19618 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:40.773592  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:40.776273 19499 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:40.755885 19644 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:40.735889 19582 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.806993 19665 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.807696 19665 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:40.808058 19665 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.760826 19664 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:40.809132 19582 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:40.803133  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54000.8us
I20260709 12:27:40.770912 19207 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:40.795492 19270 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.811094 19207 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.812580 10669 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:27:40.806937 19644 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:40.816637 19632 raft_consensus.cc:3055] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:40.817047 19632 raft_consensus.cc:740] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:40.817226  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43722.7us
I20260709 12:27:40.822639 19664 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:40.817948 19632 consensus_queue.cc:260] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.824398 19632 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:40.826586 19651 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.832137 19270 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.823235 19650 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:40.842108 19650 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:40.842569 19650 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:40.844973 19376 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:40.837882 19665 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.835755  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.847048 19665 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:40.849175  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.836318 19678 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:40.851032 19499 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:27:40.851564 19499 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:40.837056 19651 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:40.852499 19499 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.781275 19661 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:40.853771 19661 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:40.783565 19256 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:27:40.855435 19256 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:40.855787 19256 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:40.856637 19256 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.857456 10670 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:40.857934 19256 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.859395 19653 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:40.859069 19661 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.859782 19653 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:40.860749 19653 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.861899 19653 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:40.859081 10676 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:40.828491 19581 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:40.789451 19617 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:40.867216 10672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 1991) took 1018 ms. Trace:
I20260709 12:27:40.883002 10672 rpcz_store.cc:276] 0709 12:27:39.849288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:39.849685 (+   397us) service_pool.cc:224] Handling call
0709 12:27:40.823781 (+974096us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b5400544f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:40.867132 (+ 43351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1401757952}
W20260709 12:27:40.869351 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.869210 19163 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.874894 19678 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:27:40.875496 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:40.875922 19651 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:40.877570 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:40.881094 10099 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 33.
W20260709 12:27:40.772992  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 115711us
I20260709 12:27:40.885782  9987 leader_election.cc:400] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
W20260709 12:27:40.887378  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 112501us
I20260709 12:27:40.890967  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:40.889696  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.891155 10668 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:40.899171  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:40.900161  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.901294  9987 leader_election.cc:403] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:27:40.898976 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:40.903223  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.904866 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:40.905628 10671 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:40.910410  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:40.915550 10097 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:27:40.890803 19376 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.028s
W20260709 12:27:40.917554 10642 leader_election.cc:336] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.619s (SENT)
I20260709 12:27:40.919354 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:40.914877 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.921598 10674 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:40.914220  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:40.919926 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:40.842389 19655 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:40.886749 19678 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.887497 19617 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:40.928224 10096 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:40.917613 19656 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 23. Current term is 24.
I20260709 12:27:40.921507 19435 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:40.953351 10090 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:40.925237 19618 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:40.893537 19163 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.927335 19669 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:40.948596  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:40.965010  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 100924us
I20260709 12:27:40.937348 19617 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:40.899219 19651 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.934926 19678 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:40.915194 19679 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
W20260709 12:27:40.980409 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 85742.9us
I20260709 12:27:40.957418 19676 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:27:40.926409 19667 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:40.939136 19523 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:40.977418 19374 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.001s
I20260709 12:27:40.952222 19525 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:40.989640 19328 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.515s	user 0.006s	sys 0.005s
I20260709 12:27:40.979286 19664 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:40.967818 19618 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:40.976631 19120 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:41.003427  9945 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:41.009830  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:41.011121  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.021004 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:40.992066 19584 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:40.993997 19345 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:41.027993 19650 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.031064 19693 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:41.059458 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:40.996150 19676 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:41.061717  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50672.5us
I20260709 12:27:41.069725 10671 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:41.072364 19118 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:27:41.072839 19684 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:27:41.064081 18941 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:41.074704 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 69889.2us
I20260709 12:27:41.070753 19584 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.075601 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:41.077008 19584 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.077217 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:41.078083  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:40.995393 19374 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.094645 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:41.094995  9948 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
W20260709 12:27:41.095386 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 326722us
I20260709 12:27:41.095644 10671 raft_consensus.cc:2368] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:41.011909 19284 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.049s
I20260709 12:27:41.001046 19328 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.096662 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:41.060361 19345 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:41.001235 19525 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.020171 19664 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.105540 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.071645 19693 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:41.107867 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.109534 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.111595 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.123325 10024 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:41.117221 19665 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:41.123574  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:41.092214 19649 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:27:41.137805  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 47798.3us
I20260709 12:27:41.119531 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.119980 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:41.120381 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.124666 10673 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:41.139665 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.074723 18941 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:41.103340 19270 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.139088 10668 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:41.143836 10669 raft_consensus.cc:2410] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:27:41.142306 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:41.140072 19665 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.153677 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:41.151578 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:41.154913 10670 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:41.106902 19345 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:41.123369 19120 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:41.158035  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.158998 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:41.160163 10022 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
W20260709 12:27:41.151508 19674 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:27:41.199611  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:41.124831 19693 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:41.140955 19656 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:41.150919 18941 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:41.125480 19632 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:27:41.167905 10642 leader_election.cc:336] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.025s (SENT)
I20260709 12:27:41.142347 19649 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:41.206373  9948 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:27:41.207376 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.204897 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.207633  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:41.162761 19499 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:41.209196 19499 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:41.209936 19691 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:27:41.124396 19618 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 14, Current term: 25, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.204294 19699 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:41.210945 19328 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.211604 19699 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.212498  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.105008 19644 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.215651 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.205427 19655 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:41.219578 19690 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
W20260709 12:27:41.220152 19688 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 12:27:41.223307 19650 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.199784 19550 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:41.210964 19667 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.231293 10023 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:41.206076 19656 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:41.254763 19656 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:41.239320 19163 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:41.251724 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 83884.6us
I20260709 12:27:41.264004 19665 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.241170 19582 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:41.251631 19666 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:41.269439  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:41.268796 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.271077  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:41.272462  9944 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:41.273164  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.273743  9943 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:41.273800  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.275130 19120 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.284071 19678 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.253211 10024 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:41.253943  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:41.290073 19656 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.292405 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.292749 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.156391 19694 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.088s
I20260709 12:27:41.297070 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.297744 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.311527 10025 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:41.230329 19663 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:41.298533 19649 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:41.294023  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 155227us
I20260709 12:27:41.326026 19452 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:41.328490  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.329295  9949 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:41.329659  9943 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
W20260709 12:27:41.330054 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.299044 19345 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.299638 19651 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.299360 18941 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.207527 19676 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.252132 19374 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.303025 19617 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.305342 19699 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.305904  9940 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:41.312629 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.361157 10019 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:27:41.362110 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.363042 10019 raft_consensus.cc:2368] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25.
I20260709 12:27:41.363801 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.364213 10022 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:41.364528 10019 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 31: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:41.365401 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.365178 19689 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:27:41.365734 19499 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.365964 10019 raft_consensus.cc:2376] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 29. Re-sending same reply.
I20260709 12:27:41.366410 19689 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:41.366894 19689 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:41.367055 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.367879 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.368429 10018 raft_consensus.cc:2376] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 23. Re-sending same reply.
I20260709 12:27:41.361183 10020 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:41.328740 19452 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:41.335857 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:41.336010 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.336179 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.348551 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.371306 10017 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:41.331910 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.372123 10024 raft_consensus.cc:2376] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 25. Re-sending same reply.
I20260709 12:27:41.352294 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.373184 10025 raft_consensus.cc:2376] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 31. Re-sending same reply.
I20260709 12:27:41.357473 19678 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.358268 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.358951 19697 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.017s
I20260709 12:27:41.367118 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.385030 10019 raft_consensus.cc:2376] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cd983415389444e292b2330ef630850a in term 25. Re-sending same reply.
I20260709 12:27:41.370498 10677 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:41.385951 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.387432 10019 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:27:41.388319 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.389428 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.350533 10021 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:41.391913 10016 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 31. Current term is 32. Ops: []
I20260709 12:27:41.393193 10091 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:41.337735  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:41.377980 10098 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:41.378077 19523 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.378803  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:41.379037 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.381633 19662 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.395193 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.395954 10670 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:41.394815  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:41.397823  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 60214.4us
I20260709 12:27:41.400017 10016 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: [22.17-24.18]
I20260709 12:27:41.399785 10677 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:41.350905 19270 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
I20260709 12:27:41.404920 19662 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.406885 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.367698 19689 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.402433  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.408044 10669 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:27:41.421551 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:41.424057 10676 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 22: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:41.426180 10675 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:41.363788 19617 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.428750  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:41.422719  9987 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:41.382696 19452 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:41.435685 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.435844 10099 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:41.442340 10020 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:41.437019 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:41.437758 10670 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:41.439188 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.418960 19697 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:41.427683 19678 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:41.407192 19662 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.440958 19667 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:41.443472 19616 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:41.446429 19689 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:41.445458 19584 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:41.472360 19670 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:41.451874 19678 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:41.473942  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45267.1us
I20260709 12:27:41.475389  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.478304  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:41.478734 19667 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:41.485721  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.486383 19667 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:41.486931 19667 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:41.472930 19670 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.487756 19667 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.488958 19670 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.472000 19616 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:41.451220 19435 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.027s
I20260709 12:27:41.473272 19345 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:41.495419 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.498204 10020 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:41.496292  9912 leader_election.cc:400] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25.
I20260709 12:27:41.477003 19678 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:41.497229 19616 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:41.510393 10018 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:41.516505 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:41.523548 19656 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.458767 19697 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:41.538558  9940 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 18. (term mismatch)
I20260709 12:27:41.539059  9940 pending_rounds.cc:85] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 17
I20260709 12:27:41.539278  9940 pending_rounds.cc:107] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 24.18
I20260709 12:27:41.539543  9940 raft_consensus.cc:2889] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:41.542132 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.492641 19452 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.543589 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.544416 10017 raft_consensus.cc:2368] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
W20260709 12:27:41.544752  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 52863.2us
I20260709 12:27:41.546034  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:41.547463 19705 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:27:41.552268 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.554623 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:41.555437 19693 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.557757 19693 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.558524  9947 raft_consensus.cc:1240] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:41.574703  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:41.579617  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.582342  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:41.587563 19682 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.626s	user 0.004s	sys 0.000s
I20260709 12:27:41.588770 10093 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4515) took 1059 ms. Trace:
I20260709 12:27:41.589118 10093 rpcz_store.cc:276] 0709 12:27:40.529869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.530000 (+   131us) service_pool.cc:224] Handling call
0709 12:27:40.530329 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:40.530754 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:40.530780 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:40.776204 (+245424us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b2c00115cd8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:40.776635 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:27:40.943907 (+167272us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:40.943968 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:40.944067 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:41.562114 (+618047us) spinlock_profiling.cc:243] Waited 3.65 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:41.588576 (+ 26462us) raft_consensus.cc:1685] finished
0709 12:27:41.588596 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:41.588715 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":432,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"spinlock_wait_cycles":14593664,"thread_start_us":153817,"threads_started":1,"wal-append.queue_time_us":184764}
I20260709 12:27:41.591291  9940 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:41.572229 19452 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:41.597010 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:41.599187 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:41.603555  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.607856 19435 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:41.608924 19422 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:41.612574  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 264485us
I20260709 12:27:41.610360 19422 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:41.618178 19632 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:41.618734 19632 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:41.620936  9914 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:41.614171 19693 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 29.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:27:41.631397 19120 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:41.581054 19667 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:41.643862  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 97943.7us
I20260709 12:27:41.645323  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.642874 19680 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:41.632778 19689 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:41.658833 19678 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:41.671357  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.671478  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.659483  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:41.654729 10018 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 15. (term mismatch)
I20260709 12:27:41.675726 10018 pending_rounds.cc:85] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 14
I20260709 12:27:41.676402 10018 pending_rounds.cc:107] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 26.15
I20260709 12:27:41.677747 10018 raft_consensus.cc:2889] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:41.654958 19333 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:41.655201 19680 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:41.604648 19697 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.680537 19678 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:41.697642 19680 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.699059 19680 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:41.705451 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:41.707603 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:41.712318 10062 connection.cc:256] RPC call timeout handler was delayed by -1.282607506000204s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
W20260709 12:27:41.712766 10062 connection.cc:256] RPC call timeout handler was delayed by -1.2790075060001982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
I20260709 12:27:41.713721 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.692950 19525 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.718221 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.694325 19694 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:41.732044  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72617.7us
I20260709 12:27:41.736482 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:41.729398 19345 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:41.747229 19328 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:41.748579 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.739285 19284 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:41.840205 19284 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:41.840758 19284 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:41.732544 19694 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:41.842975 19694 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:41.847347  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 113058us
I20260709 12:27:41.849288  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.850114 19693 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:41.849905 19616 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:41.850859 19693 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:27:41.862941 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 132030us
I20260709 12:27:41.865607 19632 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:41.876108  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.897435 10062 leader_election.cc:400] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:41.897837 10062 leader_election.cc:403] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:41.898627 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:41.906034 19649 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:41.907413 19632 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:41.912928  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.907677 19452 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 27
W20260709 12:27:41.921623 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:41.921572  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 45576.6us
I20260709 12:27:41.908254 19685 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:41.925436  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.927824 19376 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:41.929234 19376 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:41.930085 19376 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:27:41.932202  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.932919 19616 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:41.923280 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:41.939355 19667 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:41.941599  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3507) took 1063 ms. Trace:
I20260709 12:27:41.942168  9945 rpcz_store.cc:276] 0709 12:27:40.877740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:40.886399 (+  8659us) service_pool.cc:224] Handling call
0709 12:27:40.887825 (+  1426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.009187 (+121362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.941271 (+932084us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540009b5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:41.941517 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265456768}
I20260709 12:27:41.945013 18941 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:41.945904 19654 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:41.946211 19667 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:41.948406 10668 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:41.949018 10022 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:41.950457 10668 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:41.951046  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:41.952651 10668 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:41.954005 10025 raft_consensus.cc:1240] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:41.950366  9941 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:41.956784  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:41.957695  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:41.958453 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:41.960103 10025 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:41.962936 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:41.964185 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:41.964125 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:41.965600 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:41.893296 19345 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:27:41.958185 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:41.930552 19376 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:41.993575 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:41.992177 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:41.996502 10096 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:41.957633 10090 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:27:41.954829 19694 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.014562 19694 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:41.978199 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57385.8us
W20260709 12:27:42.016309  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 58801.2us
I20260709 12:27:42.017705 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:42.024013 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.024948 10092 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:42.028471 10094 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:41.995085 18941 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:42.036720 10093 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:42.036865 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.034387 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.020538 19686 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:27:42.020182 19693 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:41.990799 19661 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:27:42.027174 19429 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:42.037708 19328 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:42.043919 19542 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:42.017424 19733 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.057s
I20260709 12:27:42.046761 19686 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 21 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:42.066700 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.069914 10099 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
W20260709 12:27:42.068176 19719 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:27:42.051834 19429 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.053771 19693 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:42.083055 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62563us
I20260709 12:27:42.079025 10092 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:42.082077 19632 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.073240 10094 raft_consensus.cc:2410] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
W20260709 12:27:42.083411 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 125294us
I20260709 12:27:42.084072 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.050484 19723 consensus_queue.cc:1059] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:27:42.105432 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.058084 19542 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:42.098711 19649 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:42.104496 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:42.109045 10640 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:42.044080 19697 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:27:42.047787 19698 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:27:42.112376 19429 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.113817 19429 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:42.114622 19669 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:42.084582 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:42.118104 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.109134 19542 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:42.108248 19649 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.130198 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.130483 19669 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:42.130517 19723 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.131309 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:42.132864 19706 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.654s	user 0.012s	sys 0.000s
I20260709 12:27:42.116394 19685 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:42.138072 19644 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.144737 19284 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.150796 19693 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.153086 10098 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:42.155859 19452 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:42.156987 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.123234 19654 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:42.136796 19661 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.158959 19654 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.159273 19661 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.160082 19654 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.164083 10090 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:42.162315 19661 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:42.171226 19664 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:42.152307 19284 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.108765 19632 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.127568 19735 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.135233 19733 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 12:27:42.186137 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.195108  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:42.198297 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:42.175118 19649 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.169402 19723 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.209256 19723 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.217413 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.224596 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.197667 19663 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:27:42.219156 19669 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.183749 19284 consensus_queue.cc:1059] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:27:42.238412 19649 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.239056 19632 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:42.230283 19686 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.247778 19686 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:27:42.248423 19632 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.248935 19632 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.251408 19632 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:42.252663 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:42.254899 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.256297 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.257380 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.258814 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:42.259063  9829 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7e7b0b17a4d646809d4dd46a0268f2eb with cas_config_opid_index 7: ChangeConfig:REMOVE_PEER failed with leader 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260709 12:27:42.260445 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.261859 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.205030 19542 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.262979 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:42.263617 10675 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:42.263850 10670 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:42.265139 10676 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:42.267297 19616 raft_consensus.cc:3055] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:42.267728 19616 raft_consensus.cc:740] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:42.268563 19616 consensus_queue.cc:260] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.269999 19616 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.275161 10675 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:42.277740  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.278060  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.278467  9945 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
W20260709 12:27:42.300345 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 153566us
W20260709 12:27:42.301733 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.305135  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.262452 10098 raft_consensus.cc:2376] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24. Re-sending same reply.
I20260709 12:27:42.308576 10675 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:42.309041 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.306787 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:42.313807 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:42.318066  9946 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:42.318400 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 65841.6us
I20260709 12:27:42.319144 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:42.320228 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.323458 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.327539  9947 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:42.321524 19661 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:42.338501 19736 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:42.340891 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.343250 19735 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.347813 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:42.359338  9946 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 28 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:42.324529 19663 raft_consensus.cc:3055] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:42.380986 19663 raft_consensus.cc:740] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.381773 10095 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:42.381863 19663 consensus_queue.cc:260] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.383302 19663 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:42.389334 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.390199 10090 raft_consensus.cc:2410] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 27 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:27:42.391666 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4220) took 1004 ms. Trace:
I20260709 12:27:42.392033 10017 rpcz_store.cc:276] 0709 12:27:41.387199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.559335 (+172136us) service_pool.cc:224] Handling call
0709 12:27:41.594312 (+ 34977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.594778 (+   466us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:41.595101 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 12:27:41.689686 (+ 94585us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:41.691811 (+  2125us) log.cc:844] Serialized 26 byte log entry
0709 12:27:41.692424 (+   613us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:41.692658 (+   234us) log.cc:844] Serialized 14 byte log entry
0709 12:27:41.693438 (+   780us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:41.693503 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.086942 (+393439us) raft_consensus.cc:1685] finished
0709 12:27:42.086961 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.275313 (+188352us) spinlock_profiling.cc:243] Waited 987 us on lock 0x7b18003c5f20. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:42.391505 (+116192us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b5400216bd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:42.391613 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125,"prepare.queue_time_us":896,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":78,"raft.queue_time_us":5925,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":4174,"spinlock_wait_cycles":82717568,"thread_start_us":94148,"threads_started":1,"wal-append.queue_time_us":302530}
I20260709 12:27:42.396529 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.397444 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 34 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.357273 19736 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.398072 10094 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 33.
I20260709 12:27:42.356938 19284 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:42.325729 19651 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:42.325519 19452 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:42.409492 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:42.399181  9946 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:42.410385 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.332252 19542 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:42.411676 19452 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:42.418671 19452 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:42.418983 19678 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.419379 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.419806 19701 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.420270 10097 raft_consensus.cc:2410] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 28 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:27:42.420362 19701 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.422312 19736 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.423527 19736 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:42.424109 19676 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:42.424961 19523 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.425185 19499 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:42.425621 19499 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:42.425496 19523 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.412081 19655 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:42.427420 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4215) took 1041 ms. Trace:
W20260709 12:27:42.427729  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.429147 19120 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:42.429622 19120 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:42.429664 19584 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:42.419519 19678 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.430802 19120 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.426489 19499 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.432415 19319 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:42.432942 19319 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.434973 10018 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:42.435595 10020 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: [29.18-29.18]
I20260709 12:27:42.438268 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:42.439052 10016 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:42.427922 10019 rpcz_store.cc:276] 0709 12:27:41.385823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.414172 (+ 28349us) service_pool.cc:224] Handling call
0709 12:27:41.423415 (+  9243us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:41.426802 (+  3387us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:27:41.569489 (+142687us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b18002dc340. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:27:41.648926 (+ 79437us) log.cc:844] Serialized 14 byte log entry
0709 12:27:42.417652 (+768726us) log.cc:844] Serialized 14 byte log entry
0709 12:27:42.417835 (+   183us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:42.418310 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:27:42.418447 (+   137us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:42.418480 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.418550 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:42.427214 (+  8664us) raft_consensus.cc:1685] finished
0709 12:27:42.427235 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:42.427365 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":596887,"prepare.queue_time_us":143265,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":585351,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":6234,"spinlock_wait_cycles":5146752,"thread_start_us":238379,"threads_started":2,"wal-append.queue_time_us":290178}
W20260709 12:27:42.452057 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 104354us
I20260709 12:27:42.453097 10062 leader_election.cc:400] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:42.453105  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:42.453613 10018 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 31. Current term is 32. Ops: []
I20260709 12:27:42.456369 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.456810  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:42.459686 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1812) took 1350 ms. Trace:
I20260709 12:27:42.460078 10021 rpcz_store.cc:276] 0709 12:27:41.108922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.109152 (+   230us) service_pool.cc:224] Handling call
0709 12:27:42.459490 (+1350338us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540038ccd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:42.459632 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522354432}
W20260709 12:27:42.461557 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.454101 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:42.466351 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:42.467113 10019 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:42.429265 19676 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:42.469740 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:42.470067 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 146669us
W20260709 12:27:42.476210 10642 leader_election.cc:336] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.636s (SENT)
I20260709 12:27:42.476753 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:42.480818 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4525) took 1139 ms. Trace:
I20260709 12:27:42.481254 10091 rpcz_store.cc:276] 0709 12:27:41.341504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.341752 (+   248us) service_pool.cc:224] Handling call
0709 12:27:41.342169 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:41.393583 (+ 51414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.480613 (+1087030us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b540014b4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:42.480762 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":116901632}
I20260709 12:27:42.459444 19656 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:42.459311 19499 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:42.532809 10024 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 21. Current term is 22. Ops: []
I20260709 12:27:42.488847 10090 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:42.452032 19584 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.490101 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:42.491501 19523 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.534519 10091 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:27:42.503569 19676 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.536506 10094 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:42.507201 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:42.507864 19651 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:42.513582 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:42.545770 10670 raft_consensus.cc:2368] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27.
W20260709 12:27:42.544184  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73258us
I20260709 12:27:42.515038 10668 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:27:42.546898  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:42.485706 19699 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:42.519783 10672 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:27:42.520617 19376 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.522408 10674 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:27:42.522370  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.557112  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.523998 10671 raft_consensus.cc:1240] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:42.524621 19345 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:42.561494 19345 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.562114 19662 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:42.562670 19662 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.562482 19345 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.564014 19345 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:42.563723 19662 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.525949 19678 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:42.567072 19581 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:42.570645 19120 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:42.572173 19376 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:42.574733 19699 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.577010 19699 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:42.579027 19651 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.579532 19651 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.581624 19651 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:42.583815 19584 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.584242 19662 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.586422 19676 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.592137 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:42.592818 10018 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:42.593130 10020 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:42.593472 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:42.595062 10674 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 20. (term mismatch)
I20260709 12:27:42.595427 10674 pending_rounds.cc:85] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 19
I20260709 12:27:42.595659 10674 pending_rounds.cc:107] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 32.20
I20260709 12:27:42.595645 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:42.595444 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:42.596347 10674 raft_consensus.cc:2889] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.611742 10023 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:42.614289 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:42.608981 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:42.615180 10098 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:42.618697 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:42.619844 10674 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 14. (term mismatch)
I20260709 12:27:42.615610 10668 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:42.620365 10674 pending_rounds.cc:85] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 13
I20260709 12:27:42.620760 10674 pending_rounds.cc:107] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:27:42.621109 10674 raft_consensus.cc:2889] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:42.598450  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:42.529063 10642 leader_election.cc:336] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.725s (SENT)
W20260709 12:27:42.529505 19682 log.cc:927] Time spent T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:27:42.529659  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:42.636443  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.531051 19319 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.485116 19734 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:42.471249 19525 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:42.503214  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.651394 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:42.651758  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.652176 10672 raft_consensus.cc:2410] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:27:42.541023 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4210) took 1170 ms. Trace:
I20260709 12:27:42.516062 19701 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.656594 10095 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 23 index: 19. (index mismatch)
I20260709 12:27:42.653101 10024 rpcz_store.cc:276] 0709 12:27:41.370240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:41.372931 (+  2691us) service_pool.cc:224] Handling call
0709 12:27:42.461061 (+1088130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:42.533176 (+ 72115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:42.540383 (+  7207us) spinlock_profiling.cc:243] Waited 71.5 ms on lock 0x7b540038ccd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:42.540846 (+   463us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540038ccd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:42.540967 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2630562560}
I20260709 12:27:42.554270 19664 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.553104 10022 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:42.559785 10019 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
W20260709 12:27:42.560465  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.560853 19284 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:42.630419 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.514469 19452 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.601415 10020 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:27:42.589517 19655 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:42.642863 19523 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:27:42.643785  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45490.1us
I20260709 12:27:42.635376  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.674170  9946 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:42.646924 19581 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:42.554426 19656 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:42.635100  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.657869 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:42.518162 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:42.665879 10025 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:42.620788 19676 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:27:42.670126 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 141092us
I20260709 12:27:42.684741 10675 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:42.670935 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:42.672935 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:42.686285 10672 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:42.683696  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:42.675806 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 185782us
I20260709 12:27:42.683226  9947 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:42.689210 10671 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:42.674891  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.668721 19664 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.675895 19748 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:42.675998 19685 consensus_queue.cc:1059] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.003s
I20260709 12:27:42.554061 19649 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.113s
W20260709 12:27:42.688217 10642 leader_election.cc:336] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.760s (SENT)
I20260709 12:27:42.695686 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:42.696736 19661 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.691486 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:42.700510 10642 leader_election.cc:336] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.805s (SENT)
I20260709 12:27:42.701181 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:42.705078  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.649094 18941 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:27:42.706837  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:42.715080 10642 leader_election.cc:336] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.522s (SENT)
I20260709 12:27:42.715591 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.716374  9990 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.650899  9834 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 12 to 18, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 18 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:42.717908  9990 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.731715  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.733114 10016 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:42.733243  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:42.674397 19616 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.054s
I20260709 12:27:42.739069 19656 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:42.739249 18941 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:42.739760 18941 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.740643 18941 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.741753 18941 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:42.730162 19655 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:42.743037 19655 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.729840 19661 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:42.748061 19661 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:42.748492 19661 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:42.729233 19733 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 12:27:42.752130 19734 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:42.752715 19734 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:42.754865 19584 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:27:42.738426  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:42.756008 19664 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.756685 19664 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:42.760829  9946 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:27:42.757131 19664 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:42.753825 19734 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.768376 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:42.712924 19676 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:42.795365 19676 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.736646 19701 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:42.795585 19734 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:42.796219 19701 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:42.796742 19701 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.798823 19676 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:42.786029 19581 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:42.799755 19581 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.801831 19581 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.802322 19581 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.787207 19650 raft_consensus.cc:3055] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:42.793896 19452 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:42.757539 19435 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.037s
I20260709 12:27:42.730389 19542 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.767779 19664 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:42.736069 19499 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.757789 19319 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:27:42.796221 19644 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:42.735318 19678 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:27:42.713054 19523 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:42.767055 19328 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:42.730863 19525 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:42.807121 19701 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.807101  9834 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 15 to 22, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:42.813429 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:42.821873  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:42.762517 19748 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:42.812666 19650 raft_consensus.cc:740] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:42.835557  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 172689us
I20260709 12:27:42.843603 10022 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:27:42.795590 19429 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
I20260709 12:27:42.828321 19644 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:42.888396 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:42.889379  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:42.898207  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:42.819566 19581 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.912813  9946 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:42.850483 19735 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:42.913466  9912 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:42.916054  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.914208  9834 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 18 to 21, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:42.920367  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:42.918813  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 251701us
W20260709 12:27:42.905287 19746 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:42.894831 19616 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:42.930927  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.838390 19328 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.933777  9946 raft_consensus.cc:2393] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:42.935447  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.932292  9834 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 16 to 19, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 19 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:42.918355 19735 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:42.956324  9834 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 13 to 17, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 17 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:42.959081 19616 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:42.959686 19616 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:42.960638  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:42.962790  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:42.963104 19703 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:42.964254 19701 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:42.964455  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:42.965252 19701 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:42.967290 19118 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:27:42.969175 19118 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:42.969758 19118 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:42.958716  9834 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 20 to 24. New cstate: current_term: 24 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:42.971784 19581 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:42.972506 19644 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.972693  9834 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:42.973935 19644 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:42.973829 19676 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:42.975993 19328 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:42.977336 19736 consensus_meta.cc:287] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.553s	user 0.010s	sys 0.001s
I20260709 12:27:42.981599 19656 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:42.980940 19118 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.974869  9834 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 19 to 21, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 21 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:42.951153 19735 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:42.987435  9837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 326) took 35283 ms (client timeout 15000 ms). Trace:
I20260709 12:27:42.960644 19616 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.930956 19435 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:43.012490 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.012392 19435 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:42.934558 19650 consensus_queue.cc:260] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.015079 19650 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:43.005265 10098 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:42.969987 19661 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.010286  9834 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 21 to 24. New cstate: current_term: 24 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:27:43.026192  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 270921us
W20260709 12:27:43.033008 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 340417us
I20260709 12:27:43.034775 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:43.039476  9834 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 18 to 24, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 24 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:43.047389 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:43.049038 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:43.050102 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 335032us
W20260709 12:27:43.053987 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:43.067910 10642 leader_election.cc:336] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 1.702s (SENT)
I20260709 12:27:43.069026  9834 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 21 to 25, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 25 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:43.073551 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:43.088630 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:43.094465 10676 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:43.098065  9834 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 17 to 19, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 19 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:43.067145 19736 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:43.078100 10671 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:43.103031  9912 leader_election.cc:400] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27.
I20260709 12:27:43.103523  9912 leader_election.cc:403] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:27:43.067425 19651 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.070865 19429 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:43.070432 19328 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:43.112084 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:43.101992 19744 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:27:43.081827 19669 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.070842 19748 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:43.136248 10017 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:27:43.118144 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:43.136215 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:43.139295 10019 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 30: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:43.140766 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:43.140719 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:43.141734 10024 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 31.
I20260709 12:27:43.117606  9834 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 20 to 23, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 23 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:27:43.113999  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 178086us
I20260709 12:27:43.045202 19656 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 12:27:43.142524 10671 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:43.145591 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.137545 19756 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:43.095217 19345 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:27:43.177964 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:43.134934 19693 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27.
I20260709 12:27:43.163757 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:43.172278 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:43.172453 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.172676  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.172785  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.118492 19452 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:43.175390 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:43.151978  9912 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:43.135519 19499 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.172075 19699 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:27:43.138191 19723 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.191759 10018 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:43.193140 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:43.211148 10023 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 22: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:43.193125 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:43.144173 19435 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:43.099784 19655 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:43.168118 19632 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:43.118151 19678 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:43.007256  9837 rpcz_store.cc:269] 0709 12:27:07.568627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:13.859938 (+6291311us) service_pool.cc:224] Handling call
0709 12:27:42.836434 (+28976496us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:35.178768 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:35.232270 (+ 53502us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:35.232332 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:27:35.232366 (+    34us) tablet.cc:662] Decoding operations
0709 12:27:35.234060 (+  1694us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:35.234160 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:35.234193 (+    33us) tablet.cc:685] Acquiring locks for 18 operations
0709 12:27:35.502126 (+267933us) tablet.cc:701] Row locks acquired
0709 12:27:35.502203 (+    77us) write_op.cc:260] PREPARE: finished
0709 12:27:35.502389 (+   186us) write_op.cc:270] Start()
0709 12:27:35.502559 (+   170us) write_op.cc:276] Timestamp: P: 1783600055502306 usec, L: 0
0709 12:27:35.502585 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:27:35.503380 (+   795us) log.cc:844] Serialized 7152 byte log entry
0709 12:27:37.383049 (+1879669us) op_driver.cc:464] REPLICATION: finished
0709 12:27:37.584418 (+201369us) write_op.cc:301] APPLY: starting
0709 12:27:37.584531 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:37.996016 (+411485us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:37.996183 (+   167us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:38.355423 (+359240us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:38.448171 (+ 92748us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b180036a4e0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:38.466410 (+ 18239us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=36,key_file_lookups=36,delta_file_lookups=0,mrs_lookups=0
0709 12:27:38.466497 (+    87us) write_op.cc:312] APPLY: finished
0709 12:27:38.492247 (+ 25750us) log.cc:844] Serialized 164 byte log entry
0709 12:27:38.923845 (+431598us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:38.993448 (+ 69603us) write_op.cc:454] Released schema lock
0709 12:27:38.993530 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:38.994132 (+   602us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:38.994173 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":89,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":201051,"apply.run_cpu_time_us":38982,"apply.run_wall_time_us":1409661,"cfile_cache_hit":74,"cfile_cache_hit_bytes":79403,"num_ops":18,"prepare.queue_time_us":521667,"prepare.run_cpu_time_us":29063,"prepare.run_wall_time_us":511360,"raft.queue_time_us":1049243,"raft.run_cpu_time_us":5479,"raft.run_wall_time_us":207592,"replication_time_us":1880257,"spinlock_wait_cycles":14776576,"thread_start_us":1905259,"threads_started":5,"wal-append.queue_time_us":859505,"wal-append.run_cpu_time_us":39946,"wal-append.run_wall_time_us":3401835}]]}
I20260709 12:27:43.189987 19764 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:43.119326 19651 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.219571 10098 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:27:43.192116 19676 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:43.146207 19748 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.140857 19669 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.274518 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.277717 10098 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:43.217589 19699 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.276973 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.280025 10094 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:43.207998 19701 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:43.290868  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 139001us
W20260709 12:27:43.291499 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.205902  9834 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:43.295665 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.296934 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.297987 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:43.270501 19754 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:43.300416 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.301218 10098 raft_consensus.cc:2376] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27. Re-sending same reply.
I20260709 12:27:43.302894 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.303794 10098 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:43.308544 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.309443 10092 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:43.311178 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.312132 10092 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:43.314393 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.315076 10094 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:43.305874 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.333779 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:43.334699 10098 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:43.336606 10668 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:43.339104  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:43.191623 19735 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.334563 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:43.337990  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46391us
I20260709 12:27:43.338361  9834 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 16 to 22, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:43.217049 19763 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.105s
I20260709 12:27:43.285069 19699 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:43.217877 19499 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:43.351289  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 171502us
I20260709 12:27:43.313534 19616 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:27:43.359453 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 162702us
I20260709 12:27:43.224082 19771 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.123s
I20260709 12:27:43.219027 19723 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:43.284723 19656 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:43.370656 10674 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:27:43.371044 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.371738 10670 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:43.254946 19678 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:43.364360 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 185334us
I20260709 12:27:43.371230 10093 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:43.344964 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:43.377406  9834 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 12 to 23, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 23 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:27:43.388332 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.353284 19661 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:43.390558 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.291594 19764 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.362649 19616 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.295218 19701 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:43.383508 19664 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.553s	user 0.006s	sys 0.004s
I20260709 12:27:43.408021 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:43.404567  9941 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:43.426134 10022 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:43.423092 10098 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:43.418586  9834 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 26 to 27, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 27 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:43.435178 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:43.436419 10672 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:43.439292  9834 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 22 to 26, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 26 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:43.447964 10019 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:43.442332  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:43.425436 19616 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:43.383260 19656 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:43.457921  9948 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:43.459250  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:43.460841  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:43.382874 19772 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:27:43.461709  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:43.457677 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 80887.6us
I20260709 12:27:43.397182 19693 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.063s
I20260709 12:27:43.469655 19703 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:43.450158 19435 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.486233 10676 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
W20260709 12:27:43.470876 19780 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:27:43.461354 19784 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:43.433833 19664 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.487133  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.487536  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:43.429939 19345 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
I20260709 12:27:43.470069 19766 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:43.431879 19376 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:27:43.498317  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:43.458748 19773 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.030s
I20260709 12:27:43.462209 19676 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.030s
I20260709 12:27:43.505678  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:43.508141  9948 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:43.509271 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.507303  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.510390  9940 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:43.513931  9837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 327) took 15733 ms (client timeout 15000 ms). Trace:
W20260709 12:27:43.514335  9837 rpcz_store.cc:269] 0709 12:27:27.776184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.504669 (+15728485us) service_pool.cc:211] Skipping call since client already timed out
0709 12:27:43.508855 (+  4186us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:27:43.501850 19644 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.000s
I20260709 12:27:43.516893 10024 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:43.517150  9834 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 11 to 20, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 20 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:43.518786  9834 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:43.504113 19703 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:43.525954 10019 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:43.505163 19766 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:43.493577 19768 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 12:27:43.532881  9834 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 16 to 22. New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:43.505326 19785 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:43.507272 19651 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:43.511866 19523 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:43.500070 19775 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:43.507758 19663 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:43.539628 10672 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:43.547115  9834 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 19 to 22, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 22 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:43.493225 19435 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.544749 19723 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:43.553582  9834 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 20 to 22, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:27:43.557401 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 111897us
W20260709 12:27:43.557888 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:43.545327 19766 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:43.564432 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:43.567546 19779 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:27:43.512015 19787 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.041s
W20260709 12:27:43.552858 19777 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 12:27:43.599215  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.599893  9943 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:43.600330 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:43.583729  9834 catalog_manager.cc:5697] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 16 to 22, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:43.600986 10025 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
W20260709 12:27:43.601977 19778 log.cc:927] Time spent T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:27:43.601994 19644 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.602777 19644 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.604830 19644 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:43.552533 19775 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.546471 19328 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 30.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:27:43.548303 19735 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:43.562645 19499 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:43.612617 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 110604us
I20260709 12:27:43.562307 19523 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:43.596122 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:43.595737 19345 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:43.599052 19723 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.609076 19785 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:27:43.630968 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 467291us
I20260709 12:27:43.631898 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:43.628428  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108700us
W20260709 12:27:43.622422 19776 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.193s	user 0.001s	sys 0.001s
I20260709 12:27:43.654538 10640 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:43.561643 19663 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:43.643537 10098 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:43.646220 19735 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:43.648312 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.651612 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:43.628532 19736 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.653276 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:43.660787 10091 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:43.612789 19775 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:43.659884  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:43.663358 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:43.680377 10022 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:43.663291 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:43.664402 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:43.627338 19678 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:43.641664 19785 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.686265 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:43.652645  9912 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:43.689604  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:43.615499 19644 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.045s
I20260709 12:27:43.691869 10017 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:43.661300 19735 consensus_queue.cc:1243] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer 4083ce96cefb47d0be1d8e99602b7ce7 log is divergent from this leader: its last log entry 21.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:27:43.661280 19661 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:43.660307 19651 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:27:43.660844 19789 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:27:43.670749 19701 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:43.671443 19764 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:43.641660 19345 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.654829 19723 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:43.685595 19790 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:43.699551 19376 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:43.733539 19376 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.733942 19376 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:43.723097 19792 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.037s
I20260709 12:27:43.739044 19525 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:43.742666 19523 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:43.744031 19788 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.182s	user 0.001s	sys 0.000s
I20260709 12:27:43.746825 19794 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:27:43.747799 19663 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:43.727186 19664 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:43.694458 19678 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:27:43.728168 19701 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:43.751333 19435 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:43.752295 19723 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:43.753048 19650 consensus_meta.cc:287] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.737s	user 0.008s	sys 0.002s
I20260709 12:27:43.695371 19785 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:27:43.755115 19791 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:27:43.724244 19795 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:43.756119 19795 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:43.756510 19795 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:43.759025 19760 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:43.683962 19736 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.761641 19736 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:43.764016 19345 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:43.731837 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78645.7us
W20260709 12:27:43.728322 19782 log.cc:927] Time spent T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
I20260709 12:27:43.764779  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.774837  9946 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 32.
I20260709 12:27:43.765090  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.765648 19790 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:43.767377  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.769228 19429 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:43.772683 19616 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.771235 19699 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.720675  9948 raft_consensus.cc:2468] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:43.775918  9943 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:43.776803 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:43.781544 10643 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:43.792805  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 172282us
I20260709 12:27:43.794040  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:43.794953 19581 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:43.795449 19581 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.795893 19581 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:43.798297  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:43.799122  9943 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:43.805737 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:43.806692 19345 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:43.807111 19345 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:43.807480 19345 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:43.787022 19651 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.815184 10671 raft_consensus.cc:1240] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 33. Current term is 34. Ops: [33.20-33.20]
I20260709 12:27:43.817173  9945 raft_consensus.cc:2368] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24.
I20260709 12:27:43.774348 19632 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:43.785805 19429 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:43.817893 19760 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:43.787145 19735 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.040s
W20260709 12:27:43.825601  9830 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader cd983415389444e292b2330ef630850a (127.8.137.193:34909) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica cd983415389444e292b2330ef630850a is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 25 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:43.822856 19768 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:43.842913 19766 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:43.848308 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 161064us
I20260709 12:27:43.850008 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.840075 19760 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.821198 19651 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.860330 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:43.866762 19800 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:43.866971 19810 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:43.867561 19810 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:43.867467 19800 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.868124 19810 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:43.870030 19800 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:43.866117 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:43.870817 19803 raft_consensus.cc:2804] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:43.875936  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:43.892402  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 191265us
I20260709 12:27:43.892855 10668 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:43.895694 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.895808 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.896374 10672 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:43.897302 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.900385 19819 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:43.900892 19819 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:43.901261 19819 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:43.902652 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.903244 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:43.903753 10676 raft_consensus.cc:2393] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:43.904249 10091 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:43.908813 10091 pending_rounds.cc:85] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 17
I20260709 12:27:43.909493 10091 pending_rounds.cc:107] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 21.18
I20260709 12:27:43.909828 10091 raft_consensus.cc:2889] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:43.912531 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:43.922446 19825 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.003s
I20260709 12:27:43.929247 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.857776 19760 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:43.952409 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 34 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:43.957126 19618 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:43.957743 19618 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:43.958859 19618 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:43.960714 19618 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:43.964558 19656 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:43.983935  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:43.986606 19733 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:43.987362 19733 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:43.997346 19748 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:43.999257 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:43.997377 19733 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.002581 19733 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.003742 19733 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.007619 19733 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.008554 19733 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.008939 18941 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.009500 18941 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.011518 19328 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:44.011996 19328 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.012333 19328 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:44.012419 19747 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.012986 19747 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.013429 18941 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.015111 19747 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.009403 19733 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.025164 19784 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.025825 19784 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.026446 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:44.027288 10677 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:44.028760 19542 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.029747 19792 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.029498 19542 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.030277 19792 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.030293 19686 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:44.031771 19686 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:44.032648 19792 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.032680 19686 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.034291 19686 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.029042 19784 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:44.037866 19783 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.422s	user 0.000s	sys 0.003s
I20260709 12:27:44.038957 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.040181 19542 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.040769 19655 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:44.010798  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.048652 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.049446 10016 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:44.045064  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:44.056638 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57458.5us
I20260709 12:27:44.068655 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.069592 10025 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:27:44.075099 10020 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:44.079849  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:44.085845  9945 raft_consensus.cc:2368] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29.
I20260709 12:27:44.049794  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:43.850776 19768 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.088101 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.126557 10016 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 32: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:44.089993 10643 leader_election.cc:400] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24.
I20260709 12:27:44.126937 10643 leader_election.cc:403] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 12:27:44.100752  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 116944us
I20260709 12:27:44.101554 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:43.883435 19800 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.205s
I20260709 12:27:44.112418 19703 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:44.121841 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.109467  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:44.131036  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 81323.2us
I20260709 12:27:43.932538 19429 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.085465 19734 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.137481 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.145833 10091 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:44.138187 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.147145 10095 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
W20260709 12:27:44.138486 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48638.9us
I20260709 12:27:44.140774 10020 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:44.149415 19734 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.152453 19734 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.153209 19758 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:44.153030 19819 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.153905 19758 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.154482 19758 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:44.163429 19766 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.165879  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.171013 19804 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.171787 19804 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.174624 18941 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.175181 18941 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.177078 18941 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.177547 18941 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.177990 18941 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.108043  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.142673 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:44.140728 10094 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:44.143307  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.181919 10672 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: [29.20-29.20]
W20260709 12:27:44.182845  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54898.5us
I20260709 12:27:44.187829  9991 leader_election.cc:400] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29.
I20260709 12:27:44.188261  9991 leader_election.cc:403] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:27:44.107368 19803 raft_consensus.cc:697] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:44.085943 19733 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.106271 19756 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.142014  9944 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:44.193033  9945 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
W20260709 12:27:44.193001  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 112234us
I20260709 12:27:44.195959  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.203176  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.204020  9945 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:44.201858  9942 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:44.208879 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.215623 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.220414 10098 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:44.225831  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:44.235555 19811 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 12:27:44.239398 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:44.239876  9943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:27:44.240420  9943 raft_consensus.cc:606] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:44.240670 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.241276 10099 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
W20260709 12:27:44.241626 19376 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.000s
I20260709 12:27:44.250494  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.250797  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.252781  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.253460  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.254287  9943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7253570248b8481791bf2187007dd5df"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:27:44.192248 19773 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.254891  9943 raft_consensus.cc:606] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:44.255599  9943 raft_consensus.cc:614] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:44.255414 19773 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.258690  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
W20260709 12:27:44.259141 19775 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.003s
I20260709 12:27:44.259598 19775 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.260488  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:44.261341  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:44.261785 10094 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:44.106573 19795 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.267486  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.269966 19795 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.273792 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.274952 19823 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:44.275424 19823 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.275604 10091 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:44.275897 19823 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.278020 19651 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.224041 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.195592 19656 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:44.283442 19785 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.529s	user 0.009s	sys 0.000s
I20260709 12:27:44.287734 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.317025 10643 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.293701  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:44.215549 19756 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.147518 19703 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.296777 19804 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.182475 19747 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:44.192512 19699 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:44.344583 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:44.300547  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 86620us
I20260709 12:27:44.311702  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.346728  9947 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:44.312855 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.347997 10095 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:27:44.262240 19775 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.192101 19772 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:44.326781 19581 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.530s	user 0.008s	sys 0.004s
I20260709 12:27:44.328154 18941 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:44.329998 19345 consensus_meta.cc:287] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.000s
I20260709 12:27:44.330431 19781 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:44.331211 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.336018 10091 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:44.189361 19616 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.340430 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.366458 10024 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:44.242192 19376 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.351420  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.367705  9944 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:44.346086  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.363137 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:44.341284 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 198742us
W20260709 12:27:44.372977  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.370397 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:44.258534 19773 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.298874 19790 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.342048 19703 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:44.341897 19651 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:44.417470 10640 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 2.148s (SENT) [suppressed 24 similar messages]
I20260709 12:27:44.352993 19584 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.345407 19734 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:44.359788 19662 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.386809  9949 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:44.193367 19830 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.195s
I20260709 12:27:44.382900 19663 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:44.369906 19781 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.438864 19781 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.445794  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.365828 19581 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.448783 19581 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.393309 19762 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 24.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.095s
I20260709 12:27:44.394184 19824 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.452306 19824 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.334306 19823 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.455006 10022 raft_consensus.cc:2368] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:27:44.456945 19823 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:44.395972 19808 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.257s	user 0.000s	sys 0.000s
I20260709 12:27:44.457916 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1828) took 1287 ms. Trace:
I20260709 12:27:44.458427 10019 rpcz_store.cc:276] 0709 12:27:43.170179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.170350 (+   171us) service_pool.cc:224] Handling call
0709 12:27:44.457710 (+1287360us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b540038e5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:44.457849 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":592674176}
I20260709 12:27:44.460068 10017 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:44.406733 19801 raft_consensus.cc:3060] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:44.380853 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:44.420527 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:44.464605 10091 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:44.380678 19775 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:44.430984 10094 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:44.434722 10023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.027s	sys 0.043s
W20260709 12:27:44.465754 10023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.027s	sys 0.043s
I20260709 12:27:44.466176  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:44.324142 19785 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:44.468834  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.414460 19651 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.476927 18941 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.477638 18941 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.369748 19345 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.482288 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:44.480206 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 99431.3us
I20260709 12:27:44.483268 10094 raft_consensus.cc:2393] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 30: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:44.485822 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:44.365998 19772 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.473268 19723 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:44.494872 10675 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:27:44.489992 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:44.504338 10676 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:44.504578  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:44.500527  9942 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:44.502626 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:44.505954 10094 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.468701 19775 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.391162 19803 consensus_queue.cc:237] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.510968 19785 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.511567 19693 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:44.511988 19345 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.431274 19584 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.516390 19584 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.517189 19756 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.518652 19819 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.485275 19651 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:44.519589 19810 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.001s
I20260709 12:27:44.519994 19723 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.520437 19723 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:44.521452 19758 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.523708 19758 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.523979 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.524816 10092 raft_consensus.cc:2410] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:27:44.528013 19376 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.529495 18941 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.534582 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.536649 10092 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:44.537822 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:44.535910 19772 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.538934 10092 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:44.542558  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.543653 19686 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:44.545043 19686 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:44.545367 19686 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.545668 19686 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:44.542791 10094 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:44.548805 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:44.551622  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.553033  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:44.566999 10094 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:27:44.567391  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.569813 10643 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.567930  9942 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 32.
I20260709 12:27:44.568286 10092 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
W20260709 12:27:44.575959 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:44.575450  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:44.579401 18941 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.580195 18941 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.554970 10091 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
W20260709 12:27:44.585441 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 168018us
I20260709 12:27:44.492118 19800 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:44.466960 19662 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.491835 19699 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:44.617797 19699 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.620352 19699 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.625376 19768 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.500628 19786 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:44.626278 19786 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.626783 19786 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:44.632678 19662 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.636885 19429 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:27:44.638942 19328 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.626s	user 0.006s	sys 0.007s
I20260709 12:27:44.639183  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.639854  9942 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:27:44.640990  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:44.641620  9944 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:44.607933  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.602504 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.585204 19814 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.611269 18941 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:44.585083 19632 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.612263 19756 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:44.589330 19819 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:44.691937 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89569.1us
I20260709 12:27:44.634142  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:44.606393  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 135793us
I20260709 12:27:44.608755 19758 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:44.554888 19775 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:44.644634  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 443683us
I20260709 12:27:44.673332 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:44.675966 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.639510 19328 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.702554 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:44.703105 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:44.691437 18941 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:44.703824 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:44.704011 18941 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:44.606066 19747 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 23. Current term is 24.
I20260709 12:27:44.643079  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.609123 19376 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.696997 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:44.520027 19810 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.730903 10024 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:44.691843 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 34 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:44.732138 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.732405 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.728243 18941 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.735576 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.737051 10099 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: [23.19-23.19]
I20260709 12:27:44.733599 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.740231 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.698559 19669 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.708123 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:44.752099 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:44.711166 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:44.752894 10020 raft_consensus.cc:2393] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 32: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:44.672900 19768 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:44.713305 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:44.714905  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.715282  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.717329  9944 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:44.717999  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.719125  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:44.678344 19584 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:44.727900  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 93852.4us
I20260709 12:27:44.703337 10017 raft_consensus.cc:2368] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:27:44.732235 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:44.696604 19452 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:44.760270  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:44.687954 10093 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.000s	sys 0.012s
W20260709 12:27:44.761739 10093 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.000s	sys 0.013s
I20260709 12:27:44.756198  9941 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:44.737944 19618 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:44.750016 10022 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:44.762100  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.734193 19799 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.763785 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.763325 19766 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:44.705935  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:44.766531 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53362us
I20260709 12:27:44.767745 10640 leader_election.cc:400] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
W20260709 12:27:44.767935  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62131.3us
I20260709 12:27:44.735247 19819 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:44.771045 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:44.766623 19499 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:44.697753 19814 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.697120 19632 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.712811 19758 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.800900  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:44.771801 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:44.813119 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 43515.5us
I20260709 12:27:44.737413 19581 consensus_queue.cc:1059] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.036s
I20260709 12:27:44.828670 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:44.840909 10672 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.830281 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:44.835786 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64851us
I20260709 12:27:44.737709 19771 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.039s
I20260709 12:27:44.826774 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:44.766654 19766 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:44.764577 19618 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:44.760177 19824 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:44.839293 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:44.764426 19768 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:44.753597 19686 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.838923 19654 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:44.737545 19376 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.910465 19376 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:44.850497  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 49719us
I20260709 12:27:44.912827  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:44.772333 19676 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:44.915867 10670 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:44.852819 10676 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:44.811354 19825 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:44.840406 19795 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:44.799571 19699 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:44.843658 10673 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:44.936195 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:44.904362 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:44.905129 19768 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:44.907065 19836 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:27:44.943562 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:44.773345 19799 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.850370 19766 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:44.850165 19618 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:44.954792 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:44.955698 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:44.960911 10673 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:44.961102 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:44.891899 19654 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:44.944931 19820 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:27:44.973407 19789 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:44.974159 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1837) took 1121 ms. Trace:
I20260709 12:27:44.974040 19789 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.964030 19828 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:44.968919 19733 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:44.945699 19499 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:44.974546 10023 rpcz_store.cc:276] 0709 12:27:43.853126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.853247 (+   121us) service_pool.cc:224] Handling call
0709 12:27:43.853582 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.860559 (+  6977us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:43.860579 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:43.861057 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.466167 (+605110us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:44.466209 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.466275 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:44.973936 (+507661us) raft_consensus.cc:1685] finished
0709 12:27:44.973956 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:44.974096 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":151760,"raft.queue_time_us":24144,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":51909,"thread_start_us":590641,"threads_started":1,"wal-append.queue_time_us":828422}
I20260709 12:27:44.949292 19825 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:44.969039 19800 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:44.976522 10022 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:44.976795 19654 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:44.974575 19799 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:44.949107 19824 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:44.993207 19651 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:44.949611 19759 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:44.958316 19581 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:45.002084 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:45.006338  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 363318us
I20260709 12:27:44.949291 19662 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:27:44.970213 19823 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:27:44.997217 19499 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:45.021777 10099 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:45.023197 10672 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
W20260709 12:27:45.023790  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 315327us
I20260709 12:27:44.961640 19819 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.081743 10096 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:27:45.034001 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:45.097839 10091 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:45.039777 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3415) took 1733 ms. Trace:
W20260709 12:27:45.030627  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:45.083571  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 163144us
I20260709 12:27:45.106287  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.107859  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:45.108654  9946 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.105103  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:45.014644 19581 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:45.112270  9943 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:27:45.099759 10090 rpcz_store.cc:276] 0709 12:27:43.306567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.306749 (+   182us) service_pool.cc:224] Handling call
0709 12:27:43.307139 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:43.316332 (+  9193us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:27:43.953459 (+637127us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b18000b79a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf 00007f4e09570038
0709 12:27:44.201166 (+247707us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.604015 (+402849us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.646337 (+ 42322us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.665821 (+ 19484us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:44.683242 (+ 17421us) log.cc:844] Serialized 14 byte log entry
0709 12:27:44.692505 (+  9263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.692688 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.039581 (+346893us) raft_consensus.cc:1685] finished
0709 12:27:45.039604 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.039724 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34997,"prepare.queue_time_us":611204,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":97,"raft.queue_time_us":55653,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":50332,"spinlock_wait_cycles":6950656,"thread_start_us":535896,"threads_started":2,"wal-append.queue_time_us":702942}
I20260709 12:27:45.111809  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.140213  9945 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:45.049532 19760 consensus_meta.cc:287] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 1.099s	user 0.010s	sys 0.002s
W20260709 12:27:45.034521 19845 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
W20260709 12:27:45.042722 19838 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
W20260709 12:27:45.043160 19781 consensus_meta.cc:287] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.003s
W20260709 12:27:45.041659 19703 consensus_meta.cc:287] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.630s	user 0.004s	sys 0.004s
I20260709 12:27:45.149082 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4247) took 1078 ms. Trace:
I20260709 12:27:45.162163 10025 rpcz_store.cc:276] 0709 12:27:44.070950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.071471 (+   521us) service_pool.cc:224] Handling call
0709 12:27:44.071931 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.122976 (+ 51045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.123005 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.123462 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.364902 (+241440us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b18001f14a0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:44.768807 (+403905us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:44.768879 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:44.768972 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.142644 (+373672us) raft_consensus.cc:1685] finished
0709 12:27:45.142669 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.149016 (+  6347us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50582,"raft.queue_time_us":51924,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":11868,"spinlock_wait_cycles":43418880,"thread_start_us":403677,"threads_started":1,"wal-append.queue_time_us":807792}
I20260709 12:27:45.029861 19499 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:45.145818 19581 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:45.152276  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:45.173806  9987 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:45.153734 19525 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:45.175040 19525 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.177099 19525 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.177598 19525 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:45.177455 19654 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.178236 19525 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:44.769456 19584 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.164840 19781 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.180176 19766 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.180642 19584 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.159075 19771 consensus_queue.cc:1059] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 22.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:27:45.181294 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1840) took 1144 ms. Trace:
I20260709 12:27:45.181648 19781 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.181705 10018 rpcz_store.cc:276] 0709 12:27:44.037312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.037852 (+   540us) service_pool.cc:224] Handling call
0709 12:27:44.038173 (+   321us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:44.113459 (+ 75286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.113504 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:44.114152 (+   648us) log.cc:844] Serialized 52 byte log entry
0709 12:27:44.550010 (+435858us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:45.012828 (+462818us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.013285 (+   457us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.030180 (+ 16895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.030256 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.180722 (+150466us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b1800398da0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:45.181072 (+   350us) raft_consensus.cc:1685] finished
0709 12:27:45.181094 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.181235 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70618,"raft.queue_time_us":107890,"raft.run_cpu_time_us":410,"raft.run_wall_time_us":23138,"spinlock_wait_cycles":53999488,"thread_start_us":751748,"threads_started":2,"wal-append.queue_time_us":589748}
I20260709 12:27:45.035045 19841 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.185520 19841 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.034765 19786 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.187976 19376 raft_consensus.cc:993] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:27:45.188592 19786 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:45.189347 19801 consensus_meta.cc:287] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.728s	user 0.011s	sys 0.000s
I20260709 12:27:45.189900 19801 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29.
I20260709 12:27:45.190677 19798 raft_consensus.cc:3055] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:27:45.191121 19798 raft_consensus.cc:740] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:45.182353 19766 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.196928 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 2029) took 1285 ms. Trace:
I20260709 12:27:45.039615 19789 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.089943 19656 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.030069 19759 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:45.040478 19758 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.039777 19632 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.040915 19651 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.041865 19686 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.051041 19825 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:45.046059 19814 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.143271 19328 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.165839 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:45.147887  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:45.197331 10668 rpcz_store.cc:276] 0709 12:27:43.912032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.912225 (+   193us) service_pool.cc:224] Handling call
0709 12:27:45.196737 (+1284512us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b5400610154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:45.196868 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2134552832}
I20260709 12:27:45.039017 19828 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.119s
I20260709 12:27:45.222302  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:45.222424  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:45.227130  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79434.2us
I20260709 12:27:45.233062  9940 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.233403  9946 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:45.228140 10091 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:45.199695 19525 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.155222 19669 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.062s
I20260709 12:27:45.154088 19760 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.236951 19814 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:45.166512 19703 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.236913 19525 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.239297 19676 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:45.217840 19656 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.306190 10098 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:45.240455 19584 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.223201 19856 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.087s
I20260709 12:27:45.285885 19644 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:45.321275 10091 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:45.323812 19758 raft_consensus.cc:1081] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:27:45.368607 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:45.322845 19814 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:45.278486 19798 consensus_queue.cc:260] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.20, Last appended by leader: 20, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.362766 18941 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:45.441105  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 242222us
W20260709 12:27:45.396739 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 492123us
I20260709 12:27:45.445792 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:45.444618 18941 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.455662 19816 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:27:45.456754  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 283031us
I20260709 12:27:45.485986 10640 leader_election.cc:400] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:27:45.400137 19785 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:45.524575 19785 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:45.525503 19785 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.396834 19650 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Stepping down as leader of term 31
W20260709 12:27:45.519726  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 56608.1us
I20260709 12:27:45.526860 19650 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:45.527961 19650 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.528537 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:45.529083 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:45.529424 10091 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:45.525523 18941 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.401973 19789 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:27:45.357666 19644 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.537894 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.530416 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:45.531718 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79127.4us
I20260709 12:27:45.533522 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:45.540565 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:45.534855 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.549467 10672 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:45.549587 10673 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:45.548354 10093 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.327332 19584 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.559062 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.555402 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.559722 10018 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:45.559948 10674 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:45.539400 10668 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:45.557760 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:45.558053 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:45.325021 19759 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.561086 10094 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.485412 19798 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:45.483397 19787 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:45.573442 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.573686 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.570174 10672 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:45.523561 19841 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.523284 19865 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:45.571898 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.530042 19650 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:45.587288 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.581426 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:45.558384 19855 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:45.583833 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:45.595953 10642 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:45.590263 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.584452 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:45.585564 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.575918 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.589056 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.598964 10023 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
W20260709 12:27:45.578984 19877 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:27:45.559775 19785 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:45.592842 10020 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:45.629523 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:45.630164 10019 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.450395 19676 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.592185 10671 raft_consensus.cc:2393] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:45.622529 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.643731 10669 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:45.623320 18941 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:45.644495 18941 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.644872 18941 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:45.596979 10025 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:45.623734 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.646422 10023 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:45.593981 19768 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.651202 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.652007 10023 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:45.652858 10093 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:45.625069 10668 raft_consensus.cc:2410] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 31 because replica has last-logged OpId of term: 29 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:27:45.583716 19801 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:45.584403 19786 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:45.631994 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.663859 10020 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
I20260709 12:27:45.585848 10677 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:45.635906 10094 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:45.574563 19790 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:27:45.653041 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:45.660336  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:45.640816 19825 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:45.675190  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:45.684253 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.685093 10025 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:45.600286 19856 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:27:45.579563 19787 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.667435 19823 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.575158 19867 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.087s
I20260709 12:27:45.675580 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:45.674356  9940 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:45.592003 19865 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.695443 19854 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:45.671943 19654 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:45.695150 19618 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.719s	user 0.005s	sys 0.006s
I20260709 12:27:45.690884  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:45.704269 10020 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:45.574189 19814 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.667196 19376 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 12:27:45.622759 19855 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.711427 10019 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:45.718585 18941 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.695595 19616 raft_consensus.cc:3055] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:45.693825 19790 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:45.742044 19790 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.742492 19790 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.710001 19669 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:45.671700 19801 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.693373 19703 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.733608 19837 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:45.719960 19869 log.cc:927] Time spent T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.126s	user 0.000s	sys 0.002s
I20260709 12:27:45.709481 19854 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:45.734850 19861 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:45.734490  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:45.737788 19723 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 1.217s	user 0.003s	sys 0.010s
W20260709 12:27:45.741271 19581 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.000s
I20260709 12:27:45.731813 19616 raft_consensus.cc:740] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:45.710812 19863 log.cc:927] Time spent T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.323s	user 0.001s	sys 0.000s
I20260709 12:27:45.664275 19768 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.704993 19766 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:45.752614 18941 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:45.684067 19825 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.768608  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:45.720772 19618 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:45.769165 19858 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.246s	user 0.001s	sys 0.001s
W20260709 12:27:45.781435 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 185579us
I20260709 12:27:45.767940 19861 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.767699 19854 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.771307 19758 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:45.794341 18941 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.794984 18941 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.790776 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:45.781828 19766 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:45.782248 19866 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:45.786916 19584 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.794413 19861 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.767454 19837 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.804306 18941 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:45.804975 10019 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:45.775089 19581 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.798065 19857 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:27:45.809595 19880 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:27:45.834416 19581 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.836056 19825 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.816797 19857 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.836549 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4563) took 1474 ms. Trace:
I20260709 12:27:45.836676 19837 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.814723 19871 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:45.839176 10096 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4562) took 1514 ms (client timeout 1584 ms). Trace:
W20260709 12:27:45.839603 10096 rpcz_store.cc:269] 0709 12:27:44.324532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.324716 (+   184us) service_pool.cc:224] Handling call
0709 12:27:45.839000 (+1514284us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b54002fbf54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:45.839117 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1646129536}
I20260709 12:27:45.794699 19758 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:45.842906 19758 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:45.803613 19616 consensus_queue.cc:260] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.845006 19616 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:45.847457 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3027) took 1110 ms. Trace:
I20260709 12:27:45.847875 10021 rpcz_store.cc:276] 0709 12:27:44.737146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.738036 (+   890us) service_pool.cc:224] Handling call
0709 12:27:44.754725 (+ 16689us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:44.755108 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.755136 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:44.765432 (+ 10296us) log.cc:844] Serialized 52 byte log entry
0709 12:27:45.159924 (+394492us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:45.560157 (+400233us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.566386 (+  6229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.566469 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.847241 (+280772us) raft_consensus.cc:1685] finished
0709 12:27:45.847261 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.847399 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5917,"prepare.queue_time_us":196145,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":12309,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":10352,"thread_start_us":633654,"threads_started":2,"wal-append.queue_time_us":449631}
I20260709 12:27:45.849882 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.855981 19748 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:45.856271 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.856611 19748 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.859293 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 3031) took 1088 ms. Trace:
I20260709 12:27:45.859526 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.859171 19766 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.859815 10022 rpcz_store.cc:276] 0709 12:27:44.771379 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.771535 (+   156us) service_pool.cc:224] Handling call
0709 12:27:45.859099 (+1087564us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54001f7554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:45.859228 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320267136}
I20260709 12:27:45.860307 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 3024) took 1129 ms. Trace:
I20260709 12:27:45.860707 10016 rpcz_store.cc:276] 0709 12:27:44.730779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.733292 (+  2513us) service_pool.cc:224] Handling call
0709 12:27:45.859421 (+1126129us) spinlock_profiling.cc:243] Waited 950 us on lock 0x7b5800003398. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431eee 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:27:45.860251 (+   830us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2184960}
I20260709 12:27:45.861150 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.861615 19618 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.861696 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:45.806583 19866 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.775717 19723 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.818001 19849 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:45.817523 19787 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:45.819784 19768 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.821274 19801 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.823925 19855 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.827824 19644 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.830294 19790 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.836750 19857 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:45.820286 19760 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.856964 10676 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:45.850487 10021 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:45.862537 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:45.887568 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.888276 10021 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:45.863507 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:45.871153  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.865967 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42034 (request call id 1703) took 1171 ms. Trace:
I20260709 12:27:45.887514 10022 raft_consensus.cc:2376] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27. Re-sending same reply.
I20260709 12:27:45.886997 19703 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.874825 19849 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:45.894515 19703 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.895252 19849 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.896318 19849 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:45.891662 19844 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:45.867208 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3429) took 1663 ms. Trace:
I20260709 12:27:45.897799 19844 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.869777  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.898864 19663 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:45.899338 19663 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:45.891834 19584 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:45.900949  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 2176) took 1326 ms. Trace:
I20260709 12:27:45.901284  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.901330  9947 rpcz_store.cc:276] 0709 12:27:44.577422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.577894 (+   472us) service_pool.cc:224] Handling call
0709 12:27:44.635597 (+ 57703us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:44.879355 (+243758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.879388 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:44.879433 (+    45us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:44.879465 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.900714 (+1021249us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540003e654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:45.900769 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.900894 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560063744}
I20260709 12:27:45.870409  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:45.814093 19499 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.601s	user 0.006s	sys 0.001s
I20260709 12:27:45.874306  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.880307  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.863306 10024 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
W20260709 12:27:45.881219 10098 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37540 (request call id 4560) took 1659 ms (client timeout 1782 ms). Trace:
I20260709 12:27:45.905167  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:45.905438 10098 rpcz_store.cc:269] 0709 12:27:44.221943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.223703 (+  1760us) service_pool.cc:224] Handling call
0709 12:27:45.881026 (+1657323us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400305a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:45.881163 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2337424256}
I20260709 12:27:45.870095  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:45.900099  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:45.886132  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.870875  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:45.906545  9941 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:45.897845 10097 rpcz_store.cc:276] 0709 12:27:44.204261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.205613 (+  1352us) service_pool.cc:224] Handling call
0709 12:27:44.682319 (+476706us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b5800111c98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:27:44.682610 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.702747 (+ 20137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.702782 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.703273 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.213634 (+510361us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:45.213676 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.214337 (+   661us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.866934 (+652597us) raft_consensus.cc:1685] finished
0709 12:27:45.866953 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.867152 (+   199us) inbound_call.cc:177] Queueing success response
0709 12:27:45.879117 (+ 11965us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
Metrics: {"mutex_wait_us":7959,"raft.queue_time_us":8182,"raft.run_cpu_time_us":11880,"raft.run_wall_time_us":1182404,"spinlock_wait_cycles":51866240,"thread_start_us":440898,"threads_started":1,"wal-append.queue_time_us":635258}
I20260709 12:27:45.904709 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:45.902822 19801 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:45.868788 19656 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.877929  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.931411  9946 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
W20260709 12:27:45.873310  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.907819 10640 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:45.876153 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:37528 (request call id 3437) took 1200 ms. Trace:
I20260709 12:27:45.911324  9945 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:45.933485 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:45.933274 10092 rpcz_store.cc:276] 0709 12:27:44.675423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.675628 (+   205us) service_pool.cc:224] Handling call
0709 12:27:45.875962 (+1200334us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400164c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:45.876097 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1436228352}
I20260709 12:27:45.934082 10673 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:45.934639 18941 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:45.935114 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:45.906762 10020 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:45.935851 10021 raft_consensus.cc:3060] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.936326 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:45.936992 10020 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:45.939028 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:45.893777 10675 rpcz_store.cc:276] 0709 12:27:44.694230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.694360 (+   130us) service_pool.cc:224] Handling call
0709 12:27:44.694735 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:45.228861 (+534126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:45.228911 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:45.229523 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.575169 (+345646us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:45.752695 (+177526us) log.cc:844] Serialized 14 byte log entry
0709 12:27:45.753095 (+   400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.865586 (+112491us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b54007b0cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:45.865634 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.865682 (+    48us) raft_consensus.cc:1685] finished
0709 12:27:45.865782 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.865916 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69392,"prepare.queue_time_us":183891,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":18563,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":16426,"spinlock_wait_cycles":379542912,"thread_start_us":707240,"threads_started":3,"wal-append.queue_time_us":449589}
W20260709 12:27:45.912333 19890 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:27:45.836947 10095 rpcz_store.cc:276] 0709 12:27:44.361870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:44.362032 (+   162us) service_pool.cc:224] Handling call
0709 12:27:44.381126 (+ 19094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:44.633700 (+252574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:44.633743 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:44.634344 (+   601us) log.cc:844] Serialized 26 byte log entry
0709 12:27:44.999956 (+365612us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:45.000020 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.836140 (+836120us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b5400304654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:45.836213 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:45.836281 (+    68us) raft_consensus.cc:1685] finished
0709 12:27:45.836294 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:45.836493 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":708,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"spinlock_wait_cycles":579776768,"thread_start_us":365090,"threads_started":1,"wal-append.queue_time_us":537373}
I20260709 12:27:45.886505 19120 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:45.929299 10016 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:45.879568 19865 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:45.873204 19775 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.140s	user 0.006s	sys 0.005s
I20260709 12:27:45.878314 19810 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:45.882010 19790 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.900956 19584 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.935142 18941 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.974982  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:45.869506 19866 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.941555  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:45.941728 19801 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.900167 19663 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.989084 19663 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:45.960742 19825 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:45.961540 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:45.912545 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:45.991384 10676 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:45.992251 19825 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.997866  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.999044 19825 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:45.999506  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:45.999096 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:46.000975  9914 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.001338  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:46.001379 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:46.002199 19814 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:46.003293 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:45.903319 19435 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:46.004935 19773 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:45.911547 19812 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.005457 19857 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:45.918264 19499 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:45.958447 19847 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.988s	user 0.002s	sys 0.000s
I20260709 12:27:45.876137 19723 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.906860 19734 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:45.931185 19654 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.027345  9943 raft_consensus.cc:2393] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 31: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:45.982565  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:45.941080 19523 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:45.962132 19632 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:45.952961 19656 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:46.003692  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.045030 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.003845 19856 raft_consensus.cc:2764] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:27:46.004667 19435 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.005594 19812 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:45.916872  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:46.043298 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:27:46.049022  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 132237us
W20260709 12:27:46.051174  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.007390 10017 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:45.886478 19748 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:45.969334 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:46.055887 10016 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:46.008112 10020 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:27:46.057703 10020 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:45.912564 19525 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.586s	user 0.005s	sys 0.005s
I20260709 12:27:46.017052 10673 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:46.018340 19895 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:45.967139 19120 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:46.022539 18941 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:46.025008 10090 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:46.025933 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.028746  9945 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 22.
I20260709 12:27:46.002087 19758 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.028934 10675 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:46.031215 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.031589 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.033737 19844 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:46.034432 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.075901 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.076202 10097 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:46.076678 10095 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:46.036442 10676 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: [29.20-29.20]
I20260709 12:27:46.036782 10025 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 31. Current term is 32. Ops: []
I20260709 12:27:46.079900 19435 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.082865 19812 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:46.069164 19525 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.085377 19801 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:46.086972 19499 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:46.086513 19525 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.090740 10017 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:46.092303 19854 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.038674 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:46.106976 19766 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:46.107312 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.108536 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.108669 19857 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.109117 10098 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:27:46.109998 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.039278 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:46.110991 10094 raft_consensus.cc:2368] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26.
I20260709 12:27:46.111070  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.111768 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.112378  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:46.112824 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.041998  9941 raft_consensus.cc:2468] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 33.
I20260709 12:27:46.005504 19773 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:46.114580 19773 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.074018 19895 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.047439 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.094825 10017 raft_consensus.cc:1275] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:27:46.118046 10093 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:46.118996 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.119524 10093 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:46.020929 19669 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:45.981019 19775 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.069809 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:46.039569 19790 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:46.133563  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:46.074159 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:46.094789 19861 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.061323 19632 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:46.117661 19895 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:46.097657 19758 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.098276  9944 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:46.098074  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.075304 19736 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:46.045406 19654 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.099366 19120 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.044188 19523 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.035970 19734 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.123229 10093 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:27:46.057791 19656 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.174999 19650 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:46.179355 19650 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.180058 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:46.180450 19656 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.180713 10097 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:46.182024 19650 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.182304 19734 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.104794 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:46.105538 19814 raft_consensus.cc:2764] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:27:46.062558 19651 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:46.114996  9991 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:46.184163  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 69252.6us
I20260709 12:27:46.040774 10021 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:46.185097 10091 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:46.121810 10098 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:46.125375  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.185848  9948 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:27:46.126037  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:46.131207 10090 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:46.122749 10670 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:46.134217  9947 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:46.140607  9949 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:46.188433  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.189123  9949 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:27:46.147358 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:46.149040  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:46.144922 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:46.190968 10668 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:46.150837 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:46.191277 10025 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:46.191642 10676 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:46.150864  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:46.193416  9941 raft_consensus.cc:2410] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 24 because replica has last-logged OpId of term: 23 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 12:27:46.163631  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.163843 10018 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:27:46.194620  9945 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:46.176115 10093 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:46.104161 10023 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:46.133668 19856 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:46.121201 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:46.021808 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:46.201462 10091 raft_consensus.cc:1240] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: [23.19-23.19]
W20260709 12:27:46.201936 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 180188us
I20260709 12:27:46.061939 19723 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:46.066255 19823 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:46.195569 10093 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:46.142223 19644 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:46.214329  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:46.200109 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 95422.7us
I20260709 12:27:46.154956  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.190026  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.146327 19703 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.207625  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.208932  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.148801 19795 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:46.212636 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:46.223232 10094 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:46.236340  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81441.3us
I20260709 12:27:46.227234  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.226392 10025 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
W20260709 12:27:46.227093 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 317679us
I20260709 12:27:46.216337 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:46.242043 10021 raft_consensus.cc:2393] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 23: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:27:46.223999 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 102920us
I20260709 12:27:46.243645 10643 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.224092  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:46.244589  9941 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:46.245167 10098 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:46.221607  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:46.249753  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.253981 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:46.255717 10097 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:27:46.148967 19790 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.260711 10674 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:46.261535 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:46.258518  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:46.261482 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:46.262976 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:46.213281 19656 raft_consensus.cc:2804] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:27:46.269538 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:46.325544 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.164336 19771 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.077s
I20260709 12:27:46.263345 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:46.213791 19120 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:46.235864 19756 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.291s	user 0.006s	sys 0.014s
I20260709 12:27:46.325985 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.326481 10091 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:46.326800 18941 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:46.325459 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 56014.7us
I20260709 12:27:46.329257 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:46.237360 19789 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:46.163277 19768 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.076s
W20260709 12:27:46.331862  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 73417.5us
I20260709 12:27:46.215376 19734 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:46.313102 10093 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.223765 19864 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:46.252045 19703 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:27:46.279515  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.407303 19789 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:46.215747 19856 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.457212 19789 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.335034 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.459065 10673 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 30: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:46.459295 19856 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:46.175323 19736 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:46.338403 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 80174.5us
I20260709 12:27:46.411798 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.416595 19654 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.462090 19654 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:46.438539 10093 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:46.463683 19861 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.462572 19654 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.465713 19654 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.469458 19736 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:46.364728 19581 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:46.375751 19656 raft_consensus.cc:697] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:46.477298 19656 consensus_queue.cc:237] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.477905  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.366286 19799 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.479152 19799 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.481341 19799 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.482036 19864 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.484407 19864 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.347465 19632 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.375972 19734 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.488405 19734 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:46.330474 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 60387.1us
W20260709 12:27:46.496116 19872 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.833s	user 0.003s	sys 0.000s
I20260709 12:27:46.497291 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4575) took 1464 ms. Trace:
I20260709 12:27:46.343180  9941 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:46.271071  9912 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.345304 10025 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:46.497660 10099 rpcz_store.cc:276] 0709 12:27:45.033243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.045055 (+ 11812us) service_pool.cc:224] Handling call
0709 12:27:45.045411 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:45.045785 (+   374us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:45.045818 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:45.046331 (+   513us) log.cc:844] Serialized 26 byte log entry
0709 12:27:45.403643 (+357312us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:45.403701 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:45.403795 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:46.495510 (+1091715us) spinlock_profiling.cc:243] Waited 2.59 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:46.497016 (+  1506us) raft_consensus.cc:1685] finished
0709 12:27:46.497039 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:46.497233 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1328,"raft.queue_time_us":7905,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8342,"spinlock_wait_cycles":5958400,"thread_start_us":356874,"threads_started":1,"wal-append.queue_time_us":485687}
I20260709 12:27:46.498937  9941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:34882
W20260709 12:27:46.334990  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48279.1us
I20260709 12:27:46.258325 19798 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:27:46.502103 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:46.360698 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:46.503597  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:46.503986 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 143434us
I20260709 12:27:46.246830 19785 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:46.362444 18941 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.505615 18941 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.363747 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.363830 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:46.366083 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:46.247707 19795 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.264037 19616 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:46.367939 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:46.548844 10675 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:46.515368  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:46.270392 19828 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:46.268942 19525 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:46.242750 19651 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:46.333884 19823 raft_consensus.cc:2804] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:46.323047 19762 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:46.583914 19823 raft_consensus.cc:697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:46.584018 19762 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.584872 19823 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.332129 19772 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:46.586712 19867 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:46.248665 19723 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:46.332473 19899 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:46.242100 19644 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:46.360734 19756 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.425s	user 0.007s	sys 0.014s
I20260709 12:27:46.407801 10095 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 31. Current term is 32. Ops: []
W20260709 12:27:46.499414  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 228532us
I20260709 12:27:46.456043 19703 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:46.499491  9941 raft_consensus.cc:606] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:46.510783 10090 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:46.337363 19854 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.334787 19676 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:46.521895  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.629295 10677 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:46.632699 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.633134 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:46.545861 10668 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:46.269119 19686 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.115s
I20260709 12:27:46.269295 19866 consensus_queue.cc:1048] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.115s
I20260709 12:27:46.558076 10676 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
W20260709 12:27:46.560225  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 44976.5us
I20260709 12:27:46.469123 19775 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:46.441283 19849 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.000s
I20260709 12:27:46.547801 19616 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.549491 19581 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:46.561087 19828 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:46.563095  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:46.580681 19525 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.581934 19651 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.583326 19825 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:46.589141  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.333818 19758 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.180s
I20260709 12:27:46.591715  9945 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:27:46.615972 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50087.2us
I20260709 12:27:46.616490 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.587155 19772 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:46.561833 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 59843.4us
I20260709 12:27:46.574926 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.685108 10669 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 32: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:46.550122 19771 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:46.687703  9945 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:46.688266 19892 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:27:46.510254 19798 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:46.689152 19798 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:46.689500 19773 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.690165 19773 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.619433 10092 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:46.695631 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.695919 19723 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.696934 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.697556 10092 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:46.504994 10062 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:46.566318  9947 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:46.699559 19825 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:46.699928 19825 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:46.700255 19825 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:46.700416 19883 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:46.626485 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:46.701327 19883 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:46.701692 19883 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:46.406805 19898 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:46.627779 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.702490 19703 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.628039 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:46.575515 19859 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.942s	user 0.001s	sys 0.001s
I20260709 12:27:46.644259 18941 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.705390 10065 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:46.628213 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:46.684104 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:46.707379 19772 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.709487 19781 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:46.656956  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:46.617939 19736 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.567209 19887 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.710439 19736 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.620431 19867 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.712666 19736 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.712746 19319 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:27:46.714102 10673 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:46.716202 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:46.664466  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:46.669039  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:46.617133 19760 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:46.626861 19812 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:46.672986  9943 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:46.679662 19903 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:27:46.681099  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 118107us
I20260709 12:27:46.673988 19616 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.432093 19790 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:46.706912 10090 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
W20260709 12:27:46.621516 19766 consensus_meta.cc:287] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.514s	user 0.011s	sys 0.000s
W20260709 12:27:46.718508  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61627.9us
I20260709 12:27:46.719265 10092 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:46.718762 10065 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:46.725978  9987 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:46.644860 19762 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:46.815078 10019 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:46.728055 18941 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:46.720589 19736 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.668524 19854 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:46.677723 19828 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:46.838356 10097 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.018s	sys 0.014s
W20260709 12:27:46.838719 10097 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.543s	user 0.018s	sys 0.014s
I20260709 12:27:46.698457 19584 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.688604 19855 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:46.739719 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.672583 19768 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:46.655470 19676 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.762480  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:46.671571 19775 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.065s
W20260709 12:27:46.765281  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 47685.3us
I20260709 12:27:46.758718 19771 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:46.765450  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:46.817888  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 91945.5us
I20260709 12:27:46.732281 19760 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.826483 10092 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:46.837455 18941 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:27:46.837817 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 119129us
I20260709 12:27:46.846506  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:46.721028 19898 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:46.681759 19581 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:46.852591  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 90184.3us
I20260709 12:27:46.836726 19766 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:46.877300  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:46.728586 19814 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:46.867367  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.878387  9944 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:46.879263  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:46.879820  9944 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:27:46.880136  9944 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:46.868257  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.883266 10673 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
I20260709 12:27:46.869196  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.885870  9948 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 22.
W20260709 12:27:46.874287  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 108902us
W20260709 12:27:46.887377 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 382447us
I20260709 12:27:46.888137  9990 leader_election.cc:400] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26.
I20260709 12:27:46.888454  9990 leader_election.cc:403] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:27:46.876003 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:46.889612 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.882532 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.891149 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:46.882479 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.894492 10673 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 34: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:46.880812  9944 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.896656  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.756392 19903 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:46.898623  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.898998 10094 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:46.885288 19584 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:46.892773 10675 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:46.757167 19644 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.767977 19525 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:46.711848 19867 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.902575  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.903522  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:46.904258 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:46.959125 10020 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:46.907509  9944 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:27:46.816802 19759 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:27:46.850747 19773 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:46.861366 19856 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:46.913271 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.916162 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.843151 19855 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:46.838762 19736 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.884011 19881 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:46.923790  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:46.927774 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.948120 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:46.848445 19854 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.906713 19849 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:46.901809 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:46.906112 19816 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:46.852221 19768 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:46.710712 19887 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.994629 19854 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:46.995078 19901 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
I20260709 12:27:46.996016  9949 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:46.971349 19896 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:46.702517 19883 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:46.968230  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:46.884066 19799 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.065s
I20260709 12:27:46.924510 19584 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:46.908223 19861 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.055s
I20260709 12:27:46.984385 10675 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:46.996500 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:46.995231 19816 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:46.961982  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58556.6us
I20260709 12:27:47.022543 10025 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 23.
I20260709 12:27:47.020936 10020 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:47.044917 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.045977  9944 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 32.
I20260709 12:27:47.046214 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.047528 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.048771 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:47.052980 10020 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:46.997671 19896 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:46.997620 19881 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:47.053464  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:47.056861 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: true dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:47.056972  9990 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:47.020126 19584 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:47.058131 19896 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.076300 19887 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.080101 19919 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:47.080806 19919 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:47.061362 19916 log.cc:927] Time spent T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:27:47.061300 19881 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.091238 19760 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:47.062202 19914 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 12:27:47.062767 19734 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.574s	user 0.012s	sys 0.000s
I20260709 12:27:47.062804 19618 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:27:47.063859 19917 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:27:47.065734 19775 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:47.067999 19857 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 32 current term: 32 is higher.
W20260709 12:27:47.103770  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.068610 10020 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:47.069649  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:47.072507 10025 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 32: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:27:47.056473 19120 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.686s	user 0.012s	sys 0.000s
I20260709 12:27:47.094152 18941 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:47.014451 19747 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.045s
W20260709 12:27:47.114939  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 45379.6us
I20260709 12:27:46.964682 19771 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.070737 19867 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:47.109992  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 197775us
I20260709 12:27:47.110890 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:47.113468  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 196306us
I20260709 12:27:47.115149  9940 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
W20260709 12:27:47.117130 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:47.066499 19920 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:27:47.116317  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:47.121537 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.121090  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.100596 19775 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.113328 18941 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.122327 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.125680  9990 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.128072  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:47.129515 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.130295 10025 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 34.
I20260709 12:27:47.095947 19734 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.110737 19814 raft_consensus.cc:2764] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election decision vote started in defunct term 28: won
I20260709 12:27:47.115140 19676 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.168920 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 2035) took 1138 ms. Trace:
I20260709 12:27:47.169335 10674 rpcz_store.cc:276] 0709 12:27:46.030826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.030987 (+   161us) service_pool.cc:224] Handling call
0709 12:27:47.168757 (+1137770us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540045ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:47.168862 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434981632}
I20260709 12:27:47.170230 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.170910 10091 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:47.171322 10017 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:27:47.171684 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.172868 10643 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:47.175203 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4273) took 1346 ms. Trace:
I20260709 12:27:47.175606 10016 rpcz_store.cc:276] 0709 12:27:45.828570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.066516 (+237946us) service_pool.cc:224] Handling call
0709 12:27:46.066858 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.080548 (+ 13690us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:27:46.344031 (+263483us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.657557 (+313526us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:46.657637 (+    80us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:46.658858 (+  1221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.174978 (+516120us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b54001c6e54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:47.175025 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.175150 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4082,"prepare.queue_time_us":19481,"prepare.run_cpu_time_us":191,"prepare.run_wall_time_us":187,"raft.queue_time_us":27441,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":4960,"spinlock_wait_cycles":30986368,"thread_start_us":537912,"threads_started":2,"wal-append.queue_time_us":350784}
I20260709 12:27:47.178640  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:47.182039 19676 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:47.189726 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4268) took 1393 ms. Trace:
I20260709 12:27:47.191591  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 3575) took 1094 ms. Trace:
I20260709 12:27:47.190151 10022 rpcz_store.cc:276] 0709 12:27:45.796127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.918436 (+122309us) service_pool.cc:224] Handling call
0709 12:27:45.918822 (+   386us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:45.919181 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:45.919209 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:45.919752 (+   543us) log.cc:844] Serialized 52 byte log entry
0709 12:27:46.230722 (+310970us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:46.323587 (+ 92865us) log.cc:844] Serialized 14 byte log entry
0709 12:27:46.349019 (+ 25432us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:46.349128 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:47.168519 (+819391us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b1800334000. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e06469522 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:47.186094 (+ 17575us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x7b1800334000. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1
0709 12:27:47.186751 (+   657us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b1800334000. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:27:47.189504 (+  2753us) raft_consensus.cc:1685] finished
0709 12:27:47.189526 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.189659 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23498,"prepare.queue_time_us":92698,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":20487,"raft.queue_time_us":51935,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":30219,"spinlock_wait_cycles":412335872,"thread_start_us":299966,"threads_started":1,"wal-append.queue_time_us":406096}
I20260709 12:27:47.192051  9943 rpcz_store.cc:276] 0709 12:27:46.097658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.097813 (+   155us) service_pool.cc:224] Handling call
0709 12:27:47.191389 (+1093576us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b5400208354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:47.191502 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1166122880}
I20260709 12:27:47.194495 19768 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.195685  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 3277) took 1070 ms. Trace:
I20260709 12:27:47.195739 19736 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.132236 19525 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:27:47.135717  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.110224 19319 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26.
I20260709 12:27:47.139281 19759 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:47.140045 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.140777 19775 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.140944 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.137236 19857 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.004s
I20260709 12:27:47.142043 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.131242 19723 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:47.113106 19798 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.137236 19772 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.009s
I20260709 12:27:47.143831 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.145284 19812 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:47.151180 19915 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.166s	user 0.002s	sys 0.000s
I20260709 12:27:47.153292 19616 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:47.153877 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.154013 19867 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:47.156057 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.255762 19523 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.157801 19650 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.215935 19651 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.256544 19650 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.257085 19651 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:47.159876 19795 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.589s	user 0.003s	sys 0.006s
I20260709 12:27:47.259291 19651 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.213313 19723 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:47.259457 19795 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.259984 19723 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.261621 19795 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.123149 19747 consensus_queue.cc:1243] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 28.20 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 19
I20260709 12:27:47.132030 19581 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.122889 19618 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.263916 19798 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.243782 19812 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.265991 19828 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:47.163079 10019 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 4269) took 1366 ms (client timeout 1557 ms). Trace:
W20260709 12:27:47.271188 10019 rpcz_store.cc:269] 0709 12:27:45.796400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:45.808825 (+ 12425us) service_pool.cc:224] Handling call
0709 12:27:47.162866 (+1354041us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54001f7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:47.163003 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081554816}
I20260709 12:27:47.163416  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:47.164908  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:47.272783  9944 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:47.112401 19903 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:47.281805  9944 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:47.284353  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:47.286947  9947 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:47.287149  9945 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:47.165618 19816 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.165477 19855 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.196223  9942 rpcz_store.cc:276] 0709 12:27:46.125698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.125809 (+   111us) service_pool.cc:224] Handling call
0709 12:27:47.195495 (+1069686us) spinlock_profiling.cc:243] Waited 925 ms on lock 0x7b540009c4d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:47.195627 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2126361856}
W20260709 12:27:47.198549  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62897.3us
I20260709 12:27:47.294238  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.296132  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:47.135069 19825 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:47.210350 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.129092 19919 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.313359 19798 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:47.313823 19798 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.314227 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.314296 19798 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:47.303339 19319 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.012s
I20260709 12:27:47.287297 19747 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.316185 19747 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.303074 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.320134 19669 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:47.321322  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:47.174431 19748 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:47.322784 19748 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:47.323240 19748 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:47.326964 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.327592 10672 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:47.238405 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.246410 10095 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4583) took 1166 ms. Trace:
I20260709 12:27:47.330437 10095 rpcz_store.cc:276] 0709 12:27:46.079648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.153859 (+ 74211us) service_pool.cc:224] Handling call
0709 12:27:46.178328 (+ 24469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.622690 (+444362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.246176 (+623486us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400341f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:47.246349 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468292352}
I20260709 12:27:47.249271 19927 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:47.331549 19927 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:47.331041 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:47.332043 19927 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:47.332908 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 35 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.333514 19828 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:47.250954 19734 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.211148 19790 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:47.162883  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:47.334861 10093 raft_consensus.cc:2368] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31.
I20260709 12:27:47.335474 19825 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:47.267596  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58240.4us
I20260709 12:27:47.339062 19616 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.339877 19581 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:47.340379 19581 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:47.340794 19581 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:47.341220  9914 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.267742 19812 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:47.268571  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.275265 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:47.278105  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61631.5us
I20260709 12:27:47.343077 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.343655 19616 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:47.290876 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.303073 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.345063 10668 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 34.
W20260709 12:27:47.345508  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.349093 19896 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:47.349467 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.318404 19747 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.351336  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.333673 10025 raft_consensus.cc:2410] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 35 because replica has last-logged OpId of term: 33 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 19.
I20260709 12:27:47.212219 19759 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:47.280037 19856 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:47.308631 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.363382 10099 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:27:47.363673 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.364466 10090 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:47.365607 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:47.331037 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.371243 10016 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:47.354112  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:47.355064 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.373886 10094 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
W20260709 12:27:47.355705 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 53677.1us
I20260709 12:27:47.356407  9941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab"
dest_uuid: "cd983415389444e292b2330ef630850a"
mode: GRACEFUL
new_leader_uuid: "9aa345bcd43b44088114b7b6f2830b17"
 from {username='slave'} at 127.0.0.1:34882
I20260709 12:27:47.356942  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:47.382848 10094 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:27:47.385783 10094 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:47.386122 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:47.292475 19768 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.290269 19787 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.390645  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:47.394205 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.302729 19650 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.389744  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:47.387315 19319 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:47.350844 19748 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.387976 19759 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:47.397753 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:47.351804 19584 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:47.457360  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66845.3us
I20260709 12:27:47.458159  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:47.460242 10097 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:27:47.462152 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64497.6us
I20260709 12:27:47.444820 10675 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 32: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:47.447033 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:47.403034 10090 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:47.418448  9949 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
W20260709 12:27:47.451210  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 61533us
I20260709 12:27:47.444229 10016 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:47.509429 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:47.510357  9991 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.8.137.254:38721, user_credentials={real_user=slave}} blocked reactor thread for 51510.9us
W20260709 12:27:47.463969  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 76166.8us
I20260709 12:27:47.511576  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.513051  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.534498  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:47.532032 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:47.536388 10098 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:47.511426  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:47.546041  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:47.397935 19790 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:47.552731 10023 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:47.387488 19887 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.122s
W20260709 12:27:47.557348 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.559239  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:47.449810 19319 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:47.561163  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 48792us
I20260709 12:27:47.570835 10673 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:47.571156  9990 leader_election.cc:400] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31.
W20260709 12:27:47.571558 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:47.561167  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:47.565892 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.459987 19771 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:47.575079 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1865) took 1314 ms. Trace:
I20260709 12:27:47.575093  9946 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:27:47.569434 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:47.644395 10020 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.044s	sys 0.068s
W20260709 12:27:47.644855 10020 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.044s	sys 0.068s
I20260709 12:27:47.448264 19841 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.114s
I20260709 12:27:47.533598 19856 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:47.464267 19120 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:47.550647 19881 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:47.673758  9946 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:27:47.664511 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:47.679176 10065 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:47.682029 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:47.575598 10021 rpcz_store.cc:276] 0709 12:27:46.260423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.380141 (+119718us) service_pool.cc:224] Handling call
0709 12:27:46.439748 (+ 59607us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:46.440041 (+   293us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:27:46.589846 (+149805us) log.cc:844] Serialized 14 byte log entry
0709 12:27:47.198713 (+608867us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b1800207ca0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f623a 00007f4e09560a2a
0709 12:27:47.574768 (+376055us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:47.574819 (+    51us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:47.574860 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.574920 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:47.575025 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":76970,"prepare.queue_time_us":149701,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":2361,"spinlock_wait_cycles":56022784,"thread_start_us":361515,"threads_started":1}
I20260709 12:27:47.653795 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:47.657020 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:47.560842 19844 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:47.560030 19790 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:47.456218 19787 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:47.664803 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 95447.1us
I20260709 12:27:47.459779 19768 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.568782 19656 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:47.705322 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:47.578959 19814 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:47.582813 19319 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:47.727346 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:47.727898 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45994.1us
I20260709 12:27:47.729671  9945 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:27:47.550892 19644 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.161s
I20260709 12:27:47.739991 19787 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:47.740697 19881 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:47.741521 19814 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.701036 19919 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:47.706837 19654 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:47.569052 19773 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.153s
I20260709 12:27:47.691977 19781 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:47.796175 19814 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.642292 19867 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.079s
I20260709 12:27:47.578217 19857 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.149s
I20260709 12:27:47.711199 19841 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:47.710988 19933 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:47.744134 19632 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:47.769671 19656 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:47.550374 19772 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.169s
I20260709 12:27:47.798480 19919 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.690919 19771 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:47.800792 19654 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:47.812558 19723 consensus_meta.cc:287] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.552s	user 0.008s	sys 0.002s
I20260709 12:27:47.803951 19781 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:47.845741 19771 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:47.710387 19790 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:47.850850 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 83378.3us
W20260709 12:27:47.859318  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 313292us
I20260709 12:27:47.860972  9941 raft_consensus.cc:606] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Received request to transfer leadership to TS 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:47.888357  9941 raft_consensus.cc:614] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:47.870833 19913 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:47.820845 19787 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.837153 19919 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:47.853160 19654 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:47.875252 19912 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:27:47.812879 19927 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.871059 19632 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:47.889312  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:47.659729 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:47.872611 19781 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:47.899070 19795 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:27:47.900804 19937 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:27:47.903772 19762 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:47.905160 19933 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:47.839375 19881 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:47.919965 19656 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:47.890686 10065 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:47.924181 18941 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.016s	sys 0.030s
W20260709 12:27:47.972532 18941 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.720s	user 0.016s	sys 0.030s
I20260709 12:27:47.875582 19748 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.852541 19841 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:47.871706 19723 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:47.958523 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 231250us
I20260709 12:27:47.986510 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:47.909986 19896 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.942682 19584 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:47.975005 19768 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:47.977423 19935 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:47.980219 19825 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:47.991497 19810 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:47.961856 19762 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:47.945048 19787 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.006513 19748 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:47.972196 19933 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.018998 19723 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.010411 19825 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:48.023433 19825 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.010480 19810 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.028586 19883 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:48.029508 19810 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.048686 19933 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.050697 19927 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:48.084759  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 537591us
I20260709 12:27:48.097635  9948 raft_consensus.cc:1240] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:48.103056  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 213765us
I20260709 12:27:48.104573  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:48.107369  9987 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:48.119560 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4288) took 1204 ms. Trace:
I20260709 12:27:48.120191 10018 rpcz_store.cc:276] 0709 12:27:46.917572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.917722 (+   150us) service_pool.cc:224] Handling call
0709 12:27:46.918111 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:46.949418 (+ 31307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:46.949448 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:46.956692 (+  7244us) log.cc:844] Serialized 26 byte log entry
0709 12:27:47.160706 (+204014us) spinlock_profiling.cc:243] Waited 27.8 ms on lock 0x7b180025c540. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:47.457268 (+296562us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:27:47.457305 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.910795 (+453490us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b54001f7554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:47.910850 (+    55us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.118914 (+208064us) raft_consensus.cc:1685] finished
0709 12:27:48.118968 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.119295 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20360,"raft.queue_time_us":20709,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":529,"spinlock_wait_cycles":87872128,"thread_start_us":296394,"threads_started":1,"wal-append.queue_time_us":1001969}
I20260709 12:27:48.128913 19977 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:47.980036 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.135989  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:48.137053 19977 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.130259 19975 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:48.139540 19975 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:27:47.979661 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89074.1us
I20260709 12:27:48.140604 19975 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election won for term 35
W20260709 12:27:47.969875 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 310224us
I20260709 12:27:48.141096 19975 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.141582 19975 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:27:48.141932 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:48.143597 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:48.143832 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.144788 10090 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:48.131821 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3052) took 1152 ms. Trace:
I20260709 12:27:48.146799 10020 rpcz_store.cc:276] 0709 12:27:46.979092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.128622 (+149530us) service_pool.cc:224] Handling call
0709 12:27:47.128954 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:47.129132 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:47.645729 (+516597us) log.cc:844] Serialized 14 byte log entry
0709 12:27:48.130853 (+485124us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:48.130994 (+   141us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:48.131161 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.131306 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.131582 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12714,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"thread_start_us":1000208,"threads_started":3,"wal-append.queue_time_us":395252}
I20260709 12:27:48.148102 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:48.149353 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:48.134120 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.159081 10090 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:48.172360 19787 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:48.179122 19762 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.184576 10093 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:48.182273 10098 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:27:48.221848 19977 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.225965 19120 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:48.227149 19756 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:48.228901 19768 raft_consensus.cc:993] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: : Instructing follower 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:27:48.233788 18941 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:48.235455 19933 consensus_queue.cc:1059] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:48.244745 19651 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.052032 19841 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.247795 19781 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.230100 19775 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:48.282212  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 125520us
I20260709 12:27:48.281617 19812 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:48.296016 19812 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.296454 19812 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:48.299299 10091 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:48.301360 19781 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:48.302094 19798 consensus_meta.cc:287] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.987s	user 0.007s	sys 0.008s
I20260709 12:27:48.302538 19775 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.302664 19798 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.310091 10017 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:48.311594 19618 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.047s	user 0.015s	sys 0.003s
I20260709 12:27:48.312161 19618 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:48.316798 10677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 2061) took 1014 ms. Trace:
I20260709 12:27:48.320628 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:48.321434 19747 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:48.321993 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:48.322333 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:48.298772 19651 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.330852 19747 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:27:48.325444  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.326478 19651 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:48.330710 19979 raft_consensus.cc:3055] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 LEADER]: Stepping down as leader of term 32
W20260709 12:27:48.332770 19319 consensus_meta.cc:287] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.006s
I20260709 12:27:48.334847 19979 raft_consensus.cc:740] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:48.335064 19319 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.337157 19618 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:48.336159 19979 consensus_queue.cc:260] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.338007 19618 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.338642 19979 raft_consensus.cc:3060] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:48.340611 19618 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.317303 10677 rpcz_store.cc:276] 0709 12:27:47.302305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.303677 (+  1372us) service_pool.cc:224] Handling call
0709 12:27:47.303996 (+   319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:47.328485 (+ 24489us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:47.328516 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:47.328979 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:27:47.810028 (+481049us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:47.810085 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.315898 (+505813us) spinlock_profiling.cc:243] Waited 24.1 ms on lock 0x7b5400611a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:48.316003 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.316083 (+    80us) raft_consensus.cc:1685] finished
0709 12:27:48.316096 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.316516 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2821,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"spinlock_wait_cycles":55517440,"thread_start_us":480667,"threads_started":1,"wal-append.queue_time_us":654185}
I20260709 12:27:48.342758 19319 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.306372 19775 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.344177 19841 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.352711 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.353259 19841 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.354282 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.356007 19841 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.344132 19747 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.357582 19747 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:48.324628 19798 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.346105  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:48.359310 19654 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.359972 19841 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:48.360852 10095 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:27:48.361984 19654 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:48.363821  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:48.364159 19975 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.364984 19812 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.367386 19812 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:48.367798  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:48.375398 19319 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:27:48.376276 19319 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:48.378628 19841 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:48.378507 19319 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:48.379391 19940 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.572s	user 0.000s	sys 0.005s
I20260709 12:27:48.379766  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:48.382035 19841 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:48.383580 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:48.329319 19919 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.407007 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:48.407734 19814 consensus_meta.cc:287] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.587s	user 0.014s	sys 0.000s
I20260709 12:27:48.408072 10671 raft_consensus.cc:2410] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 29 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:27:48.408767 10093 raft_consensus.cc:2368] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
I20260709 12:27:48.408241 19814 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.411036 19978 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:48.412472 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:48.413151 19812 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:48.413867 19812 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.414793 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:48.414996 10092 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:48.415624 10017 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:27:48.415975 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:48.406410 19919 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.416574 10092 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:48.416647 19812 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.417840 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.418968 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.419647 10090 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 26
W20260709 12:27:48.420213 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 277987us
I20260709 12:27:48.424261 19756 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:48.424475 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.424822 19756 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:48.425220 19759 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.892s	user 0.013s	sys 0.004s
I20260709 12:27:48.425810 19759 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.425256 10091 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:48.416718 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:48.428794  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:48.431489  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 146142us
I20260709 12:27:48.432816 10096 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:27:48.418545 10092 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:27:48.421236 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:48.440559 19525 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:48.441241 19525 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:48.427453 19616 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:48.433971 19686 raft_consensus.cc:1081] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Signalling peer 4083ce96cefb47d0be1d8e99602b7ce7 to start an election
I20260709 12:27:48.442983 19616 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:48.447744 10677 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:48.449154 19525 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.426115 19756 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.463223 19975 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.479918 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:48.467223 10672 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:48.467363 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:48.468155 19958 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:48.469695 10675 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:27:48.465359  9914 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:48.467410 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:48.474319 19855 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:27:48.457325 19790 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.516s	user 0.011s	sys 0.003s
I20260709 12:27:48.496019  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.497167 19748 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.498006 19748 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.498621 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:48.499370 10670 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:48.497241  9941 raft_consensus.cc:2410] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:27:48.501030 19748 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.466513 19837 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:27:48.504539 19676 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.022s
I20260709 12:27:48.505724 19913 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:48.506295 19913 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:48.510987 19723 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:48.511549 19723 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.511974 19723 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:48.512555 19790 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.472582 19919 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:48.525617 19656 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:48.526263 19968 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.530468 19844 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:48.531184 19844 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:48.526947 19927 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.531709 19844 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:48.532091 19927 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.530522 19656 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.530812 19968 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.480445 19814 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:48.518150 19581 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.177s	user 0.014s	sys 0.008s
I20260709 12:27:48.543335  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:48.538671 10671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42028 (request call id 2047) took 1653 ms (client timeout 2189 ms). Trace:
W20260709 12:27:48.551447 10671 rpcz_store.cc:269] 0709 12:27:46.881967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:46.882126 (+   159us) service_pool.cc:224] Handling call
0709 12:27:48.533519 (+1651393us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b5400501ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:48.535509 (+  1990us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3487171840}
I20260709 12:27:48.557013  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.557528 10674 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 28 index: 20. (term mismatch)
I20260709 12:27:48.557960 10674 pending_rounds.cc:85] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 19
I20260709 12:27:48.558408 10674 pending_rounds.cc:107] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 27.20
I20260709 12:27:48.558454 10096 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 19. (term mismatch)
I20260709 12:27:48.558794 10674 raft_consensus.cc:2889] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:48.558924 10096 pending_rounds.cc:85] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 18
I20260709 12:27:48.559252 10096 pending_rounds.cc:107] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 32.19
I20260709 12:27:48.557946  9948 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:48.559500 10096 raft_consensus.cc:2889] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:27:48.561184 19982 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.237s	user 0.003s	sys 0.000s
I20260709 12:27:48.562656 19883 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:48.563364 19883 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.585588 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:48.535290 19775 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:48.590416 10090 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:48.593858 10017 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:48.597100 19944 log.cc:927] Time spent T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.810s	user 0.002s	sys 0.001s
W20260709 12:27:48.598320 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 177258us
I20260709 12:27:48.598670 19790 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.535111 19120 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:27:48.601315 10673 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:48.565768 19883 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:48.599735  9990 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:48.602306 19978 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:48.602308  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:48.602929 19957 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.624s	user 0.000s	sys 0.002s
W20260709 12:27:48.611240  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.543361 19975 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
I20260709 12:27:48.611711 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:48.617271  9946 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:27:48.618018  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:48.618908 19775 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.623970 19927 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.626827 19758 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:48.627540 19758 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.629756 19758 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.631768  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.632807  9948 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:48.632900  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:48.634625  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:48.634363  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:27:48.635596 19968 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:48.637521 19968 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:48.638223 19968 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.639027  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:27:48.635031  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.643226 19762 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:48.645113 10090 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:48.647990 19775 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.648631 19775 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.648764 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3472) took 1261 ms. Trace:
I20260709 12:27:48.649171 19775 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.621656 19841 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.649255 10099 rpcz_store.cc:276] 0709 12:27:47.387622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.448814 (+ 61192us) service_pool.cc:224] Handling call
0709 12:27:47.454917 (+  6103us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:47.455252 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:47.455281 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:47.455728 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 12:27:47.836046 (+380318us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:47.836103 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:47.836202 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:48.326294 (+490092us) spinlock_profiling.cc:243] Waited 43.1 ms on lock 0x7b4800180258. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:27:48.597732 (+271438us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:48.648421 (+ 50689us) raft_consensus.cc:1685] finished
0709 12:27:48.648442 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:48.648590 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4163,"raft.run_cpu_time_us":3998,"raft.run_wall_time_us":860764,"spinlock_wait_cycles":122832384,"thread_start_us":285233,"threads_started":1,"wal-append.queue_time_us":452041}
I20260709 12:27:48.653836 19968 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.654277  9942 raft_consensus.cc:1275] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:27:48.655362 10022 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:27:48.659953 19616 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.661180 19616 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:48.662858  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 67080.9us
I20260709 12:27:48.663236  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:48.664654  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:48.667863 19812 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:48.668710 19881 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:48.668545 19812 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.669762 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:48.664783 19857 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.008s
I20260709 12:27:48.664783 19975 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:48.674111 19975 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:48.674508 19975 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:48.665263 19814 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:27:48.671756 10021 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:27:48.687211  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:48.680919 19771 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.745s	user 0.016s	sys 0.002s
I20260709 12:27:48.689165 19771 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.690779 10091 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:48.693749  9945 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:48.667243 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.678910 19981 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:48.665117 19854 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:27:48.679534 10023 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:48.679850 19676 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:48.680864 10016 raft_consensus.cc:1240] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:48.683374  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.684233  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.684440  9947 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:27:48.566870 19581 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.639082 19759 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:48.708339  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.712317 10018 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
W20260709 12:27:48.712672  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46119us
W20260709 12:27:48.722851 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.723984 19964 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.724558 19964 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.726150 19771 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.702615 19881 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:27:48.728492 19319 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:48.728962 19319 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:48.729571 18941 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:48.730233 18941 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:48.731367 18941 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.732239 19913 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.732630 19812 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.666678 19656 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.723848  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:48.729485  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:48.718849 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:48.742424 10092 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 35.
I20260709 12:27:48.742631  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.743350 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:48.743410  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.767055  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.768333 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:48.768712 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:48.768996 10018 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:48.769390 10091 raft_consensus.cc:2393] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:48.771986 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 163343us
I20260709 12:27:48.772372  9943 raft_consensus.cc:2410] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 21.
I20260709 12:27:48.775332 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:48.756310 19981 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:48.797227  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:48.793370  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:48.798657 10016 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:48.801190 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:48.797498 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 2055) took 1514 ms. Trace:
I20260709 12:27:48.789321 19881 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:48.803534 10674 rpcz_store.cc:276] 0709 12:27:47.272130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:47.272241 (+   111us) service_pool.cc:224] Handling call
0709 12:27:48.535990 (+1263749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.559411 (+ 23421us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.721638 (+162227us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b5400501ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:48.783493 (+ 61855us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400501ed0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:48.786556 (+  3063us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2900140288}
I20260709 12:27:48.808768  9990 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:48.809055  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 90060.2us
I20260709 12:27:48.809783 19523 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:27:48.810295 19523 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 35 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:48.810822 19903 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:48.811254 19523 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.20, Last appended by leader: 20, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.808759 10025 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:48.815382  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:48.809202 19977 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:48.817284  9946 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:48.796623 19676 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.808758  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:48.805683 19964 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:48.831537 10021 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:48.847194  9914 leader_election.cc:400] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
I20260709 12:27:48.847596  9914 leader_election.cc:403] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:27:48.851279  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:48.852846 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:48.858079  9942 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:48.823310 19958 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:48.868664 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:48.872689  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.849730  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.846431 19977 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:48.876637  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 71635.5us
I20260709 12:27:48.894604 19799 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:48.895365 19799 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.898722 10092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483"
dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:37540
I20260709 12:27:48.899039  9948 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:48.899379 10099 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:27:48.900965 19525 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:48.901348  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:48.902551  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.903082 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:48.903318  9947 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:48.904767  9943 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:48.906407 10092 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:27:48.906838 10092 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:48.909250  9942 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
W20260709 12:27:48.887229 19988 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:27:48.876291 19913 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:48.904342  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:48.898343 19676 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.908340 19981 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.927386 19976 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:27:48.928316 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:48.928840  9948 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 31.
I20260709 12:27:48.928910 19632 consensus_queue.cc:1059] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:27:48.930042 19964 raft_consensus.cc:3055] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:48.930401 19964 raft_consensus.cc:740] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:48.931582 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:48.931370 19964 consensus_queue.cc:260] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:48.932624 19964 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:48.936159 19581 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.938854  9947 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:48.939448  9940 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:48.908989 19958 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.876526 19844 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:48.943816 10022 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:27:48.945093 19844 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.893599 19775 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:48.946236 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 145827us
I20260709 12:27:48.908821 19881 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:48.947371 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:48.930414 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.943240 19958 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.953274 19913 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:48.957635 19654 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:48.958537 19654 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:48.962831  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 141611us
I20260709 12:27:48.963198  9949 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:48.927685 19676 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:48.973404 10091 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:48.979588  9943 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:27:48.983142 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:48.979256 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:48.961694 19654 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:48.995689 10091 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:48.997043  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:48.998399  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:48.999321 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:48.962455 19927 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.004299 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:49.006814  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:49.009070 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.009438  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:49.009714 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.009924  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:49.012255 10670 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 19. (term mismatch)
I20260709 12:27:49.012735 10670 pending_rounds.cc:85] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 18
I20260709 12:27:49.013299 10670 pending_rounds.cc:107] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 25.19
I20260709 12:27:49.013617 10670 raft_consensus.cc:2889] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:49.001989 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:49.022059 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:49.029371 19723 consensus_meta.cc:287] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.517s	user 0.006s	sys 0.002s
W20260709 12:27:49.030099 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.034057 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.032454  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:49.034827  9949 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:49.010102 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:49.030762  9948 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 24.
I20260709 12:27:49.037268 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:49.043057  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:49.053643  9948 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:49.051819  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:48.984826 19651 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.061s
I20260709 12:27:49.067736 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:49.066107 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:49.075306  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:49.076074 10092 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:49.077888  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 90887.3us
I20260709 12:27:49.071825 10668 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:49.078538 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:49.081123  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72596.1us
I20260709 12:27:49.082201 19977 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.088683 19772 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:49.089159 19756 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:49.094776 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 96754.6us
I20260709 12:27:49.095278 19984 raft_consensus.cc:2764] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader pre-election decision vote started in defunct term 33: won
I20260709 12:27:49.097486 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:49.139088  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:49.053345 19790 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.100282  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:49.101107  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49412.9us
I20260709 12:27:49.111464 10099 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:49.112645 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45025.3us
I20260709 12:27:49.115607 19823 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.077491 19581 consensus_queue.cc:1048] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.039s
I20260709 12:27:49.116977 19881 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24.
I20260709 12:27:49.117873 19883 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:49.036135 19927 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.062712 19844 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:49.123512 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:49.047220 19723 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.127228  9943 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:27:49.129855 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:49.132356 19913 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 12:27:49.139022  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54705.3us
I20260709 12:27:49.140105 10019 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:49.147143  9945 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:49.138710 19841 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:49.150516  9948 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: [27.18-27.18]
I20260709 12:27:49.149329  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:49.152501 19823 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.174130  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:49.177953 10092 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:49.181857 10091 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:49.154628 19883 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.185539 10062 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:49.194106 19823 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:49.203406 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:49.204216 10674 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:27:49.203610 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:49.205221 10672 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.204161 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:49.206030 10669 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:49.207638 18941 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:49.208171 18941 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:49.214829  9941 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:49.134389 19772 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.217628  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:49.221508 10668 raft_consensus.cc:2393] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 30: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:49.182914 19981 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:49.230207  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 131298us
I20260709 12:27:49.237916 10670 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:27:49.239908 19799 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:49.255162 19789 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.050s
I20260709 12:27:49.160337 19790 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.222031 19903 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 12:27:49.205576 19984 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:27:49.263257  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.215868 19120 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:49.182893 19651 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:49.206321 19883 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:49.305212 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 75935.9us
I20260709 12:27:49.302721 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:49.215222 19841 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:49.312201 19584 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:27:49.312741  9940 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 32 index: 23. (index mismatch)
I20260709 12:27:49.269695 19913 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:49.326779 19816 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:49.336596 10674 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 35.
I20260709 12:27:49.329012 10065 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:49.223127 19844 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.097s
I20260709 12:27:49.346575 10672 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:49.307322 19855 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.025s
I20260709 12:27:49.327663 19523 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.327775 19584 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:49.366393 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 54932.3us
I20260709 12:27:49.330843 19984 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:49.358255 18941 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.371886 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:49.333039 19816 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:49.392846 19816 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.310563 19618 consensus_queue.cc:1059] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:27:49.349311 19798 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.374504 19961 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:49.375592 19968 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:49.397600 19798 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.398253 19961 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:49.376595 19841 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.379902 19913 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:49.352170 19686 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.382915 19984 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.386022 19790 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:49.389024 19723 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:49.349483 19837 consensus_queue.cc:1059] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:27:49.327634 19771 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 12:27:49.391180 18941 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.391558 19654 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:49.394428 19816 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.366485 19584 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.398625 19961 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:49.410022  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: true dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:49.402681 19903 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.016s
I20260709 12:27:49.350775 19651 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.366976 19120 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.044s
I20260709 12:27:49.421746  9940 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:27:49.422772  9949 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:49.427428  9949 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:49.398561 19927 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:49.421869 19654 raft_consensus.cc:697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:49.410178 19686 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.444417 19798 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:49.420644 19523 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.446127 19686 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:49.447929 19523 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:49.420867 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:49.442943  9949 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.462393  9949 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.481302  9946 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
W20260709 12:27:49.458719 19975 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.784s	user 0.011s	sys 0.001s
W20260709 12:27:49.485752 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67928.3us
I20260709 12:27:49.480896 19978 raft_consensus.cc:3055] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:49.489508 19913 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:49.508309  9949 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:49.479391 19883 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:49.528959 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 157199us
I20260709 12:27:49.500483 19978 raft_consensus.cc:740] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:49.495479 19975 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.515545 19913 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:49.576628  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 234874us
I20260709 12:27:49.584501 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:49.496872 19961 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:49.565553 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 254723us
I20260709 12:27:49.599263 19823 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:27:49.600711 20012 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:27:49.612262 19651 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:49.612556 19903 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:49.614948 19975 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:49.616254 19913 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.619905 19975 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.620687 19975 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:49.624560 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 189914us
W20260709 12:27:49.625945 10642 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:33983 timed out after 2.044s (SENT) [suppressed 3 similar messages]
I20260709 12:27:49.626945 19883 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:49.631992 19961 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.637934 19723 raft_consensus.cc:3055] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:49.636292 19772 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:49.638411 19723 raft_consensus.cc:740] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:27:49.667240 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 101632us
I20260709 12:27:49.668125 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:49.669232  9944 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:49.669535 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:49.672005  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 79411.8us
I20260709 12:27:49.675846 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 35 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.678650 10098 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 35: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:49.676410 10092 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:49.677078 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.677810 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.677503 19723 consensus_queue.cc:260] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.681849 19723 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.535584 19654 consensus_queue.cc:237] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.14, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.686623 19978 consensus_queue.cc:260] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 30.15, Last appended by leader: 15, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.688028 19978 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:49.693480 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:49.701545  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.703051  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:49.694303 19883 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:49.707572 10091 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:49.708611  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:49.721072 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 224918us
W20260709 12:27:49.754707 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:49.761758  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.709792 19816 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:49.771028 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.772862  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:49.710209 19883 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:49.774114 10016 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:27:49.771231 10640 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:49.775242 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:49.707319 19772 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
I20260709 12:27:49.777647 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:49.777843 19861 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:49.778328 19686 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:49.778729 19686 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:49.779589 19686 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.780761 19686 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:49.782711 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.782953 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.783411 10092 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:49.783707 10091 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:49.784868 10098 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:49.786027 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.786721 10096 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 23.
I20260709 12:27:49.791090 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.791963 10094 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:49.792786 19981 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:49.794569 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:49.794976 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.795322 10018 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:27:49.794246 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.776814 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:49.777139 10668 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:49.798755 20018 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:49.802227 19981 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:49.804299 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:49.805006 10669 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.806727 19651 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:49.806942  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:49.753124 19975 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:49.807186 19651 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:49.807869 19651 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.809686 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:49.811347  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:49.775656  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:49.706012 20017 consensus_queue.cc:1059] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:27:49.814975  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:49.723044 19723 raft_consensus.cc:2764] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:27:49.773576 19837 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:27:49.819164  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 44972.8us
I20260709 12:27:49.819936  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:49.812532 19883 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:49.772254 19816 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:49.833112 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.836730 10095 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.837680  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:49.832180 19961 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:49.827709 19841 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:49.827898 19981 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:49.820690 19975 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:49.831902 20017 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:49.845337 10668 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:49.839157 19816 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:49.834333 19523 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:49.848946 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:49.849195 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.849294 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:49.851208 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.855351  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:49.849251 10677 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 17. (term mismatch)
I20260709 12:27:49.856326 10677 pending_rounds.cc:85] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 16
I20260709 12:27:49.856575 10677 pending_rounds.cc:107] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:27:49.856968 10677 raft_consensus.cc:2889] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:49.858247 10669 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:49.834401 19861 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:27:49.847410 19964 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:49.843796 19981 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:49.866410 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.852128 19676 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:49.839771 19961 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.851454 19523 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:49.841578 19790 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.022s
W20260709 12:27:49.870335 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 95986.6us
I20260709 12:27:49.873222 19981 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:49.883224  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:49.887742 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.878656 19523 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:49.886469 10065 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:49.863623 19584 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.902153 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.887058 19980 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:49.904984  9912 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:49.893445 19772 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:49.918457 19319 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:49.919986 19319 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:49.923218 20028 log.cc:927] Time spent T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:27:49.926918  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.927466 19523 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.930419 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:49.931097 10094 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:49.908334 19913 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:49.896250 19855 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.016s
W20260709 12:27:49.935456  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:49.887202 19964 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:49.940148 10673 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:49.913573 19772 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:49.937186 19686 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:27:50.035379 20029 log.cc:927] Time spent T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
I20260709 12:27:50.034926 19772 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:49.947683 19913 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:50.116886 19913 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.040948 20016 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:50.081271 19837 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.160952 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:49.937374 19676 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.213s
W20260709 12:27:50.177567 10640 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.220093 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.158416 20017 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:50.239136 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:50.232023 19961 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:50.291523 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:50.161536 20016 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:50.302232 10098 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
W20260709 12:27:50.281399  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 197256us
I20260709 12:27:50.235800 19837 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.312819 19837 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:50.323834 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.325773  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.243383 19686 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 25 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:50.328063  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:50.293193 19790 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:50.329968  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:50.346879 10094 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
W20260709 12:27:50.329807 19992 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.333s	user 0.005s	sys 0.015s
W20260709 12:27:50.351549 20038 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:27:50.347673  9947 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
W20260709 12:27:50.352063 20037 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.411s	user 0.001s	sys 0.000s
I20260709 12:27:50.353209  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:50.356132 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.356491  9945 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 3295) took 1671 ms (client timeout 2055 ms). Trace:
I20260709 12:27:50.323416 19584 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:50.360109 19790 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.361088 19841 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:50.364267  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.366698  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:50.260986 20017 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:50.380210  9945 rpcz_store.cc:269] 0709 12:27:48.685380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.777219 (+ 91839us) service_pool.cc:224] Handling call
0709 12:27:50.356229 (+1579010us) spinlock_profiling.cc:243] Waited 238 ms on lock 0x7b54000b6154. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:50.356390 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":547367680}
I20260709 12:27:50.405966  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:50.407131 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:50.409267 19841 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:50.409778 10095 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.558s	user 0.006s	sys 0.005s
I20260709 12:27:50.410810 10095 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:50.417950  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 2216) took 1823 ms. Trace:
I20260709 12:27:50.419471 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:50.420140 10021 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:50.420454 10096 raft_consensus.cc:1240] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: [30.15-30.15]
W20260709 12:27:50.396997 19816 consensus_meta.cc:287] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.536s	user 0.012s	sys 0.000s
I20260709 12:27:50.422365 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:50.418321  9941 rpcz_store.cc:276] 0709 12:27:48.594430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.594577 (+   147us) service_pool.cc:224] Handling call
0709 12:27:50.417746 (+1823169us) spinlock_profiling.cc:243] Waited 601 ms on lock 0x7b54000b5254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:50.417896 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1381866368}
W20260709 12:27:50.408236 20041 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:27:50.430717  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:50.438115 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:50.433427 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:50.441263  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:50.417526 19584 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.456404  9941 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:27:50.459779 10096 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:27:50.496062 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:50.430191 19961 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:50.451727 19964 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:50.436539 19816 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.570820 10097 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3489) took 1789 ms. Trace:
W20260709 12:27:50.553092 20037 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:27:50.481451 19584 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.656416 10677 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: []
W20260709 12:27:50.658291  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65820.9us
I20260709 12:27:50.438318 19686 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 23.19, Last appended by leader: 19, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:50.660997  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.661412 10096 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
W20260709 12:27:50.661895  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.664279  9942 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:27:50.695379  9944 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 3594) took 2178 ms (client timeout 2153 ms). Trace:
W20260709 12:27:50.695966  9944 rpcz_store.cc:269] 0709 12:27:48.488752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.513947 (+ 25195us) service_pool.cc:224] Handling call
0709 12:27:50.663507 (+2149560us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001b8354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:50.663653 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308089216}
W20260709 12:27:50.664074 19883 consensus_meta.cc:287] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.819s	user 0.006s	sys 0.007s
I20260709 12:27:50.733251  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:50.703995 10025 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:50.572876 10097 rpcz_store.cc:276] 0709 12:27:48.641407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.641934 (+   527us) service_pool.cc:224] Handling call
0709 12:27:48.647292 (+  5358us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:48.668345 (+ 21053us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:48.668376 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:48.668877 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:27:48.855929 (+187052us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:48.855969 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:48.856031 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:50.301165 (+1445134us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:50.411123 (+109958us) raft_consensus.cc:1685] finished
0709 12:27:50.411153 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.431016 (+ 19863us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20679,"raft.queue_time_us":25879,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":101,"spinlock_wait_cycles":11106048,"thread_start_us":186643,"threads_started":1,"wal-append.queue_time_us":327449}
I20260709 12:27:50.734050  9944 raft_consensus.cc:2376] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25. Re-sending same reply.
W20260709 12:27:50.707167 20020 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
W20260709 12:27:50.756574 19651 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.948s	user 0.011s	sys 0.002s
W20260709 12:27:50.771566 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 276367us
I20260709 12:27:50.776304 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:50.758939 20050 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.762897 20054 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:50.764972 19913 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:50.788506 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4637) took 1864 ms. Trace:
I20260709 12:27:50.790830  9942 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3618) took 1465 ms. Trace:
W20260709 12:27:50.792236  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:50.793093 19981 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.892s	user 0.009s	sys 0.000s
I20260709 12:27:50.741580 19883 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.797031  9942 rpcz_store.cc:276] 0709 12:27:49.325439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.325575 (+   136us) service_pool.cc:224] Handling call
0709 12:27:50.663851 (+1338276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.664803 (+   952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.790504 (+125701us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54001b8350. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:50.790728 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3077122304}
I20260709 12:27:50.802430 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:27:50.798236 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 316337us
I20260709 12:27:50.795701 10090 rpcz_store.cc:276] 0709 12:27:48.924343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.929646 (+  5303us) service_pool.cc:224] Handling call
0709 12:27:48.929938 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.974672 (+ 44734us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:48.974719 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:48.974774 (+    55us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:48.974838 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.788180 (+1813342us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b5400161f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:50.788247 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.788419 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76574976}
I20260709 12:27:50.773249 19651 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.793917 20055 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.821508 10674 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:50.802112 20065 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:50.827809 19772 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.000s
I20260709 12:27:50.809432 19790 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:50.806017 19981 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.823347 19841 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:50.872869 10676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 2067) took 1907 ms. Trace:
W20260709 12:27:50.875696 20057 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:27:50.879398 20056 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:27:50.879779 19964 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:50.880888 19584 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:50.795928 20054 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.788972 20050 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.816810 20017 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:50.886948 10676 rpcz_store.cc:276] 0709 12:27:48.965551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.965739 (+   188us) service_pool.cc:224] Handling call
0709 12:27:49.001463 (+ 35724us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:49.025395 (+ 23932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:49.025430 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:49.025952 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.319638 (+1293686us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b1800364a20. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:50.578581 (+258943us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:50.578662 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.578760 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:50.872545 (+293785us) raft_consensus.cc:1685] finished
0709 12:27:50.872569 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.872802 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3602,"raft.queue_time_us":13511,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":32228096,"thread_start_us":258721,"threads_started":1,"wal-append.queue_time_us":1817772}
I20260709 12:27:50.802315 19913 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:50.856897 20026 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.012s	sys 0.002s
I20260709 12:27:50.891374 20059 raft_consensus.cc:2764] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:27:50.892313 20070 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:50.893999 19978 consensus_meta.cc:287] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.206s	user 0.015s	sys 0.000s
I20260709 12:27:50.894070 20018 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:50.926996  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:50.856642 20055 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:50.897640 20063 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:27:50.900903 19319 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.981s	user 0.005s	sys 0.006s
I20260709 12:27:50.855295 19772 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:50.864989 20065 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.906242 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4309) took 1147 ms. Trace:
I20260709 12:27:50.915975 20016 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:50.909590 19964 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:50.923702 20026 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.005s	user 0.012s	sys 0.005s
W20260709 12:27:50.909469 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 103564us
W20260709 12:27:50.947785 10017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.131s	user 0.045s	sys 0.081s
I20260709 12:27:50.918756 19977 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.932077 20066 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.938555 10020 rpcz_store.cc:276] 0709 12:27:49.758874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.759152 (+   278us) service_pool.cc:224] Handling call
0709 12:27:49.775337 (+ 16185us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:49.834885 (+ 59548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:49.834940 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:50.454885 (+619945us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b2c00046bc8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:50.455423 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:27:50.768677 (+313254us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:50.770094 (+  1417us) log.cc:844] Serialized 14 byte log entry
0709 12:27:50.848204 (+ 78110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:50.848293 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:50.905864 (+ 57571us) raft_consensus.cc:1685] finished
0709 12:27:50.905983 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:50.906148 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4585,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":61660,"raft.run_cpu_time_us":380,"raft.run_wall_time_us":38122,"spinlock_wait_cycles":6116736,"thread_start_us":279937,"threads_started":2,"wal-append.queue_time_us":321607}
I20260709 12:27:50.944141 10675 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:27:50.945134 19772 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:50.945261 20060 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:50.950562  9948 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:27:50.955518 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:50.961468 10021 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:50.965160 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:50.966857 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:50.967649 10019 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:50.941162 20052 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:50.969594 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 186974us
I20260709 12:27:50.960307 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:50.946928 19964 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:50.985155  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:50.987138 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:50.987774 19913 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:50.962566 20066 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:50.988994 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:50.989854 10091 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:50.990661 20054 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:50.991184 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:50.977226 19935 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.991923 10097 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:50.932461 20018 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.992408 19935 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.979676 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:50.994644 19935 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:50.996748 19964 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.999517 19964 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:50.982121 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:50.956537 19790 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:50.994333 20018 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.000458 19790 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.003733 19790 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:50.982941 20069 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:51.004717 20069 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.005787 10673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 2080) took 1163 ms. Trace:
I20260709 12:27:51.006335 10673 rpcz_store.cc:276] 0709 12:27:49.842169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.842312 (+   143us) service_pool.cc:224] Handling call
0709 12:27:51.005540 (+1163228us) spinlock_profiling.cc:243] Waited 89 ms on lock 0x7b54007d7b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:51.005704 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":204622720}
I20260709 12:27:51.007323 20069 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:50.984223 20055 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:50.991366 10094 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:27:50.977008 10669 raft_consensus.cc:1240] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:51.013401  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3317) took 1067 ms. Trace:
W20260709 12:27:51.013444  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.017714  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.018496  9946 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:51.018616 19927 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.018726 19837 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.019182 19927 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.019465 19837 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.020569 19980 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.021098 19980 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.021925 19837 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.021418 19927 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.023429 20047 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.023924 20047 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.025665 19837 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:51.025915 20047 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.026326 19837 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:50.946775 20016 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.994251 19976 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:27:51.039872  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:50.952466 20059 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.033252 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:51.044114 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.046082  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:51.046509 10017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.230s	user 0.045s	sys 0.081s
I20260709 12:27:50.959637 19978 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:51.050808  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.047854  9941 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:51.055462  9944 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
W20260709 12:27:51.055867  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.051815 10668 raft_consensus.cc:2368] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25.
I20260709 12:27:51.052208  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.057085  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:51.023268 19980 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:51.059842 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.064621 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.065469 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.066306 10091 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:51.068089 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:51.084281 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4294) took 2439 ms. Trace:
I20260709 12:27:51.013859  9943 rpcz_store.cc:276] 0709 12:27:49.946419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.946668 (+   249us) service_pool.cc:224] Handling call
0709 12:27:50.036328 (+ 89660us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.036505 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:50.246362 (+209857us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.012838 (+766476us) spinlock_profiling.cc:243] Waited 3.15 ms on lock 0x7b180049da40. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:27:51.013056 (+   218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.013105 (+    49us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:51.013151 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.013217 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.013335 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33189,"prepare.queue_time_us":127021,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":766339,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":7237888,"thread_start_us":135438,"threads_started":1}
I20260709 12:27:51.089553 10668 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:51.090298  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:51.091856 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:51.095300  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:51.096124  9947 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:50.936368 20063 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.100085 19855 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.100495 19768 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:51.100658  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54477.6us
I20260709 12:27:51.103085  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.084860 10017 rpcz_store.cc:276] 0709 12:27:48.622291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:48.639777 (+ 17486us) service_pool.cc:224] Handling call
0709 12:27:48.640227 (+   450us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:48.674904 (+ 34677us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:27:48.789724 (+114820us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.055275 (+2265551us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.055331 (+    56us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:51.055383 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.055458 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.055546 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48009,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":8186,"thread_start_us":2309112,"threads_started":2,"wal-append.queue_time_us":978601,"wal-append.run_cpu_time_us":1182,"wal-append.run_wall_time_us":1001210}
I20260709 12:27:51.108057  9942 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 29 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:27:51.108520 10094 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 24. Current term is 26. Ops: []
I20260709 12:27:51.109025  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.110021  9946 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:51.091183  9940 raft_consensus.cc:2376] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25. Re-sending same reply.
I20260709 12:27:50.938653 20070 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.051551 19798 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.117949 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.108103 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.109596  9942 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:51.107511  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:51.115954 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:51.116328  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:51.126158  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.054466 19903 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:50.968977 19977 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.056921 19775 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:51.134500 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:51.137439 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.139806 10668 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:51.107537 19964 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:50.974184 20060 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.144018 10016 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: []
I20260709 12:27:51.043464 20052 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.148860  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:51.150270  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:51.150663  9946 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:51.151149 10677 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 12:27:51.133075 20021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.207s	user 0.025s	sys 0.037s
I20260709 12:27:51.047683 20016 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.115967 20018 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.152432 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:51.083448 19978 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.120937 20026 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:51.153119 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.154824  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:51.158272  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:51.153153 20021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.228s	user 0.025s	sys 0.040s
I20260709 12:27:51.121444 19927 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:51.141225 19768 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.125054 19855 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.054570 20059 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.121589 19841 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:27:51.140291 19772 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:51.194237 10093 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:27:51.185590 10024 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:51.160339  9990 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:51.230243 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63719.4us
I20260709 12:27:51.176985  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:51.303315  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 143041us
I20260709 12:27:51.216205  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:51.163060 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.304328  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:51.196681 10019 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:51.167176 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: true dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:51.183593 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.199530 10022 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:27:51.202858 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:51.306054  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53699.5us
I20260709 12:27:51.172199 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:51.189932 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47924.5us
I20260709 12:27:51.306438 10021 raft_consensus.cc:2468] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:51.205394  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.150388 20076 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.199460 10017 raft_consensus.cc:1240] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:27:51.309423 10673 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:27:51.318099 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.318719  9945 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:51.319936 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.305678  9947 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:51.318881  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:51.195091 20027 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.312s	user 0.006s	sys 0.012s
I20260709 12:27:51.157603 19981 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.159937 19976 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:51.169943 19977 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:51.319389 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 184075us
I20260709 12:27:51.319060  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:51.178683 20060 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.155295 19964 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.162670 19790 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:51.202394 19816 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:51.324187 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.233680 20017 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.189816 20073 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:51.163460 20040 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:51.192306 20016 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.320487 20081 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:51.152027 19903 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.361783 20091 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:51.362640 19903 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.201498 20065 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.176744 20063 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:51.187022 19978 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.328078 20067 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.005s
I20260709 12:27:51.162825 20078 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.166s
I20260709 12:27:51.328469 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:51.376241 10021 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:51.341118 20055 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:51.173111 19837 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.305483 20069 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:51.304530  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 88418.6us
I20260709 12:27:51.391474 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.331434  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.392609  9945 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:51.331578  9990 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:51.335575  9946 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:51.314970 19823 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.310410 20072 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:51.310001 19654 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:51.187000 20050 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.318632  9914 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:51.320533 19984 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:51.317829 19772 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:51.415601 20072 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.415949 19772 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:51.342532 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 170418us
I20260709 12:27:51.417928 20072 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.418135 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.419152 19768 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.419495 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.163085 20071 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.200s
I20260709 12:27:51.421396 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1893) took 1005 ms. Trace:
I20260709 12:27:51.356969 10019 raft_consensus.cc:2393] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:51.421752 10025 rpcz_store.cc:276] 0709 12:27:50.415860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.416530 (+   670us) service_pool.cc:224] Handling call
0709 12:27:51.421214 (+1004684us) spinlock_profiling.cc:243] Waited 263 ms on lock 0x7b54001388d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:51.421313 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":605976960}
I20260709 12:27:51.422569 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.407867  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.422932 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 29 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.423755 10025 raft_consensus.cc:2410] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 32 because replica has last-logged OpId of term: 30 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 14.
I20260709 12:27:51.423832 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:51.200033 20026 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.424582 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.425674 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.426106 20026 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:51.172448 20054 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.168s
I20260709 12:27:51.427955 20046 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.428544 20046 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.187443 20075 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.188s
I20260709 12:27:51.148258 19798 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.350835 19790 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.433341 19790 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:51.208377 20018 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.341848 19980 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:51.440855 10020 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:27:51.378842 19756 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:51.442664 19903 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:51.380590 10676 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:27:51.380774  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3631) took 1041 ms. Trace:
I20260709 12:27:51.444082  9949 rpcz_store.cc:276] 0709 12:27:50.339570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.581971 (+242401us) service_pool.cc:224] Handling call
0709 12:27:50.666453 (+ 84482us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:51.028444 (+361991us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:27:51.028481 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:51.029018 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:27:51.353476 (+324458us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:51.353508 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.353595 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:51.380570 (+ 26975us) raft_consensus.cc:1685] finished
0709 12:27:51.380589 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:51.380714 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":330462,"raft.queue_time_us":334937,"spinlock_wait_cycles":18302336,"thread_start_us":324047,"threads_started":1,"wal-append.queue_time_us":349546}
I20260709 12:27:51.446209 20085 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:51.381139 20080 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:51.446967 20080 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.447269 20054 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:51.387401 10021 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:51.447817 20054 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:51.448462 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.448709 20054 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.449728 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.450421 19980 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.451203 19964 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:51.451340 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.451488  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3318) took 1503 ms. Trace:
I20260709 12:27:51.451902  9948 rpcz_store.cc:276] 0709 12:27:49.947683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.947903 (+   220us) service_pool.cc:224] Handling call
0709 12:27:50.040062 (+ 92159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.969565 (+929503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.451257 (+481692us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54001b9754. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:51.451431 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2085101312}
I20260709 12:27:51.452396 19855 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.453640 20075 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:51.453819 20059 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.360231 20017 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.454643 20065 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:27:51.389832 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.456137 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.456357 20017 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.391785  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:51.456902 10670 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:51.441020 19977 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:51.457412  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65710.5us
I20260709 12:27:51.457886 10019 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:51.457577 19977 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.174793 20070 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:51.398058  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 66556.8us
I20260709 12:27:51.358300 19650 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:51.459690 19841 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:51.459807  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:51.335959 19319 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.040s
I20260709 12:27:51.461182  9991 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:51.461683 19319 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:27:51.396387 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.462766 10672 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:51.463251 19976 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:51.463678 20091 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:51.465647 19584 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:51.466041 19584 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:51.323182 20076 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.466308 10640 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:51.467466 10640 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:51.468135 19975 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:51.468263 20076 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:51.468437 19964 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:51.408583 10675 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:27:51.469034 19964 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:51.469882  9947 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:51.408321 19654 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.403539 19823 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.471366 10019 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 33.
I20260709 12:27:51.472424 19654 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.400887 20044 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.424074  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.435721 10669 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:51.470003 19964 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.436597 10062 leader_election.cc:400] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25.
I20260709 12:27:51.336119 19799 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.049s
I20260709 12:27:51.331045 19927 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.452581 19980 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.442737 19756 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:51.494243 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:51.507444  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.499166 20046 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.379527 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 2074) took 1656 ms. Trace:
I20260709 12:27:51.447152 20086 consensus_queue.cc:1059] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:27:51.447407 20080 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:51.508138 20059 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:51.485394  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.508827 20059 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:51.509248 20059 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:51.512923 10674 rpcz_store.cc:276] 0709 12:27:49.722845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.754634 (+ 31789us) service_pool.cc:224] Handling call
0709 12:27:49.755033 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:50.846874 (+1091841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:51.379126 (+532252us) spinlock_profiling.cc:243] Waited 1.06 s on lock 0x7b5400690d54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:51.379463 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2444514688}
I20260709 12:27:51.508098 19756 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.489677  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.515086 19756 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:51.516770 20066 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.517976 19977 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.454190 20075 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.481840  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.520562 20075 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:51.401257  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 82738.1us
I20260709 12:27:51.500797 20052 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.504963 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.403774 20081 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:51.414788 19861 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:51.523833 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:51.524009 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44788.7us
I20260709 12:27:51.524200 19861 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.524698 10025 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:51.450344 10021 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:51.531527 10065 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:51.532218 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.503564 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.532745 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.533305 10017 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:51.533699 20051 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:51.452479 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:51.484663 20068 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.488761 20018 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.534701 20078 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:51.534859 20068 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.535733 20078 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:51.536795 20078 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.537457 20068 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.524789 19861 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:51.407824 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 96272.9us
I20260709 12:27:51.534255 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.540623 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:51.540961 10017 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:27:51.540321 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 211145us
I20260709 12:27:51.542461 10062 leader_election.cc:403] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:51.542304 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:27:51.543335  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64998.7us
I20260709 12:27:51.442042 10024 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:51.525468 19975 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.450038 20054 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:51.508301  9943 raft_consensus.cc:2393] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:51.523906 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:51.548058 10020 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:51.526633 20086 raft_consensus.cc:3055] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:51.468262 20060 raft_consensus.cc:2804] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:51.517976 20059 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.530786 20044 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.544298 20018 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:51.533577 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.391661 10673 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:51.534226 20051 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.553334 10021 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:51.472530 19823 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.542090 10643 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:51.545949  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:51.554486 18941 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:51.550660  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:51.555866  9945 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:51.556174  9949 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:51.541847 19980 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:51.547485 19975 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:51.483794 19523 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:51.587265 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45260us
W20260709 12:27:51.560658 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 124188us
W20260709 12:27:51.608395 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.560447 10676 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:51.619406 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.627146 10017 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
W20260709 12:27:51.628376 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87814us
I20260709 12:27:51.629477 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:51.549526 19654 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25.
I20260709 12:27:51.549546 20086 raft_consensus.cc:740] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:51.631310  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:51.634639 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:51.634516  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.635617 10643 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:51.637831  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.633833 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:51.553357 20051 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:51.641721 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:51.642144 20059 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.643343 20044 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.546900 20068 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:27:51.485278 20069 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:51.649384 20069 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.651456 20070 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:51.652182 20070 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.655411 20096 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:51.654421 20070 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.656610 19861 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.648841 20018 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:27:51.657570 19654 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:51.658280 19654 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.651539 20069 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.657641 20018 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:51.660089 20018 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.661088 19654 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:51.656958 20070 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.661864 20070 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.663254 20045 consensus_queue.cc:1059] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:51.664182 20070 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.664340 19855 raft_consensus.cc:3055] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:51.665079 19855 raft_consensus.cc:740] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:51.666039 19855 consensus_queue.cc:260] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.667052 19855 raft_consensus.cc:3060] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:51.667303 20044 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:51.668919 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.586699 20076 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:27:51.670360 20076 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:51.670789 19723 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:51.671188 19723 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:51.672029 19723 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.670075 10673 raft_consensus.cc:2410] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 29 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:27:51.673348 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.673969 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.674810 10674 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:51.675638 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.675794 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:51.676172 10674 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:51.674081 10672 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:51.677646 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.677368 20076 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:51.678870  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.617192 19980 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.587149 20060 raft_consensus.cc:697] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:51.679517 20075 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:51.680572 20075 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:51.680474 20060 consensus_queue.cc:237] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.681363 19980 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.617540 20052 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.065s
I20260709 12:27:51.596603 20080 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.644111 20050 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:51.683801 20050 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.683794 10016 raft_consensus.cc:2368] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:51.684305 20050 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:51.684065 20086 consensus_queue.cc:260] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.684801 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.644428 19771 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:51.685946 19771 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.686281 19771 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:51.687875 19861 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:51.690618 20080 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.691529 10677 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:51.692097 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.692749 10674 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:51.692780 19584 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.633366 20072 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:51.694270 20072 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:51.694726 20072 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:51.633183 20071 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:51.699816 20086 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:51.700440 20075 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:51.701896 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:51.560534 20081 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:51.698251 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.678956  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:51.542472 19790 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.705928 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.707042  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.707278 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.709784 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.716331 10096 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:51.710079 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.716543 10020 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:51.678567 20076 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:51.716590  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:51.641597 20061 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:51.715301 10093 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:51.721881 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.723260 10020 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 29 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:27:51.725492 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.735776 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.738025 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.738732 10092 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:51.743031 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:51.744655 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.694159 19584 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:51.747967 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.748965 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.647621 20017 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:51.740221 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.778329 10017 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:51.751118 10022 raft_consensus.cc:1240] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:27:51.752200 10095 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
W20260709 12:27:51.773705 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71892.7us
I20260709 12:27:51.774794 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.780071 10672 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:51.775789 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.781499 10020 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 31: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:51.787109 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.719082 20026 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:51.790728 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.789139 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.791390 10094 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 18.
I20260709 12:27:51.636080 20063 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.117s
I20260709 12:27:51.719214 20016 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:51.739711 20081 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:51.678653 19913 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.103s
I20260709 12:27:51.803381 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.796188 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.797302 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.742270 20068 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:51.786922 20075 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:27:51.800562 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.809159 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.809247 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:51.811866 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.814755 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.815927 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:51.816906  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:51.791143 20026 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:51.824296  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.829149 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.830735  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.823913 10020 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:51.830996 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.823400 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.829797 10093 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:51.831868 10096 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:51.833243 10017 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 32.
I20260709 12:27:51.748420 19771 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.746109 20061 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:51.834585 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.844156 10093 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:51.834769 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.837165 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.842056 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.844404 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:51.846725 10672 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:51.848414 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.845082 10096 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:27:51.849300 10097 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:51.850510 10020 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 25: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:51.833964 20046 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:51.851125 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:51.851475 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:51.827025 20075 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:51.852128 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.867905  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.852327 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 35 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.852977 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.864436 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:51.856762 20026 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.806865 20016 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:51.863184 10025 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 32. Current term is 33. Ops: []
I20260709 12:27:51.810959 19935 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:27:51.852977 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:51.870889 10020 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:51.868425  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.827469 20081 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:51.872363 10099 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:51.872231 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.872723 10672 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 23 index: 19. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:27:51.864405 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.876734 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.877133 10675 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:51.877900  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.878472 10098 raft_consensus.cc:2393] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 35: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:51.869475 10093 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:51.842904 19686 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:51.886005 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:51.896317 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.897698 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.898092  9948 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 31.
I20260709 12:27:51.898977 20096 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:51.886207  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.890044 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:51.939672 10090 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:51.891561 20016 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:51.898221 20085 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:51.887578 10017 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:51.947371 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:51.959012 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:51.972055 10094 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 35 index: 21. (index mismatch)
I20260709 12:27:51.837579 20078 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:27:51.978168 10091 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:51.883515 19758 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:51.988698 19927 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.990731 19771 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.992053 19790 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:51.899468 20096 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:51.971470 20052 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:51.959937 20085 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:51.998937 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:52.003278  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 125496us
W20260709 12:27:52.003810  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 117679us
I20260709 12:27:52.037195 10020 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:52.038071 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.023311 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 36 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.030000  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:52.048143 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:52.056875 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 169523us
I20260709 12:27:52.058542  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.059303 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:51.983558  9974 heartbeater.cc:646] Failed to heartbeat to 127.8.137.254:38721 (8 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.8.137.254:38721 timed out after 15.000s (SENT)
I20260709 12:27:52.062029 10017 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:27:52.063390 10017 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:52.011006 20096 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:52.032238 20085 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.003747 19861 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:52.066381 19861 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.066802 19861 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:52.056777 19980 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.012s
I20260709 12:27:51.980091 19823 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:52.064440 10020 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:27:52.003451 19855 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:52.072211 19975 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.073951 10020 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:27:52.060592 19771 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:51.994567 20072 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.075274 10020 raft_consensus.cc:1275] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 30 index: 15. Preceding OpId from leader: term: 31 index: 15. (term mismatch)
I20260709 12:27:52.075799 10020 pending_rounds.cc:85] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 14
I20260709 12:27:52.076066 10020 pending_rounds.cc:107] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 30.15
I20260709 12:27:52.076306 10673 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:52.076283 10020 raft_consensus.cc:2889] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:52.077471 20072 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.075266 19771 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.043022 20046 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:52.080735  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:51.974615 20090 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:52.078694 10020 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:52.082861 20090 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:52.084122 19758 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:27:52.084527 20090 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.084797 19758 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 33 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:52.085846 19758 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.092227 19771 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.092934 19650 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:52.020251 20081 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.097427 19837 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:52.074393 19975 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.055351 19768 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:52.102448 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:52.105425 19903 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:52.013728 20050 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.021232 20052 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.099985 19768 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.113086 20050 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.113137 19768 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.113786 19651 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.114985 10674 raft_consensus.cc:2368] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28.
I20260709 12:27:52.099392 20026 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:51.995146 20075 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.120406 20016 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:52.137948 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:52.114231 20085 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.163033 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:52.166289 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:27:52.170315 10062 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.132952 19651 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:52.175107 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.183282 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:52.185410 10093 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:52.183393 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.186441 10672 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:52.186972 10099 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.184108 10671 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:52.170496 20016 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:52.171362 20049 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:27:52.191257 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.192981 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.193722 10093 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.174867 19823 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:52.172034 20086 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:52.184504 20050 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:27:52.201416 19823 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:52.211584 19823 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.211879 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:52.212546 10671 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.221282 19650 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:52.222002  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3325) took 1168 ms. Trace:
I20260709 12:27:52.222892 20070 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:52.223219 10669 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42034 (request call id 1731) took 1117 ms. Trace:
I20260709 12:27:52.223282 20070 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:52.223687 10669 rpcz_store.cc:276] 0709 12:27:51.106303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.106491 (+   188us) service_pool.cc:224] Handling call
0709 12:27:52.223054 (+1116563us) spinlock_profiling.cc:243] Waited 296 ms on lock 0x7b540045ae54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:52.223155 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":681452800}
I20260709 12:27:52.224220 20070 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.224967 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:52.225122 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.225378 20070 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:52.225574 10669 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.226745 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.226933 20052 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:52.186491 19651 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.227373 10016 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:52.229182 19651 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:52.229249  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 197801us
I20260709 12:27:52.207760 20086 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.230007 10022 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:52.230829 20085 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.228704 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:52.232100 20086 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:52.193056 20049 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:52.232906 20049 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:27:52.233762 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63545.3us
I20260709 12:27:52.234453 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.235031 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:52.236276  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:52.236843 10065 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:52.237113 19584 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.239321 20050 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:52.239879 20050 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:52.240756 20050 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.243785  9948 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:27:52.244904  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.189507 20016 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:52.245730 20016 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.247519 18941 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:52.244936 19584 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:52.222401  9947 rpcz_store.cc:276] 0709 12:27:51.053123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.328990 (+275867us) service_pool.cc:224] Handling call
0709 12:27:51.329275 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.470250 (+140975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.221774 (+751524us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54001a36d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:52.221944 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322610432}
I20260709 12:27:52.251245  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.253355 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:52.254551 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.255002 20021 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:52.255589 20021 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:52.256104 20021 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.236920 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:52.257175 19935 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:52.257599 19935 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:52.257975 19935 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:52.258985  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.230474  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.260370  9912 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:52.261198  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:52.261992 19120 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:52.262724 20108 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:52.263289 20108 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:52.264164 20108 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:52.271243 20105 log.cc:927] Time spent T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:27:52.270998  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.280922 19861 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.284581 19823 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:52.285702 20032 log.cc:927] Time spent T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 2.357s	user 0.018s	sys 0.028s
W20260709 12:27:52.291201 20017 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
I20260709 12:27:52.291709 20017 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:52.293047  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.293833  9945 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:52.295009  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.291100 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4310) took 2531 ms. Trace:
I20260709 12:27:52.298729  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.299693 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3067) took 1150 ms. Trace:
I20260709 12:27:52.300221 10025 rpcz_store.cc:276] 0709 12:27:51.151164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.534781 (+383617us) service_pool.cc:224] Handling call
0709 12:27:51.863003 (+328222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.876764 (+ 13761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.279560 (+402796us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54001c6450. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:52.299631 (+ 20071us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":754113792}
I20260709 12:27:52.299489  9945 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:52.290460 19723 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.302245 10669 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:52.306880 10093 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:52.307596 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.299041 10023 rpcz_store.cc:276] 0709 12:27:49.759842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:49.770305 (+ 10463us) service_pool.cc:224] Handling call
0709 12:27:49.786944 (+ 16639us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:49.787438 (+   494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:49.787482 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:49.788106 (+   624us) log.cc:844] Serialized 52 byte log entry
0709 12:27:49.854238 (+ 66132us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:49.888605 (+ 34367us) log.cc:844] Serialized 14 byte log entry
0709 12:27:49.891893 (+  3288us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:49.891971 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:52.285028 (+2393057us) spinlock_profiling.cc:243] Waited 8.57 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:52.290795 (+  5767us) raft_consensus.cc:1685] finished
0709 12:27:52.290902 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.291041 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2732,"prepare.queue_time_us":118920,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":69,"raft.queue_time_us":5717,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":5336,"spinlock_wait_cycles":19705216,"thread_start_us":82551,"threads_started":2,"wal-append.queue_time_us":117740}
I20260709 12:27:52.208789 19927 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:52.313709 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:52.313900 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.328774 10097 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.311362  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:52.330233 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:52.332181 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.332927 10022 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:27:52.333735 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:52.336994  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.338747 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:52.309130 20054 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.757s	user 0.007s	sys 0.004s
I20260709 12:27:52.347285 10017 raft_consensus.cc:2368] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:52.347112 10670 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
W20260709 12:27:52.309741 20051 consensus_meta.cc:287] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.572s	user 0.008s	sys 0.002s
I20260709 12:27:52.307870 19855 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:52.356009 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:52.358991 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:52.358618  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:52.362223 10016 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:27:52.356455 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.365207 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:52.368513 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 36 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:52.286444 20090 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.361394  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.374666  9945 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.350888 19981 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:52.350936 19120 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:52.378283 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.406009  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.406878 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4314) took 1042 ms. Trace:
I20260709 12:27:52.350339 19837 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.008s
I20260709 12:27:52.381204 20096 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.407289 10019 rpcz_store.cc:276] 0709 12:27:51.364599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.862308 (+497709us) service_pool.cc:224] Handling call
0709 12:27:51.864041 (+  1733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.405983 (+541942us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:52.406013 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:52.406050 (+    37us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:52.406082 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.406653 (+   571us) spinlock_profiling.cc:243] Waited 542 ms on lock 0x7b54001c5a54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:52.406702 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.406823 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245916416}
I20260709 12:27:52.408016 19837 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:52.408468 19837 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:52.409211 20096 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:52.388586 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.410171  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.410341 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.410465  9946 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:27:52.358144 19771 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:52.356611 19823 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.411355 19771 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.372196 19756 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:52.413527 20061 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.413762 19756 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:52.414199 19756 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.391736 19861 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.392524 19768 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.420310 19768 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:52.396721 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1895) took 1447 ms. Trace:
I20260709 12:27:52.422166 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.421427 10018 rpcz_store.cc:276] 0709 12:27:50.949113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.949266 (+   153us) service_pool.cc:224] Handling call
0709 12:27:50.966373 (+ 17107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:51.031353 (+ 64980us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:27:51.050991 (+ 19638us) log.cc:844] Serialized 14 byte log entry
0709 12:27:51.137430 (+ 86439us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:51.137484 (+    54us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:51.137532 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:52.396507 (+1258975us) spinlock_profiling.cc:243] Waited 64.7 ms on lock 0x7b540038e0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:52.396553 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:52.396667 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20977,"prepare.queue_time_us":19898,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":21330,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":12655,"spinlock_wait_cycles":148923264,"thread_start_us":65087,"threads_started":1,"wal-append.queue_time_us":128235,"wal-append.run_cpu_time_us":1308,"wal-append.run_wall_time_us":1005490}
I20260709 12:27:52.423188 19798 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:52.401923  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.434181  9948 raft_consensus.cc:3060] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.402879  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.413569 19771 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.379134 20044 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.065s
I20260709 12:27:52.362131 20051 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.409329 19837 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.350632 19927 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.398051 10671 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:52.428824  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.429286  9940 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:27:52.449993  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.429606  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.431089 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:52.440809 20044 raft_consensus.cc:3055] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:52.448302  9949 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:52.452003 10668 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:52.352779 20072 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.032s
W20260709 12:27:52.456810 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46574.2us
I20260709 12:27:52.456988 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:52.457283 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.470048 10093 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:27:52.457624 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 79419.7us
I20260709 12:27:52.470978 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:52.458513 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.472050 10091 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.460233  9945 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:52.461400 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.455647 10643 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:52.474300 10090 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.477810  9948 raft_consensus.cc:2468] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:52.484824  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.485550 10092 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:52.486001 18941 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:52.486367  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:52.487493  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:52.451524 20071 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.481654 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:52.491397  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:52.491871  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.453013 20044 raft_consensus.cc:740] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:52.497120 10643 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:52.520341 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.467564 20054 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:52.528302 10090 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:52.455173 19981 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:52.456084 19771 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:52.480232 20065 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:52.494585 20052 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:52.494472 19676 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:52.440779 19756 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.534101 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:52.520476 19837 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.537395 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:52.517452 19883 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:52.506475 20071 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:52.544710  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.492583 19964 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:52.522780 20081 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:52.521066 20116 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:52.551350 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.529908 20018 raft_consensus.cc:2804] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:52.538713 20075 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:52.516881 19823 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.028s
I20260709 12:27:52.570151  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.495788 20078 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
W20260709 12:27:52.623803 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 141451us
I20260709 12:27:52.575001  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.629848  9947 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:52.577986  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.630810  9942 raft_consensus.cc:3060] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:52.544116 19676 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:52.579193  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.580631 19756 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:52.583442 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:52.584448 20091 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:27:52.584762 19935 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.585584 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:52.586445  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:52.633340  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46974.4us
I20260709 12:27:52.634621  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:52.587631 20021 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:52.594079 20076 consensus_meta.cc:287] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.856s	user 0.011s	sys 0.000s
I20260709 12:27:52.594395 20045 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:52.594985 20115 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:27:52.597018 20044 consensus_queue.cc:260] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.600440 10676 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 18. (term mismatch)
I20260709 12:27:52.644280 10676 pending_rounds.cc:85] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 17
I20260709 12:27:52.644579 10676 pending_rounds.cc:107] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 27.18
W20260709 12:27:52.644636  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 74616us
I20260709 12:27:52.645011 10676 raft_consensus.cc:2889] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:52.601204 20051 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:52.546784 19771 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:52.602102 19981 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.590026 20049 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.602698 10093 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 18. Preceding OpId from leader: term: 25 index: 20. (index mismatch)
I20260709 12:27:52.652894 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:52.603683 10024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1896) took 1654 ms (client timeout 1896 ms). Trace:
I20260709 12:27:52.522037 10098 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.607452 10097 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:52.654970 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:52.611665 20104 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.513s	user 0.003s	sys 0.000s
I20260709 12:27:52.606383 10091 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:52.612317  9838 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 16 to 22, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 22 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:52.614897  9944 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:52.615744 20016 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:52.616473 20098 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:27:52.617847 10669 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:52.622238 19927 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:52.539579 20065 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:52.623131 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.625649  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:52.627876 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:52.628814 10021 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:52.496441 19798 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.045s
W20260709 12:27:52.639219 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 168366us
W20260709 12:27:52.669399 20121 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
W20260709 12:27:52.646179  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.635861 19676 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:52.671974  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.635970 20018 raft_consensus.cc:697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:52.626835 20116 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:52.656522 10090 raft_consensus.cc:1240] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:52.641952 20081 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:52.661299 19771 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:52.661891 20126 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
W20260709 12:27:52.653844 10024 rpcz_store.cc:269] 0709 12:27:50.949490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:50.949637 (+   147us) service_pool.cc:224] Handling call
0709 12:27:52.603487 (+1653850us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54001392d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:52.603600 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":505086848}
I20260709 12:27:52.702905 20090 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:52.649925  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 124224us
I20260709 12:27:52.711145 20114 raft_consensus.cc:2764] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:27:52.711506 20016 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:52.676888 19723 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:52.707573  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.677539  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.678018 10062 leader_election.cc:400] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28.
I20260709 12:27:52.650403  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:52.713613 10062 leader_election.cc:403] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:52.702392  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:52.714375 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43954.4us
I20260709 12:27:52.714430  9914 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.663234 19584 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:52.651036 19837 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:52.698855 20116 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.662441 19976 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:52.671280 20065 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:52.665762  9942 raft_consensus.cc:2468] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:52.721230  9941 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:52.723435  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:52.732044  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.734040  9947 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:52.700351 20081 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:52.737076 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.712472 19723 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:52.737452  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 87147.4us
I20260709 12:27:52.739405  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:52.738339  9990 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:52.707901 20049 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:52.721778 19584 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:52.781755  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.744659  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.745862 10096 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:52.746914 20134 consensus_queue.cc:1059] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:27:52.747053 19654 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:52.749315 10095 raft_consensus.cc:1240] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 32. Current term is 33. Ops: []
I20260709 12:27:52.751323 20018 consensus_queue.cc:237] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.755090 10097 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:52.767963 19975 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:52.769565 19841 raft_consensus.cc:2804] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:52.771363 19976 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:52.649224 20045 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.774845  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:52.727620 19903 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28.
I20260709 12:27:52.726693 19686 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:52.775835 20080 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:52.701859 19927 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:27:52.710150 20021 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:52.709033 19935 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:52.783411 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 46464us
I20260709 12:27:52.820318 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:52.821676 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:52.799558 10091 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:27:52.729246 19772 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:52.725692 19798 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.074s
I20260709 12:27:52.743010 20081 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:52.782557 20070 consensus_meta.cc:287] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.000s
W20260709 12:27:52.817997 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.806710 20016 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:52.829823 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:52.815969  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 76633.9us
I20260709 12:27:52.841820  9940 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:52.808391 20134 consensus_queue.cc:1243] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer cd983415389444e292b2330ef630850a log is divergent from this leader: its last log entry 26.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:27:52.849066 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:52.849822  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 111589us
I20260709 12:27:52.831328 20049 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:52.832804 20078 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:52.810207 19861 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.038s
I20260709 12:27:52.817827 19980 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.036s
I20260709 12:27:52.871206  9940 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:52.874397 10062 leader_election.cc:400] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:52.875358 10062 leader_election.cc:403] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:52.835590 20118 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:27:52.845242 19855 consensus_queue.cc:1059] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.031s
I20260709 12:27:52.858258 20134 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:52.878573 20134 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:52.879130 20134 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:52.835713 20054 consensus_queue.cc:1048] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:27:52.835410 20026 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:27:52.851452 19935 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:52.891798 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:52.844708 19841 raft_consensus.cc:697] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:52.894181  9943 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 31.
I20260709 12:27:52.896787 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:52.898823 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:52.903734 10025 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:52.868623 20017 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.037s
I20260709 12:27:52.905448 20016 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.004s
I20260709 12:27:52.906976 10018 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:52.908679 20026 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:52.909211 20026 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:52.902012 19756 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 12:27:52.905673 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:52.889536 19723 consensus_queue.cc:1059] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.030s
I20260709 12:27:52.919847 19723 raft_consensus.cc:3055] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:27:52.920243 19723 raft_consensus.cc:740] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:52.920991 19723 consensus_queue.cc:260] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.914521 20017 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:52.923183 20017 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:52.923715 10023 raft_consensus.cc:2393] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:52.924050 20017 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.918207 20049 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:52.925293 20017 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.910254 20026 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.925712 20071 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.926416 20026 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:52.928750 20116 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.929785 10092 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:52.902802 20118 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:27:52.917474 20045 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:52.936930 10098 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:52.938930  9941 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:52.939218 10642 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:52.939987 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:52.940758 10670 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.944645 10094 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
W20260709 12:27:52.946954 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.949132 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:52.951296 10674 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:52.953332 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.953567 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:52.954001 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.954741 10097 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:52.961515 10642 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:52.959465  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54207.6us
W20260709 12:27:52.964715  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:52.973003 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:52.973678 10669 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:52.975108 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:52.959005 20085 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:52.973654 19964 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:52.984613 19964 raft_consensus.cc:697] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:52.985697 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:52.985530 19964 consensus_queue.cc:237] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.985911 20116 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:52.986650 20059 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:52.950989 19676 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.988574 20065 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.989593 19676 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:52.990763 20065 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:52.991495 20071 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:52.993763 20081 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.994879 10090 raft_consensus.cc:3055] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:52.995258 10090 raft_consensus.cc:740] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:52.995975 10090 consensus_queue.cc:260] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:52.997057 10090 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:52.996846 20134 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:52.987730 20046 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:52.994899 10016 raft_consensus.cc:1240] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 18. Current term is 19. Ops: []
I20260709 12:27:53.000236 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:53.001518  9946 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:53.003054  9941 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:53.000185 10642 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:53.004535  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:53.005169 20071 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:53.005905 20071 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:53.006840 20071 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.002691 20046 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:53.014285 20063 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:53.014819 20063 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:53.015686 20063 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.016812 20063 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:53.018113 19772 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:53.018767 19772 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.021552 20045 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.023316 20085 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:53.010694 10090 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:27:53.025092 10097 raft_consensus.cc:2468] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:53.026958 10090 pending_rounds.cc:85] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 14
I20260709 12:27:53.028883 20134 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.029126 10065 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:53.048278 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:53.047327 10642 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:53.221897 10669 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 22.
W20260709 12:27:53.222052 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 174851us
I20260709 12:27:53.141949  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:53.150108 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:53.223449 10676 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:53.022189 20045 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.154578 10019 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:53.225939  9944 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:53.225884 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.158205 10025 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:53.141852 20118 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.162669 10018 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:27:53.021832 19772 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.170742 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.179378 10023 raft_consensus.cc:1240] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:53.185442 20081 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:53.195557 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.197038 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.198370 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.233155 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:53.215659 10090 pending_rounds.cc:107] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:27:53.220592 19618 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:53.235846 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 206787us
I20260709 12:27:53.236469 10022 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:53.238969 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:52.497853  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:27:53.234555 10090 raft_consensus.cc:2889] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:53.147670 19816 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.077s
I20260709 12:27:53.237380 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:53.250144 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:53.256822 10022 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:27:53.225698 19676 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.252282 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.254720 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:53.262049 10024 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 24.
I20260709 12:27:53.232589 19651 raft_consensus.cc:2804] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:53.014317 20046 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.262441 10019 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:53.163364 19841 consensus_queue.cc:237] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.243607 20065 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:27:53.244212 19618 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.271744  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:53.253888 20134 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:27:53.242856 20049 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.025s
I20260709 12:27:53.236299 20118 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.263254 20021 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:53.283442 20054 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:53.282389 19771 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:27:53.294440 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44365.6us
I20260709 12:27:53.267745 19676 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.255743 20078 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.028s
I20260709 12:27:53.255502 19654 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:27:53.304229 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:53.308043 10668 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 2110) took 1084 ms. Trace:
W20260709 12:27:53.311488 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 53631.4us
I20260709 12:27:53.311512 20118 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.282660 19651 raft_consensus.cc:697] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:53.308849 10668 rpcz_store.cc:276] 0709 12:27:52.223953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.258208 (+ 34255us) service_pool.cc:224] Handling call
0709 12:27:52.282650 (+ 24442us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.455554 (+172904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:53.307742 (+852188us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b5400408dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:53.307971 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":382062976}
I20260709 12:27:53.283582 20065 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:53.271118 20108 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:27:53.303591 19771 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.343371 19771 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:27:53.345630 20133 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 12:27:53.346093 20049 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:53.324546 10017 raft_consensus.cc:2393] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 31: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:53.346616 20049 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:53.327368 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:53.347868 10668 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:53.329164 19676 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:53.330400 19651 consensus_queue.cc:237] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.352099 20045 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.354916  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.347560 20049 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.357553  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:53.336820  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:53.367885 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:53.324671 20078 consensus_queue.cc:1243] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:27:53.372017 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:53.373133 10670 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:53.376070 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:53.377838 20090 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:53.378715 10674 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:27:53.355356 20065 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.376940 10022 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:53.381151 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:53.378288 20078 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.385169 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:53.319267 20070 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 12:27:53.347590 20133 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:53.283300 20085 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.405366  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:53.382028 20108 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:27:53.412213 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.380570 20021 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:53.423714 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.426036  9941 raft_consensus.cc:1240] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:53.385031 20159 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:27:53.430394 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.379282 20114 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.044s
I20260709 12:27:53.431951 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.436957 10024 raft_consensus.cc:1275] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:27:53.421552 20108 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:53.411460 20161 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:53.424088 20081 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:53.438303 20026 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.511s	user 0.010s	sys 0.000s
I20260709 12:27:53.394546 20078 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.443104 20108 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:27:53.472575 20131 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.752s	user 0.002s	sys 0.001s
I20260709 12:27:53.445574 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:53.438208 20159 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:53.404088 20133 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.461570 19913 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.423969 20155 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.071s
W20260709 12:27:53.471771 20150 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:27:53.438271 19676 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:27:53.479839 20021 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.474311 20078 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.441318 20054 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:53.483474 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:53.484603 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:53.494956 10669 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:53.486788 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:53.443837 20161 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.476987 20134 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:53.507085 10640 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:53.487989 19816 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:53.503800 10062 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:53.486248 20090 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:53.504132 20018 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:53.476974 19771 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.485566 20159 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.488129 19913 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.521315 20114 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:53.520716 20090 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.527088 20114 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.527570 20114 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:53.528782 20090 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:53.499536 20161 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:53.529665 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:53.500049 20078 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:53.530964 19841 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.531122 20078 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.531490 19841 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.503919 20054 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:53.533603 19841 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:53.520362 19823 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:53.535023  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.513841 20071 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 12:27:53.540282 20108 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.540992 19913 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:53.543433  9947 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:27:53.545022 10676 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:53.546322 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:53.554119  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.559657 19676 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.560389 19676 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.499778 20134 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:53.534102 20054 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.569595 20134 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.570889 20134 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:53.574162 10673 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 28. Ops: []
I20260709 12:27:53.575232 20091 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:53.575832 20091 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.576704 20118 raft_consensus.cc:3055] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:53.577102 20118 raft_consensus.cc:740] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:53.577102 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:53.578094 20118 consensus_queue.cc:260] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.607615 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:53.608824 10673 raft_consensus.cc:2393] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:53.659698  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.660549  9947 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:53.663064  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:53.666713  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 74651.6us
I20260709 12:27:53.671484 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:53.672129 10019 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 33.
I20260709 12:27:53.706753  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.706827 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:53.710198  9946 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:53.644556 20114 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.718228  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.669152 20135 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.055s
I20260709 12:27:53.725903 20091 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.731003 19676 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:53.735554 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.702135 19976 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:27:53.750697  9943 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:53.742161 20075 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:53.757328 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:53.745201 20175 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:53.745921 20045 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:53.749315  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:53.760056 10096 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:27:53.737242 20114 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:53.751605  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.755064  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:53.755685 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:53.757565 20075 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:53.758044 10091 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:27:53.743011 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:53.758519 10095 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:53.763998 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:53.751650 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:53.764380  9943 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:27:53.767639 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:53.767850  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:53.765960 10673 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:53.765192 10092 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:53.763029  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.765261 20018 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:53.778926 19976 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:53.779371 19976 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:53.780164 19976 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.781361 19976 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:53.772718 20070 consensus_queue.cc:1059] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.014s
I20260709 12:27:53.772504 20159 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.784027  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:53.759074 20134 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:27:53.786815  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.790851  9949 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 27 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:53.792385  9946 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 22. Current term is 23. Ops: []
I20260709 12:27:53.787663  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:53.800407  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:53.801158  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.801985  9948 raft_consensus.cc:2410] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 32 because replica has last-logged OpId of term: 30 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
I20260709 12:27:53.762143 20182 raft_consensus.cc:476] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Not starting pre-election -- already a leader
I20260709 12:27:53.806967  9946 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: [25.14-25.14]
I20260709 12:27:53.804209 20185 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:27:53.794909 19120 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:53.807811 19120 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:53.805029  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.813242  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.813884  9948 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:53.814106  9943 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:53.812216  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.815915  9940 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:53.818936  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.821501 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:53.796070 19771 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.823581 10673 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:53.823747 10065 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:53.802682 19756 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:53.828015 20070 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:53.829663  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:53.805816 20135 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:53.824676 10677 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:53.772666 20175 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.803730 20157 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:53.797629 20134 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.815768 20071 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.032s
I20260709 12:27:53.845566 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:53.815923 20017 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:27:53.816069 20060 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.021s
I20260709 12:27:53.851707 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:53.846798 19120 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.829639 19841 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.021s
I20260709 12:27:53.861945 19120 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:53.838514 20070 raft_consensus.cc:697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:53.822556 20065 consensus_queue.cc:1059] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:27:53.858338 20159 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:53.868742 20183 log.cc:927] Time spent T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:27:53.872102 20157 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:27:53.872574 20157 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:53.873425 20157 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.875862 20134 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:53.877871 20161 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.880990 19584 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.882198 19584 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:53.882460 20157 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:53.885005 19650 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.885524 19650 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.887167 20070 consensus_queue.cc:237] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.870056 20159 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:53.888983 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:53.889158 20159 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:53.889580 10023 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:53.847230 19756 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:53.902010 20178 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:53.906885 20178 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.907226 19756 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 27.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.909132 20178 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:53.862826 20162 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:53.906006 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:53.910902 10022 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:53.914374 10023 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
W20260709 12:27:53.915948  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 86343.8us
I20260709 12:27:53.916683  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:53.918673  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:53.919977 20190 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:53.920501 20190 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:53.920028 19841 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:53.922192 19841 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.922577 19841 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:53.923267 20190 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:53.925099  9942 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:53.910290 20162 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:53.931862 20075 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.945029 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:53.945775 10093 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:53.951395 19651 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:53.951911 19651 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:53.952098 20175 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.929121 20161 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.963670 19651 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.22, Last appended by leader: 22, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:53.965260 19651 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:53.928006 19650 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:53.969614 10099 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:53.957643 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:53.975926 10093 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:53.984139 20046 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:53.987686 20078 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:53.992785 10673 raft_consensus.cc:1240] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:53.993942 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:53.994688 20133 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:53.995393 10675 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:53.996630  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:53.999521 20157 consensus_queue.cc:1059] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:53.999934  9987 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:54.000317 20157 consensus_queue.cc:1243] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer 6c7bd3fa1bfa431c89fc9c9096452a76 log is divergent from this leader: its last log entry 29.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:27:54.000841 20157 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:54.001499 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 2108) took 1031 ms. Trace:
I20260709 12:27:54.001700 19976 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:54.007930 20118 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:54.010974 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:54.012941 20071 consensus_queue.cc:1048] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:27:54.013563 20157 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:54.014149 20157 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.014448 20071 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:54.016240 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.021719 10091 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:54.021898 20081 raft_consensus.cc:3055] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:54.016040 20096 consensus_queue.cc:1059] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.006s
I20260709 12:27:54.022331 20081 raft_consensus.cc:740] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:54.023110 20071 raft_consensus.cc:3055] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:27:54.023481 20071 raft_consensus.cc:740] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:54.023573  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.015643 19976 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:54.023718 20081 consensus_queue.cc:260] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.024564 19976 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.025344 20081 raft_consensus.cc:3060] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:54.027861 10023 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:27:54.030773 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:54.034708 19976 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.035017 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.019270 10097 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:54.036655 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:54.037242 10020 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:54.038543 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:54.039122 10020 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:54.016157 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.001993 10674 rpcz_store.cc:276] 0709 12:27:52.970512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:52.970709 (+   197us) service_pool.cc:224] Handling call
0709 12:27:53.375711 (+405002us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:53.379243 (+  3532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.000444 (+621201us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b540058f2d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:54.001288 (+   844us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b540058f2d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:54.001443 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":936711296}
I20260709 12:27:54.051829 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:54.053696 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:54.053989 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:54.026427  9834 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 334) took 43323 ms (client timeout 15000 ms). Trace:
I20260709 12:27:54.057140  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.057356  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:54.055022 10674 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:53.943279 20162 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:54.059120  9945 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:54.062973 19976 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:54.063212 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:54.064142 20071 consensus_queue.cc:260] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.065400 20071 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:27:54.059774 20060 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:54.067682 20060 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.068951 10097 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:27:54.070185 20060 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:54.070827 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.070848 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.070237 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:54.077383 10094 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:54.080117 10093 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:54.086081 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:54.092512 10018 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 23.
I20260709 12:27:54.078740  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:54.099318 10097 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:54.104221  9945 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 18. (term mismatch)
I20260709 12:27:54.104796  9945 pending_rounds.cc:85] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Aborting all ops after (but not including) 17
I20260709 12:27:54.105053  9945 pending_rounds.cc:107] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Aborting uncommitted NO_OP operation due to leader change: 28.18
I20260709 12:27:54.105325  9945 raft_consensus.cc:2889] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:54.020025 20091 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:54.018153 20044 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:54.110673  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:54.111881 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51198.2us
I20260709 12:27:54.017323 20108 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.014993 19618 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:54.169744 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62742.6us
W20260709 12:27:54.203784 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 117782us
I20260709 12:27:54.193501 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:54.208876 10675 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:54.210246 10668 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:54.214684 20026 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:54.215533 20182 raft_consensus.cc:3055] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:27:54.215988 20182 raft_consensus.cc:740] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:54.216889 20182 consensus_queue.cc:260] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.218204 20182 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:54.220214 10674 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:27:54.221807 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:54.235904 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 63442.5us
I20260709 12:27:54.115831 20091 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:54.189705 20118 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:54.243768 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:27:54.245677 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:54.231667 20160 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.737s	user 0.003s	sys 0.000s
W20260709 12:27:54.249280  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 138722us
I20260709 12:27:54.242561 20193 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:54.267942  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:54.268261 20193 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:54.268785 20193 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:54.270870 19120 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:27:54.242619 20177 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:54.254455 19768 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:54.271474 19120 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:54.271858 19768 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.272347 19768 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:54.272535 19120 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:54.254671 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.274808 20194 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.275334 20194 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.275777 19913 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.276336 19913 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:54.258754 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53573.2us
I20260709 12:27:54.259804 10675 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:54.278019 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:54.277482 20194 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.279484 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:54.280426 19654 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:54.280274 20159 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.281876 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:54.283339 10024 raft_consensus.cc:1240] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 27. Current term is 28. Ops: []
W20260709 12:27:54.242447 20174 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
W20260709 12:27:54.270457  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.286172 20190 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.286886 20190 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.290171  9947 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:54.291183 19650 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.291765 19650 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.282369 20159 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.291313 20075 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.291127 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.294162 10099 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:54.242712 20195 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.296702 20195 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:54.241338 20170 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.300s	user 0.002s	sys 0.000s
I20260709 12:27:54.299860  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.265461 19841 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.266893 20085 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.018s
I20260709 12:27:54.247225 20191 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.017s
I20260709 12:27:54.300915  9943 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:27:54.304617 10094 raft_consensus.cc:1275] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:27:54.295567 20190 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.307192 10670 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
W20260709 12:27:54.248872 20171 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:27:54.235039 19756 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:54.278386 19913 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.315878 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:54.309136  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:54.326025 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:54.326985 10017 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:54.330812 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:54.331678 10675 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:54.307699 19654 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:27:54.335081 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:54.340385 10017 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:54.316617 19756 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.318461 20155 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:54.347466  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 333137us
I20260709 12:27:54.364948  9990 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:54.334824 19771 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.366313 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:54.320559 20190 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:54.368762 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54075.2us
W20260709 12:27:54.373829  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.372061  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:54.379982  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:54.380652 10017 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:27:54.381719  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:54.382021 10024 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:27:54.382568 19798 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:27:54.382942 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:54.383292 19798 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:54.384399 19798 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:54.345665 20195 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.394778 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:54.363134 20155 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.347754 19756 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:54.341622 20194 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:27:54.393997 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:54.391673 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:54.375959 19903 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:54.408996 20017 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:54.409833 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.405848 20060 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.004s
I20260709 12:27:54.412570 20017 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.372524 20091 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.414259 10096 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:54.414742 20091 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:54.415194  9914 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:54.417428 20189 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:54.417855 20189 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:54.419014 10097 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:27:54.418732 20189 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.420307 20189 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:54.451704  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.403749 20155 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:54.374097 19771 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.473603  9945 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:27:54.447189 20090 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:54.461875  9948 raft_consensus.cc:1240] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:27:54.459734 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:54.402874 20190 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:54.055951  9834 rpcz_store.cc:269] 0709 12:27:10.702731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:24.639445 (+13936714us) service_pool.cc:224] Handling call
0709 12:27:54.026184 (+29386739us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:43.843766 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:43.844527 (+   761us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:43.844586 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:27:43.844603 (+    17us) tablet.cc:662] Decoding operations
0709 12:27:44.289902 (+445299us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:44.290036 (+   134us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:44.290080 (+    44us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:27:44.293101 (+  3021us) tablet.cc:701] Row locks acquired
0709 12:27:44.293162 (+    61us) write_op.cc:260] PREPARE: finished
0709 12:27:44.293418 (+   256us) write_op.cc:270] Start()
0709 12:27:44.293623 (+   205us) write_op.cc:276] Timestamp: P: 1783600064293319 usec, L: 0
0709 12:27:44.293651 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:27:44.346209 (+ 52558us) log.cc:844] Serialized 9576 byte log entry
0709 12:27:45.956391 (+1610182us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:46.715029 (+758638us) op_driver.cc:464] REPLICATION: finished
0709 12:27:47.247370 (+532341us) write_op.cc:301] APPLY: starting
0709 12:27:47.247484 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:47.994075 (+746591us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:47.994106 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:48.617099 (+622993us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:48.617591 (+   492us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=0,mrs_lookups=0
0709 12:27:48.617675 (+    84us) write_op.cc:312] APPLY: finished
0709 12:27:48.618351 (+   676us) log.cc:844] Serialized 212 byte log entry
0709 12:27:48.861426 (+243075us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:49.021821 (+160395us) write_op.cc:454] Released schema lock
0709 12:27:49.021949 (+   128us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:49.022561 (+   612us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:49.022623 (+    62us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1742925,"reported_tablets":47,"spinlock_wait_cycles":9984,"child_traces":[["op",{"apply.queue_time_us":528311,"apply.run_cpu_time_us":72672,"apply.run_wall_time_us":1775151,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104495,"num_ops":24,"prepare.queue_time_us":100613,"prepare.run_cpu_time_us":19144,"prepare.run_wall_time_us":877202,"raft.queue_time_us":526620,"raft.run_cpu_time_us":11392,"raft.run_wall_time_us":457038,"replication_time_us":2375383,"spinlock_wait_cycles":46935424,"thread_start_us":1098755,"threads_started":5,"wal-append.queue_time_us":1030485,"wal-append.run_cpu_time_us":29677,"wal-append.run_wall_time_us":3429897}]]}
I20260709 12:27:54.442430 19903 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:54.480826  9945 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:27:54.481949 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:54.481750 20177 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:27:54.485899 10677 raft_consensus.cc:2410] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 29 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
I20260709 12:27:54.476789 20017 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.462455 20116 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:27:54.479447 20190 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.456452 19976 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.494747  9947 raft_consensus.cc:1275] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:27:54.497557 19841 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.512117 20168 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:27:54.472934 20118 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.030s
I20260709 12:27:54.514564 19771 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.519644 20090 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:54.521459  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 143036us
I20260709 12:27:54.500561 10668 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
W20260709 12:27:54.478608 19651 consensus_meta.cc:287] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.513s	user 0.012s	sys 0.000s
I20260709 12:27:54.503242 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:54.447616 20071 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.059s
W20260709 12:27:54.527701  9834 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 317) took 18593 ms (client timeout 15000 ms). Trace:
W20260709 12:27:54.528116  9834 rpcz_store.cc:269] 0709 12:27:35.934005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.527504 (+18593499us) service_pool.cc:211] Skipping call since client already timed out
0709 12:27:54.527628 (+   124us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:27:54.507766 20193 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.481033 20060 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.485245 20191 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:27:54.509367 20070 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:54.510638 20190 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:54.511106 19976 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:54.531500 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:54.532066 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:54.532862  9946 raft_consensus.cc:1275] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:27:54.538319  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:54.558702  9948 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:54.546228 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:54.548259 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:54.553429 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:54.563333 20114 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:54.564150 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:54.565030 20159 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:54.564060 20114 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:54.566474 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:54.570829  9944 raft_consensus.cc:1240] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:54.530305 19756 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.577337 10677 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 29 index: 22. (term mismatch)
W20260709 12:27:54.578819  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 161285us
I20260709 12:27:54.579087 10677 pending_rounds.cc:85] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting all ops after (but not including) 21
I20260709 12:27:54.579610 10677 pending_rounds.cc:107] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting uncommitted NO_OP operation due to leader change: 28.22
I20260709 12:27:54.579949 10677 raft_consensus.cc:2889] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:54.585259 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:54.585820 10668 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:27:54.587289  9948 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:54.552196 20070 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:54.593570  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:54.594508  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:54.551501 20118 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.047s
I20260709 12:27:54.569949 20194 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:54.568876 20177 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:27:54.576182 19771 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.555842 20185 consensus_queue.cc:1059] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:27:54.551245 20060 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.607681 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.601303 20069 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:54.593325 20154 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:54.593498 20096 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:54.636430 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.646097  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:54.637322  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 255657us
I20260709 12:27:54.599920 20070 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:54.648643 20069 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:54.649065 20217 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.003s
I20260709 12:27:54.649796 20069 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.650776 20200 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.059s
I20260709 12:27:54.648180  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:54.656092 20210 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:54.660108 19927 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:54.663228 19823 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.663952 19823 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:54.667228 20205 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:27:54.668128 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.668002 20070 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.668900 19768 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.669313 19976 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.672523 19823 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.669226 10091 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:54.673153 19723 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:54.652503 20185 consensus_queue.cc:1243] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:27:54.674758 20216 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:54.675315 20216 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.675714 20216 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:54.676743 10670 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:27:54.677745 20060 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.679772 20193 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.680246 20189 raft_consensus.cc:3055] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:54.680688 20189 raft_consensus.cc:740] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:54.681723 20189 consensus_queue.cc:260] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.603401 20016 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.044s
I20260709 12:27:54.684593 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:54.686285  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.687220  9946 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:54.690769  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:54.683504 20017 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.600850 19903 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.692112 20060 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:54.637742 20194 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.640002 19771 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.705972 20080 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:54.705808 20200 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:54.716640 20189 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:27:54.731462  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 60873.3us
I20260709 12:27:54.717006 20060 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.759546 20211 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:54.760617 20211 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.733832 20080 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:54.765826  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.714399 20017 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.773133  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:54.757539  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:54.746133 20060 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:54.740861 20081 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:54.749897 20213 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:54.770723 20080 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:54.746778 20200 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.786473 19976 consensus_queue.cc:1048] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.044s
I20260709 12:27:54.786128 20213 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.811828 20211 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.829795 20070 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.784297 20081 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:54.859433 20162 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.753s	user 0.010s	sys 0.007s
I20260709 12:27:54.871551 20190 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.872344 19771 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.873106 20017 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.874396 20194 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.875630 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.783744 20069 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.031s
I20260709 12:27:54.876883 20182 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.886224  9946 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:54.903826  9943 raft_consensus.cc:1275] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:27:54.907089 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:54.881062 20081 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.905017 19984 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.920578  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.880591 20155 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.923981 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:54.927909 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.929013 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.918286 20116 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:54.932703 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.933100  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:54.920331 19756 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:54.937472 10022 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:27:54.941367 10093 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:54.924496 20217 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.918448 20118 consensus_queue.cc:1048] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.022s
I20260709 12:27:54.948794 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.932729 19723 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.917941 20182 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.905347 20162 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.951081 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:54.951387  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:54.955922 20200 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.956812 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:54.963382  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.964906 10092 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:27:54.965204 10093 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:54.967935  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:54.948429 19976 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:54.944214 20116 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:54.973502 10024 raft_consensus.cc:1240] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:54.931677 19984 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.969151 20200 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:54.974658 20070 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:54.975276 20070 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:54.976919 19984 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:54.977059 20070 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:54.977592  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:54.978405 10016 raft_consensus.cc:1275] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:27:54.979050  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.972294 19841 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:54.980818  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:54.981447  9943 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:54.972519 20096 consensus_queue.cc:1048] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.009s
I20260709 12:27:54.983626 10020 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:54.995563 20155 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:54.998953  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:55.000440  9943 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:54.950066 20217 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.964159 19723 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:54.995752 20080 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.002281 20200 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.008s
I20260709 12:27:55.010931 20116 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.011452 20157 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.020465 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3071) took 3700 ms. Trace:
I20260709 12:27:55.044230 10021 rpcz_store.cc:276] 0709 12:27:51.320164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:51.560393 (+240229us) service_pool.cc:224] Handling call
0709 12:27:52.603952 (+1043559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:52.669267 (+ 65315us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.148241 (+1478974us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b54001392d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:55.020255 (+872014us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54001392d0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:55.020399 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2455947904}
I20260709 12:27:55.045835 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:55.025249 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:55.038002 10093 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:55.047056 10092 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:55.047765 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.048846 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:55.049384 10093 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:55.047823 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.050042 10092 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:55.051257 10092 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:27:55.054909 20207 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:27:55.057529 10093 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:55.058923  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:55.059772 20225 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:55.060302 20225 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:55.061059 20225 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.046428 10021 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:55.088615 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.005604 20070 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:55.089597 10094 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:55.099483  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:55.102427 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:55.082535 10021 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:55.103943 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:55.103955  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 131382us
I20260709 12:27:55.110431 10095 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:55.101439 20190 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.104012 20207 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:55.124711  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:55.103928 20235 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.112887 20155 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:55.115034 20114 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.093874 20070 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.102943 20216 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.127247 20207 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:55.144246 20207 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:55.144325 20080 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:55.148212  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:55.149062  9940 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:55.153141 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:55.117668 20081 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 27.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 12:27:55.138994 20177 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:55.130455 20060 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.141912 20155 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:55.139423 20114 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:55.183322 20233 log.cc:927] Time spent T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:27:55.153752 20182 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:55.190037 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55597.7us
I20260709 12:27:55.193780 10097 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:55.135339 20235 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.209370 20114 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.210235 20155 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.230134  9940 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:55.237879  9991 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:55.243538  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:55.234736 20080 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:55.253896 20080 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.246186 20236 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:55.255163 20236 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:55.256434 20236 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.256903 20235 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.257071 20080 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.258889 20060 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:55.234516 20182 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.263353 20182 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.272655  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:55.272965  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:55.273720  9946 raft_consensus.cc:2410] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:27:55.235601 19723 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:55.276046  9943 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
W20260709 12:27:55.263939 20198 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.826s	user 0.003s	sys 0.000s
W20260709 12:27:55.400396  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89269.2us
I20260709 12:27:55.312592 20168 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:55.400008 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:55.400064  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:55.402223 10097 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:55.204769 20177 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:55.417140 20242 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.179s	user 0.006s	sys 0.002s
I20260709 12:27:55.440033  9940 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:55.419189 20235 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.408604 19723 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.443255 20190 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:55.417788 20168 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:55.479014 20253 log.cc:927] Time spent T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:27:55.477432 19723 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.480140 20189 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:55.485109 20257 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.496075 10090 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4693) took 1577 ms. Trace:
I20260709 12:27:55.511929 20257 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.511942 10090 rpcz_store.cc:276] 0709 12:27:53.919048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.919275 (+   227us) service_pool.cc:224] Handling call
0709 12:27:53.919587 (+   312us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:53.919976 (+   389us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:27:53.920285 (+   309us) log.cc:844] Serialized 14 byte log entry
0709 12:27:54.246259 (+325974us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:54.495767 (+249508us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b2c00115cd8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:54.496211 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 12:27:54.496401 (+   190us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:27:54.496433 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.496508 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.263394 (+766886us) spinlock_profiling.cc:243] Waited 2.83 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:55.495896 (+232502us) raft_consensus.cc:1685] finished
0709 12:27:55.495918 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.496024 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43383,"prepare.queue_time_us":1023,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":4388,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":50035,"spinlock_wait_cycles":132637440,"thread_start_us":325645,"threads_started":1,"wal-append.queue_time_us":451896}
W20260709 12:27:55.497089 20181 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.762s	user 0.004s	sys 0.001s
I20260709 12:27:55.507079 19723 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:55.477551 20235 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:55.508040 20246 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:27:55.509552 20168 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:55.516706  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 2274) took 1460 ms. Trace:
I20260709 12:27:55.488869 20190 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.532212 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:55.514530 20257 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:55.514956 20249 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:27:55.494264 20216 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.533890  9944 rpcz_store.cc:276] 0709 12:27:54.056287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.056419 (+   132us) service_pool.cc:224] Handling call
0709 12:27:54.056675 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.576652 (+519977us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.516442 (+939790us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b54001b9254. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:55.516624 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182011776}
I20260709 12:27:55.538961 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.543061 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.516511 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.548642 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.516448 20189 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.549345 10095 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:55.537361 20091 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:55.539121 20236 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:55.538664 20168 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.559300 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:55.547155 20225 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:55.592164 19984 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:55.592968 19984 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.594826 19903 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:27:55.588675  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:55.599628  9944 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:55.575804 20096 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:55.575055 20177 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:55.620501  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 101717us
I20260709 12:27:55.565272 20091 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.607844 19772 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:55.614769  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:55.616286 19984 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.617901 20237 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:55.618892 10020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:27:55.629623 10020 raft_consensus.cc:606] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:55.567250 20236 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.568099 20168 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.625953  9948 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:55.626969  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41138 (request call id 3660) took 1828 ms. Trace:
I20260709 12:27:55.571305 20225 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.633095  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:55.634259 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:55.634804 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:55.643793 10668 raft_consensus.cc:2368] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30.
I20260709 12:27:55.643934 10675 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:55.607764 19756 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:55.618940 20085 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.637998  9949 rpcz_store.cc:276] 0709 12:27:53.798622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.799156 (+   534us) service_pool.cc:224] Handling call
0709 12:27:53.803395 (+  4239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:53.948835 (+145440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.948868 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:53.948911 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:53.948942 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.626748 (+1677806us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400096354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:55.626794 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.626915 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333911808}
I20260709 12:27:55.639281 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:55.642220 20258 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:27:55.642614 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:55.625316 20116 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:55.646062 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:55.646257  9940 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:27:55.725039 10675 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:55.624516 20247 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:55.663595 10093 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:55.664623  9948 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:27:55.612484 20096 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.730679 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:55.732666 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:55.748867 10062 leader_election.cc:400] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30.
I20260709 12:27:55.749977 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:55.635789 20237 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.631290 19772 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:55.746417 20264 log.cc:927] Time spent T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:27:55.754076 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:55.737784 20096 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:55.744583 20255 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:55.668870 19756 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:55.753099 20181 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:27:55.753928 20060 consensus_queue.cc:1059] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.061s
I20260709 12:27:55.762144 20235 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.822719 20277 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:55.841684 20277 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.844220 20277 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.824873 20276 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:55.754907 20026 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:55.821165 20217 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:55.845678 20276 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.758944 20189 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:55.847808 20070 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.848276 20276 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:55.853719 20261 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
I20260709 12:27:55.856218 19841 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:55.856720 19841 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:55.857615 19841 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.866905 19650 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.867888 20225 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:55.872033 20237 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:55.873677 20236 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:55.875118 20190 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:55.875685 20162 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:55.876672 20267 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:27:55.779454 19984 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:55.885408 19984 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.885792 19984 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:55.760232 20213 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:55.833128 20266 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 12:27:55.834184 20270 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
W20260709 12:27:55.840269 20203 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 1.395s	user 0.000s	sys 0.003s
I20260709 12:27:55.726008 20085 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:55.808045 20239 log.cc:927] Time spent T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 12:27:55.832123 20265 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:27:55.888557 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:55.891021 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.776157 20268 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:55.921809 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.809932 20255 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:55.747771 20116 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.910771 10095 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:55.912288 10094 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
W20260709 12:27:55.887630 20207 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.001s
I20260709 12:27:55.908694 19841 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:55.747516 20247 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:55.938978 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.940441 10669 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:55.940919 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.947078 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.951464 10096 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:55.914881 20070 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:55.951097 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 303552us
I20260709 12:27:55.958002 10643 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:55.937707 20255 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:55.963096 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:55.977098 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.993062 10098 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4696) took 2021 ms. Trace:
I20260709 12:27:55.977908 10094 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 33.
I20260709 12:27:56.017355  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:55.941036 20207 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:55.971668 20210 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.024210  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:56.019233 10098 rpcz_store.cc:276] 0709 12:27:53.971909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:53.972076 (+   167us) service_pool.cc:224] Handling call
0709 12:27:53.972516 (+   440us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:53.972971 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:53.973012 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:53.973598 (+   586us) log.cc:844] Serialized 26 byte log entry
0709 12:27:54.300163 (+326565us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:54.300207 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:54.300282 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:55.839445 (+1539163us) spinlock_profiling.cc:243] Waited 5.94 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:27:55.975331 (+135886us) raft_consensus.cc:1685] finished
0709 12:27:55.975360 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:55.992974 (+ 17614us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4834,"raft.queue_time_us":69431,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":18995,"spinlock_wait_cycles":13666176,"thread_start_us":326235,"threads_started":1,"wal-append.queue_time_us":419765}
I20260709 12:27:56.029992  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.026381  9948 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:55.996948 20276 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:56.037822 10672 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:27:55.992164 20277 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.035737  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.039654  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.040550  9948 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:55.939884 20268 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.041319 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:56.015763 20190 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:55.979431 20071 consensus_queue.cc:1059] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:27:56.000708 20091 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.047302 19798 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.059026 19772 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.058784 20190 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.059284 19798 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.060083 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:55.976136 20070 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.061453 19798 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.063026 20070 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.063516 20085 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.043648 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:56.064847 10673 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:56.018236 20168 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:56.018675 20192 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:56.055434 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1922) took 1145 ms. Trace:
I20260709 12:27:56.074383 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.048044 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.052148 19654 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:56.052464 20182 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:56.042472 19618 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:56.044314 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.085338 10676 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:56.034816 20210 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.085825 10095 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:56.078764 10091 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:56.079670 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.086519 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.082784 10672 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:56.073678 10016 rpcz_store.cc:276] 0709 12:27:54.910318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.980134 (+ 69816us) service_pool.cc:224] Handling call
0709 12:27:54.980393 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.854822 (+874429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:55.854869 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:55.854917 (+    48us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:55.854953 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.055217 (+200264us) spinlock_profiling.cc:243] Waited 874 ms on lock 0x7b54001f5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:56.055262 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.055377 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2010538240}
I20260709 12:27:56.087222 10098 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:27:56.091416 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.094347 10673 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:56.097302 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.096117  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:56.070927 20216 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.072185 20200 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.105234 20200 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:56.105904 20274 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:27:56.090293 19676 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.017s
I20260709 12:27:56.103089 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.083247 19823 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.077633 19654 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:56.107594 20200 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.107926 20210 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.111960  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.100564 20235 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:56.115468 20204 log.cc:927] Time spent T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 1.670s	user 0.004s	sys 0.000s
I20260709 12:27:56.115190 20235 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.117403 20235 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.083475 20276 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:56.121701  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.125197  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3388) took 1413 ms. Trace:
I20260709 12:27:56.126406  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.125597  9941 rpcz_store.cc:276] 0709 12:27:54.711729 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.711922 (+   193us) service_pool.cc:224] Handling call
0709 12:27:54.727358 (+ 15436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.733067 (+  5709us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.733117 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:55.616814 (+883697us) spinlock_profiling.cc:243] Waited 4.84 ms on lock 0x7b1800352a80. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064372a6 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a
0709 12:27:55.617806 (+   992us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b2c00012928. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:55.618287 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:27:55.746850 (+128563us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:27:55.746905 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:55.747003 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:56.124993 (+377990us) raft_consensus.cc:1685] finished
0709 12:27:56.125008 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.125115 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5183,"raft.queue_time_us":8106,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":20437,"spinlock_wait_cycles":18037504,"thread_start_us":128124,"threads_started":1,"wal-append.queue_time_us":294798}
I20260709 12:27:56.128959 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.131546 10020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:27:56.045965 20277 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.107646 19823 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.150408 19823 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.132206 10020 raft_consensus.cc:606] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:56.151054 20277 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.151352 10020 raft_consensus.cc:614] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:56.151058 10670 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 19.
I20260709 12:27:56.121529 19618 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:56.134289 19676 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:56.153258 19676 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:56.153658 19676 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:56.152078  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 2279) took 1701 ms. Trace:
I20260709 12:27:56.154937  9947 rpcz_store.cc:276] 0709 12:27:54.450212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.450904 (+   692us) service_pool.cc:224] Handling call
0709 12:27:54.471152 (+ 20248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.502853 (+ 31701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.151871 (+1649018us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b540018a1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:56.152029 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53371648}
I20260709 12:27:56.088382 19976 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:27:56.130896 20081 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:27:56.138490 20247 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.155366 20018 raft_consensus.cc:3055] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:56.131603 19756 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.140085 20116 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:56.204357  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.200171  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:56.204968 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.208987  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.209744  9941 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 33.
W20260709 12:27:56.212218  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:56.214999  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:56.151382 20211 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:56.085111 20192 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.220083 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.220647 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:56.133040 20235 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:56.231968  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.121201 20216 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.274772 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.275782 10019 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:56.276477 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.280479 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:56.281256 10019 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:56.244474 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:56.210542 20090 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.202392 20018 raft_consensus.cc:740] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:56.287422 10020 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:56.310552 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:56.317262 10022 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:56.312500 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.318365 10025 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:56.314496 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.213658 19756 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.324326 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:56.325029 10017 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:56.328382 20118 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.328948 20118 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.329154 19654 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.330988  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.331532 20081 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:56.332604 10019 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
W20260709 12:27:56.311452 20297 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:27:56.333178  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.223301 20140 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.333833  9948 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:56.333995 20140 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.334841 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.335930 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.336108 20140 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.336944 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.337610  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.337975 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.338187  9940 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:56.334870 20090 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.339217 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.340374 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.340631 20288 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.340945 10019 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:56.341162 20090 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.342442  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.342571 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:56.342727 20298 log.cc:927] Time spent T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:27:56.343097  9947 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:56.142297 20276 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.341840 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.347987 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.348835 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:56.351279 20293 log.cc:927] Time spent T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 12:27:56.216692 20292 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:27:56.221823 20211 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:56.353118 20279 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:56.353297 20211 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.353667 20279 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.355249 20018 consensus_queue.cc:260] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:56.355885 20283 log.cc:927] Time spent T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.532s	user 0.000s	sys 0.002s
I20260709 12:27:56.356470 20018 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:56.356824 20234 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:56.356895 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:56.357750 20060 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:56.358294 20060 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.358734 20237 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.359278 20237 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.359844 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.359359 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.360849 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.361332 20237 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:56.361776 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.360963 20257 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:27:56.362268 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.362579 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.362860 10672 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:27:56.357501 20234 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.358376 20287 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:56.360476 10671 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:56.363946 20287 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.364928 20234 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:56.365300 20118 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.360467 20060 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.361224 20085 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.005s
I20260709 12:27:56.316282 20235 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.353847 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.367797 20268 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.368237 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.369170  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.369278 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.369853 10016 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 24.
I20260709 12:27:56.371402  9941 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:56.372133 20216 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.372675 20216 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:56.373608 19756 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.361027 10673 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:56.375909 19756 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.377154 20235 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.366302 20287 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.363425 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:56.378167 10020 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:56.367991 19676 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.380689 19676 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.381201 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.382802 10016 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:27:56.384402  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3392) took 1481 ms. Trace:
I20260709 12:27:56.219667 19798 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:56.341588 20282 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:56.405936 10017 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:56.352152 20192 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:56.405776 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 195466us
I20260709 12:27:56.408304 10017 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:27:56.243521 20277 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:56.413597 10065 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.420655 20159 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:27:56.421144 20159 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.421545 20159 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:56.421542 20085 consensus_queue.cc:1048] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:56.386077 10025 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:56.384788  9945 rpcz_store.cc:276] 0709 12:27:54.902945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.903823 (+   878us) service_pool.cc:224] Handling call
0709 12:27:54.904186 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:55.862481 (+958295us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:55.862520 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:55.862567 (+    47us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:27:55.862631 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.384178 (+521547us) spinlock_profiling.cc:243] Waited 958 ms on lock 0x7b5400222ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:56.384222 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.384331 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2203385216}
I20260709 12:27:56.370540  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:56.424644  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54174.4us
I20260709 12:27:56.348803 10019 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:56.425298  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.426307  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.430904 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:56.382287 19984 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.431830 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.432647 10094 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:27:56.409291 20282 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.406621 20046 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.453649 10019 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 12:27:56.411397 20096 consensus_meta.cc:287] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
I20260709 12:27:56.443320  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.460827  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.431574 19756 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:56.457444 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.468972 10095 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:56.465698  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:56.440690  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 224889us
I20260709 12:27:56.467304 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.462685 20046 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.470781 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.527447 10677 raft_consensus.cc:2393] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:56.528416 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.528683 20046 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.529044 10677 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 23.
I20260709 12:27:56.529954 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.466490 19772 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:56.530416  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.429021 20277 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.531466 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.532220 10677 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:56.532382 20279 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.341142 20288 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:56.536793 10097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.024s	sys 0.019s
W20260709 12:27:56.537114 10097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.024s	sys 0.019s
I20260709 12:27:56.476693 20225 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:27:56.447328 20045 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:27:56.449357 20255 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.478320  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.478971 20175 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:27:56.543159  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 64929.1us
I20260709 12:27:56.544485  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.455788 20282 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:56.481827 20211 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:56.493336 20190 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:56.501535 20299 log.cc:927] Time spent T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 12:27:56.445644 20210 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:56.506016 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.591912 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.506920 19654 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:56.596853 20154 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:56.597420 20154 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.607029 19976 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.607658 19976 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.611799 20288 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.542659 20175 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.614388 20175 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:56.580493 19650 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:56.620568 20255 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.623610 19976 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.590286 20307 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.626019 20307 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.628415 20307 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:56.507362 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.632234 10676 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:56.509831 20070 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.409444 19798 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.511060 20207 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.511379 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.512450 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.468019 20216 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:56.517920 20159 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.518623 19756 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.522014 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.684496 10023 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:27:56.522915 10022 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:56.685737 10020 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:27:56.691692  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:56.525185 10096 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:56.472393 19768 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.534377 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.704525 10677 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
W20260709 12:27:56.537952  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 323010us
I20260709 12:27:56.476555 19984 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.712656 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.680995 20304 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.595080 20286 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.099s
I20260709 12:27:56.558456  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.559047  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.727651  9941 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:27:56.687340 19756 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:56.468593 20096 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.588481 10674 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:56.555282 19772 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:27:56.733964 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:56.533424 20277 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.731578 20304 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.717770 20072 raft_consensus.cc:3055] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:56.736470 20072 raft_consensus.cc:740] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:56.736613 20304 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:56.736749 10676 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:27:56.712750 20200 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.591877  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.737412 20200 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.737993 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.595208  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.539897 20045 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.633313 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:56.630455 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 164248us
I20260709 12:27:56.712982 19976 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.636198 10097 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 33. Current term is 34. Ops: []
I20260709 12:27:56.767889 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:56.654500 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.770325 10097 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:56.667606 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.675709  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.640332 10668 raft_consensus.cc:2393] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 25: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:56.677309 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.652652  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.681386 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.771906 10673 raft_consensus.cc:1240] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:56.772171 10091 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 18.
I20260709 12:27:56.683908 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.554293  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.716866  9946 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:27:56.676393 20162 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.556249 10024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3118) took 2165 ms. Trace:
I20260709 12:27:56.724150  9948 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:56.777743 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.560262  9943 raft_consensus.cc:1240] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:27:56.688256 20216 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:56.738580 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.792322 10098 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:56.740995 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.792918 10671 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
W20260709 12:27:56.792678 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52097.5us
I20260709 12:27:56.640102 20070 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.631484  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.795120  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.751643 20045 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:56.747545 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.745946 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:56.799127 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51677.3us
I20260709 12:27:56.792233 10024 rpcz_store.cc:276] 0709 12:27:54.390572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.392333 (+  1761us) service_pool.cc:224] Handling call
0709 12:27:54.392598 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:54.975293 (+582695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.545657 (+1570364us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54001379d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:56.556170 (+ 10513us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1335554432}
I20260709 12:27:56.757480 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.802130 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.755882 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.633612  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.752658 20159 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:56.770234 10095 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:56.804651 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.675282  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:56.805454 10094 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:27:56.805862  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 130706us
I20260709 12:27:56.806262  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.807911  9948 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:56.808498  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:56.774933  9988 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.025s	sys 0.057s
I20260709 12:27:56.809399 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:56.809665  9988 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.025s	sys 0.057s
I20260709 12:27:56.761890 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.777096 10022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:27:56.743409 20308 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:27:56.795753  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 164359us
I20260709 12:27:56.817258 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.817723  9914 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.747406 10025 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:56.798018 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:56.803557  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 170013us
I20260709 12:27:56.774510 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:56.798895  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.767479 20292 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.818864 10023 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 23: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:56.821614 10092 raft_consensus.cc:2393] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:56.821128 20142 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:56.818969 10091 raft_consensus.cc:2368] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28.
I20260709 12:27:56.818955 10022 raft_consensus.cc:606] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:56.826617 10022 raft_consensus.cc:614] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:56.820092 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.778272 19798 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.829875 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.819947 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:56.832374  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.827875  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.833994 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:56.834509 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 88769.1us
I20260709 12:27:56.835280 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:56.835527  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:56.824648 19768 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:56.807337 20207 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:56.846909 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.777614 20090 consensus_queue.cc:1048] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.045s
I20260709 12:27:56.843914  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:56.845086  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:56.821241 20052 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:56.842336  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 43558.8us
I20260709 12:27:56.847069  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:56.808403 20255 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:56.815444 20225 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:56.737437 20072 consensus_queue.cc:260] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.833544 19975 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:56.892294 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3121) took 2420 ms. Trace:
W20260709 12:27:56.892748 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57554us
I20260709 12:27:56.900323  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:56.892964  9943 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:56.804225 20235 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:27:56.850796 20114 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:56.899701 20052 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.895692 20225 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.935973 20225 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:56.936432  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:56.853626 19981 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:56.937048 20096 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.850955 20081 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:56.937615  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.937861 20081 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:56.845934 20292 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.938934 20081 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.795248 20162 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.941102 20292 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.941956 19984 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:56.942353 20162 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.851341 20207 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.825273 20275 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:56.943233 20207 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:56.826054 20308 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.943619 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.943650 20275 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.944787 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.945390 10021 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:56.945509 20308 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.825438 20277 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:56.946026 20275 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.937094 20052 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.953997 20225 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.954708 19654 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:56.851539 20140 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:56.955567 19654 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:56.955953 19654 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:56.896605 20271 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 33.20, Next index: 21, Last known committed idx: 18, Time since last communication: 0.065s
I20260709 12:27:56.956722 20271 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:27:56.956877 19654 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.957433 20271 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 33 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:56.853396 20190 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:56.958106 19654 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:56.958289 20190 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:56.959081 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.959297 20190 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:56.901361 10018 rpcz_store.cc:276] 0709 12:27:54.471461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.471584 (+   123us) service_pool.cc:224] Handling call
0709 12:27:54.471872 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:54.472316 (+   444us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.472358 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:54.479514 (+  7156us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.383678 (+1904164us) spinlock_profiling.cc:243] Waited 13.3 ms on lock 0x7b18002fbe40. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:27:56.676954 (+293276us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:56.851456 (+174502us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.851712 (+   256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.851819 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:56.851861 (+    42us) raft_consensus.cc:1685] finished
0709 12:27:56.892093 (+ 40232us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:56.892238 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3736,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":13308,"spinlock_wait_cycles":30661888,"thread_start_us":447837,"threads_started":2,"wal-append.queue_time_us":2227253}
I20260709 12:27:56.942000  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.961380  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:56.843163 20159 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:56.932987  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.935439 20114 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.993634 20114 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:56.939335 20070 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.942947 20282 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:56.900705 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.999189 10025 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:56.943158 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.827178 20154 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.000370 20282 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:56.948376 20309 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:56.949015 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:56.949208  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 105393us
I20260709 12:27:57.001562  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:56.949580 20016 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:57.002305 20016 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:57.004623 20016 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:57.007823 20200 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:56.859653 19768 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:57.012483 19768 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:56.949998 20312 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
I20260709 12:27:56.834105 19841 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:56.846081 20091 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:56.759040 19976 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:56.894012  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47004.8us
I20260709 12:27:56.959939 10024 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:27:56.927460  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 82478.5us
I20260709 12:27:56.962810  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:56.964264  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 64024.4us
I20260709 12:27:56.848261 19837 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:56.967243 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:56.968308  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:57.032914  9990 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:56.892716 20255 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:27:56.948172 20302 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 12:27:57.000838 10019 raft_consensus.cc:2410] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 29 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
I20260709 12:27:57.050508 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:57.017168  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:56.930007 19772 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:57.026175 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:57.055423 10095 raft_consensus.cc:2368] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 33. Current term is 34.
I20260709 12:27:57.030354  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:57.033200  9947 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:57.055915  9945 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:27:56.902387 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:57.010193 20207 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:57.018334 20311 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:27:57.036470 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:57.057228 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 154924us
I20260709 12:27:57.076298 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:57.074265  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43985.9us
I20260709 12:27:57.081496  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:57.084508 10024 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
W20260709 12:27:57.123773 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50029.5us
I20260709 12:27:57.021687 20091 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:56.930434 19975 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.128993 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:57.129659 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:57.051566 20255 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:57.136565 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:57.140122 10098 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:27:57.058825 20213 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:57.145030 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:57.157054 10095 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 30 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:27:57.150767  9948 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:56.901523 20216 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.123603 20189 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:57.131419 20091 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:57.127976 20155 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:57.031612 19841 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.159812 10016 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 19.
I20260709 12:27:57.127820 20210 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:27:57.158030 20175 consensus_meta.cc:287] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.543s	user 0.010s	sys 0.000s
W20260709 12:27:57.179392 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.184651 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:57.213625 10668 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.206110 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:57.215566 10642 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:57.129914  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:57.194249 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:57.196867 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:57.219551 10669 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:57.202338 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:57.224596 10674 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:57.132629 20307 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:27:57.127602 19650 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.052s
I20260709 12:27:57.178861 20155 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.152665 19984 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.059s
I20260709 12:27:57.151744 20016 consensus_queue.cc:1059] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.059s
I20260709 12:27:57.039280 20271 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.20, Last appended by leader: 20, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:57.233618 19823 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:57.248483 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:57.250859 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.258293 20168 raft_consensus.cc:3055] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:27:57.258796 20168 raft_consensus.cc:740] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:57.259670 20168 consensus_queue.cc:260] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:57.261144 20168 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.229555 20257 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:27:57.266824 20257 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:57.267208 20257 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.272078 20225 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:57.279306 19976 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.291893 19841 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.227831 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:57.317482  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 187699us
I20260709 12:27:57.321224  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:57.306535 10668 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:57.209046  9947 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:27:57.330958 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:57.385703 19975 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.241120 20155 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:57.405974 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 178362us
I20260709 12:27:57.414384 10098 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
W20260709 12:27:57.415997 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 85169.8us
W20260709 12:27:57.419054  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 97924.3us
I20260709 12:27:57.420486  9912 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:57.421873  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:57.417300 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:57.431504 10099 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4700) took 3035 ms. Trace:
I20260709 12:27:57.431926 10099 rpcz_store.cc:276] 0709 12:27:54.370983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.371434 (+   451us) service_pool.cc:224] Handling call
0709 12:27:54.371775 (+   341us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:27:54.390788 (+ 19013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:54.390821 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:27:56.148042 (+1757221us) spinlock_profiling.cc:243] Waited 9.36 ms on lock 0x7b2c00115cd8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:27:56.148667 (+   625us) log.cc:844] Serialized 52 byte log entry
0709 12:27:56.540369 (+391702us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:27:56.596440 (+ 56071us) log.cc:844] Serialized 14 byte log entry
0709 12:27:56.596671 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.596759 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.405891 (+809132us) raft_consensus.cc:1685] finished
0709 12:27:57.405972 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.406119 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65437,"prepare.queue_time_us":76087,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":24685,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":24659,"spinlock_wait_cycles":21536640,"thread_start_us":372830,"threads_started":1,"wal-append.queue_time_us":564681}
I20260709 12:27:57.436896  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:57.441242 10674 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:27:57.226104 20175 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.469077  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:57.585098 10021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.041s	sys 0.033s
W20260709 12:27:57.585681 10021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.041s	sys 0.033s
W20260709 12:27:57.594396  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 125392us
W20260709 12:27:57.594774 10017 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.017s	sys 0.014s
W20260709 12:27:57.596827 10017 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.017s	sys 0.015s
I20260709 12:27:57.430625 20018 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:57.602254 19768 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:57.603037 19768 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:27:57.609869 19756 consensus_meta.cc:287] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.862s	user 0.007s	sys 0.002s
I20260709 12:27:57.612115 20307 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:27:57.427312 20045 consensus_meta.cc:287] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.000s
I20260709 12:27:57.473758 20210 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:27:57.620198  9942 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:57.473590 20159 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:57.474134 20189 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:57.623420 20210 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:57.628177 20210 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:27:57.464063 20081 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.023s	sys 0.049s
I20260709 12:27:57.430773 20275 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:57.469081 20075 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:27:57.626282 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:57.637773 20081 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.023s	sys 0.049s
I20260709 12:27:57.430950 20286 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:57.467288 19837 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:57.652980  9946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.018s	sys 0.014s
W20260709 12:27:57.655084  9946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.019s	sys 0.014s
I20260709 12:27:57.433917 20225 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:57.635327 20159 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:57.610395 19756 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.645493 20075 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:27:57.678586 10097 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.101s	sys 0.137s
W20260709 12:27:57.678984 10097 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.623s	user 0.101s	sys 0.137s
I20260709 12:27:57.665505 20159 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:57.626202 20257 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.654883 19837 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:57.663754 20207 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.702637  9949 raft_consensus.cc:2368] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28.
I20260709 12:27:57.676153 20075 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:27:57.714367 10018 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.008s	sys 0.008s
W20260709 12:27:57.714766 10018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.008s	sys 0.008s
I20260709 12:27:57.654330 20045 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:57.745911 10063 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:57.747797 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 311994us
W20260709 12:27:57.745314 20302 log.cc:927] Time spent T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:27:57.747793 20114 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.000s
I20260709 12:27:57.722332 19837 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:57.665505 20286 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:57.770702 20175 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.778223 19837 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.779979 19756 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.785157 20210 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.761948  9944 raft_consensus.cc:2368] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 33. Current term is 34.
I20260709 12:27:57.787621 19837 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:57.759400  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 162977us
I20260709 12:27:57.796559 20286 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:57.797798 20255 consensus_meta.cc:287] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.640s	user 0.005s	sys 0.003s
W20260709 12:27:57.799913 20168 consensus_meta.cc:287] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.538s	user 0.010s	sys 0.001s
W20260709 12:27:57.802034 20091 consensus_meta.cc:287] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.000s
I20260709 12:27:57.764560 20114 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.817772 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:57.802058 20045 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:57.819932 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:57.823562 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:57.825615 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:57.852443 10673 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:57.832530 20257 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:57.832571 19756 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:57.853751  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:57.818473 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:57.855257 10668 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:57.847294 20075 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.862851 20075 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.849488 20271 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:57.864629  9948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 2305) took 1008 ms. Trace:
I20260709 12:27:57.865056  9948 rpcz_store.cc:276] 0709 12:27:56.856171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.856448 (+   277us) service_pool.cc:224] Handling call
0709 12:27:56.929291 (+ 72843us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.157719 (+228428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:57.864343 (+706624us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54000b7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:57.864562 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":508654336}
I20260709 12:27:57.850183 20114 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.850564 20091 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.866780 10675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42028 (request call id 2124) took 1904 ms. Trace:
I20260709 12:27:57.867175 10675 rpcz_store.cc:276] 0709 12:27:55.962248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:55.962385 (+   137us) service_pool.cc:224] Handling call
0709 12:27:56.488389 (+526004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:56.488818 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.488844 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:56.489268 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 12:27:56.697398 (+208130us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:27:56.697438 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:56.697499 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:57.010002 (+312503us) raft_consensus.cc:1685] finished
0709 12:27:57.010025 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:57.866497 (+856472us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b54007b0cd0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:57.866716 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13212,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":100,"spinlock_wait_cycles":1133265280,"thread_start_us":207784,"threads_started":1,"wal-append.queue_time_us":438719}
I20260709 12:27:57.868476 20091 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.832229 20210 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.829753 20159 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.848333 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:57.871970 10668 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:57.900853 20155 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.902171 20159 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.903898 20155 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:57.904109  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 50446us
I20260709 12:27:57.906853 20210 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:57.877655 19756 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.917026 19756 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:57.901335 20213 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:57.930745 20213 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:57.935236  9912 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:57.935025 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 185991us
I20260709 12:27:57.940250 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:27:57.936214 20320 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:27:57.841362 20255 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:57.948307 10095 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:57.958705 10097 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 19. (term mismatch)
I20260709 12:27:57.959138 10097 pending_rounds.cc:85] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 18
I20260709 12:27:57.959375 10097 pending_rounds.cc:107] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 29.19
I20260709 12:27:57.959609 10097 raft_consensus.cc:2889] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:57.960237 20255 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:57.961161 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:57.942387 20271 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:57.965946 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:57.966344 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:57.967805 10097 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:57.968120 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:57.969542 10090 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:57.971094 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:57.971477 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:57.971792 10090 raft_consensus.cc:2393] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 34: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:57.972023 10097 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:57.975207  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:57.977243 20257 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:57.979596  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:57.985276 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:57.986011 10097 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:57.986272 19723 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:27:57.988111 10642 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:57.988796  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 3704) took 1345 ms. Trace:
I20260709 12:27:57.989169  9949 rpcz_store.cc:276] 0709 12:27:56.643968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.644159 (+   191us) service_pool.cc:224] Handling call
0709 12:27:57.988616 (+1344457us) spinlock_profiling.cc:243] Waited 931 ms on lock 0x7b54000b5c54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:57.988743 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2141876480}
I20260709 12:27:57.968119 20271 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:57.990556  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:57.984331 19837 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:57.991532 20257 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.002s
I20260709 12:27:57.993166 19837 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:57.993925 20257 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:57.994272 20257 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:57.994949 20276 raft_consensus.cc:2804] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:27:57.995487 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 1933) took 1057 ms. Trace:
I20260709 12:27:57.995548 20276 raft_consensus.cc:697] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:57.995241 20257 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.996606 20257 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:57.996630 20276 consensus_queue.cc:237] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:57.996027 10016 rpcz_store.cc:276] 0709 12:27:56.937809 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.946791 (+  8982us) service_pool.cc:224] Handling call
0709 12:27:57.995308 (+1048517us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b540011b054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:57.995429 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":291120000}
I20260709 12:27:58.001341 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:58.002619  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41138 (request call id 3712) took 1208 ms. Trace:
I20260709 12:27:58.003996  9944 rpcz_store.cc:276] 0709 12:27:56.794481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.794867 (+   386us) service_pool.cc:224] Handling call
0709 12:27:58.002377 (+1207510us) spinlock_profiling.cc:243] Waited 965 ms on lock 0x7b5400189cd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:58.002533 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2219327872}
I20260709 12:27:58.004179 10065 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:58.005286 20016 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:58.005789 20016 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:58.006736 19756 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.007367 19756 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.012604  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:58.013840  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:58.015810  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:58.016392  9941 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:58.024354  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:58.024753 10640 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:58.025489  9946 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:58.027115  9944 raft_consensus.cc:2393] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:58.028297 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:58.029949 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:58.035907 20018 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.036566 20018 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.039184  9948 raft_consensus.cc:1240] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:58.043241 19756 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.026711 20347 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:58.044466 20347 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.022341 20255 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.046993 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.047263 20255 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.022459 20096 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.047688 10677 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:58.048348 20096 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.049503 20255 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:58.050011 20288 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:58.026733 19823 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:58.050539 20096 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:58.039578 20238 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:58.052958  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:58.053663  9941 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:58.006704 20016 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.058974  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.063611  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.067322 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1923) took 1426 ms. Trace:
I20260709 12:27:58.067355  9948 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:27:58.068951  9944 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:58.033655 20287 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:27:58.068116 10092 rpcz_store.cc:276] 0709 12:27:56.634566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.667320 (+ 32754us) service_pool.cc:224] Handling call
0709 12:27:58.060399 (+1393079us) spinlock_profiling.cc:243] Waited 45.1 ms on lock 0x7b540015bdd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:58.060527 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":103824256}
I20260709 12:27:58.073611 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:58.074431 20118 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:27:58.059538 20190 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.918s	user 0.007s	sys 0.015s
I20260709 12:27:58.052675 20347 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:58.079658 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:58.083673 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 53803us
I20260709 12:27:58.058267 20213 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.087091 20018 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.071877 19723 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:58.087847 19723 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.076519 20271 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.088260 19723 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:58.088260 20213 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.063635 20346 consensus_queue.cc:1059] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.029s
I20260709 12:27:58.092010 20346 raft_consensus.cc:3055] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:58.092479 20346 raft_consensus.cc:740] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:27:58.095976  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 1.09445e+06us
I20260709 12:27:58.096846  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:58.100909  9987 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:58.101258 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.101979 10092 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
W20260709 12:27:58.077653 20190 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.007s	sys 0.015s
I20260709 12:27:58.075024 20118 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:58.086236 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:58.120009  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 1.08708e+06us
I20260709 12:27:58.120417 10640 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:58.093654 20346 consensus_queue.cc:260] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.123843 19756 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.125967 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:59744 (request call id 51) took 1345 ms. Trace:
I20260709 12:27:58.119119 20118 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:58.122411 20235 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:58.136210 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1943) took 1195 ms. Trace:
I20260709 12:27:58.137001 19756 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.136657 10017 rpcz_store.cc:276] 0709 12:27:56.970348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:57.001572 (+ 31224us) service_pool.cc:224] Handling call
0709 12:27:57.001852 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.002041 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:27:57.002317 (+   276us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.135872 (+1133555us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:58.135922 (+    50us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:27:58.135964 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.136036 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.136150 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":538593,"prepare.queue_time_us":126267,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":6257,"thread_start_us":594588,"threads_started":1,"wal-append.queue_time_us":709645}
I20260709 12:27:58.139361 19981 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:58.139922 19981 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.143280 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1939) took 1203 ms. Trace:
I20260709 12:27:58.143687 10021 rpcz_store.cc:276] 0709 12:27:56.939841 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.946010 (+  6169us) service_pool.cc:224] Handling call
0709 12:27:56.946320 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:56.947943 (+  1623us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.947978 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:56.948541 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:27:57.655823 (+707282us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:57.675063 (+ 19240us) log.cc:844] Serialized 14 byte log entry
0709 12:27:57.851548 (+176485us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b4800002958. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0e47faa1 00007f4e0e476b37 00007f4e0e4769ca 00007f4e0e56e26f 00007f4e0e56df37 00007f4e0e58255c 00007f4e0e582508 00007f4e0e582488 00007f4e0e582443 00007f4e0e5816ec 00007f4e0d4aba96
0709 12:27:58.142806 (+291258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.142901 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:58.142945 (+    44us) raft_consensus.cc:1685] finished
0709 12:27:58.143042 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.143205 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":467450,"prepare.queue_time_us":19021,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2020,"raft.queue_time_us":1442,"raft.run_cpu_time_us":8294,"raft.run_wall_time_us":903626,"spinlock_wait_cycles":57401216,"thread_start_us":644782,"threads_started":1,"wal-append.queue_time_us":798415}
I20260709 12:27:58.146237 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1941) took 1205 ms. Trace:
I20260709 12:27:58.100203 20309 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:58.146844 20309 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.147297 20309 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.147804 10025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 1942) took 1207 ms. Trace:
I20260709 12:27:58.146756 10018 rpcz_store.cc:276] 0709 12:27:56.940377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.961322 (+ 20945us) service_pool.cc:224] Handling call
0709 12:27:56.961613 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:27:56.961941 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.961969 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:27:56.962373 (+   404us) log.cc:844] Serialized 26 byte log entry
0709 12:27:57.715104 (+752731us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:57.715393 (+   289us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.145859 (+430466us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.145953 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:27:58.146004 (+    51us) raft_consensus.cc:1685] finished
0709 12:27:58.146021 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.146169 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1520384,"prepare.queue_time_us":146,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":432933,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":34713,"thread_start_us":752438,"threads_started":1,"wal-append.queue_time_us":837497}
I20260709 12:27:58.149166 19618 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:27:58.151667  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 50829.5us
I20260709 12:27:58.151887 20069 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:27:58.153092 10640 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:58.154749 20080 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:58.155277 20080 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.156332 19651 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:58.156754 19651 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.157116 19651 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:58.157934 20080 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.159853 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:58.161245 20255 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:58.161634 20255 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.148201 10025 rpcz_store.cc:276] 0709 12:27:56.940646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.999893 (+ 59247us) service_pool.cc:224] Handling call
0709 12:27:57.000193 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:57.233959 (+233766us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:27:57.299583 (+ 65624us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.145891 (+846308us) log.cc:844] Serialized 14 byte log entry
0709 12:27:58.146064 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:58.146109 (+    45us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:27:58.146154 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.147572 (+  1418us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b540038ccd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:58.147619 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.147746 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":439293,"prepare.queue_time_us":68534,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":75,"raft.queue_time_us":442989,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":537082112,"thread_start_us":406337,"threads_started":1,"wal-append.queue_time_us":460149}
I20260709 12:27:58.161985 20255 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.155571 20238 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:27:58.134845  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:58.125526 20287 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:27:58.158514 20080 consensus_queue.cc:1059] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.011s
I20260709 12:27:58.126387 10022 rpcz_store.cc:276] 0709 12:27:56.776759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.776929 (+   170us) service_pool.cc:224] Handling call
0709 12:27:58.121788 (+1344859us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b5400137ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09550853 00007f4e0e51e60a 00007f4e07aa47d4 00007f4e07aa4762 00007f4e07aa4691 00007f4e07aa460c 00007f4e07aa38c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:58.121917 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15252864}
I20260709 12:27:58.154274 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:58.171501 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.182021 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.172135 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:58.182192 10018 raft_consensus.cc:3060] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:27:58.173242 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.176337 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.184118 10016 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Advancing to term 34
I20260709 12:27:58.174829 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:58.179669 10640 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.534s (SENT) [suppressed 9 similar messages]
I20260709 12:27:58.172313 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.173164 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.183393 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.185483 10021 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:27:58.187454 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.188127 10021 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 19: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:58.178170 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.188777 10023 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:27:58.171161 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.170006 10065 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:58.173393  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52340.3us
I20260709 12:27:58.195325  9990 leader_election.cc:400] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28.
I20260709 12:27:58.195616  9990 leader_election.cc:403] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:58.199972 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.208370 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.209110 10024 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:58.210000 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.211351 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:58.215377 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45457.6us
I20260709 12:27:58.217702 10024 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:58.238361 10018 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:58.241684 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.168612 20346 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:58.167250 20282 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:58.242913 10021 raft_consensus.cc:1240] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:27:58.178035 20292 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:58.168741 19618 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:58.167776 19981 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.173099 20235 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:58.164305 19756 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.246454  9990 leader_election.cc:400] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 33. Current term is 34.
I20260709 12:27:58.168853 19975 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:58.163631 20238 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.181726 20354 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:27:58.248822 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.244836 10024 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:58.280784 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.245393 19976 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28.
I20260709 12:27:58.162547 19723 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.247689 20287 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:58.256079 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.257287 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37540 (request call id 4741) took 1440 ms. Trace:
I20260709 12:27:58.214461 19768 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.271512  9990 leader_election.cc:403] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:27:58.224295 20347 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.226397 20070 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:58.298796 10091 rpcz_store.cc:276] 0709 12:27:56.816901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.842701 (+ 25800us) service_pool.cc:224] Handling call
0709 12:27:56.848962 (+  6261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:27:56.992644 (+143682us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:27:56.992699 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:27:56.992761 (+    62us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:27:56.992808 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:27:58.256979 (+1264171us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54002fce54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:27:58.257047 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:27:58.257214 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":329828608}
I20260709 12:27:58.274580 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.291977 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:58.294246 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.299959  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:58.299197  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 52942.1us
I20260709 12:27:58.322031 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.324020  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.325043 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:58.323503 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 143859us
I20260709 12:27:58.324770  9944 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:58.279528 20238 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:58.329802 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.325299  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.331847 10018 raft_consensus.cc:2393] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:58.327759 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:58.331849 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.334535 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.335558 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.336468 10024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:27:58.336921 10024 raft_consensus.cc:606] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Received request to transfer leadership to TS 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:58.327008  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:58.339542 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.302922 20070 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.338817 19841 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:27:58.344260 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.269408 20292 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.341244 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:27:58.347877 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:58.342746 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.294903 19975 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.271118 19618 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.357200  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.371567  9943 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:58.372498  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.373557  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.325340 20338 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:27:58.350116  9942 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38784 (request call id 2300) took 1792 ms (client timeout 1534 ms). Trace:
I20260709 12:27:58.374800 20338 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:27:58.374969 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.375259 20338 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:58.374943  9942 rpcz_store.cc:269] 0709 12:27:56.558098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:56.558221 (+   123us) service_pool.cc:224] Handling call
0709 12:27:58.349935 (+1791714us) spinlock_profiling.cc:243] Waited 891 ms on lock 0x7b540003e654. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:27:58.350054 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2049500416}
I20260709 12:27:58.350919 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:58.351438 20114 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:58.378208 20114 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.378583 20114 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:58.321851 19768 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.352267 20304 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:58.327877 20307 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.382911 20304 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:58.383498 20304 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:58.383395 20307 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.384433 19768 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.384857  9941 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:58.385442 20307 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:58.353897  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.354682  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.387486 20212 raft_consensus.cc:3055] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:58.354933 19772 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.387889 20212 raft_consensus.cc:740] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:58.388202 19772 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.388722 20212 consensus_queue.cc:260] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.355643 20210 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.391004 20307 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:58.390844 20210 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:58.391479 20351 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:27:58.393038 20210 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.347575 20345 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:58.361003  9940 raft_consensus.cc:1240] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:58.321620 20347 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:58.395362 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44520.8us
I20260709 12:27:58.396396 20347 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:58.326027 20016 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:58.397119  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.397818 20340 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.397184  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.399928 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:58.400667 20016 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:58.390353 19772 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.408260 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.408371 20338 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.409574 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.410169 10017 raft_consensus.cc:3060] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:27:58.412647 20304 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.414624 10093 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:27:58.414785 20304 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:58.416442 10097 raft_consensus.cc:1240] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
W20260709 12:27:58.417155 20355 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:27:58.418200 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.418890 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.419646 10097 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:27:58.420529 10024 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 16. (term mismatch)
I20260709 12:27:58.420894 10024 pending_rounds.cc:85] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 15
I20260709 12:27:58.421131 10024 pending_rounds.cc:107] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:27:58.421478 10024 raft_consensus.cc:2889] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:27:58.422210 20338 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.341739 19823 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:58.347079 20288 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.323478 20287 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.346552 20070 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:58.441186 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:58.364185  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:58.364562  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.365006 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.366636 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.366918 20292 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.367156 20072 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.449173  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:58.449751 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.390237 20212 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.368670 19651 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.368858  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.343573 20296 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.371047 19618 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.293243 20235 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.479843 19975 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.481305 20235 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.343116 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.482547 10092 raft_consensus.cc:2393] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 20: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:58.483183 10025 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:27:58.427541 10091 raft_consensus.cc:1240] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:58.428457 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.429554 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.431274 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.432870  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:58.435904 10016 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 34.
I20260709 12:27:58.485749 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.487170 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.487897 10016 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:27:58.436468 10017 raft_consensus.cc:2468] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20.
I20260709 12:27:58.398401 20340 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.438341 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:58.439607 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.497591 10675 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:27:58.440730 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.448269 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.498642 10668 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:27:58.354734 19841 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:58.449034 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:58.368331 20309 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.455631 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.357061 19723 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.390081 20114 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.458667 20271 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:58.463661 10062 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:27:58.520896 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:58.521610 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 58039.2us
I20260709 12:27:58.469161 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:58.349959 20276 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.110s
W20260709 12:27:58.502472 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 138566us
I20260709 12:27:58.504359 10672 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:58.449219 20288 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.442453 20190 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.043s
I20260709 12:27:58.445499 19772 consensus_queue.cc:1059] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 29.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.047s
I20260709 12:27:58.446074 19823 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.491705 19981 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election lost for term 33. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 33. Current term is 34.
I20260709 12:27:58.504106 19984 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:58.484470 20296 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.495952 20213 consensus_queue.cc:1059] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:27:58.543046 19841 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.530166 20341 raft_consensus.cc:2804] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:27:58.547899 20341 raft_consensus.cc:697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:58.548921 20341 consensus_queue.cc:237] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.549602 20340 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:58.520329 20238 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.537263 20257 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:58.559114 20069 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:58.562319 20287 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.563941 20288 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.565550 20309 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.569216 20296 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:27:58.574144 20369 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:27:58.575941 20190 raft_consensus.cc:3055] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:27:58.576417 20190 raft_consensus.cc:740] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 23 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:58.577308 20190 consensus_queue.cc:260] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.578547 20190 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:58.563966 20255 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.563952 20238 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.450886  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
W20260709 12:27:58.624351  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 173374us
I20260709 12:27:58.629067  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:58.435822  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:27:58.631801  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 195897us
I20260709 12:27:58.642532  9987 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:58.493780  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 163550us
I20260709 12:27:58.647290  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:58.342031 10020 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:58.653393 10021 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:58.655175 10022 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:58.657594 10023 raft_consensus.cc:2410] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for term 29 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:27:58.659907  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.665689  9944 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:27:58.672086  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.673863  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.674016 10022 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
W20260709 12:27:58.530586 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 61540.3us
I20260709 12:27:58.692457  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.545208 19984 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.697325 19651 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.697630 20114 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.625948  9841 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 16 to 20. New cstate: current_term: 20 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:58.700109 10677 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:58.700208 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 169528us
I20260709 12:27:58.700444 19823 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.701104 19723 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:58.702787 20052 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:58.703692 19772 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:58.703624 20069 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:58.708871 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.709004 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.711105 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.712217 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.710235 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:27:58.699190 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 166207us
W20260709 12:27:58.727421  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94578.7us
I20260709 12:27:58.732862  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.741618 20255 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.743064 19756 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:27:58.743613 19756 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.744079 19756 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:58.749018 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:58.751883  9841 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a reported cstate change: term changed from 19 to 23, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 23 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:58.755865 10019 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:58.757071 20118 consensus_meta.cc:287] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.629s	user 0.011s	sys 0.000s
I20260709 12:27:58.758983 10643 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:58.772730 20118 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.773466 19823 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:58.774083 20291 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:58.774708 20291 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.775122 20291 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.775728 20338 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:58.778354 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:58.779357 10098 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:58.780143 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:58.780828 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:58.737931 19772 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.787173 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.787043 20069 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.782522 19772 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.735216 19723 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:58.789101 19723 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:58.790558 20069 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:58.791996 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 51087.6us
I20260709 12:27:58.788307 19984 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:58.763096 10025 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 34.
I20260709 12:27:58.795081 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.781980 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:58.786927 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.799713  9944 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:27:58.802023  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.803756 10024 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:58.805653 20052 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:58.806480 20052 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.754583  9841 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 20 to 22. New cstate: current_term: 22 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:58.808332 20304 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:27:58.808758 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.809036 20304 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.811151 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 34 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:58.762099  9838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 217) took 81142 ms (client timeout 15000 ms). Trace:
I20260709 12:27:58.810712  9841 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a reported cstate change: term changed from 16 to 23, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 23 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:58.774024 19823 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:58.814211 20052 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:58.814508 19823 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:58.814772 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.815668 10091 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:27:58.799762 10674 raft_consensus.cc:3060] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:27:58.816941 10672 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:58.805109 19772 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:58.823032 19837 raft_consensus.cc:3055] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:58.823606 19837 raft_consensus.cc:740] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:58.824189 10096 raft_consensus.cc:2393] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 34: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:58.824528 19772 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.824882 19837 consensus_queue.cc:260] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:58.827155 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:58.831254  9943 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 34.
I20260709 12:27:58.834408 20343 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:27:58.843321  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
W20260709 12:27:58.847143 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 98539.1us
I20260709 12:27:58.852301 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:58.830711  9841 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 21 to 25, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 25 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:58.862097 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:58.858543 10677 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:27:58.859604 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:58.873140 20304 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.873413 19976 consensus_queue.cc:1059] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:27:58.874752 19976 raft_consensus.cc:3055] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:27:58.875237 19976 raft_consensus.cc:740] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:58.876636 19976 consensus_queue.cc:260] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.883224 19837 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:58.878297 19976 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.865872  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:58.891716 10674 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:27:58.870674 20118 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:58.828298 19984 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.056s
I20260709 12:27:58.897169 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.896781 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.900841 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:58.904433 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:58.904592 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:27:58.908881  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 278319us
I20260709 12:27:58.908288 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:58.915124 10019 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:27:58.916039 10025 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
W20260709 12:27:58.919100  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 53481.3us
I20260709 12:27:58.907317  9841 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:27:58.921244 10017 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:58.935676 20271 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:58.936406 10097 raft_consensus.cc:2393] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:58.936869 10022 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:58.923306 20052 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:27:58.941999 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 82521.5us
I20260709 12:27:58.946525  9991 leader_election.cc:400] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 33. Current term is 34.
W20260709 12:27:58.948360  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 116355us
I20260709 12:27:58.945758 10098 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:27:58.953922 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:58.954865 10094 raft_consensus.cc:2368] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:27:58.954777  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:58.943207 20072 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:58.941818 20052 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:58.966753 19768 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:58.967072  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.990396  9943 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
W20260709 12:27:58.976747 20070 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.004s
I20260709 12:27:58.979318 19723 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:58.983562  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:58.984872 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:58.987689 10022 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
W20260709 12:27:58.811874  9838 rpcz_store.cc:269] 0709 12:26:37.619126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:37.696135 (+ 77009us) service_pool.cc:224] Handling call
0709 12:27:58.761796 (+81065661us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:54.569646 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:54.680792 (+111146us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:54.680850 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:27:54.680874 (+    24us) tablet.cc:662] Decoding operations
0709 12:27:54.681317 (+   443us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:54.681425 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:54.681452 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:27:54.681885 (+   433us) tablet.cc:701] Row locks acquired
0709 12:27:54.681933 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:27:54.682108 (+   175us) write_op.cc:270] Start()
0709 12:27:54.682252 (+   144us) write_op.cc:276] Timestamp: P: 1783600074682036 usec, L: 0
0709 12:27:54.682277 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:27:54.683017 (+   740us) log.cc:844] Serialized 536 byte log entry
0709 12:27:55.143740 (+460723us) op_driver.cc:464] REPLICATION: finished
0709 12:27:55.273286 (+129546us) write_op.cc:301] APPLY: starting
0709 12:27:55.312291 (+ 39005us) tablet.cc:1367] starting BulkCheckPresence
0709 12:27:55.547723 (+235432us) tablet.cc:1370] finished BulkCheckPresence
0709 12:27:55.549601 (+  1878us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:27:55.627308 (+ 77707us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:27:55.648328 (+ 21020us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:27:55.667923 (+ 19595us) write_op.cc:312] APPLY: finished
0709 12:27:55.745440 (+ 77517us) log.cc:844] Serialized 25 byte log entry
0709 12:27:55.760371 (+ 14931us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:55.760478 (+   107us) write_op.cc:454] Released schema lock
0709 12:27:55.760522 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:27:55.761024 (+   502us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:27:55.761054 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1582104,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":96887,"apply.run_cpu_time_us":14131,"apply.run_wall_time_us":511609,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":267406,"prepare.run_cpu_time_us":39733,"prepare.run_wall_time_us":401980,"raft.queue_time_us":157537,"raft.run_cpu_time_us":15411,"raft.run_wall_time_us":48957,"replication_time_us":461285,"thread_start_us":568547,"threads_started":4,"wal-append.queue_time_us":285832,"wal-append.run_cpu_time_us":16937,"wal-append.run_wall_time_us":1852507}]]}
I20260709 12:27:58.989553 19823 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.004842 19823 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:59.005765 10675 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 30. Current term is 31. Ops: [30.19-30.19]
I20260709 12:27:59.010035 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:59.011051 10094 raft_consensus.cc:2410] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 16.
I20260709 12:27:59.012945  9841 catalog_manager.cc:5697] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 15 to 19, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 19 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:27:58.987185 20052 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:59.028807 19723 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:59.029438  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:59.032743 19975 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.033488 19975 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.035396 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:59.035734 20345 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:27:59.036208 20345 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:59.037211 20345 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.038331 20345 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:58.997622  9991 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:27:59.050521  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57437us
I20260709 12:27:59.051882  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:59.058692  9946 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:59.081930 10017 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:59.079244  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.085948  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.093432 10097 raft_consensus.cc:2368] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
W20260709 12:27:59.098896  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 142347us
I20260709 12:27:59.043136 20070 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:59.107949 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 148898us
W20260709 12:27:59.108942  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 57235.7us
I20260709 12:27:59.109381  9940 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:59.110569  9948 raft_consensus.cc:2393] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 33: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:59.112954 10092 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 30. Current term is 31. Ops: []
I20260709 12:27:59.114568 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:59.116513  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.118214  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.120918 19756 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.126171 20070 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:59.127290 10099 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:59.130146 19837 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:27:59.131172 19837 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:59.131529 19837 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:59.132481 19837 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.133121  9991 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:59.133888 19837 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:59.134899 19976 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:59.136013 20155 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.136587 20155 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.145783  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:59.142763 19981 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:59.142520 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:59.151441 19723 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:27:59.167584 10676 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:59.160476 20052 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.173790 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:59.174616 10674 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.178128 19976 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:27:59.179289 19975 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:27:59.180326  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 45734.4us
I20260709 12:27:59.171361 20271 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:27:59.189783 10674 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:59.188887 19841 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:27:59.194710 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:59.200130 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:59.185168 20155 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:59.202651  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:59.192082 19756 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:59.211308 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:27:59.211735  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 93663.6us
I20260709 12:27:59.189206 20069 consensus_queue.cc:1059] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.018s
I20260709 12:27:59.220250 19975 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:59.227995 19964 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:27:59.230471  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:27:59.232698  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.233332  9942 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:59.238922  9940 raft_consensus.cc:1240] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 25. Current term is 26. Ops: []
I20260709 12:27:59.239051  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:59.241711  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:27:59.242215  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 43809.6us
I20260709 12:27:59.245955 10640 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.249033  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.245827 19723 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:27:59.250327 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.237785 20291 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.257272 20213 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:59.257752 20155 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:27:59.260558 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:27:59.264190 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.262454 20155 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.257232 19964 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:59.269609  9940 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:27:59.267838 20155 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:59.281728 20384 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:27:59.274996  9841 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a reported cstate change: term changed from 14 to 22, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 22 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:27:59.278456  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.263909 20345 consensus_queue.cc:1059] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:27:59.275789  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:59.289546 20052 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:59.290781  9841 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a reported cstate change: term changed from 22 to 27, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 27 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:27:59.301220 20235 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:59.304111 20070 raft_consensus.cc:3055] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:27:59.304718 20070 raft_consensus.cc:740] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:59.308917 19981 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:59.309993 20052 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:59.316455 10676 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:27:59.322878  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:27:59.324607 20383 log.cc:927] Time spent T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.097s	user 0.000s	sys 0.002s
I20260709 12:27:59.324743 19981 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:27:59.326828 19975 raft_consensus.cc:3055] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:27:59.311798 20052 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:59.327240 19975 raft_consensus.cc:740] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:27:59.306104 20070 consensus_queue.cc:260] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.332502 20304 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:59.333335  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:59.336651  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:59.337491  9948 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.336810  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:59.340471  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79764.1us
I20260709 12:27:59.342739 20235 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:27:59.343243 20235 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:59.331863 20345 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.350136 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:59.359805 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:59.363189 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:59.365408 10020 raft_consensus.cc:2368] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:27:59.329952 19975 consensus_queue.cc:260] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.362913 20235 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:59.368932 10017 raft_consensus.cc:2376] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 20. Re-sending same reply.
I20260709 12:27:59.375385  9991 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:59.379032  9948 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:59.367548 20070 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:27:59.383298 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:27:59.383222  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 48382.3us
I20260709 12:27:59.386654 10090 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:27:59.390304  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78927.9us
I20260709 12:27:59.391394 20213 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:27:59.393340  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.393522 20213 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:59.395292 19756 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:27:59.368628 19981 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:27:59.406049 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:59.406831 10020 raft_consensus.cc:2368] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:27:59.405184 19975 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:27:59.414958 10062 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:59.416889 20114 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:27:59.417001 10062 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.417543 20114 raft_consensus.cc:697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:59.418579 20114 consensus_queue.cc:237] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.419976 20069 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:59.420852 20291 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:59.421546 19984 raft_consensus.cc:2804] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:27:59.422086 19984 raft_consensus.cc:697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:59.423127 19984 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.338398 20052 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.431393 20052 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:27:59.435053 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:59.436914 10017 raft_consensus.cc:1275] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:27:59.438364  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:59.447804 20291 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:59.448696 20291 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.452934 20155 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:27:59.455044  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 88432.4us
I20260709 12:27:59.456465  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:59.457222  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.457866  9948 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:27:59.459201 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.460187 20090 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:59.460701 20090 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:59.461141 20090 raft_consensus.cc:3060] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:59.464946 20282 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:27:59.465415 20282 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:59.465799 20282 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:27:59.473513  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78917.9us
I20260709 12:27:59.474536  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.475723 20291 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:59.481654 20090 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.486390  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:59.366748 20345 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.495507 20345 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.496062  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:59.496675  9949 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.499010  9948 raft_consensus.cc:2468] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:27:59.501596 10643 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.502878 20155 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:59.507020 19981 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:27:59.508838 19964 raft_consensus.cc:2804] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:59.509438 19964 raft_consensus.cc:697] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:59.510010 19823 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.510527 19823 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.510440 19964 consensus_queue.cc:237] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 29.22, Last appended by leader: 22, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.512893 19823 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:59.515781  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.516055 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:27:59.507165  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50805.4us
I20260709 12:27:59.517805  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:59.524238  9948 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:59.526232  9949 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:27:59.537021 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.537751 10065 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.510604 20257 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:27:59.523159 20090 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:27:59.545878 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 128976us
I20260709 12:27:59.546906 20155 raft_consensus.cc:2804] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:59.546970 20069 raft_consensus.cc:2764] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election decision vote started in defunct term 33: won
I20260709 12:27:59.559473 20291 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:59.568261 20291 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.566005 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:59.568686 20155 raft_consensus.cc:697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:59.569473 10675 raft_consensus.cc:2393] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:27:59.568732 20291 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:59.569846 20155 consensus_queue.cc:237] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.570928 20271 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:59.572177 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:59.573645 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:59.574888 20118 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:27:59.575604 20118 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:27:59.564389 20257 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:27:59.566893  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 49209.1us
I20260709 12:27:59.558411 20213 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.579016 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:59.593276  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:59.592028  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 117621us
I20260709 12:27:59.597491 20291 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.590333 19823 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:59.599206 19823 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.601564 19823 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:59.602372 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:59.605113  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.605804  9942 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:27:59.607669 10643 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.603412 19976 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.585389 20213 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.614383 20069 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.618659 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:59.619560 10675 raft_consensus.cc:2410] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:27:59.609032 19976 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.611080 19823 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:27:59.625212 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:59.626156  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:27:59.631577 10025 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:27:59.632695  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 48014.1us
I20260709 12:27:59.635376  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.636147  9948 raft_consensus.cc:1240] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 32. Current term is 33. Ops: []
I20260709 12:27:59.636659  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.637351  9949 raft_consensus.cc:2393] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:27:59.637459 10676 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:59.638343 20390 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:59.638844 20390 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:59.639235 20390 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:59.640235 19976 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.643194 20257 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:59.643656 20090 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.643836  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:27:59.651006 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48773.1us
W20260709 12:27:59.651590  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 43799.1us
I20260709 12:27:59.652316 20235 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.653091 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:59.653311 20213 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.649204 20282 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.644217 20090 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.664826  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:59.665657  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.670205 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:59.670660 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:59.671454 10020 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:27:59.671629 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:59.672267 10099 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:27:59.672863 20090 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:27:59.672572 20235 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.674710  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:27:59.674865 20291 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:59.675688 19975 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:27:59.676534  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:59.678123  9942 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:27:59.676970 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.676987  9944 raft_consensus.cc:1240] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:27:59.678514  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:27:59.629441 19823 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:59.679915  9990 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:27:59.682009 10025 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:27:59.684120 20235 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:27:59.690709 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:27:59.665680 20282 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.692269 10090 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:27:59.696115  9948 raft_consensus.cc:1240] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 28. Current term is 29. Ops: []
I20260709 12:27:59.696494 10640 leader_election.cc:400] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:27:59.696661  9912 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:59.697582 10640 leader_election.cc:403] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:27:59.689589 19823 raft_consensus.cc:3060] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:27:59.690316 20052 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:59.731215 10640 leader_election.cc:400] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:27:59.731913 10022 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:27:59.735427 10640 leader_election.cc:403] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:27:59.730741 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:59.737031 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:27:59.730794 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:27:59.740450 20072 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:27:59.740603 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:59.741427 20271 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.741838 19723 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.736984 19756 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.741986 20271 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.742398 19723 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.742575 19756 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.744668 19723 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.744987 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:27:59.744920 20271 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.747219 10674 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:27:59.747730 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:59.748123 10676 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:27:59.748440 10677 raft_consensus.cc:2393] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 30: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:27:59.704675 20282 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:27:59.749348 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:59.750761 10022 raft_consensus.cc:1240] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: []
I20260709 12:27:59.754191 19837 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:27:59.754464 19772 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:27:59.756958 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:59.703784 20213 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:59.758323 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:59.758656 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:27:59.759097 10676 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:27:59.759308 10677 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:59.762122 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:59.762873 10099 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:27:59.767526 20390 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.705510 20052 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.744632 19756 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.758386 20213 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:59.738655 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:59.770928 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:27:59.771675  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.703536 10025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d8453bea911c4756add721edebd5d124"
dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
mode: GRACEFUL
new_leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
 from {username='slave'} at 127.0.0.1:59744
I20260709 12:27:59.770507 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:59.768710 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:59.787992 10025 raft_consensus.cc:606] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Received request to transfer leadership to TS 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:27:59.788367 10025 raft_consensus.cc:614] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:27:59.790900 19756 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:27:59.791334 19756 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.782871 20070 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:27:59.791721 19756 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:59.791922 20070 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:27:59.788570 10090 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:27:59.772049  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.701025 20090 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:27:59.769980  9942 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:27:59.727811 19976 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:27:59.787895 10020 raft_consensus.cc:2410] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 27 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:27:59.803956  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 108251us
W20260709 12:27:59.805404 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66862.3us
I20260709 12:27:59.806558 20271 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:27:59.806876 20072 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:27:59.808238 20090 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:27:59.811342  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.811980 20390 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:27:59.812438 20210 consensus_queue.cc:1059] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:27:59.803385 20070 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.20, Last appended by leader: 20, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.812947 19837 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:59.813908  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.813076  9912 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.785422 20052 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:27:59.825631 20210 raft_consensus.cc:3055] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:27:59.773110 19772 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.043s
I20260709 12:27:59.826872  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:59.829523 10643 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.831362  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:27:59.836580  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:27:59.837810 10640 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:27:59.816062 20282 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:27:59.843791 19837 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:27:59.844238 19837 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:27:59.844588 19837 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:27:59.846935 20404 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:27:59.830476 20210 raft_consensus.cc:740] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:27:59.836606 20090 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:27:59.835578 19772 consensus_queue.cc:1243] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [LEADER]: Peer 9aa345bcd43b44088114b7b6f2830b17 log is divergent from this leader: its last log entry 26.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
W20260709 12:27:59.849556 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 153121us
I20260709 12:27:59.850067 19981 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:27:59.850745 20309 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:27:59.854405  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.838615 20235 raft_consensus.cc:3060] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:27:59.860252 20238 raft_consensus.cc:2804] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:27:59.858925 20282 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.868520 19981 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.860340 20213 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.870885 19981 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.871701 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:59.872545 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:27:59.874583 20282 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.836089 19975 consensus_queue.cc:1059] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 32.23, Next index: 24, Last known committed idx: 23, Time since last communication: 0.024s
I20260709 12:27:59.835476 19976 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.888113 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:59.895783 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:27:59.886739 20090 raft_consensus.cc:697] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:59.902119 20235 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:27:59.903155  9940 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:27:59.888139 20238 raft_consensus.cc:697] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:27:59.903988 20282 raft_consensus.cc:3055] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:27:59.925946  9942 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:27:59.905064 19837 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:27:59.914387 20210 consensus_queue.cc:260] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:27:59.919139 20072 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:27:59.952123  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 111071us
I20260709 12:27:59.899889 20070 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:27:59.922489 20391 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:27:59.975744  9991 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:27:59.982702  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:27:59.925462 20282 raft_consensus.cc:740] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:27:59.985519  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 172515us
I20260709 12:28:00.016712  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.953702 20345 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:00.018081 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:28:00.017690 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 179920us
I20260709 12:27:59.959642 20155 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.057s
I20260709 12:27:59.983294 19823 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.016541 20412 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:00.036938  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:27:59.953816 20114 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:27:59.921381 20238 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:00.045337 20401 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:28:00.037016 20345 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:00.052786 20345 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:00.054646 19823 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:00.027494 20257 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.055519 20257 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.057946 20257 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:00.059420 19976 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:27:59.987308 20090 consensus_queue.cc:237] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.059062 20282 consensus_queue.cc:260] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.060889 20282 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:00.061637 20069 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:00.037447 20309 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 12:28:00.062521 10017 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
W20260709 12:28:00.063496 20400 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
I20260709 12:28:00.041078 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:00.066133 19837 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.028723 20235 consensus_queue.cc:1059] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.038s
I20260709 12:28:00.067682 20235 raft_consensus.cc:3055] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:00.068034 20235 raft_consensus.cc:740] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:00.068898 20235 consensus_queue.cc:260] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.027740 20213 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:00.070950 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:00.071018 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:00.071822 10025 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:28:00.072718  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.077997 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:28:00.090878  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 54070.5us
I20260709 12:28:00.091668  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:00.092362  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.094628 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.095276 10676 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:00.095557 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:00.096083 10090 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:00.104858 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:00.107447 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:00.036612 20070 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:00.110921 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.024003 19981 consensus_queue.cc:1059] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.053s
I20260709 12:28:00.111572 10672 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:00.112504 10676 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:00.113653 10675 raft_consensus.cc:2393] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 31: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:00.114828 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.119056 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.066221 20391 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:00.122344 10025 pending_rounds.cc:85] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 18
I20260709 12:28:00.125130 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:00.095796 19837 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:00.122720 10025 pending_rounds.cc:107] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 12:28:00.128453 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.128754 10025 raft_consensus.cc:2889] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:00.129511 10668 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:00.129149 10671 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:28:00.134542 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:00.135669 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:00.138561 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:00.142565  9942 raft_consensus.cc:2468] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 24.
I20260709 12:28:00.143792  9940 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:28:00.145726 10643 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:00.151015 10020 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:00.152139 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:00.133242 20052 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.155332 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:28:00.149118 20406 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:28:00.160939 20072 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:00.172091 10019 raft_consensus.cc:2368] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30.
I20260709 12:28:00.176358 10023 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:00.164467 20090 raft_consensus.cc:2804] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:28:00.186264 10022 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
W20260709 12:28:00.188102 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 63043.4us
I20260709 12:28:00.188493 10023 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:28:00.188894 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:00.192229 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:00.196275 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:00.196342 20072 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:00.197567 20072 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:00.198100 20072 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:00.177316 19837 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:28:00.206859 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:00.206311 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 95338us
I20260709 12:28:00.209136  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:00.210263 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:00.212226  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:00.163537 20414 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.039s
I20260709 12:28:00.215561  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:00.219553 10642 leader_election.cc:400] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 28 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 28. Current term is 29.
I20260709 12:28:00.194473 19981 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.226382 10017 raft_consensus.cc:2468] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:28:00.205984 20090 raft_consensus.cc:697] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:00.229408  9942 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 25. Current term is 26. Ops: []
I20260709 12:28:00.230300  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.231644  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:28:00.231889 20399 log.cc:927] Time spent T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 12:28:00.235786 20309 raft_consensus.cc:3055] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:00.236289 20309 raft_consensus.cc:740] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:00.236320 20072 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.220842 20069 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:00.237344 20309 consensus_queue.cc:260] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:00.238080 20069 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:00.217321 19837 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.238708 19981 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:00.240598 19837 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.240664 19981 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:00.205329 20345 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:00.280136 10090 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:28:00.245551 10020 raft_consensus.cc:1275] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
W20260709 12:28:00.306037 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 86619.4us
I20260709 12:28:00.308995 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.311061 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:00.312553 20391 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:00.308382  9914 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:28:00.313215  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 97760us
I20260709 12:28:00.228205 19976 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:00.325642 19976 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:00.328652 19976 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:00.319990  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:28:00.320439 20416 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:00.321031 20419 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
W20260709 12:28:00.322247 20417 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 12:28:00.324019 20072 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:00.324983 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 131326us
I20260709 12:28:00.306607 20238 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:00.316771 19837 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:00.356345 20421 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:00.339874 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 194251us
I20260709 12:28:00.357939 20070 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:00.315858 20420 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:28:00.382414 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:28:00.383996  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 199566us
W20260709 12:28:00.376989 19756 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.585s	user 0.009s	sys 0.003s
I20260709 12:28:00.375207 20210 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:00.391634  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:28:00.392812  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:00.378785 19837 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:00.395573 19837 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.374998 20070 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.398814 19823 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:28:00.401443 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:00.385078 20090 consensus_queue.cc:237] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.383471 20052 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.360471 20238 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.349958 20345 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:00.409720 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 102684us
I20260709 12:28:00.462327 10642 leader_election.cc:400] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
W20260709 12:28:00.512094  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:00.524525 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62437.5us
I20260709 12:28:00.524829  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:00.402719 19756 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.528235 19756 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.535948 20238 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:00.537717 19976 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:00.402513 20390 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:00.402801 20420 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:28:00.546983  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:00.543612 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 226827us
I20260709 12:28:00.568462 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.564698 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:00.569839 10018 raft_consensus.cc:2393] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 24: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:00.574541 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.578634 10099 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 31: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:00.583273 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:00.588053 10094 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:28:00.561902 20304 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:00.592206 20304 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:00.573192 20430 log.cc:927] Time spent T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.213s	user 0.000s	sys 0.001s
I20260709 12:28:00.564616 20421 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:00.538436 19976 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:00.594540 20421 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.596235 19976 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:00.596669 20421 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:00.573274 20420 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:00.599433 20420 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.628273  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.629022  9949 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:28:00.594492 20304 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:00.617899  9991 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:00.637511 20427 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:28:00.638203 20422 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 12:28:00.646514  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:28:00.682919  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 65115.2us
I20260709 12:28:00.693104  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:00.694680 20416 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.695365 20416 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:00.691004 20427 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.613610 20257 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:28:00.707368 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 129912us
I20260709 12:28:00.707698 20257 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:00.711040 20257 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.712215 20257 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:00.562376 20391 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.722498 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:00.720932 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:00.610738 20434 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.094s
I20260709 12:28:00.717729 20416 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:00.747213  9942 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:00.734347 20052 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:00.736930 20304 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:00.750630 19976 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.737133 20390 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:00.767540  9943 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:28:00.758008 19984 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:28:00.773641  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 80651.1us
I20260709 12:28:00.764389 20434 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:00.775107  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:00.772441 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.764657 20070 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:28:00.782919 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 60531.4us
I20260709 12:28:00.792351 10094 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:28:00.795186  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:00.795606 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:00.802210 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:00.764706 20304 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.802883 10671 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:00.806149 10643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.806946 20052 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:00.807071 20426 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.088s
I20260709 12:28:00.772857 20390 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:00.807766 10641 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:00.809394 20390 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.771951 19976 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.811146 20440 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:28:00.797760 20282 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.003s
I20260709 12:28:00.796221 19981 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:00.822252  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.822956  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.829394 20444 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.829650 20447 consensus_queue.cc:1048] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:28:00.830181 20444 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:00.832363 20438 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:28:00.814996 20052 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.833642 20052 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:00.835354 20439 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:28:00.810169 20390 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:28:00.841156 10671 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:28:00.845508 10020 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:00.846856  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.851047  9987 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:00.812474 19976 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.856168 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.853876 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:00.802742 20429 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:00.885417 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.884690 10671 raft_consensus.cc:1275] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:00.885175 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.802762 20437 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:00.894476 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:00.894897 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:00.793926 20070 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.895941 10022 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 33 index: 20. Preceding OpId from leader: term: 34 index: 21. (index mismatch)
I20260709 12:28:00.896003 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.896935 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.897548 10025 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:00.823536 19981 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:00.849601 20416 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:28:00.848803 20444 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.887265  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.909474 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:00.895118 10095 raft_consensus.cc:3060] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:28:00.886277 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.906168 10092 raft_consensus.cc:2368] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:28:00.852928 20427 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.780993 20434 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.911566 10090 raft_consensus.cc:2393] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 30: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:00.908533  9946 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:28:00.912122 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:00.912352 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.912746 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.908430 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:00.915803 10019 raft_consensus.cc:1275] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:00.915970 10093 raft_consensus.cc:1275] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:28:00.913527 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:00.906324 19981 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:00.924885  9991 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:00.901952 20416 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:00.913875 20345 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.937191 20434 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.938093 20445 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:00.938973 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:00.940048 10670 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:00.941845 20448 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:00.942356 20448 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:00.942746 20448 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:00.946897 20070 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:00.948194 20459 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:00.932999 20429 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:00.955181 20429 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:00.955080 20416 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:00.955824 20429 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:00.956998 20416 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:00.970718 10020 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:00.918151 10018 raft_consensus.cc:2368] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:00.967662  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:00.986003  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:00.905148 20437 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:00.995415  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.000463  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.000800 20413 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.001365 20413 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:01.002904 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 92452.5us
I20260709 12:28:01.003681 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:01.005759  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.006539  9943 raft_consensus.cc:2368] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:00.946940 19772 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:00.971683 20445 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:01.021184 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:01.022500  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.030831 20434 consensus_queue.cc:1059] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:01.024549  9947 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:01.033113 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:01.041023  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:00.989631 20414 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.034s
I20260709 12:28:01.067075 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.023409 20235 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:01.055832 10062 leader_election.cc:400] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30.
I20260709 12:28:01.080013 10062 leader_election.cc:403] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:01.077037 10018 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:01.021364 20070 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:01.032395 20282 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:28:01.087596 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 124065us
W20260709 12:28:01.084828  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.033880 20455 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:01.091384  9947 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:28:01.037032 20437 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.082046 20434 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:01.076889 20420 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.069933 20412 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:01.093303  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:01.038764 20448 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.057755 20390 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:28:01.053222 19772 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.091465 20052 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.094265 19984 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30.
I20260709 12:28:01.137039 10095 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:28:01.094098 20304 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.090361 20282 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.117740 20412 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:01.171952 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 81877.5us
I20260709 12:28:01.173169 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.113375 20449 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:01.134464 20426 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:01.186303 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.135115 20455 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:01.174789 20282 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:01.115115 20420 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.178508 20412 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:01.243158 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 150671us
I20260709 12:28:01.164201 20462 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:01.179872 20210 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:01.136205 20345 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.061s
I20260709 12:28:01.135833 20070 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.057s
I20260709 12:28:01.162005 20465 consensus_queue.cc:1048] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.115s
I20260709 12:28:01.273862 10095 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:28:01.262794 20257 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:01.177440 20449 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.169631 20445 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:01.291923 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48035.7us
I20260709 12:28:01.297513 10062 leader_election.cc:400] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:01.297878 10062 leader_election.cc:403] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:01.275590 20434 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
I20260709 12:28:01.281874 20210 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.314807 20468 consensus_queue.cc:1059] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.029s
I20260709 12:28:01.316391 20436 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:01.317149 20436 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.299080 19984 consensus_queue.cc:1048] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.023s
I20260709 12:28:01.299501 20257 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.322078 20257 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.322629 20070 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
W20260709 12:28:01.324013 20464 log.cc:927] Time spent T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:28:01.324168 20391 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:01.324820 20391 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.327103 20467 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.327734 20467 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.329131 20391 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.330026 20467 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.344209 20429 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.275856 20462 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.348117 20420 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.349237 20282 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.349910 20462 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.354400 20456 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:01.356966  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3473) took 1312 ms. Trace:
I20260709 12:28:01.356828 20456 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.359380 20448 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.359400 19981 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:01.362128  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:01.298454 20469 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:28:01.404734 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 155687us
I20260709 12:28:01.357393  9944 rpcz_store.cc:276] 0709 12:28:00.044676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.098768 (+ 54092us) service_pool.cc:224] Handling call
0709 12:28:00.117053 (+ 18285us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:00.722262 (+605209us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b180040ea60. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e0954ebfb 00007f4e0955964b
0709 12:28:00.722402 (+   140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.722430 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:00.749797 (+ 27367us) log.cc:844] Serialized 52 byte log entry
0709 12:28:00.909660 (+159863us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:00.909706 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.356442 (+446736us) spinlock_profiling.cc:243] Waited 45.2 ms on lock 0x7b54000b6b54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:28:01.356517 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.356584 (+    67us) raft_consensus.cc:1685] finished
0709 12:28:01.356605 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.356854 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9935,"raft.queue_time_us":49334,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3265,"spinlock_wait_cycles":141523840,"thread_start_us":193359,"threads_started":2,"wal-append.queue_time_us":203613}
I20260709 12:28:01.421916 10674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:42042 (request call id 2172) took 1169 ms. Trace:
I20260709 12:28:01.422330 10674 rpcz_store.cc:276] 0709 12:28:00.246115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:00.307602 (+ 61487us) service_pool.cc:224] Handling call
0709 12:28:00.318531 (+ 10929us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:00.335419 (+ 16888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:00.335468 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:00.360727 (+ 25259us) log.cc:844] Serialized 26 byte log entry
0709 12:28:01.035290 (+674563us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b18001f9720. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0644be07 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06468c02 00007f4e090f020b 00007f4e090f5d23 00007f4e090f60d1 00007f4e09511a0f
0709 12:28:01.095199 (+ 59909us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:01.095257 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:01.095353 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:01.314473 (+219120us) raft_consensus.cc:1685] finished
0709 12:28:01.314506 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:01.415655 (+101149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13621,"raft.queue_time_us":44124,"raft.run_wall_time_us":4145,"spinlock_wait_cycles":13102208,"thread_start_us":59688,"threads_started":1,"wal-append.queue_time_us":799230}
I20260709 12:28:01.431313 10065 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:01.434548 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:01.435218 10668 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:01.403055 20235 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.447386 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:01.451761 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 285972us
I20260709 12:28:01.466244 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:01.468080 10672 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:01.477845 10065 leader_election.cc:400] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:01.303165 20455 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.484037 20309 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:01.422289 20469 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.487962 20469 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.489260 20282 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.456029 20235 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.494401 10672 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:28:01.464790 20070 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:01.519790 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.521829 20435 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:01.496995 20468 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:01.529824 20460 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.447s	user 0.004s	sys 0.000s
I20260709 12:28:01.530517 20468 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.547602 20468 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.548236 19984 raft_consensus.cc:3055] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:01.548955 19984 raft_consensus.cc:740] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:01.553706 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.555500 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.558029 10668 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:28:01.557066 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:01.559927 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.560377 10676 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:28:01.560555 20449 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.561698 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.563422 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.566377 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:01.571307 20282 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.587101 20282 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.589608 20282 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.590853 20413 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.591379 20449 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.592090 19975 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.591890 20449 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.592557 19975 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.595399 20444 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.595860 20444 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.597986 20444 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.596755 20449 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.598984 20465 consensus_queue.cc:1048] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.014s
W20260709 12:28:01.601369 20416 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.644s	user 0.015s	sys 0.000s
I20260709 12:28:01.571483 20412 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.605788 20456 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.608103 19975 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.610569 20416 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:01.611284 20416 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.613919 20416 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.587090 20304 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:01.567935 19841 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:01.623575 19841 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.625955 19841 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.570865 19984 consensus_queue.cc:260] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.629961 20429 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.630434 20412 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.634091  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:01.645015 19981 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.647159 19772 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.650862 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.651615 19772 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:01.651779 10672 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 33.
I20260709 12:28:01.652036 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:01.652209 19772 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.652976 20404 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.653456 20404 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.655143 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.655540 20404 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.657292 20413 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:01.657619  9912 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:01.657914 20413 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.659715 20428 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:01.660211 20413 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.660331 20428 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:01.661422 20428 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.661804 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.636802  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:01.662501 10672 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:01.662876  9941 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:01.637816  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.639483  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.665025  9943 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 33.
I20260709 12:28:01.572899 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:01.640406  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.665939 20473 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:01.666246  9945 raft_consensus.cc:2410] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 25 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:28:01.638566  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.641458  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.666878  9948 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
W20260709 12:28:01.667137 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 94369.9us
I20260709 12:28:01.667353  9946 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 26: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:28:01.654446 19772 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.642303 20114 raft_consensus.cc:3055] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:01.652606 10671 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:01.668390 20114 raft_consensus.cc:740] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:01.669387 20114 consensus_queue.cc:260] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.670434 20114 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:01.676352  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:01.677052  9944 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:01.574934 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:01.668399 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:01.682919 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:01.678941 20235 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.688738 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.690876 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.691514 10677 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:01.680042  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:01.693598  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:01.694007 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:01.694903 10672 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 32
W20260709 12:28:01.694953 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 120135us
I20260709 12:28:01.697750 10671 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:01.698949 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.703151 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.710763 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:01.674314 20210 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.710840  9987 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:01.713697 10642 leader_election.cc:400] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:28:01.714131 10642 leader_election.cc:403] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:28:01.715695 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:01.701215  9942 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:28:01.718438  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:01.723555 10093 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:28:01.727212 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:01.728873 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:01.733002 20457 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:01.733780 20457 consensus_queue.cc:1048] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:28:01.737697 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:01.740598 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.689177 20414 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:01.662873 20436 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.703213 20235 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:01.744786  9991 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:01.694480 20473 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:01.748718 20479 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:01.723563 20429 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.723317 20309 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:01.726022 20345 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.748821 20069 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:01.749200 20155 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:01.784726 20480 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:01.786898 20155 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.789254 20155 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.790084 10018 raft_consensus.cc:1275] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:28:01.784353 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.793736 20271 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:01.794420 20271 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.796561  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.750662 10091 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:01.751684 10669 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 30. Current term is 31. Ops: [30.19-30.19]
I20260709 12:28:01.755256 20210 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.755828 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.747654 20426 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:01.756588 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.757053 10670 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:28:01.758145 20052 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:01.762869 20403 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:01.763526 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:01.713271 20427 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:01.764773  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:01.732051 20448 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:01.785391 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.785238 20345 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.764899 19984 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:01.784812 20309 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.784869 10668 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:01.803179  9944 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:28:01.803936  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.833081  9942 raft_consensus.cc:2393] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 29: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:28:01.807518  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.833175 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.730552 20424 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:28:01.777483 10672 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:28:01.837481 10095 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:01.815024  9941 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:28:01.837491  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.819155  9940 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:01.769009 20404 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.821391 10099 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:28:01.821579  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.823325  9945 raft_consensus.cc:1275] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:01.825202 20485 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.007s
I20260709 12:28:01.826349 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:01.826807 20069 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.814072 20429 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.873786 20069 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.876541 20069 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:01.766813 20235 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:01.830783  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 66094.5us
I20260709 12:28:01.877638 20345 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:01.799229 20436 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:01.878945 20436 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:01.879922 20436 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.882105 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.747167 20445 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:01.882927 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.883219 20445 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.883605 10091 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:28:01.820113 10021 raft_consensus.cc:2368] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 31. Current term is 32.
I20260709 12:28:01.885782 20481 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:01.882943  9987 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:01.887606 10674 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:28:01.887914 19976 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:01.888384 19976 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:01.889279 19976 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.889709 20404 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.811780 20426 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.892941 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.893694 20426 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.844971  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.895043 10090 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:28:01.895040  9942 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:28:01.896116 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.834954  9991 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:28:01.897217  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62394.7us
I20260709 12:28:01.817631 20489 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.817864 20052 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.850435 20479 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.009s
I20260709 12:28:01.909929 20479 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:01.910331 20479 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:01.911119 20479 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.912346 20479 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:01.913138 20421 consensus_queue.cc:1048] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:01.894004 20445 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:01.916764 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.917371 20489 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:01.850652 20155 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.010s
I20260709 12:28:01.885828  9914 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:01.836244 19841 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:01.867300 10025 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:01.868177 20448 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.836230 20309 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:01.869758 20457 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:28:01.818154 20210 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.877452 20235 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:01.883687 10671 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:01.840592  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.882784 10097 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:28:01.948127  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.900760  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.949863  9942 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:28:01.950243 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.849179 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:01.951695 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 102602us
I20260709 12:28:01.887614  9987 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:01.903056  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.907338 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.904989 20069 raft_consensus.cc:2804] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:01.849959 20114 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:01.933432 10090 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:01.934437  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.937278 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.947295 20291 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:01.965696  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 78144.9us
W20260709 12:28:01.967029  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81287.8us
I20260709 12:28:01.967172 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.955621 10023 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:28:01.973238 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.973490 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.974099 10091 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:01.922868 20052 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:01.982970  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:01.974262 10090 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:28:01.984414 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:01.991997 10640 leader_election.cc:400] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 31. Current term is 32.
I20260709 12:28:01.992344 10640 leader_election.cc:403] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:01.992465 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.992640 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.993140 10024 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:28:01.928872 19841 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:01.929076 20445 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:01.995004 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.995388 10640 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:02.003921 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:02.005271 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:01.976511 20427 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:01.977264 19981 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.010906 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:02.011279 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:02.012919 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.997521 20473 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 31. Current term is 32.
I20260709 12:28:02.017016 10090 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 30: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:01.997735 19841 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:02.014134 20414 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:01.983417 20416 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:01.992985 19975 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:02.023101 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:01.991008 20485 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:02.021970 10091 raft_consensus.cc:3060] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:02.024922 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:01.993611 20069 raft_consensus.cc:697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:02.027413 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:01.983337 20426 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.029091  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.015579  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:28:02.029671 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 57988.8us
I20260709 12:28:02.028457 10090 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:28:02.001050 20458 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.037365 10094 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 31: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:02.036280 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:02.029868  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.039361  9940 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 31.
I20260709 12:28:02.054164 10024 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:28:01.983929 20114 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:01.990947 20291 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.030989 20424 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:02.061997 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:02.062242 20481 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:28:02.062553  9841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 316) took 55315 ms (client timeout 15000 ms). Trace:
I20260709 12:28:01.997398 19772 consensus_queue.cc:1059] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.041s
I20260709 12:28:02.029254 20271 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:02.063100 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.062659 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:02.063508 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:02.062734 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:02.055598  9838 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 24 to 29, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 29 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.003872 20451 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:28:02.060778 19975 raft_consensus.cc:697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:02.076823 10024 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:28:02.084126 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:02.071962  9841 rpcz_store.cc:269] 0709 12:27:06.746855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:06.748753 (+  1898us) service_pool.cc:224] Handling call
0709 12:27:06.766854 (+ 18101us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094c0090 00007f4e0955ef3a 00007f4e0f314e9c 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc 00007f4e0d8d3392 00007f4e0789148c 00007f4e078908c6
0709 12:28:02.062475 (+55295621us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:27:59.674027 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:27:59.674718 (+   691us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:27:59.674766 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:27:59.674785 (+    19us) tablet.cc:662] Decoding operations
0709 12:27:59.675769 (+   984us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:27:59.675864 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:27:59.675891 (+    27us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:27:59.676728 (+   837us) tablet.cc:701] Row locks acquired
0709 12:27:59.689899 (+ 13171us) write_op.cc:260] PREPARE: finished
0709 12:27:59.698198 (+  8299us) write_op.cc:270] Start()
0709 12:27:59.730691 (+ 32493us) write_op.cc:276] Timestamp: P: 1783600079694189 usec, L: 0
0709 12:27:59.730747 (+    56us) op_driver.cc:348] REPLICATION: starting
0709 12:27:59.731446 (+   699us) log.cc:844] Serialized 3606 byte log entry
0709 12:28:00.064760 (+333314us) op_driver.cc:464] REPLICATION: finished
0709 12:28:00.195010 (+130250us) write_op.cc:301] APPLY: starting
0709 12:28:00.209625 (+ 14615us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:00.527707 (+318082us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:00.527886 (+   179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:01.159289 (+631403us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:01.159749 (+   460us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=0,mrs_lookups=0
0709 12:28:01.159841 (+    92us) write_op.cc:312] APPLY: finished
0709 12:28:01.160355 (+   514us) log.cc:844] Serialized 89 byte log entry
0709 12:28:01.291685 (+131330us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:01.291878 (+   193us) write_op.cc:454] Released schema lock
0709 12:28:01.294724 (+  2846us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:01.355460 (+ 60736us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:01.355512 (+    52us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":203915,"reported_tablets":48,"spinlock_wait_cycles":8807168,"child_traces":[["op",{"apply.queue_time_us":108659,"apply.run_cpu_time_us":76585,"apply.run_wall_time_us":1173262,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41765,"num_ops":9,"prepare.queue_time_us":221728,"prepare.run_cpu_time_us":8669,"prepare.run_wall_time_us":105889,"raft.queue_time_us":170272,"raft.run_cpu_time_us":25519,"raft.run_wall_time_us":103106,"replication_time_us":311235,"thread_start_us":449916,"threads_started":5,"wal-append.queue_time_us":286674,"wal-append.run_cpu_time_us":20716,"wal-append.run_wall_time_us":1151380}]]}
I20260709 12:28:02.011040 20445 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.078675  9838 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 17 to 28, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 28 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.101704  9838 catalog_manager.cc:5697] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 31, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 31 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.100438 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.098253 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.071853 20481 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.107069  9838 catalog_manager.cc:5697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 28, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 28 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:02.051280 20155 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:28:02.070917 20424 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:02.029446 19981 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.108153 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:02.103732 10096 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:28:02.037750 20416 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.080706 20429 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.059039 20426 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.068081 20458 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.095584 20427 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:02.111160 20481 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:02.137872 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:02.104986 20282 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:02.140817  9943 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:02.120770  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.110757 20213 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:02.152671  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 253948us
I20260709 12:28:02.144515 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:02.113566 20424 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:02.151109 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:02.166005  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.168466 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:02.170806 10020 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:02.169754 10023 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 34.
I20260709 12:28:02.172621  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.139050 20404 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.176090 10092 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:28:02.181761 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:02.193149  9949 raft_consensus.cc:2393] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 30: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:28:02.196951 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52571.3us
I20260709 12:28:02.200176 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:02.161458 20428 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:02.203472 20428 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.203905 20210 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:02.115854 20069 consensus_queue.cc:237] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.205919 20428 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.195021 19984 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:02.207660 19984 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.208073 19984 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:02.172426 20429 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.210980 20429 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.213171 20429 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:02.215639 20420 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:02.221510 20494 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:28:02.110615  9838 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 25. New cstate: current_term: 25 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.227051 10065 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:02.231869  9838 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 26. New cstate: current_term: 26 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.239989 10065 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:02.238991  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.257393  9838 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 24 to 27. New cstate: current_term: 27 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.160768 20427 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.232283 20436 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:02.167740 20282 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.281953  9838 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 27. New cstate: current_term: 27 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:02.172247 20213 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.289136  9838 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 19 to 25. New cstate: current_term: 25 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:02.241298 20420 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:02.234367 20070 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:02.291699  9987 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:02.298499  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.172657 19975 consensus_queue.cc:237] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.305541  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:02.298386  9838 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 20 to 27, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.307945  9838 catalog_manager.cc:5697] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 33. New cstate: current_term: 33 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:02.232527 20309 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.283288 20436 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.287194 20455 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:02.344265  9838 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 25 to 32. New cstate: current_term: 32 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:02.342252 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:02.356061  9838 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 23 to 27. New cstate: current_term: 27 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:02.354245  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 312534us
I20260709 12:28:02.353824 10669 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:28:02.385783 10019 raft_consensus.cc:2368] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 29. Current term is 30.
I20260709 12:28:02.344969 20444 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:02.342334 19976 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:02.422258 20282 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.421599 20481 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:02.428445 20496 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:28:02.429836 20448 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:02.430126 20458 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.431771 20427 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:02.432262 20481 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.417748 19841 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.432893 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:02.433733 10022 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:02.436007 19841 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:02.436204 10020 raft_consensus.cc:1240] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 LEADER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:02.437281 20416 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.437817 10020 raft_consensus.cc:1240] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:02.439877 20213 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.406347 20489 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.352258 20436 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:02.442555  9838 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:02.444648  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63978.6us
I20260709 12:28:02.459741  9838 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 23 to 28, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 28 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.467449  9838 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.464432  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:02.473013 19976 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.473780 19976 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.355499 20420 consensus_queue.cc:1059] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.116s
I20260709 12:28:02.475425 20420 raft_consensus.cc:3055] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:02.475780 20420 raft_consensus.cc:740] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:02.476640 20420 consensus_queue.cc:260] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.477869 20420 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:02.478020 10019 raft_consensus.cc:1240] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:02.344839 20070 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.457733 19984 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.480984  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.485582 10019 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:28:02.486763 20481 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:02.487278 20481 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.487711 20481 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:02.492661 19984 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:02.493953 10091 raft_consensus.cc:2468] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:28:02.465137 20458 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.302489 20424 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.469403  9838 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 19 to 27, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 27 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.470646 20291 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:02.503486  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 474279us
I20260709 12:28:02.509692  9838 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 23 to 26, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 26 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:02.481886 20445 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.482583 20413 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:02.534466  9838 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 17 to 18, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 18 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.535624  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:02.550262  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.550554  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:02.551229  9940 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:02.548840  9838 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 19 to 24, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 24 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:02.534468 20479 consensus_meta.cc:287] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.622s	user 0.007s	sys 0.004s
I20260709 12:28:02.566002  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.567565  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.567850  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.511098 20458 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.574096  9838 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 25, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 25 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:28:02.565639 20235 consensus_meta.cc:287] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.001s
I20260709 12:28:02.555317 20210 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:02.554719 20416 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:02.560484 20489 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:02.611356  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.612080  9942 raft_consensus.cc:2468] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:02.547875 20445 consensus_queue.cc:1059] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.024s
I20260709 12:28:02.557375  9990 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:02.581169 20436 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.608009  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:02.616421  9948 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:02.608330 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 313583us
I20260709 12:28:02.599552  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.617751  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.618392  9942 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
W20260709 12:28:02.607533  9914 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.000s	sys 0.019s
W20260709 12:28:02.619511  9914 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.000s	sys 0.020s
W20260709 12:28:02.620536  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 590753us
W20260709 12:28:02.626070 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 386168us
I20260709 12:28:02.627003 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:28:02.610549 20052 consensus_meta.cc:287] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.612s	user 0.007s	sys 0.004s
I20260709 12:28:02.600382 20480 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:02.632635 20420 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:02.663673 20052 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:02.668638  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 46556.6us
I20260709 12:28:02.671073 20506 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:02.670814 20503 raft_consensus.cc:3055] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:02.736320  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.615770 20235 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.614809 20489 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:28:02.671205  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 113913us
I20260709 12:28:02.630643 20210 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:02.731648  9948 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:28:02.700774 20458 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:02.703774 20271 raft_consensus.cc:3055] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:02.636557  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.633395 20516 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.072s
I20260709 12:28:02.665369 20436 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.701721 20424 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:02.733172 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 103077us
W20260709 12:28:02.733623 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 100569us
I20260709 12:28:02.612768  9838 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 26. New cstate: current_term: 26 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:02.734661  9942 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:28:02.764357  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.735083  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:02.738921  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:02.634920 20451 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.101s
I20260709 12:28:02.756146 10021 raft_consensus.cc:1240] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:02.761119  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:02.769833  9941 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:28:02.740991 20485 raft_consensus.cc:2764] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 29: won
I20260709 12:28:02.774542 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:02.738286 20503 raft_consensus.cc:740] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:02.772419  9838 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 25, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 25 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.741684 20489 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:02.767946  9949 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:28:02.732990 20481 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.775590  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.781375  9838 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 20 to 26. New cstate: current_term: 26 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:02.754041 20271 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:02.787662  9991 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.793244 10092 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57396 (request call id 1979) took 1020 ms. Trace:
W20260709 12:28:02.796589 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:02.807883 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.800187 20517 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:02.779690 20502 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:02.776670 20428 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:02.807451 10092 rpcz_store.cc:276] 0709 12:28:01.772908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.773023 (+   115us) service_pool.cc:224] Handling call
0709 12:28:02.793031 (+1020008us) spinlock_profiling.cc:243] Waited 385 ms on lock 0x7b54002fc954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:02.793157 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":884960640}
I20260709 12:28:02.812769 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.819464 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.819404  9838 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 28, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 28 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.785396 20420 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:02.821857 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:02.836532 10093 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:02.834081 10097 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:02.838405 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.821843  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:02.842399  9838 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 25, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 25 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.839706 10095 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:28:02.846508 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.836746 10090 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 25.
I20260709 12:28:02.839054 10097 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:28:02.844385  9838 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 26, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 26 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:02.850566  9838 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 23 to 25. New cstate: current_term: 25 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.853569  9838 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 20 to 26, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 26 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:02.802838 20444 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:02.852530 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.858120 10090 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 21. (term mismatch)
I20260709 12:28:02.858479 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.858659 10090 pending_rounds.cc:85] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 20
I20260709 12:28:02.859189 10090 pending_rounds.cc:107] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 28.21
I20260709 12:28:02.859472 10090 raft_consensus.cc:2889] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:02.860834 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.857941 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:02.822202 20502 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.863255 10677 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:28:02.839385 20420 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.837134 20428 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:02.849113 20516 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:02.854682 20069 raft_consensus.cc:2804] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:02.882303 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.862097  9838 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 30, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 30 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:28:02.876940 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 116453us
I20260709 12:28:02.879897 10668 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:02.878720 20420 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:02.858174 20444 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:02.906980 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.908464 20451 consensus_queue.cc:1048] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:02.911562  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:02.896808  9838 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 24 to 28, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 28 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.915086 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:02.868072 20502 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:02.877933 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 130997us
I20260709 12:28:02.919884 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.921079 10640 leader_election.cc:400] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 29. Current term is 30.
I20260709 12:28:02.921052 10095 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 25.
I20260709 12:28:02.922034 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:02.881597 20428 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:02.927973 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:28:02.923411  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81875.8us
W20260709 12:28:02.916899 20507 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:28:02.922889  9838 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 32, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 32 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:02.916955 20515 log.cc:927] Time spent T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:28:02.925182 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 33 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.835914 20517 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.917217 20444 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:02.926716 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.930709  9838 catalog_manager.cc:5697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 28, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 28 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:02.934679  9838 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 28. New cstate: current_term: 28 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:02.935921 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.936532 10095 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:28:02.923034 20069 raft_consensus.cc:697] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:28:02.939456  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 151867us
I20260709 12:28:02.941339  9991 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.921568 10640 leader_election.cc:403] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:02.948625  9990 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:02.954365 10091 raft_consensus.cc:1240] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:02.957011 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:02.957851 10090 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:28:02.957752 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 92923.7us
W20260709 12:28:02.956995  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 87445.3us
I20260709 12:28:02.963099  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:02.964258  9944 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:02.903093 20271 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:02.949781 20052 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:02.959885 20424 raft_consensus.cc:3060] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:28:02.977378  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.952276 20282 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:02.965662 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:02.952911 20445 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.027s
W20260709 12:28:02.982133 20512 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.179s	user 0.001s	sys 0.001s
W20260709 12:28:02.989631  9913 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:02.991930  9944 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 34.
I20260709 12:28:02.993108  9991 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:02.994298 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:02.982630 19841 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:02.989059 20282 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.017283  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:02.997056 20271 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:02.997488 20479 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:02.980523 20532 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.035s
W20260709 12:28:03.051403 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.008890 20309 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.012310 20052 raft_consensus.cc:2804] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:03.009402 19841 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.012595 20282 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:03.014499 20345 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:03.054961  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:03.073772  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 62195.7us
I20260709 12:28:03.100533  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:03.109813  9944 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:28:03.100539 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:03.061841 20213 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:02.999451 20069 consensus_queue.cc:237] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.013954 20534 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.094s
I20260709 12:28:03.131004 20530 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.131174  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.105953 19841 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:03.150133 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:03.156561  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:03.166266 10090 raft_consensus.cc:2393] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:03.182451 10093 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:28:03.151371 10018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 4440) took 1184 ms. Trace:
I20260709 12:28:03.188622 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:03.183347 10018 rpcz_store.cc:276] 0709 12:28:01.966800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.966936 (+   136us) service_pool.cc:224] Handling call
0709 12:28:03.151129 (+1184193us) spinlock_profiling.cc:243] Waited 613 ms on lock 0x7b54002d5854. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:03.151286 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1410294016}
I20260709 12:28:03.129897 20213 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.134419 10023 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:03.129623 20052 raft_consensus.cc:697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:03.196592 20437 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:28:03.194582 20519 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:28:03.210381 10674 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:28:03.212145 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:03.163200 20445 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.214210 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:03.216209 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:03.218247 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 34 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:03.244685 10672 raft_consensus.cc:3060] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:03.201357 20213 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:03.228972 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:03.235145 20052 consensus_queue.cc:237] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 32.23, Last appended by leader: 23, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:03.236462 20537 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:28:03.238166 20535 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
W20260709 12:28:03.242866  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 86439.5us
I20260709 12:28:03.268571  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:03.264122  9944 raft_consensus.cc:2393] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 28: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:28:03.264998 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:03.249723 10673 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:28:03.212816 20517 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:03.226856 20527 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.256s	user 0.000s	sys 0.001s
I20260709 12:28:03.151602 20530 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.251592 20549 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:03.282339  9941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.283330  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.281913 10674 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
W20260709 12:28:03.133976 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.282025  9945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 2376) took 1106 ms. Trace:
I20260709 12:28:03.277724  9914 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:03.306901 20545 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:03.307510 20545 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:03.264726 20532 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.226428 20437 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.300083 20517 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.274506 20540 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:03.295481  9945 rpcz_store.cc:276] 0709 12:28:02.201467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.201650 (+   183us) service_pool.cc:224] Handling call
0709 12:28:02.202103 (+   453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.228113 (+ 26010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.228155 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.228697 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.746979 (+518282us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:02.747018 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.747087 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.281807 (+534720us) raft_consensus.cc:1685] finished
0709 12:28:03.281829 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.281960 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":109,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":6873,"thread_start_us":458305,"threads_started":1,"wal-append.queue_time_us":668242}
I20260709 12:28:03.271477 20458 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.325414 20517 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:03.345289 20509 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.495s	user 0.001s	sys 0.000s
I20260709 12:28:03.346783  9946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3496) took 1493 ms. Trace:
W20260709 12:28:03.346782 20548 log.cc:927] Time spent T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:28:03.347358  9946 rpcz_store.cc:276] 0709 12:28:01.853016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.853137 (+   121us) service_pool.cc:224] Handling call
0709 12:28:01.922473 (+ 69336us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:01.997748 (+ 75275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:01.997784 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.032844 (+ 35060us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.577327 (+544483us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:02.577385 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.577482 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.344701 (+767219us) spinlock_profiling.cc:243] Waited 8.5 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:28:03.346354 (+  1653us) raft_consensus.cc:1685] finished
0709 12:28:03.346388 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.346707 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43718,"raft.queue_time_us":58672,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":15659,"spinlock_wait_cycles":19549952,"thread_start_us":476203,"threads_started":1,"wal-append.queue_time_us":711877}
I20260709 12:28:03.350368  9946 raft_consensus.cc:1275] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:28:03.351557 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:03.352912 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:03.353652 10019 raft_consensus.cc:2393] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:03.356986  9943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3492) took 1546 ms. Trace:
I20260709 12:28:03.309264 19975 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.386771 19975 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.335620 20532 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.392581 19976 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.396147  9945 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:28:03.354382 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:03.357457  9943 rpcz_store.cc:276] 0709 12:28:01.810259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:01.848332 (+ 38073us) service_pool.cc:224] Handling call
0709 12:28:01.906222 (+ 57890us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.162342 (+256120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.356758 (+1194416us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b54000b7054. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:28:03.356927 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":539071360}
I20260709 12:28:03.379479 20503 consensus_queue.cc:260] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.381270 20235 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 34 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.384801 20545 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:03.412436 20525 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.548s	user 0.000s	sys 0.001s
I20260709 12:28:03.417469 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3207) took 1317 ms. Trace:
I20260709 12:28:03.413127 20426 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.413712 20070 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:03.418121 20503 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:03.417905 10020 rpcz_store.cc:276] 0709 12:28:02.099805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.440318 (+340513us) service_pool.cc:224] Handling call
0709 12:28:02.440686 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.441135 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.441176 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.483101 (+ 41925us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.629417 (+146316us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:28:02.629475 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:02.629569 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.411689 (+782120us) spinlock_profiling.cc:243] Waited 4.41 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:28:03.417119 (+  5430us) raft_consensus.cc:1685] finished
0709 12:28:03.417142 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.417400 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2127,"raft.queue_time_us":1073,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":9434,"spinlock_wait_cycles":10140928,"thread_start_us":118887,"threads_started":1,"wal-append.queue_time_us":287081}
I20260709 12:28:03.394968  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.421829 20533 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.422470 20533 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.424717 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:03.427304 19981 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.431203 19984 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.438076 20070 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.335106 20540 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.438715 20070 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.433337 19984 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.453361 20510 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.460704 20480 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:03.461735 20114 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.461463 20480 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.454071 20510 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.433535 20291 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.477939 20114 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.478564 20114 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.478312 20291 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.455569 20540 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:03.404297 20458 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.511085 20235 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:03.511806 20235 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:03.512656  9912 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.194:35115 timed out after 1.730s (SENT) [suppressed 4 similar messages]
W20260709 12:28:03.519749  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.547601 20210 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.548303 20235 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.549088 20517 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.550310 20510 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.550531 20530 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.550806 20291 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.551529 20114 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.263918 20445 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.553196 20532 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.553886 20532 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 12:28:03.556427 20420 consensus_meta.cc:287] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.633s	user 0.010s	sys 0.006s
W20260709 12:28:03.557156 20489 consensus_meta.cc:287] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.768s	user 0.002s	sys 0.013s
I20260709 12:28:03.558288 20458 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.559643 19984 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.560880 20437 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.562114 20533 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:03.566306 20271 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.517s	user 0.003s	sys 0.011s
I20260709 12:28:03.575977  9944 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:28:03.575304 20489 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:03.569058 20502 consensus_meta.cc:287] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.640s	user 0.021s	sys 0.037s
I20260709 12:28:03.570457 19975 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.556985 20420 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:03.580125 20428 consensus_meta.cc:287] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.644s	user 0.011s	sys 0.003s
I20260709 12:28:03.580773 10095 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:03.580602 20428 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:03.581621  9942 raft_consensus.cc:1240] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:03.573813 20540 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:03.513332 20538 consensus_queue.cc:1059] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.070s
I20260709 12:28:03.584385 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3198) took 1539 ms. Trace:
I20260709 12:28:03.584786 10017 rpcz_store.cc:276] 0709 12:28:02.045147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.223003 (+177856us) service_pool.cc:224] Handling call
0709 12:28:02.223378 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.223748 (+   370us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.223779 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:02.224244 (+   465us) log.cc:844] Serialized 26 byte log entry
0709 12:28:02.611793 (+387549us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:28:02.849045 (+237252us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.583794 (+734749us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b18002e9d20. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287
0709 12:28:03.583928 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.584003 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.584051 (+    48us) raft_consensus.cc:1685] finished
0709 12:28:03.584149 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.584282 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":312349,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":22856,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":27395,"spinlock_wait_cycles":2610048,"thread_start_us":689235,"threads_started":3,"wal-append.queue_time_us":565489}
I20260709 12:28:03.586000 10672 raft_consensus.cc:2468] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 34.
I20260709 12:28:03.579113 10019 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:03.595005  9990 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.015s	sys 0.007s
W20260709 12:28:03.595779  9990 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.015s	sys 0.007s
W20260709 12:28:03.596151  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 640857us
I20260709 12:28:03.597317  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:03.599488 10062 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:03.589751 20480 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.601606 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:03.622648  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:03.624571 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:03.598065 20473 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.011s	sys 0.011s
I20260709 12:28:03.634927 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:03.667059 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:03.602327 20455 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.602196 20448 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.669001 20455 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.671833 20420 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:03.671936 20455 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.674328 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:03.677731 20489 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.601974 20155 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.680222 20155 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.682320 20155 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.684438 20445 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.595125 20530 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.602788 20210 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.591297 20549 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:03.669152 20448 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:03.689239 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.687507  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:28:03.690675  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 68185us
I20260709 12:28:03.664928 20309 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.682479  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.635720  9987 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:03.695202 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:03.687497 20542 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.698048 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:03.589080 20540 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.703006 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:03.689543 20473 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.011s	sys 0.011s
I20260709 12:28:03.619310 20070 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.619858 20533 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.664916 19975 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:03.741657 10016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45922 (request call id 3193) took 1698 ms. Trace:
W20260709 12:28:03.759174 20555 log.cc:927] Time spent T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:28:03.625036 20282 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:03.595870 20502 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:03.696398 10019 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:28:03.665268 20536 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:03.705072 20444 consensus_meta.cc:287] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.747s	user 0.007s	sys 0.010s
I20260709 12:28:03.757623 20532 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:03.696283 20489 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:03.753275  9949 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:28:03.750298  9943 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
W20260709 12:28:03.753402  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 117899us
W20260709 12:28:03.753384  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.742267 20445 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.768057 10021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4450) took 1766 ms. Trace:
W20260709 12:28:03.776407 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.777909  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.748447  9990 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:03.769932 20424 consensus_meta.cc:287] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.773s	user 0.019s	sys 0.037s
W20260709 12:28:03.789984  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 277365us
I20260709 12:28:03.741762 20530 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.790324  9948 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 25.
I20260709 12:28:03.770037 10016 rpcz_store.cc:276] 0709 12:28:02.043175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.043351 (+   176us) service_pool.cc:224] Handling call
0709 12:28:02.062793 (+ 19442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:28:02.549741 (+486948us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:02.549771 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:28:02.568784 (+ 19013us) log.cc:844] Serialized 52 byte log entry
0709 12:28:02.854112 (+285328us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:28:03.692306 (+838194us) spinlock_profiling.cc:243] Waited 3.54 ms on lock 0x7b18003733c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e090fe4bc 00007f4e0644c48b 00007f4e0f2a69a9 00007f4e0646869b 00007f4e0646693f 00007f4e06466455 00007f4e0cf4e3c7 00007f4e095609a7 00007f4e095700bf
0709 12:28:03.695719 (+  3413us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.695941 (+   222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.696020 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:03.696059 (+    39us) raft_consensus.cc:1685] finished
0709 12:28:03.702420 (+  6361us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.741591 (+ 39171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":76931,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":78909,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":53318,"spinlock_wait_cycles":8144128,"thread_start_us":827233,"threads_started":3,"wal-append.queue_time_us":354920}
W20260709 12:28:03.793094  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:03.789379 10021 rpcz_store.cc:276] 0709 12:28:02.001678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.004927 (+  3249us) service_pool.cc:224] Handling call
0709 12:28:02.028156 (+ 23229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:02.777766 (+749610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.767816 (+990050us) spinlock_profiling.cc:243] Waited 726 ms on lock 0x7b54001f7f54. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:28:03.767994 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1670199552}
W20260709 12:28:03.788810 20213 consensus_meta.cc:287] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: Time spent flushing consensus metadata: real 0.522s	user 0.010s	sys 0.007s
I20260709 12:28:03.750861 20210 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.768002 20437 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.793634 10095 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 19. (term mismatch)
W20260709 12:28:03.848966  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59618.2us
I20260709 12:28:03.849159 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:03.789537 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 168313us
I20260709 12:28:03.755213 20309 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.851151 10025 raft_consensus.cc:2368] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:03.850042  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:28:03.849299  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100957us
I20260709 12:28:03.873322 10099 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:28:03.873922  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:03.874676 10095 pending_rounds.cc:85] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting all ops after (but not including) 18
I20260709 12:28:03.874989 10095 pending_rounds.cc:107] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting uncommitted NO_OP operation due to leader change: 29.19
I20260709 12:28:03.874993  9990 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:03.875380 10095 raft_consensus.cc:2889] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:03.876618 10018 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:03.878353  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.788358 20481 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.881906 10669 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:03.883841 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:28:03.847831 20506 consensus_meta.cc:287] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 1.111s	user 0.013s	sys 0.009s
I20260709 12:28:03.886742  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.888378  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:03.892686  9942 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:28:03.893517  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:03.794708 20424 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 29. Current term is 30.
W20260709 12:28:03.896142 20560 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:28:03.890901 20114 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.781312 20542 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:03.900489  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:03.877938 20429 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:03.780576 20533 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.849944 20473 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.903353  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:03.850201 20503 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.905308 20428 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.912006 20052 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.912380 20436 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:03.913062 20458 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:03.912789 20052 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.913540 20458 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:03.913944 20458 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:03.914486 19841 consensus_meta.cc:287] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.718s	user 0.014s	sys 0.006s
I20260709 12:28:03.914948 19841 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.917053 19841 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.878448 20070 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:03.911813 20542 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.918884 20070 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.919426  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:03.911569 20532 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:03.920673 19981 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:03.879834 20213 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:03.921713 20536 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.924245 20536 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.925415 20213 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.923370 20533 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.926895  9941 raft_consensus.cc:2368] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:28:03.921293  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:03.929339 20436 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:03.929787 10671 raft_consensus.cc:2468] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:28:03.903739  9943 raft_consensus.cc:2368] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27.
I20260709 12:28:03.896714 20309 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.930635 20436 consensus_queue.cc:1059] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:03.788192 20444 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:03.908968  9948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:28:03.931782 20567 log.cc:927] Time spent T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:28:03.933195 20429 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.905267 20235 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:28:03.912083 20557 consensus_queue.cc:1048] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.023s
I20260709 12:28:03.934880 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:03.871536 20414 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.915215 20052 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.933781 20429 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.911717 20530 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.848157 20445 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.850399 20427 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:03.912639 20503 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:03.940338 10022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:36302 (request call id 2009) took 1156 ms. Trace:
I20260709 12:28:03.940340 10099 raft_consensus.cc:1240] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:03.942842 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:03.960608 10090 raft_consensus.cc:2368] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27.
I20260709 12:28:03.961796 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:03.963516 10094 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:03.971140  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:03.958943 10022 rpcz_store.cc:276] 0709 12:28:02.784984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.356240 (+571256us) service_pool.cc:224] Handling call
0709 12:28:03.356511 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.395719 (+ 39208us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:03.395765 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:03.395822 (+    57us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:28:03.395867 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:03.940131 (+544264us) spinlock_profiling.cc:243] Waited 38.9 ms on lock 0x7b5400217ad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:28:03.940176 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:03.940279 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":89554688}
I20260709 12:28:03.900115 20506 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:03.879345 19975 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.027s
I20260709 12:28:03.945161 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:03.937070 10023 consensus_meta.cc:287] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.726s	user 0.009s	sys 0.000s
I20260709 12:28:03.980922 10023 raft_consensus.cc:2468] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:28:03.981487 10023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:36302 (request call id 2013) took 1183 ms. Trace:
I20260709 12:28:03.982163 10020 raft_consensus.cc:1240] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 28. Current term is 29. Ops: []
I20260709 12:28:03.953892 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:03.983011 10098 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:28:03.933245 20444 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:03.882746 20437 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.989280 20536 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:03.981819 10023 rpcz_store.cc:276] 0709 12:28:02.797928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.063113 (+265185us) service_pool.cc:224] Handling call
0709 12:28:03.981427 (+918314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:28:03.996912 10017 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:28:03.992693 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:03.905617 20114 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.002899 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:03.944970 20345 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:28:04.004339  9990 leader_election.cc:400] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27.
W20260709 12:28:04.005831  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.004142 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:04.005853 10024 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:28:03.948134 20155 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.043s
I20260709 12:28:03.961588 20530 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.009068  9990 leader_election.cc:403] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:28:03.945570 20414 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:03.999105 20559 log.cc:927] Time spent T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:28:03.938323 20473 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:03.982357 19976 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:03.994814 20436 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:03.981086 20427 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:04.016173 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 67538.2us
I20260709 12:28:04.051162 10094 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:28:04.045703 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.044996  9987 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:04.009145 20448 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.056442 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:04.057097 10094 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:28:04.058203  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:04.021696 20506 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27.
W20260709 12:28:04.062418 20564 log.cc:927] Time spent T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: Append to log took a long time: real 0.287s	user 0.003s	sys 0.000s
I20260709 12:28:04.069039 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:04.069762 10090 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:28:04.062402 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:04.063751 20414 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.068248 10092 raft_consensus.cc:1275] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:28:04.073238 10096 raft_consensus.cc:1240] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 31. Current term is 32. Ops: [31.20-31.20]
I20260709 12:28:04.051321 20518 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:04.059480 20444 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:04.071969 20210 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.093202 10094 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:28:04.096273 20114 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.054204 19976 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.017503 20530 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.108989 10640 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:04.110213 20530 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:04.110996 10090 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:04.112362 19975 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:04.114126 20413 raft_consensus.cc:3055] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:28:04.118188 20070 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:04.121240 20571 log.cc:927] Time spent T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.068s	user 0.001s	sys 0.001s
I20260709 12:28:04.123647  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:28:04.125131 19976 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.125655 20424 raft_consensus.cc:2804] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:04.126184 20424 raft_consensus.cc:697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:04.127063 20424 consensus_queue.cc:237] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.132452 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.106936 20426 raft_consensus.cc:3055] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:04.081272 20518 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.114534 20413 raft_consensus.cc:740] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:04.178634 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.171597 20503 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.169008 20437 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.165858 20426 raft_consensus.cc:740] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:04.185684  9947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38784 (request call id 2384) took 1266 ms. Trace:
I20260709 12:28:04.186048  9947 rpcz_store.cc:276] 0709 12:28:02.918830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.918972 (+   142us) service_pool.cc:224] Handling call
0709 12:28:02.919313 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:02.983211 (+ 63898us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:28:03.061787 (+ 78576us) log.cc:844] Serialized 14 byte log entry
0709 12:28:03.590300 (+528513us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:03.626639 (+ 36339us) log.cc:844] Serialized 26 byte log entry
0709 12:28:03.626806 (+   167us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:28:03.627031 (+   225us) log.cc:844] Serialized 14 byte log entry
0709 12:28:04.183543 (+556512us) spinlock_profiling.cc:243] Waited 1.68 ms on lock 0x7b1800474820. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e063e3c0a 00007f4e0fb23219 00007f4e0646ba40 00007f4e064664e4 00007f4e06466455 00007f4e0954e060 00007f4e09560716 00007f4e09571287 00007f4e09571208
0709 12:28:04.183702 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.183787 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:04.185453 (+  1666us) raft_consensus.cc:1685] finished
0709 12:28:04.185475 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:04.185622 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":231872,"prepare.queue_time_us":198764,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":2674,"raft.queue_time_us":125442,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":34194,"spinlock_wait_cycles":3855104,"thread_start_us":474017,"threads_started":3,"wal-append.queue_time_us":322031}
W20260709 12:28:04.188135  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 397796us
I20260709 12:28:04.196120 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.196972 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.180598 20473 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.202055 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.203933 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:04.206774 10094 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:04.205232 10673 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:28:04.213488 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:04.214145 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.217137 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.217751 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.218334 10673 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:28:04.185288 20504 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:04.220788 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.213382 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.236364  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.184386 20413 consensus_queue.cc:260] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.221987 20213 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.220913 20533 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.225891 20504 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.240473 20445 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.243497  9941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 3515) took 1196 ms. Trace:
I20260709 12:28:04.244246 20458 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.227026 20427 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.249936  9941 rpcz_store.cc:276] 0709 12:28:03.046880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.263413 (+216533us) service_pool.cc:224] Handling call
0709 12:28:04.243215 (+979802us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400096354. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:04.243394 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480482560}
I20260709 12:28:04.259600  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.259951 20591 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:04.260550 20591 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.261101  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.262979 20413 raft_consensus.cc:3060] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:04.264602  9914 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.237654 20485 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:04.253005 20213 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.267117  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.271252 20213 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.251351 20533 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.224030 20426 consensus_queue.cc:260] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.275066 20426 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:04.267311 20485 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.275898  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.277218 20485 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.266953 20481 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:04.278939 20591 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.266091 20518 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.281162 20504 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.282954  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:04.283628  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:04.277225  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a"
I20260709 12:28:04.267832  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.287891  9942 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:04.293644  9945 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 27: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:28:04.295740 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.296650 10097 raft_consensus.cc:2468] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:28:04.301137 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.302038 10097 raft_consensus.cc:2468] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:28:04.298507 20542 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:04.307353  9946 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:04.310653 20458 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.315358 20589 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:04.315963 20589 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.316354 20589 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:04.293519 20533 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.329589 20213 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:04.336378  9942 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:04.339536 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.341383 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:04.344065 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:04.345600 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.346387 10024 raft_consensus.cc:2468] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
W20260709 12:28:04.337917 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 214298us
I20260709 12:28:04.351382 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.352448 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.351447 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:04.351859  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 62935.9us
I20260709 12:28:04.353205 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.354398 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.355785 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.359182 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:04.359496 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:04.359742 10668 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:04.360162 10018 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:04.360469  9945 raft_consensus.cc:1240] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 34. Current term is 35. Ops: []
I20260709 12:28:04.362073 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.347653 20489 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.365975 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.373265  9944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41126 (request call id 3520) took 1067 ms. Trace:
I20260709 12:28:04.373920  9944 rpcz_store.cc:276] 0709 12:28:03.303040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.320550 (+ 17510us) service_pool.cc:224] Handling call
0709 12:28:04.370660 (+1050110us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400016dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:04.370765 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355661184}
I20260709 12:28:04.375811 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:04.376209 10675 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:28:04.376492 10092 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:04.377099  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:04.379395  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.378631 20510 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:04.383186 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.384626 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.385339 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.388661 10018 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 30: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:04.389148 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.371693 20545 consensus_queue.cc:1059] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.020s
I20260709 12:28:04.392656 20545 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:04.393034 20213 raft_consensus.cc:3055] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:04.393492 20213 raft_consensus.cc:740] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:04.394522 20213 consensus_queue.cc:260] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.395685  9949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41126 (request call id 3516) took 1348 ms. Trace:
I20260709 12:28:04.396088  9949 rpcz_store.cc:276] 0709 12:28:03.047124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.355293 (+308169us) service_pool.cc:224] Handling call
0709 12:28:03.355588 (+   295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.788868 (+433280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.395455 (+606587us) spinlock_profiling.cc:243] Waited 397 ms on lock 0x7b540009bad4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:28:04.395626 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":913898880}
I20260709 12:28:04.393061 20545 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:04.398474 20545 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:04.390450 10676 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:28:04.390419 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.371803 20489 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.404330 20489 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.386780 20510 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.407984 20510 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:04.409565 20070 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:04.409735 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 347410us
I20260709 12:28:04.407994 20155 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:04.410971 20345 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:04.396212 20213 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:04.411489 20345 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.410447 20155 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:04.413307 20155 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.413815 20345 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.416232 20566 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.416776 20566 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.421520 20510 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.422228 20502 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.423285 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.424475  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.426205 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:04.431319 10093 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:04.431808 10095 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 26: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:04.431294 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.390492 10668 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:28:04.432421 10022 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:28:04.438282 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:04.442493 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.378688 20591 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.048s
I20260709 12:28:04.424775 20510 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.449478 10672 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:28:04.456053  9912 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.438017 20533 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:04.464242 20533 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:04.464953 20533 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:04.467783 10019 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45930 (request call id 4459) took 1212 ms. Trace:
I20260709 12:28:04.438273 20426 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.030s
I20260709 12:28:04.468200 10019 rpcz_store.cc:276] 0709 12:28:03.254933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.354165 (+ 99232us) service_pool.cc:224] Handling call
0709 12:28:04.467611 (+1113446us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b540038d1d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:04.467718 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415133952}
I20260709 12:28:04.438905 20529 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:04.470055 20529 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.471097  9949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.471457 10671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:42042 (request call id 2192) took 1267 ms. Trace:
I20260709 12:28:04.471805 10671 rpcz_store.cc:276] 0709 12:28:03.204138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.211892 (+  7754us) service_pool.cc:224] Handling call
0709 12:28:04.471284 (+1259392us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b5400501ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:04.471380 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1573547392}
I20260709 12:28:04.472470 20529 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.473878  9943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.475095  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.470664  9940 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.430039 20566 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.450011 20345 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:04.479205 20345 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.481364 20345 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:04.460691 20413 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:04.482033 20413 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:04.482702 10020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45930 (request call id 4453) took 1272 ms. Trace:
I20260709 12:28:04.482982 20413 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.483150 10020 rpcz_store.cc:276] 0709 12:28:03.209824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.420568 (+210744us) service_pool.cc:224] Handling call
0709 12:28:03.981929 (+561361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:03.994286 (+ 12357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:04.482469 (+488183us) spinlock_profiling.cc:243] Waited 561 ms on lock 0x7b54001c5550. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e095558ad 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:04.482623 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1289971584}
I20260709 12:28:04.485049 20429 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.460707 20282 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.486421 20516 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.486966 20516 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.489037 20516 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.489425  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:04.491006 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:04.493098  9942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.494742  9945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.495599  9945 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:28:04.496834  9946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.501145  9947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
W20260709 12:28:04.493400  9988 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.506009 10092 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:04.506260 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.507200 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.509061  9991 leader_election.cc:400] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 26 election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for earlier term 26. Current term is 27.
I20260709 12:28:04.510464  9943 raft_consensus.cc:1275] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 32 index: 23. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:28:04.515436 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.515858 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.524910 10671 raft_consensus.cc:2468] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:04.517796 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.519116  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.516105 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:04.526769 10668 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:28:04.528358  9987 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:04.527210  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.527671  9991 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:04.535856 19975 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:04.544986  9944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "cd983415389444e292b2330ef630850a" is_pre_election: true
I20260709 12:28:04.545382  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:04.547581 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.508067 20451 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:04.542367  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.507989 20426 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:04.554859 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.556056 10025 raft_consensus.cc:2393] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:04.555652 10668 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:04.559544  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.559312  9987 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:28:04.564425 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 152505us
I20260709 12:28:04.566926 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:04.567549  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.562762  9991 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.568635 20538 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:04.569274 20538 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:04.569290  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.569907 20538 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:04.570464 20592 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:04.570647  9912 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:04.571449 20542 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:04.571910 20542 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:04.572780 20542 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.534241 20591 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:04.534089 20489 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:04.563784 19975 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.016s
I20260709 12:28:04.511157 20429 consensus_queue.cc:1059] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:28:04.590945 19976 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:04.591580 19976 raft_consensus.cc:697] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:04.591619  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.592676  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:04.592602 19976 consensus_queue.cc:237] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:461: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 5
I20260709 12:28:04.554862 20213 raft_consensus.cc:2804] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:04.599323 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.602895  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.605564 10092 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:28:04.544790 20518 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:04.611506 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.595570  8743 tablet_server.cc:179] TabletServer@127.8.137.193:0 shutting down...
I20260709 12:28:04.618901 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.576771 20458 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:04.575979 20516 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:28:04.622076 10097 raft_consensus.cc:2368] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:28:04.597940 20540 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:04.623481 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.597923 20534 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:04.629856 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.596931 20213 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:04.595919 20444 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:04.594995 20503 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:04.628495 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.610457 20428 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:04.633421 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.608788 20489 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:04.576484 20445 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.048s
I20260709 12:28:04.620586 20070 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:04.594568 20545 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.658150  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.659332 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.638940 20213 raft_consensus.cc:3060] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:04.660656  9914 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:04.663353 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:04.621354 20518 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.662904 20540 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:04.672412 20489 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.672919 20538 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.639734 20503 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:04.641494 20534 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:04.669955 20533 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.688536 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.687040 10065 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.690030  9914 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:04.654868 20114 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:04.685606 20503 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:04.684005 20540 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:04.741427 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:04.743533 20603 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:28:04.736346 20602 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:28:04.743794 20600 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:28:04.736297 20584 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:04.747777 20485 raft_consensus.cc:2804] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:04.751149 20545 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.797135 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.733026 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:28:04.736290 20601 log.cc:927] Time spent T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:28:04.805142 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:04.806031  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 211915us
I20260709 12:28:04.806516 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:04.754923  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 64343.5us
I20260709 12:28:04.798818 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.799800 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.808808 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.799898 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.800596 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:04.801519 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 69172us
I20260709 12:28:04.802691 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.809643 10668 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:28:04.810465 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.811065 10023 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:28:04.754660 20602 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.732661 20566 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:04.818504 10062 leader_election.cc:400] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:04.819574 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.826083 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.823514 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:04.827330 10021 raft_consensus.cc:1240] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:28:04.827735 10016 raft_consensus.cc:2410] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for term 32 because replica has last-logged OpId of term: 31 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 19.
I20260709 12:28:04.822821 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:04.824927 10065 leader_election.cc:400] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:28:04.826692 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.828456 10065 leader_election.cc:403] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:04.828934 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.763019 20485 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.829504  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:04.830298 10062 leader_election.cc:403] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:04.740841 20114 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.837024 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:04.838419 20114 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:04.782691 20533 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:04.815492 20070 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:04.823117 20602 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:04.840246 10065 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:04.845038 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.833796 20536 raft_consensus.cc:2749] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer cd983415389444e292b2330ef630850a (127.8.137.193:34909) with higher term. Message: Invalid argument: T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27.
I20260709 12:28:04.854735 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.859519 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.779538 20589 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.867025 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.768389 20584 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:04.848287 20114 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:04.818212 20426 consensus_queue.cc:1048] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.029s
I20260709 12:28:04.835709 20538 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:04.822093 20213 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.841383 20485 raft_consensus.cc:3060] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:04.863950 10017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45922 (request call id 3216) took 1263 ms. Trace:
I20260709 12:28:04.886045 10017 rpcz_store.cc:276] 0709 12:28:03.615840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:03.686904 (+ 71064us) service_pool.cc:224] Handling call
0709 12:28:04.863783 (+1176879us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54002666d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:04.863898 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554412416}
I20260709 12:28:04.882416 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:04.847342 20070 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:04.855743 20481 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:04.833575 20566 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:04.860956 20575 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:04.844815 20427 consensus_queue.cc:1059] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: INVALID_TERM, Last received: 34.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.039s
W20260709 12:28:04.921495 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 98407.8us
I20260709 12:28:04.926321 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:05.046155 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:05.047151 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 120933us
I20260709 12:28:05.050731 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:05.051656 10669 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:28:04.865890 20517 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:28:04.893961 20070 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:05.095289 20451 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:04.901175 20575 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.123588 19975 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:04.845700 20534 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 26.21, Last appended by leader: 21, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:05.052668 20587 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.793s	user 0.000s	sys 0.009s
W20260709 12:28:05.156984 20611 log.cc:927] Time spent T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:28:05.157430 20213 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.165050 19975 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.170132 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:05.176762  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 328074us
I20260709 12:28:05.181270 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:28:05.181348 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 331005us
W20260709 12:28:05.171932 20607 log.cc:927] Time spent T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
I20260709 12:28:05.137576 20451 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:05.177245 20605 log.cc:927] Time spent T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.362s	user 0.001s	sys 0.000s
I20260709 12:28:05.053540 20538 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:05.153447 20575 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:05.219370 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:05.219697 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:05.223940 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:05.211908 20451 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:05.254841 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 205833us
I20260709 12:28:05.220695 19975 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.255775 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:05.218468 20584 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.264925 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:05.226679 20538 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.284171 20617 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:05.299763 20616 log.cc:927] Time spent T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:28:05.286651 20589 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.308847 20566 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:05.344580 20436 raft_consensus.cc:3055] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:05.345068 20436 raft_consensus.cc:740] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:05.346388 20485 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:05.348207 20606 log.cc:927] Time spent T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.601s	user 0.009s	sys 0.036s
I20260709 12:28:05.349862 20485 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.345981 20436 consensus_queue.cc:260] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.351409 20436 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:05.355674 20602 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.358527 20602 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.364876 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:05.362903 20155 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:05.363332 10669 raft_consensus.cc:1275] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
W20260709 12:28:05.370823 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 115124us
I20260709 12:28:05.371601 10642 leader_election.cc:400] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:28:05.371932 10642 leader_election.cc:403] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:28:05.370260  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 105524us
I20260709 12:28:05.383304 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:05.385499 20424 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983) with higher term. Message: Invalid argument: T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28.
I20260709 12:28:05.412599 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.388919 19841 consensus_queue.cc:1048] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.018s
I20260709 12:28:05.409386 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:05.430883 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:05.434517 20213 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:05.434283 20549 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:05.414232 20533 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:05.457815 20533 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:05.450398 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89480.5us
I20260709 12:28:05.449478 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:05.434866 20538 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:05.449550 20503 consensus_meta.cc:287] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: Time spent flushing consensus metadata: real 0.692s	user 0.006s	sys 0.005s
I20260709 12:28:05.466579 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:05.448516 20463 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:05.455818 20529 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:05.467527 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 108747us
I20260709 12:28:05.471102 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:05.467571 20617 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:05.448988 20070 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.459610 20213 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.473459 20623 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:05.513016 10065 consensus_peers.cc:597] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.513825 10065 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.514496 10065 leader_election.cc:336] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.515241 10065 consensus_peers.cc:597] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.515722 10065 leader_election.cc:336] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.516374 20213 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.517395 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.517686 20592 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.518226 20592 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:05.519337 10065 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.520040 10065 leader_election.cc:336] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:05.520383 20592 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:05.520725 10065 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.521399 10065 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.522060 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.489624 20529 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:05.527267  9827 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 9aa345bcd43b44088114b7b6f2830b17 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:05.528182 20451 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.464674 20549 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.531627 10099 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:28:05.533195 20549 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.533761 20549 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:05.534162 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:05.534606  9991 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.709s	user 0.021s	sys 0.031s
I20260709 12:28:05.468871 20503 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:05.534906  9991 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.022s	sys 0.031s
W20260709 12:28:05.535202  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 709819us
I20260709 12:28:05.536558 20503 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.548923 10098 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:28:05.550583 20070 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.557448 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.557551 20575 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.557718 20533 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.530359 20451 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.565977 20575 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:05.591799 20481 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:05.592419 10065 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.592458 20481 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.594739 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:28:05.593222 10065 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.596341 10065 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.596997 10065 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.597112 20481 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.598321 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.602313 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.603144 10094 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 31: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
W20260709 12:28:05.598285 10065 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.604229 10065 leader_election.cc:336] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.604935 10065 consensus_peers.cc:597] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.605441 10065 leader_election.cc:336] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.606015 10065 leader_election.cc:336] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:05.606487  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 82283.4us
W20260709 12:28:05.607033 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 136071us
I20260709 12:28:05.607937 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:05.610616  9991 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:05.616514  9990 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:05.616099  9991 consensus_peers.cc:597] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.615983 20583 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:05.620194 20445 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:05.620348  9990 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:05.620604 20069 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:05.620836 20445 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.621109 20069 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.621662  9990 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:05.622822 20506 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:05.623329  9990 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:05.623556 20069 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.624078 20069 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:05.624181 20426 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:05.624583 20426 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:05.624950 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:05.625349 20426 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:05.626407 20503 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:05.630666  9990 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:28:05.637061 10065 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.631839 19841 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:05.627907 20445 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.642421 20489 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.647280  9990 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:05.648638 20503 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:05.650223  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:05.651278 20534 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:05.652549  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:05.623343 20506 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.654165 20489 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.636113 20444 consensus_queue.cc:1048] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 12:28:05.658913 19841 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:05.646740 20445 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:05.674813 19841 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.675041 20445 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:05.675315 19841 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:05.675987 20445 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.680163 20506 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.681282 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:05.682010 10677 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:05.683795 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:05.684429 10024 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:05.686131 10025 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:28:05.686976 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:05.687237 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:05.687438 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:05.688063 10673 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:05.688678 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:05.636920 20069 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.714233 20069 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.703197 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:05.703723 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:05.739369 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:05.705695 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:05.706686 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:05.707396 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:05.709347 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:05.712174 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:05.743162 10017 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 26.
I20260709 12:28:05.701908 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:05.710445 20489 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.719655 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:05.733371 20414 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:28:05.734685  9987 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:05.744468 10016 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:28:05.761144 10670 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:05.765347 20629 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:05.765892 20629 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.766448 10024 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:28:05.766825 10677 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:28:05.767988 20629 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.768721 10673 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:28:05.771149 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.773204 20629 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:28:05.775242  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:05.775993 20629 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:05.777529 20426 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.780957  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:05.782148 20516 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:05.783295  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:05.784610 20516 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:05.785924  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:05.788899  9912 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:05.796643  9912 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:05.798153  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:28:05.802531 10643 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.803663  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 145928us
I20260709 12:28:05.749181 20529 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:05.780965 20518 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:05.764153 20549 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:05.763180 20510 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:05.764988 20069 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:05.804962 20591 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:05.804785 20213 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:05.803256 20629 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:05.826723 20629 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:05.826836 20213 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:05.827178 20629 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:05.828243 20213 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.805836 20538 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:05.821985  9990 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:05.822127 20510 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.842406 20480 raft_consensus.cc:3055] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:28:05.842828 20480 raft_consensus.cc:740] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:05.843686 20480 consensus_queue.cc:260] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 34.21, Last appended by leader: 21, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.844712 20624 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:05.844909 20480 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:05.759828 20414 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.842216 10017 raft_consensus.cc:1240] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 27. Current term is 28. Ops: []
I20260709 12:28:05.850219 10018 raft_consensus.cc:1275] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:05.837778 10096 raft_consensus.cc:1240] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:05.816166 20529 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.853638 20426 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.844228 20510 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.860199 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:05.875870 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.861310 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:05.861543 10016 raft_consensus.cc:1240] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 26. Current term is 27. Ops: []
I20260709 12:28:05.861723 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:05.877288 10020 raft_consensus.cc:1240] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:05.877785 10022 raft_consensus.cc:3060] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:05.863068 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:05.879448 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:05.874243 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:05.875070 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 55532.6us
I20260709 12:28:05.881197 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:05.882998 10023 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 31: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:05.892103 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.820751 20549 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.876281 20575 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.876027 20538 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:05.895758 20643 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:05.889079 20518 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:05.937625  9991 consensus_peers.cc:597] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:05.938623  9991 consensus_peers.cc:597] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.836133 20069 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.905709 20445 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.907415 20575 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.920179 20549 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.922106 20529 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.943570 20445 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.945221 20575 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.885228 19841 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.946262 20445 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.923760 20414 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:05.929962 20538 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.889464 20516 consensus_queue.cc:1059] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.017s
I20260709 12:28:05.962870 20516 raft_consensus.cc:3055] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:05.963301 20516 raft_consensus.cc:740] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:05.931114 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:05.895183 20448 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:28:05.943552  9991 consensus_peers.cc:597] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.947136 10093 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:05.929749 20647 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:28:05.982435  9991 consensus_peers.cc:597] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:05.935241 20643 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:05.986572 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:05.987234 10096 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:05.987864 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.988929 20643 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:05.989974 20510 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.990478 20510 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.992583 20510 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:05.957368 10022 raft_consensus.cc:2468] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 31.
I20260709 12:28:05.994385 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:05.994647 20436 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:05.995076 10094 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:28:05.995136 20436 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:05.996814 20629 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:05.997376 20436 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.956436 19841 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.008756 20629 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.012938  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:06.091554 19841 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:06.092015 19841 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.092367 19841 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:06.016048 10096 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:05.985471 20451 consensus_queue.cc:1048] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.044s
I20260709 12:28:05.982470 20584 consensus_queue.cc:1059] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.067s
I20260709 12:28:06.042872 20633 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:06.048235  9991 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.100332  9991 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:06.101377 10062 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:06.055150 20510 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.056905 20413 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:06.062001 20069 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:05.985291 20448 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.064610 20538 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:06.075009 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 67482.7us
I20260709 12:28:06.076360 20516 consensus_queue.cc:260] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:06.091454  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78618.9us
I20260709 12:28:06.129977  9990 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:06.014263 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.102387 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:06.131616 10095 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 31: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:06.103965 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:06.104635  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 299613us
I20260709 12:28:06.132386  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:06.104612 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 116853us
I20260709 12:28:06.134483 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:06.135468 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:06.136068 10670 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:28:06.108007 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.111931 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:06.137104 10673 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:06.118158 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:06.116217 10062 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:28:06.135577 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:06.101249  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 53092.1us
I20260709 12:28:06.145285 10062 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:06.147732 10094 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:28:06.149401 10096 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:28:06.152347 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:06.153225 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:06.116256 20647 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:06.144969  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:06.116479 20480 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.156291 10677 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 21. Preceding OpId from leader: term: 28 index: 22. (index mismatch)
I20260709 12:28:06.152045 19975 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:06.114892 20591 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.159708  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:06.162348 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.163545 10090 raft_consensus.cc:1275] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:06.155167 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:06.120190 20427 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:06.178848  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:06.170639 10670 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
W20260709 12:28:06.161530 10065 consensus_peers.cc:597] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:06.178227  9991 consensus_peers.cc:597] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.151969 20602 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:06.160645 20538 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:06.160478 20448 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:06.183233  9991 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.195874  9991 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:06.118889 20510 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.168880 20575 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:06.196609  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:06.121470 20413 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.116017 20633 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.171380 20589 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.202404  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:06.152598 20069 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:28:06.180934 20542 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:06.180214 19975 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:06.183748 20481 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:06.211314 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 56231.4us
I20260709 12:28:06.229501 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:06.174474 20429 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.029s
W20260709 12:28:06.226890  9991 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.232775  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:06.206759 10643 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.233608 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:06.195724 20624 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:28:06.237869 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:06.230659 10062 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:06.232234 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:06.184656 20516 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:28:06.235005  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 58156.6us
I20260709 12:28:06.201257 20629 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:06.180478 20591 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.223073 20545 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:06.225265 20070 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:06.179695 20480 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.241729 20569 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.258467 20569 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.248935 20448 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:06.260632 20569 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.263453 20413 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.264132 19975 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:06.255391  9991 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.244865 20444 consensus_queue.cc:1048] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.013s
I20260709 12:28:06.259626 20463 raft_consensus.cc:3055] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:28:06.266343  9838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 328) took 13880 ms (client timeout 15000 ms). Trace:
I20260709 12:28:06.272089 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.272758 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.267659  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:06.245322 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:06.240481 20549 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:06.239008 20420 raft_consensus.cc:2764] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:28:06.239368 20436 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:06.239491 20485 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:06.285985 20485 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.289299 20485 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.290360 20643 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.291832 10673 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
I20260709 12:28:06.292847 20643 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.277482 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:06.301055 10096 raft_consensus.cc:2393] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 27: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:06.278059 10097 raft_consensus.cc:2393] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 32: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:06.250761 20584 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:06.237138 20481 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.304219 20481 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.267004 20445 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:06.308001 20420 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.308578 20420 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.309299 10097 raft_consensus.cc:1240] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:06.285362 20591 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.225050 20589 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.316533 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:06.272140  9838 rpcz_store.cc:269] 0709 12:27:52.386093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:59.018097 (+6632004us) service_pool.cc:224] Handling call
0709 12:28:06.266258 (+7248161us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:03.355445 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:03.356030 (+   585us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:03.356067 (+    37us) write_op.cc:435] Acquired schema lock
0709 12:28:03.356085 (+    18us) tablet.cc:662] Decoding operations
0709 12:28:03.381912 (+ 25827us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:03.382002 (+    90us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:03.382035 (+    33us) tablet.cc:685] Acquiring locks for 34 operations
0709 12:28:03.567880 (+185845us) tablet.cc:701] Row locks acquired
0709 12:28:03.567909 (+    29us) write_op.cc:260] PREPARE: finished
0709 12:28:03.568081 (+   172us) write_op.cc:270] Start()
0709 12:28:03.568228 (+   147us) write_op.cc:276] Timestamp: P: 1783600083568025 usec, L: 0
0709 12:28:03.568250 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:28:03.569018 (+   768us) log.cc:844] Serialized 13383 byte log entry
0709 12:28:04.182871 (+613853us) op_driver.cc:464] REPLICATION: finished
0709 12:28:04.214721 (+ 31850us) write_op.cc:301] APPLY: starting
0709 12:28:04.214800 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:04.460645 (+245845us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:04.460743 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:04.672202 (+211459us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:04.672631 (+   429us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=68,key_file_lookups=68,delta_file_lookups=0,mrs_lookups=0
0709 12:28:04.672690 (+    59us) write_op.cc:312] APPLY: finished
0709 12:28:04.673253 (+   563us) log.cc:844] Serialized 292 byte log entry
0709 12:28:04.673431 (+   178us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:04.673716 (+   285us) write_op.cc:454] Released schema lock
0709 12:28:04.673785 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:04.674211 (+   426us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:04.674241 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7546,"reported_tablets":47,"child_traces":[["op",{"apply.queue_time_us":31424,"apply.run_cpu_time_us":44581,"apply.run_wall_time_us":459443,"cfile_cache_hit":138,"cfile_cache_hit_bytes":146315,"num_ops":34,"prepare.queue_time_us":222875,"prepare.run_cpu_time_us":14696,"prepare.run_wall_time_us":336469,"raft.queue_time_us":84386,"raft.run_cpu_time_us":23944,"raft.run_wall_time_us":31895,"replication_time_us":614446,"spinlock_wait_cycles":130944,"thread_start_us":351387,"threads_started":4,"wal-append.queue_time_us":179543,"wal-append.run_cpu_time_us":52336,"wal-append.run_wall_time_us":1937686}]]}
I20260709 12:28:06.271868 20463 raft_consensus.cc:740] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:06.350296 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.351475 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:28:06.351884  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 96524.2us
I20260709 12:28:06.355497 20624 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:06.355209 20463 consensus_queue.cc:260] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.356639 20463 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:06.370713 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:06.371580 10024 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 30: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:28:06.371850 20653 log.cc:927] Time spent T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:28:06.369266 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.315503 20549 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
W20260709 12:28:06.377842  9991 leader_election.cc:336] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.382997  9991 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:06.380321 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 177568us
I20260709 12:28:06.374725 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.338716 20436 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:28:06.389360  9991 leader_election.cc:336] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:06.385159 10643 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:06.399840 10643 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:06.401125 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.393476 10062 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.350826 20545 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:06.398288  9991 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:28:06.396539  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 188410us
W20260709 12:28:06.402757  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 97522.4us
I20260709 12:28:06.350289 20518 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:06.373476 20650 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:06.372352 20542 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.355538 20502 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.376646 20569 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.395397 20448 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:06.428874 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:06.383081 20575 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:06.444850 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:06.404202 10643 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.440349 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.446282 10098 raft_consensus.cc:2468] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:28:06.402686 20489 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:06.447724  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:06.455418 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:06.430971 20602 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:28:06.481746  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 64278.7us
I20260709 12:28:06.412971 20436 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.459582 20538 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.496381 10093 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:28:06.468065 20428 raft_consensus.cc:2804] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:06.430718 20518 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.433594 19841 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:06.509233 20655 log.cc:927] Time spent T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
I20260709 12:28:06.510264 19841 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:06.487721  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46151.4us
I20260709 12:28:06.486271  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:06.512274 10092 raft_consensus.cc:3055] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:06.512729 10092 raft_consensus.cc:740] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:06.513607 10092 consensus_queue.cc:260] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.522040 20420 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.500799 20538 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.524806 20538 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.495170 20429 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.030s
I20260709 12:28:06.488405 20510 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.527227 20589 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.527981 10092 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:06.527666 20545 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:06.529208 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:06.529654 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 82095.8us
I20260709 12:28:06.495716 20533 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.489082 20504 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.530416 20533 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.530957 20504 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.532627 20533 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.453225 20542 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.533587 20504 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.534621 20542 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.454427 20569 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.537309 20569 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.498178 20549 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.538443 20619 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.538502 10092 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:28:06.539158 20619 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.541607 20619 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:06.542846 20654 log.cc:927] Time spent T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 12:28:06.497896 20633 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.437526 20502 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:06.555075  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 43819.1us
W20260709 12:28:06.530309 10643 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.558634  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:28:06.559576  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 72088.3us
I20260709 12:28:06.561354 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:06.562011 10092 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:06.562901  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 173584us
W20260709 12:28:06.565488 10643 leader_election.cc:336] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:06.566262 10643 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:06.580271 10643 leader_election.cc:336] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.507758 20428 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:06.586954  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 101041us
I20260709 12:28:06.575929 20591 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:06.600092 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 206720us
I20260709 12:28:06.582324 20664 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:06.600133 20480 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:06.601266  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 85124.6us
I20260709 12:28:06.602685 20436 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.605022 20213 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:06.605103  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 46579.3us
I20260709 12:28:06.609038 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:06.617498 10643 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.593488 20429 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:06.678562  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:06.703047 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.671608 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.640547 10062 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.688442 10092 raft_consensus.cc:2468] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 33.
W20260709 12:28:06.693307 10064 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:06.673187 10021 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:06.707768 10097 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 35.
I20260709 12:28:06.690531 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.662135 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:06.709993 10016 raft_consensus.cc:2393] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 31: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:06.711182 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.711460 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.712294 19841 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:06.712808 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 72802us
I20260709 12:28:06.713570 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.714681 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.707166  9990 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:06.717643 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:06.710398 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 93057.7us
I20260709 12:28:06.711134 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.673480 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.720288 10018 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:06.714150 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.716809 10025 raft_consensus.cc:2468] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 34.
I20260709 12:28:06.721323 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.724335 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:06.725849 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:06.618999 20428 raft_consensus.cc:3060] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:06.720620 10643 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.723544 10097 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
W20260709 12:28:06.736682  9991 consensus_peers.cc:597] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.741328 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.726387 20448 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:06.720504 20420 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:06.743916  9991 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:06.726686 20069 raft_consensus.cc:2804] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:06.747295 20662 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:06.638916 20591 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.748406  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:06.748104 20662 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.730840 20155 consensus_queue.cc:1059] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.020s
I20260709 12:28:06.750308 20504 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:06.698773 20660 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:06.752566 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.757606  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.716459 20633 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:06.763087 20481 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:06.763222 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.742821 20436 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:06.764596 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.639549 20445 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.132s
I20260709 12:28:06.727097 20480 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:06.784518 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.727489 20619 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:28:06.788009 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.768560  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.753501 19975 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.747341 20069 raft_consensus.cc:697] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:06.753158 20575 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:06.751104 20504 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.795362 20504 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:06.797369 20633 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:06.798672 20069 consensus_queue.cc:237] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:06.802733 20669 log.cc:927] Time spent T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:28:06.712839 19841 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.804582 10017 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:28:06.805915 19841 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.794019 19975 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.822423 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.823465 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:06.823477  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 55028.3us
I20260709 12:28:06.836732 10099 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:28:06.764461 20436 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.760969 20660 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.775115 20481 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
I20260709 12:28:06.815128 20662 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.837615 19975 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.846346 20591 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.848946 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:06.837960 20666 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:28:06.839279 20502 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.851266 10025 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
W20260709 12:28:06.875069 20668 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:28:06.884438 20436 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:06.889508  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 63488.3us
I20260709 12:28:06.891891  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:06.902442 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:06.917850 10025 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:28:06.921974 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 36 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:06.919763 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 185672us
I20260709 12:28:06.922822 10099 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 35.
I20260709 12:28:06.917054 20545 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:06.943558 10643 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.919422 20619 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:28:06.949584 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 207379us
W20260709 12:28:06.952075 10643 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:06.943380 20569 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:06.956825 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:06.954851  9914 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:28:06.966008 20675 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:28:06.969112 20672 log.cc:927] Time spent T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:28:06.971045 20674 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 12:28:06.972569 20673 log.cc:927] Time spent T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:28:06.975023 20660 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.960080 20569 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.977512 20569 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:06.957712 20516 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:06.986835 20516 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:06.988935 20516 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:06.959903 20545 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:28:06.990096 20545 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:06.990902 20545 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:28:06.994948 20463 consensus_meta.cc:287] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.005s
I20260709 12:28:06.952248 20619 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:06.998265 20619 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.002022 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.002732 10093 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:28:07.005919 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.009053 20518 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.011315 20619 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:07.011868 20619 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.013850 20619 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.020396 20504 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.024264  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:07.026167 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.027245 20591 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:07.027736 20591 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.028069 20591 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:07.028214  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 283152us
W20260709 12:28:07.028894  9991 leader_election.cc:336] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.029299  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:07.029816 20428 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.030961 20660 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:07.032097 20428 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.036824  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:07.037396 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:07.037899 10093 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:28:07.038232 10093 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:07.040218  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.041107  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.041320 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.042650 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 36 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.043237 10020 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 34.
I20260709 12:28:07.043391 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.042280 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.038956 10093 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.18, Last appended by leader: 18, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.045742 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.045993 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:07.046347 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:07.047191  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.048628  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.049425  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:07.050735 10023 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:07.051458 10093 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:07.052307 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.053609  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.055563 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.056589  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:07.055387 10643 leader_election.cc:336] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.061714 10091 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:37528 (request call id 3711) took 1177 ms. Trace:
I20260709 12:28:07.045265 20619 raft_consensus.cc:2804] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:28:07.046825 20542 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:07.054394 20413 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:07.049889 20428 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:07.070899 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:07.071514 10096 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 32
W20260709 12:28:07.061673 10643 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.060582 20569 raft_consensus.cc:3060] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:07.061956 20549 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:07.062333 10091 rpcz_store.cc:276] 0709 12:28:05.880408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:05.880527 (+   119us) service_pool.cc:224] Handling call
0709 12:28:05.895638 (+ 15111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:06.160799 (+265161us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:28:06.279992 (+119193us) log.cc:844] Serialized 14 byte log entry
0709 12:28:06.502655 (+222663us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:06.502732 (+    77us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:28:06.502806 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:07.057256 (+554450us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540015bdd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:28:07.057302 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:07.057421 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2723,"prepare.queue_time_us":197585,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":63,"raft.queue_time_us":2869,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":124,"spinlock_wait_cycles":609308032,"thread_start_us":334844,"threads_started":2,"wal-append.queue_time_us":464118}
I20260709 12:28:07.051115 20516 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:07.065368 20619 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:07.049750 20649 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:07.086129 10093 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:28:07.075014 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:07.067431 20413 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.056926 20485 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.019s
I20260709 12:28:07.070016 20504 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.086710 20619 raft_consensus.cc:3060] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:28:07.093464 20684 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:07.087996 20545 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 36 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.095846  9912 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.098071 20650 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:28:07.098443 10065 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.115247 10676 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:07.107898 10096 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:28:07.089815 20413 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:07.117161 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 165773us
I20260709 12:28:07.118013  9912 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.122256  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.124112 20649 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:07.140146 10643 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:07.116681 20575 raft_consensus.cc:3055] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:28:07.146495 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:07.148453 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:07.149070 20602 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:07.149289 20155 consensus_queue.cc:1059] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:07.150043 20602 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:07.124396 20504 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:07.124042 20510 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:07.149523 10062 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:07.165899 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.168167 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.170331 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.171499 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.150483 20155 raft_consensus.cc:3055] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:07.143817 20575 raft_consensus.cc:740] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:07.138935 20445 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.037s
I20260709 12:28:07.190300 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:07.172696 20602 raft_consensus.cc:2804] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:07.190949 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.192041 10677 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 29. Current term is 30. Ops: []
I20260709 12:28:07.195614 10677 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Rejecting Update request from peer 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:07.196116 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 26 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.197253 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.197917 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.197333 10062 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:07.179939 20155 raft_consensus.cc:740] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:07.121564 20650 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.198962 20480 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:07.232615 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.201273 20602 raft_consensus.cc:697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:07.235795 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.219923 20643 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:07.222851 20420 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:28:07.241801 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 103809us
I20260709 12:28:07.238106 20427 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:07.237789 20680 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:07.254438 20591 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.261380 20510 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:07.261756 20591 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.230259 20575 consensus_queue.cc:260] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.244856 20643 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.264734 20545 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 36 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.253588 20155 consensus_queue.cc:260] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.239763 20686 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.031s
I20260709 12:28:07.278026 20575 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:07.288203 20569 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:07.289142 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:07.276069 20602 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.291096 20428 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:07.299592 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 36 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:07.303022 10021 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Advancing to term 36
I20260709 12:28:07.313796 10021 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 36.
I20260709 12:28:07.314368 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 36 candidate_status { last_received { term: 34 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:07.314999 10096 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:28:07.315060 10090 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:07.315454 10090 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:07.316160 10090 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.317385 10090 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:07.301270 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:07.305550 10065 leader_election.cc:336] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.319067 10065 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:07.317919 20463 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:07.319705  9914 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.319998 20463 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.321655 20428 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:07.323480 20569 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:07.323657 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.325598  9912 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:07.324071 20569 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 32 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:07.326721 20584 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.327267 20584 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.327905 20510 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:28:07.328392 20510 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 36 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:07.329453 20584 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.329221 20510 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.21, Last appended by leader: 21, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:07.330097 10643 leader_election.cc:336] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.332105 20643 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.301112 20155 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:07.300148 20650 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:07.334901 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:07.335386 20680 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:07.335536 10099 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:07.327437 20569 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:07.340939 10643 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.302214 20619 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.344971 10096 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 36.
I20260709 12:28:07.322975 20463 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.345959 20649 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.351550 20649 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.353603 20649 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.354137 10090 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:28:07.347554 20575 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:07.354765 19841 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:07.354861 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.345115 10099 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:28:07.355389 19841 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.357410  9914 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.358381 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:07.359485 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:07.367148 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.368511  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:07.362640 10643 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.369881  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.358220 20445 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:07.369877 20619 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.372793 20481 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:28:07.373665 20428 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:07.372277 20545 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:07.373344 20649 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.014s
I20260709 12:28:07.390806 19841 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.386061 20445 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:07.409054 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:07.409091 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:07.409502 10065 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.409960 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:07.414500 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.418836 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.422236 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.422976 10090 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 31.
I20260709 12:28:07.424285  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.424691 20155 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:07.425159 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:07.426539 20691 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.164s	user 0.003s	sys 0.000s
I20260709 12:28:07.426805 20428 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:07.434139 20683 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:07.434757 20683 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.437879 20683 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.438719 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:07.443849 20518 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:07.444933 20549 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.445464 20549 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.447345 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:07.449609 20617 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:07.448171 20549 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.460426 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.462826 10098 raft_consensus.cc:2393] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 32: Already voted for candidate cd983415389444e292b2330ef630850a in this term.
I20260709 12:28:07.463703 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.464407 10090 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
W20260709 12:28:07.465759 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 56289.4us
I20260709 12:28:07.470033 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.473395 20445 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.477504 10098 raft_consensus.cc:1240] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:28:07.394062 20545 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:07.492142 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.489524 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.495877 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.496487 10090 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:28:07.500653  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:07.474766 20413 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.506887 20413 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.482470 20631 consensus_queue.cc:1059] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.022s
I20260709 12:28:07.508198 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.497181 20650 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:07.510537 20650 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.512799 20650 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:07.517539 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.521037 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:07.527020 20545 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.527400 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:07.533541 10065 leader_election.cc:336] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.534723 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.533991 10065 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:07.535418 10023 raft_consensus.cc:2468] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
I20260709 12:28:07.536312 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:07.536897 10098 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 29
W20260709 12:28:07.536753  9991 consensus_peers.cc:597] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:07.538182  9991 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:07.539093  9991 leader_election.cc:336] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.540535 10642 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.523411 20649 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:07.543066 20649 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.536803 20631 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:07.547224 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
W20260709 12:28:07.543017  9991 leader_election.cc:336] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.547844 10023 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:07.543781 20686 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:07.548396  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:28:07.551791  9991 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:07.552392  9991 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:07.552912  9991 leader_election.cc:336] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:07.554808  9991 leader_election.cc:336] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.543529 20649 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:07.555456 10642 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:07.553606 20069 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:07.557415 20463 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:07.567591 20463 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:07.568542 20463 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.575140 20617 raft_consensus.cc:3055] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:07.575676 20617 raft_consensus.cc:740] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:07.584918 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:07.586632 20584 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:07.593680 20619 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:07.594928  9991 leader_election.cc:336] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.592060 10023 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:28:07.595367  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:07.596943  9912 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:07.594223 20619 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.598984 20584 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:07.599646 20584 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:07.600047 20584 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:07.600867 20619 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.602522 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:07.603438 20703 log.cc:927] Time spent T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:28:07.605820 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:07.594726 20448 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 31 current term: 31 is higher.
I20260709 12:28:07.611804 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:07.610008  9991 consensus_peers.cc:597] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:07.612274 20684 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:07.610958 20069 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:07.629922 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:07.630715  9991 leader_election.cc:336] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.632894 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.633481 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:07.633805 10643 leader_election.cc:336] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.636461 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.609884 20619 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:07.639676 20619 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:07.631978  9991 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:07.644012 19975 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:07.644697 20619 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.646193 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.664314 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.659380 20662 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:07.685333  9990 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:07.689694 10098 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
W20260709 12:28:07.686357  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 146272us
I20260709 12:28:07.691637 20647 raft_consensus.cc:2764] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 36 FOLLOWER]: Leader pre-election decision vote started in defunct term 34: won
I20260709 12:28:07.772210 20155 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:07.782037 20070 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:07.793133 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.582790 20617 consensus_queue.cc:260] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:07.797569 20705 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:28:07.687256 20662 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:07.807224  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 100463us
I20260709 12:28:07.810463 10668 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 29 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:28:07.812291 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.792583 20619 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:07.810544 20649 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.821552 20649 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.803289 20504 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:07.827142 20504 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:07.827492 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:07.828097 10098 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:07.829545 20504 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.832155 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.798481 20070 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.836426 20070 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.839998 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.846239 20633 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.846835 20633 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.848898 20633 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.792313 20155 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.851017 20155 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.853091 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.854936 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.855782 10023 raft_consensus.cc:1275] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:28:07.857523 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.858366 10099 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
I20260709 12:28:07.861088 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:07.863307 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:07.876173 10098 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:28:07.878388 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:07.884111 20070 consensus_queue.cc:1048] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:07.885007 20662 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.817968 20619 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.891606 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:07.871213 10065 leader_election.cc:336] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:07.895197 10065 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:07.897128 10024 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 36 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 34 index: 21. Preceding OpId from leader: term: 36 index: 22. (index mismatch)
I20260709 12:28:07.898708 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:07.903743 20619 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.908200 20584 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:07.910864  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:07.912371 20584 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:07.912499 20649 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:07.912972 10677 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:07.914155  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
W20260709 12:28:08.003544  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 89466.7us
I20260709 12:28:08.004261  9914 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:08.004966 20569 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:07.916985 20633 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:08.005403 20428 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:08.005939 20428 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 32 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:07.918036 20619 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:07.925789 10642 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:08.005117  9914 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:08.006577 20619 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.009018 20633 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:08.009101 20619 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:07.954938 20463 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:07.971076 20070 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:08.009788 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:08.010143 10096 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:28:08.006824 20428 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.009852 20463 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.011631 10099 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:28:08.012626 20463 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:08.013155 20463 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:08.005403 20569 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.014228 10091 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 34 index: 21. Preceding OpId from leader: term: 36 index: 22. (index mismatch)
I20260709 12:28:08.014360 20569 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:08.014703 20413 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:28:08.016166 20413 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:08.014460 20633 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.003s
W20260709 12:28:08.006918 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 81227.4us
I20260709 12:28:08.023504 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:08.023988 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:08.027740 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:08.026209 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:08.030838 10677 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 27.
W20260709 12:28:07.978917 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 118857us
I20260709 12:28:07.916402  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:08.037653  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 121313us
I20260709 12:28:08.038836  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:08.043712  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:08.046617  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 139092us
I20260709 12:28:08.049656  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:08.084954  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:08.054073 20485 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:08.046566 19975 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:08.101096 10643 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:08.051976 20649 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:08.101723  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 50072us
I20260709 12:28:08.053378 20602 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:08.102751 10643 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:08.051581 20545 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:08.047116 20633 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:08.089049 20662 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:08.164002 20575 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:08.164829 20575 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:08.146668 20633 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.129617 19975 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:08.137327 20663 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:28:08.138154 20664 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.173147 20664 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:08.180055 10090 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:08.134088 20602 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.182857 20602 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:08.184191 20664 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:08.184998 19975 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:08.185271 20617 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:08.186398 20633 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:08.129004 20649 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:08.194464 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:08.233084 20070 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:28:08.323350  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 219429us
W20260709 12:28:08.326087  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 224203us
I20260709 12:28:08.327267  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:08.198985 20516 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.119s
W20260709 12:28:08.328080  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 292559us
I20260709 12:28:08.349359  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:08.351415  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:08.232028 20569 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.328836 19975 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.363801 20427 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.358570 20715 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:08.351791 20649 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:08.366561 20427 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:08.362093 20575 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:08.330627 20070 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:08.425006 20070 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:08.367313 20631 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.426011 20631 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.428323 20631 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:08.368119 19975 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.355298 20633 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
W20260709 12:28:08.441498  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 79675.8us
I20260709 12:28:08.442981 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:08.443787 10673 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
W20260709 12:28:08.459612 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 114633us
W20260709 12:28:08.459734  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 110467us
I20260709 12:28:08.462211  9990 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:08.436813 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 87175.1us
I20260709 12:28:08.478135  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:08.526300 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 70091.9us
W20260709 12:28:08.550034 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 86455.1us
I20260709 12:28:08.445765 20575 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.451906 20723 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:08.564481 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 99450.5us
I20260709 12:28:08.565198 20575 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:08.572431 20575 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:08.573035 20723 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:08.575290 20723 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:08.578811 20729 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:08.579802 20729 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:08.585016 20427 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:08.589404 10675 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:28:08.591001 19975 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:08.593786 20569 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:08.597662 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:08.598312 10675 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:08.600020  9990 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:08.449275 20649 consensus_queue.cc:1059] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.117s
W20260709 12:28:08.653985  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54043.1us
I20260709 12:28:08.600963 10062 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:28:08.655794 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 54858.2us
W20260709 12:28:08.658622  9991 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.008s	sys 0.012s
I20260709 12:28:08.563980 20727 raft_consensus.cc:2764] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
W20260709 12:28:08.658949  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 61956.8us
I20260709 12:28:08.663085 10090 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:28:08.667449  8743 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 9974)
I20260709 12:28:08.669490 10093 raft_consensus.cc:1275] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:28:08.677472 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:08.679123 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:08.679765 10095 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:28:08.680611 10097 raft_consensus.cc:2410] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 29 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:28:08.688279 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:08.659826 20155 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:08.605756 19975 consensus_queue.cc:1059] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.080s
I20260709 12:28:08.661523 20730 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:08.682435 20070 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:08.741329 10640 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:08.693981 20155 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:28:08.746373 20720 log.cc:927] Time spent T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 12:28:08.746752 20722 log.cc:927] Time spent T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:28:08.743513 20712 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:08.760262 20155 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:08.793803 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:08.794719 10023 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:28:08.751863 20070 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.804590 19841 consensus_queue.cc:1048] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:08.779364 20736 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:08.799957  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 135039us
I20260709 12:28:08.807631 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:08.808053  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:08.816392  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:08.823429 19841 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:08.831488 19841 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.832427 19841 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:08.816150 10095 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:08.799602 20649 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:28:08.883647  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 67388.5us
W20260709 12:28:08.871627  9991 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 135 similar messages]
I20260709 12:28:08.784526 20712 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.940688 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:08.814086 20728 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:08.948912 10018 raft_consensus.cc:3055] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:28:08.901191 20733 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:28:08.806854 19975 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.095s
I20260709 12:28:08.949390 10018 raft_consensus.cc:740] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:08.952715 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:08.952775 10095 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:28:08.879465 20619 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:08.880046 20545 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:08.872255 20569 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:08.955778 20070 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:08.957022 10018 consensus_queue.cc:260] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:08.966843 10018 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:08.898380 20444 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:08.898173 20549 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:08.965106 10023 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:08.969102 20444 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:08.979242 20444 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:08.979970 20740 log.cc:927] Time spent T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 12:28:08.982278 20731 log.cc:927] Time spent T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
I20260709 12:28:08.989684 20444 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.992213 20737 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:08.992794 20737 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.993947 20716 raft_consensus.cc:3055] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:08.995164 20737 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:08.996513 20729 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:08.998065 20549 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:08.994477 20716 raft_consensus.cc:740] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 31 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:08.998790 20444 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:08.999671 20716 consensus_queue.cc:260] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.000532 20549 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.001091 20716 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:08.954514 20728 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.004894 20728 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:08.967407 20619 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.025485 20619 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:08.897516 20649 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.028918 20649 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.015810 20070 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:08.970337 20569 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.032789 20569 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.034456 20712 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:08.971100 20545 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.037272 20545 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.030377 20070 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.041093 20070 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.054118 20070 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.055135 20070 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.058193 20070 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:09.074982 20710 log.cc:927] Time spent T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: Append to log took a long time: real 1.265s	user 0.006s	sys 0.015s
I20260709 12:28:08.977931 19841 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.098838 19841 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.064186 20155 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:09.103402 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 337425us
W20260709 12:28:09.114969 10642 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.004s	sys 0.004s
W20260709 12:28:09.115537 10642 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.004s	sys 0.004s
I20260709 12:28:09.118089 20155 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.127765 20617 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.128674 20617 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.131837 20617 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:09.205405  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 385094us
I20260709 12:28:09.216826  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:09.220932 10016 raft_consensus.cc:1275] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:28:09.221551 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.224167 10017 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:28:09.224776 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.225644 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.226347 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.227339 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.228055 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.229002 10025 raft_consensus.cc:2368] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:09.229054 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.230017 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:09.230978 10062 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.005s	sys 0.009s
W20260709 12:28:09.231523 10062 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.005s	sys 0.009s
I20260709 12:28:09.225705 10024 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:28:09.234984 10021 raft_consensus.cc:2468] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:28:09.235633 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.242091 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.243362 10019 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 28: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
W20260709 12:28:09.251748 10091 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.007s	sys 0.007s
W20260709 12:28:09.252257 10091 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.008s	sys 0.007s
I20260709 12:28:09.264330 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:09.271657 10097 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:09.273123 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.274173 10098 raft_consensus.cc:2410] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 for term 31 because replica has last-logged OpId of term: 29 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 19.
W20260709 12:28:09.218791  9912 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.267516 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.268811 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.271565 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:09.279657 10095 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:09.313830 19976 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.314256 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.315562 10018 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 17. (term mismatch)
I20260709 12:28:09.316027 10018 pending_rounds.cc:85] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 16
I20260709 12:28:09.316334 10018 pending_rounds.cc:107] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:28:09.316666 10018 raft_consensus.cc:2889] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:09.316931 10676 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:09.316871 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:09.318966 20647 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 28 current term: 28 is higher.
I20260709 12:28:09.319236 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.319607 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:09.320629 10672 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:09.321542 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.322633 10675 raft_consensus.cc:2468] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 29.
I20260709 12:28:09.325717 20684 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.327948 10671 raft_consensus.cc:1240] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:28:09.330569 20732 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.330864 20448 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.331737 20732 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.331885 20448 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.334972 20569 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.335541 20663 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.336411 20663 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.339274 20726 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.336344 20569 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.340142 20726 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.346400 10640 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:09.340982 20684 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.347003 20069 raft_consensus.cc:493] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:28:09.347307  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 60575.5us
I20260709 12:28:09.348330 20069 raft_consensus.cc:515] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:09.341037  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 407842us
I20260709 12:28:09.330862 20575 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.330806 19841 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.350536 20684 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:09.362449 20448 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.359912 19976 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.370990 20713 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.371748 20713 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.372820 20726 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.373741 19975 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.374807 20660 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:09.375612 20660 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.377537 20445 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:09.368880 19841 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:09.382824  9915 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:09.335014 20463 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.383285 20643 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:09.384008 20643 raft_consensus.cc:493] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.384568 20643 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:09.384150 20463 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.385686 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:09.388932 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 66326.6us
I20260709 12:28:09.390739 20662 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:09.394029 20715 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.394992 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.409353 10098 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:09.409720 10670 raft_consensus.cc:2468] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:28:09.410542 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.409341 20715 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.412809 19975 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.415483 19975 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.416301 20715 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.389159 20529 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.373790 20612 raft_consensus.cc:493] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.407258 10095 raft_consensus.cc:2468] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:09.414145 19841 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.419224 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.359244 20732 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.395364 20713 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.399189 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.399139 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:09.433732  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54493.7us
I20260709 12:28:09.435913 10672 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 28.
I20260709 12:28:09.378211 20445 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.421702 20529 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:09.433004 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 92580.7us
I20260709 12:28:09.398554 20569 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:09.442657 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 86283.1us
I20260709 12:28:09.447976 20463 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.447304 20715 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:09.442345 10090 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.010s	sys 0.004s
I20260709 12:28:09.457564  9990 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:09.451110 20643 raft_consensus.cc:515] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.477157 19976 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.443804 20591 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.478940 20643 leader_election.cc:290] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.479274 20591 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.480007 20715 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.482035 20591 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.443113 20663 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.482504 20715 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.463321 20069 leader_election.cc:290] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.483930 10676 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
W20260709 12:28:09.486301 10022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.017s	sys 0.007s
W20260709 12:28:09.486536 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48950.5us
I20260709 12:28:09.478382 20529 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:09.488243 20643 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:28:09.493142 10090 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.010s	sys 0.004s
I20260709 12:28:09.447028 20155 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.443369  9914 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:09.443306 20575 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.496323 20753 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.497103 20753 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.498304 20575 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.494401 10091 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 26.
I20260709 12:28:09.509914 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.515146 20752 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.515235 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.516175 20752 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.519910 20445 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.519498 20155 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:09.521588 10065 leader_election.cc:336] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.440667 20612 raft_consensus.cc:515] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:09.523036  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 79852.7us
I20260709 12:28:09.523809 20762 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:09.524590 20762 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.524840 20612 leader_election.cc:290] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.526510 20715 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:09.527284 20715 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:09.528983  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:09.528725 20715 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:09.529423  9835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 216) took 109405 ms (client timeout 15000 ms). Trace:
W20260709 12:28:09.529888  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 93306.2us
I20260709 12:28:09.522210 10065 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:09.540092 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.536707 19976 raft_consensus.cc:2804] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:09.550349 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.551270 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.559027 20633 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:09.559162 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:09.559316 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.559845 10094 raft_consensus.cc:3060] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:09.560041 10098 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:28:09.561697 10099 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:09.562776 20445 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.562871 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.563498 20445 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.564507 20660 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.565097 20569 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.565990 20647 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.565830 20569 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.566612 20647 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.568138 20569 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.568784 20647 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.571005 19976 raft_consensus.cc:697] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:09.572232 19976 consensus_queue.cc:237] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.542500 20752 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.582190  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:09.583323 20569 raft_consensus.cc:2804] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:09.583899 20569 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting leader election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:09.584542 20569 raft_consensus.cc:3060] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:09.586481 20753 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.556842 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.587469 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.590574 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.591799 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 35 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.593294  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:09.545444 20684 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.556466 20726 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:09.549957 20762 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.587603 20445 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.617285 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:09.627979 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.630925 10024 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 29: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:09.634346 20617 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.635080 20617 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:09.646828 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 125315us
I20260709 12:28:09.646952 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.650460 10062 leader_election.cc:400] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:09.652400 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:09.652093 20684 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.654613 20753 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:09.658702 20649 raft_consensus.cc:2804] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:09.659214 20649 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:09.659621 20649 raft_consensus.cc:3060] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:09.660887 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:09.662846 10643 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.664340 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:09.529917  9835 rpcz_store.cc:269] 0709 12:26:20.154253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:20.199660 (+ 45407us) service_pool.cc:224] Handling call
0709 12:28:09.529167 (+109329507us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":428601,"reported_tablets":39}
I20260709 12:28:09.664808 10643 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:09.668932 10094 raft_consensus.cc:2468] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:28:09.666937 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.657673 20753 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:09.672714 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:09.673642  8743 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 9974)
I20260709 12:28:09.675508  9990 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:09.676216 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:09.655534 20684 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:09.677469 20069 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:09.662873 20765 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.015s
I20260709 12:28:09.677986 20069 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.680595 20069 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:09.680871 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.677762 10021 raft_consensus.cc:2410] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a for term 32 because replica has last-logged OpId of term: 31 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 17.
I20260709 12:28:09.677552 10676 raft_consensus.cc:2393] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
W20260709 12:28:09.691411 20777 log.cc:927] Time spent T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:28:09.678282 20753 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:09.695600  9991 leader_election.cc:336] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:09.696058 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 47649.3us
I20260709 12:28:09.696269  9991 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:09.697887  9991 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.698935 20445 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:09.671238 20663 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:09.708369 20463 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:28:09.717900  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53622.1us
I20260709 12:28:09.727267 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.728039 20155 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.729763 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.735934 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:09.736769 10019 raft_consensus.cc:2368] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:09.737012 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.737601 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.739259 10642 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:09.741451 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.746047 10020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:09.747848  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50020.1us
W20260709 12:28:09.750211 20778 log.cc:927] Time spent T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.205s	user 0.002s	sys 0.000s
I20260709 12:28:09.754658 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.754978 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.759130 20155 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.759841 20155 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:09.762100 20155 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:09.764344 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.765641 10097 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.772838 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.773415 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.774390 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.774652 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed7d787bc5ad4e8db7719c178ffbc59b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.775431 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.745379 20663 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:09.775947 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab118912551248c1985fce40a2e815b0" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.776245 20663 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.776726 20663 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.780025 20617 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:09.780849 10642 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:09.782951 20463 raft_consensus.cc:2804] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:09.783516 20463 raft_consensus.cc:697] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:09.783324 20716 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:28:09.784502 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 112964us
I20260709 12:28:09.784559 20463 consensus_queue.cc:237] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.785794 10018 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
I20260709 12:28:09.786947 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.788506 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.789176 10093 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
W20260709 12:28:09.793449  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44919.2us
W20260709 12:28:09.795689  9991 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.796154  9991 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:09.796939 20660 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:09.797629 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:09.806277 10062 leader_election.cc:400] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:09.814788 10062 leader_election.cc:403] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
I20260709 12:28:09.814956 10671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53edc906d1af41e0992f01434995f9d1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:09.815871 20155 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115) with higher term. Message: Invalid argument: T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 28. Current term is 29.
I20260709 12:28:09.816505 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:09.807909 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.808826 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.809793 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.809731 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.810482 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.810623 10021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.811769 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.811750 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.812436 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.813169 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.819579 10675 raft_consensus.cc:2393] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 32: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:09.824172 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.827906 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 35 candidate_status { last_received { term: 34 index: 24 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:09.829005  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:09.829952 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.830229 20649 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.830771  9987 leader_election.cc:304] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:09.829950 20660 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:09.831725 19841 raft_consensus.cc:2749] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:09.832724 20649 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.835307 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:09.835477 20569 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.837960 20569 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:09.839041 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:09.839946 20617 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:09.840399 20617 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.840801 20617 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:09.848816 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.854839 20633 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:09.855142 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:09.855865  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 47776.1us
W20260709 12:28:09.858722  9991 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 61348.1us
I20260709 12:28:09.859058 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:09.859432  9991 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.861002 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.862866 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:09.864907 20069 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:09.865187 20716 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:09.863063  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:09.873447  9987 leader_election.cc:304] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:09.875226 19841 raft_consensus.cc:2749] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:09.876336 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:09.877032  9987 leader_election.cc:304] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:09.886925  9914 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:28:09.898717 10065 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:09.899026 20445 raft_consensus.cc:2749] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:28:09.902275 20155 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:28:09.918522  9991 leader_election.cc:336] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.926019  9991 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:09.919240 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:09.926990  9991 leader_election.cc:336] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.927350  9991 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:09.927937 19841 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:09.928138  9991 leader_election.cc:336] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.928680  9991 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:09.854342 20753 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:09.919747 10065 leader_election.cc:336] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.931717 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:28:09.935318  9991 leader_election.cc:336] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.935843  9991 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:09.936517  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 75716.1us
I20260709 12:28:09.938335 19975 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:09.919505 20765 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:09.925559 20155 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:09.931056 20445 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:09.946055 20724 raft_consensus.cc:2804] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:28:09.949743  9991 leader_election.cc:336] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.949493 19976 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:28:09.958096  9991 leader_election.cc:336] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:09.966065  9990 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:09.972568 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 53430.4us
I20260709 12:28:09.973685 19841 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:09.950863 20155 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:09.975559  9991 leader_election.cc:336] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:09.978984  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 121475us
I20260709 12:28:09.956009 20569 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:09.959236 20724 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:28:09.998950 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 122693us
I20260709 12:28:10.002871 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:09.951531 20649 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:10.010427 20785 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:28:10.046511 20663 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.049057 20569 raft_consensus.cc:493] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:28:10.052183  9991 consensus_peers.cc:597] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:10.047948  9914 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.006398 20575 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:09.982664 20481 consensus_queue.cc:1048] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.062s
I20260709 12:28:10.034794 20744 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.009680 20724 raft_consensus.cc:3060] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:10.017098 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:10.052608 10640 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.048929 20753 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:09.998498 20716 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.046s
I20260709 12:28:10.063334 20569 raft_consensus.cc:515] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.089331 20569 leader_election.cc:290] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.064364 20591 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:10.089629 20684 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:10.064544  9914 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:10.070120 20617 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.079615 20516 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:10.083086 20069 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.133417 20726 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.070580 20649 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:10.152724 20754 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:10.153555 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 238493us
W20260709 12:28:10.154492 10065 leader_election.cc:336] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.155071 10065 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.130527 20575 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:10.157284 10643 leader_election.cc:336] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:10.161942  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 97536.2us
I20260709 12:28:10.161087 10643 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:10.148921 20591 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:10.166648 20726 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:10.169169 20643 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:10.151569 20744 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:10.179680 20617 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.184909  9987 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:10.186972 19976 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.190348  9914 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:10.193758 10643 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.194365  9914 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.196671  9987 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:10.198021  9914 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:10.200460 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 46012.9us
W20260709 12:28:10.203401 10065 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.194288 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.166878 20516 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.200228 20660 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.226859 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.227777 20788 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:10.228278 10065 leader_election.cc:336] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:10.230232 10065 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.230705 10065 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.227351 20660 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:10.201470 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 216012us
I20260709 12:28:10.234324 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:10.234915 10018 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:10.235018 20752 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:10.231844  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 159698us
I20260709 12:28:10.197299 20481 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.244416 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:10.252022 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:10.253980 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.255198 10675 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:28:10.257104 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.258430 10022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.267189 10673 raft_consensus.cc:1240] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:10.195885 20744 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.217206 20768 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.213231 20633 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:10.210494 20716 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:10.212415 20649 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.275120 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:10.209410 20663 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.271137 10677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:10.277148 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
W20260709 12:28:10.278937 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 79387.3us
W20260709 12:28:10.280665 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 50684.1us
W20260709 12:28:10.287346 10643 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:10.286448 10065 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.290203 10065 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.292490 10099 raft_consensus.cc:1275] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:28:10.297751 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:10.305152 20660 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.305703 20502 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:10.306335 20502 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.308795 20502 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.305712 20649 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.310218 20649 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:10.312414 20649 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.280877 20481 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.305008 10024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.281504 20684 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.314078 20724 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:10.339068 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52666.2us
I20260709 12:28:10.318284 20481 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:10.364256 10643 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.356886 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.367179 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.340022 20791 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:10.356709 10065 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:10.376124  9914 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 178190us
W20260709 12:28:10.376991 10643 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.377097  9914 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.377496 10643 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.382539  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:10.382092 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 57831.3us
W20260709 12:28:10.384045 10643 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.379114 20765 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.385179 20569 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:10.388969 20793 raft_consensus.cc:493] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:10.389672 20793 raft_consensus.cc:515] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.392081 20793 leader_election.cc:290] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.393575  9914 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.394869 20744 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:10.395591 20744 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.401369 20792 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:10.402000 20792 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.404202 20792 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.414491 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:10.417234 10096 raft_consensus.cc:2393] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:10.409194 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.417541 10092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41bba71b483942ed97a9e1276a90514f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:10.422787 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:10.424667 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e7b0b17a4d646809d4dd46a0268f2eb" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:10.386312 20762 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.434222 10062 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.434852 20481 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.437022 10062 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.437726 20796 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:10.439760 10092 raft_consensus.cc:2393] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd983415389444e292b2330ef630850a in current term 28: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:10.445465 10018 raft_consensus.cc:2468] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:28:10.465375 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:10.476665 10019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:10.477614 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 229148us
I20260709 12:28:10.411823 20795 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:10.483466 20789 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:10.487574 10677 raft_consensus.cc:1240] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Rejecting Update request from peer cd983415389444e292b2330ef630850a for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:10.500049 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.464449 20792 raft_consensus.cc:2749] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.507066  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.477352 20791 raft_consensus.cc:2749] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.486348 20591 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.518922 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.515520 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "062c203e13aa4527a094ec24339001e5" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:10.523710 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 222761us
I20260709 12:28:10.490164 20481 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:28:10.526091 10065 leader_election.cc:336] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.528151 10065 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:10.513998 20793 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
W20260709 12:28:10.535025 10065 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.539780 10065 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:10.541954 20791 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:10.544734 20724 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.540326 20569 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:10.533963 20591 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:10.545607 20569 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.547783 20591 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.548416 20569 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.524020 20502 consensus_queue.cc:1048] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.019s
W20260709 12:28:10.551762 20155 consensus_meta.cc:287] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.003s
I20260709 12:28:10.552219 20155 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.555208 20516 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.558267 20715 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.506887 20762 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.536950 20798 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
W20260709 12:28:10.543329 10065 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.534567 20792 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:10.565780 20793 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:10.562281 10065 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.567374 20754 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:10.568018 20793 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.569382 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.584055 20798 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.589671 20762 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.592393 20155 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.595960 20575 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:10.603789 20789 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:10.537518 20481 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.576898 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:10.619422 20798 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.630577 20649 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:10.635675 10065 leader_election.cc:336] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.646926 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:10.649953  9914 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:10.667680  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 160739us
I20260709 12:28:10.668509  9912 leader_election.cc:304] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:10.668023 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:10.669896 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 283178us
W20260709 12:28:10.671693 10643 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.672062 10643 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:10.669755 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:10.678092  8743 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 9974)
I20260709 12:28:10.677227 20649 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.679816 20808 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:10.680675 20808 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.681260 20649 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.670989 20715 raft_consensus.cc:3060] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:10.693655 20569 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:10.677968 20591 raft_consensus.cc:2749] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:10.650828 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 153228us
I20260709 12:28:10.665903 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:10.665938 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:10.663910 10065 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:10.707760 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 43931.3us
I20260709 12:28:10.677186 20617 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:28:10.714375 10065 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:10.720901 20808 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.721760 20744 consensus_queue.cc:1059] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.003s
I20260709 12:28:10.724123 20744 raft_consensus.cc:3055] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:10.724494 20744 raft_consensus.cc:740] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:10.725391 20744 consensus_queue.cc:260] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.721216 10098 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:28:10.727664 20744 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:10.728412  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 87651.1us
I20260709 12:28:10.715762  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:28:10.730767  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 154586us
I20260709 12:28:10.731628 20591 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.732265  9987 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:10.742774 20660 raft_consensus.cc:2749] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:28:10.744117 20481 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:10.748049 10062 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:28:10.756551 10065 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:10.759703  9991 consensus_peers.cc:597] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:10.763489 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:10.767159 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.767817 20752 raft_consensus.cc:2804] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:10.767895 10670 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:28:10.769233 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.770743 10099 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:10.773562 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.774503 10025 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:28:10.774756 10672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.774916 20715 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:10.773442 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.776902 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.777462 20795 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.777980 10676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.777947 20795 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.778751 20481 consensus_queue.cc:1059] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:28:10.779289  9991 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:28:10.780212  9991 leader_election.cc:336] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.780371 20795 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:10.780606  9991 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:10.781075 20481 raft_consensus.cc:3055] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:10.781481 20481 raft_consensus.cc:740] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 30 LEADER]: Becoming Follower/Learner. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:10.781836 20806 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:10.782810 20752 raft_consensus.cc:697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:28:10.784623 10643 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.784544 20752 consensus_queue.cc:237] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:10.785255 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:10.786381 20663 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:10.786872 10669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.788647 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.782850 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.766047 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:10.792414 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:10.794710  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.796366 20809 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:10.798832  9912 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.801524 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.804356 10025 raft_consensus.cc:1240] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 26. Current term is 27. Ops: []
W20260709 12:28:10.806470 10065 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.816481  9912 leader_election.cc:304] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.824365 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.829363 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "447b5d26fe15480dbb8fedf0f7a44777" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 35 candidate_status { last_received { term: 32 index: 23 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.832589 10016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.834075 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:10.837795 20716 raft_consensus.cc:2749] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:10.840725 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:10.816591 20715 raft_consensus.cc:3060] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:10.850425 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.856523 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.862663 20155 raft_consensus.cc:2749] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:10.863360 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:10.782418 20481 consensus_queue.cc:260] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.872515 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:10.880115 20813 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:10.848109 20794 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.909003 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:10.910818 10092 raft_consensus.cc:1275] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
W20260709 12:28:10.860996 10643 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:10.912971 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 52047.1us
W20260709 12:28:10.914004 10643 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:10.915628 20715 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:10.916381 20715 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:10.916857 20715 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:10.868438 20788 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:10.852373 20796 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:10.924986  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:10.960542 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:10.968747  9991 consensus_peers.cc:597] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:10.894145  9836 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 332) took 117814 ms (client timeout 15000 ms). Trace:
W20260709 12:28:10.984498  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59596us
W20260709 12:28:10.990015  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 157743us
W20260709 12:28:10.990705  9991 consensus_peers.cc:597] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:10.982178 10065 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:10.985004 20463 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:11.002373  9912 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:10.981411 20813 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.018s
I20260709 12:28:11.008066  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:10.985785  9836 rpcz_store.cc:269] 0709 12:26:13.079118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:13.081520 (+  2402us) service_pool.cc:224] Handling call
0709 12:28:10.893913 (+117812393us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2360,"reported_tablets":47}
I20260709 12:28:11.028371 20798 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:11.028371  9912 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.030563 20820 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:11.032444 20612 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:11.003659 20617 consensus_queue.cc:1059] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 12:28:11.037871 20463 raft_consensus.cc:3055] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:28:11.038250 20463 raft_consensus.cc:740] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:11.039203 20463 consensus_queue.cc:260] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.044505  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:11.045787  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.047093 20794 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:11.063930  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.010545 20808 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:11.047541 20794 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:11.060087 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:11.077641 10065 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 53204.3us
I20260709 12:28:11.076818  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:11.067608 20787 log.cc:927] Time spent T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:28:11.081318  9912 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.077630 20633 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:11.086846 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 103262us
I20260709 12:28:11.070881 20649 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:11.073436 20794 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:11.048295 20684 consensus_queue.cc:1048] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.034s
I20260709 12:28:11.092734 20762 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:28:11.093115  9912 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.082634 20569 raft_consensus.cc:2749] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:11.098896  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.105540 20715 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:11.117377 10642 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:11.116961  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.115056 20809 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:11.117722 20649 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:11.136670 20794 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.139165 20794 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:11.146023 20715 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:11.154939 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:11.155639 10675 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:11.154502 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:11.165845 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:11.170887 20481 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:11.179157 10098 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Rejecting Update request from peer 6c7bd3fa1bfa431c89fc9c9096452a76 for earlier term 30. Current term is 31. Ops: []
I20260709 12:28:11.183494 20643 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:11.186143 20463 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:11.193008 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:11.193720 10094 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:11.204566 10675 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 33.
I20260709 12:28:11.208036  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:11.213358 20790 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:28:11.213840 20790 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:11.215668  9912 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:11.217319 20793 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:11.219779 10643 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.217504 20812 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:28:11.241214 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.247037 10094 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 33.
W20260709 12:28:11.254921 10065 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:11.272799 10065 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:11.280294 20812 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 33 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:11.227214 20790 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.286185 20790 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:11.269553 10643 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:11.291343 10643 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.281929 20812 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:11.319732  9991 consensus_peers.cc:597] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:11.331794 10643 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:11.359776  9991 consensus_peers.cc:597] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:11.361990  9839 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 314) took 113398 ms (client timeout 15000 ms). Trace:
W20260709 12:28:11.362483  9839 rpcz_store.cc:269] 0709 12:26:17.963801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:17.963992 (+   191us) service_pool.cc:224] Handling call
0709 12:26:18.072211 (+108219us) spinlock_profiling.cc:243] Waited 71.6 ms on lock 0x7b5400002490. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e064320b2 00007f4e0f3ab759 00007f4e0f343a76 00007f4e0f314d58 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc
0709 12:28:11.361788 (+113289577us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":958053,"reported_tablets":48,"spinlock_wait_cycles":164689792}
I20260709 12:28:11.365186 10018 raft_consensus.cc:1275] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:28:11.367960 20796 consensus_queue.cc:1048] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:28:11.371426 10643 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:11.384336 10065 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:11.423378  9840 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 215) took 133146 ms (client timeout 15000 ms). Trace:
W20260709 12:28:11.424150  9840 rpcz_store.cc:269] 0709 12:25:58.276855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:25:58.280042 (+  3187us) service_pool.cc:224] Handling call
0709 12:28:11.423113 (+133143071us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":320,"reported_tablets":39,"spinlock_wait_cycles":8448}
W20260709 12:28:11.454939  9991 consensus_peers.cc:597] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.467613  9836 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 29 to 32, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 32 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:11.469734  9836 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 31. New cstate: current_term: 31 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.496464  9836 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 25 to 28. New cstate: current_term: 28 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.507241  9836 catalog_manager.cc:5697] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:11.511054  9991 consensus_peers.cc:597] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.511690  9836 catalog_manager.cc:5697] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 30. New cstate: current_term: 30 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.533042  9836 catalog_manager.cc:5697] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 27 to 30, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 30 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:11.534789  9836 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 27 to 35. New cstate: current_term: 35 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.543579  9836 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 19 to 29. New cstate: current_term: 29 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.566759  9836 catalog_manager.cc:5697] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 25 to 27, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 27 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:11.573738  9836 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 17 to 25, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 25 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.605667  9836 catalog_manager.cc:5697] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 32 to 34, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 34 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:11.615195  9836 catalog_manager.cc:5697] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 27 to 28, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 28 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.617036  9836 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 25 to 29, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.629886  9836 catalog_manager.cc:5697] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:11.639137 10643 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.638732  9836 catalog_manager.cc:5697] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 29, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 29 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:11.648545  9836 catalog_manager.cc:5697] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 29, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:11.673329 10643 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.677398  9836 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
W20260709 12:28:11.678553  8743 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 9974)
I20260709 12:28:11.678728  9836 catalog_manager.cc:5697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 19 to 26, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 26 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:11.680142  9836 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 25 to 26, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 26 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:11.688720 10643 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.701993  9836 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 30. New cstate: current_term: 30 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.708451  9836 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 27, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.713434  9836 catalog_manager.cc:5697] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:11.725975  9836 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 21 to 27, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 27 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:11.731211  9836 catalog_manager.cc:5697] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 25 to 29, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 29 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:11.737423  9836 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 28, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 28 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.750923 20792 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:11.751595 20792 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.753798 20792 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:11.756541 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:11.756822 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:11.757278 10675 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:28:11.757586 10018 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:11.758988 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:11.744866  9836 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 26, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 26 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:11.761000 20155 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:11.761385 20155 raft_consensus.cc:493] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:11.761703 20155 raft_consensus.cc:3060] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:11.761689  9836 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:11.764317  9836 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 32, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 32 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.776789 20155 raft_consensus.cc:515] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.779886 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:11.780652 10023 raft_consensus.cc:2393] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 27: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:11.783083 20155 leader_election.cc:290] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:11.786078 20481 raft_consensus.cc:493] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.786873 20481 raft_consensus.cc:515] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:11.789517 20481 leader_election.cc:290] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:11.789930 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "886f3cb002374eae906fdbe2d392f3e7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:11.790493 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:11.791409 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b34dc95d0414e70879dd34eaa8213de" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:11.794131  9912 leader_election.cc:304] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.795709 20516 raft_consensus.cc:2749] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:28:11.795441  9836 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 20 to 30, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 30 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.798327  9836 catalog_manager.cc:5697] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.800092  9836 catalog_manager.cc:5697] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 31. New cstate: current_term: 31 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:11.797343 10675 raft_consensus.cc:2468] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:28:11.801936  9836 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 30, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 30 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:11.803788 10062 leader_election.cc:304] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:11.805368 20155 raft_consensus.cc:2804] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:11.805907 20155 raft_consensus.cc:697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:11.806819 20155 consensus_queue.cc:237] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.854423 10098 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:28:11.856366 20715 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:28:11.891765 20715 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:11.906420  9991 consensus_peers.cc:597] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:11.912446 20715 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.914636 20715 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:11.943323  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 50402.5us
I20260709 12:28:11.944482 20481 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:11.945065 20481 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.947322 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:11.947582 10668 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:28:11.947762 20481 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:11.949450 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7b809cb22d54c31ad5985fb9f01e6ab" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:11.952665 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:11.961258 20715 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:11.961849 20715 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:11.964218 20715 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:11.956734 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:11.971674 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:11.974284 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:11.985222 20715 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:12.021050  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:12.022375 20836 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:12.035344  9912 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:12.036583 20836 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:12.038138 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:12.038882 10668 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:28:12.047226  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:12.051229 20836 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:12.051689 20836 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.052052 20836 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:12.062927 20836 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:12.063884 10643 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:12.066285 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:12.073020 20836 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.075723 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:12.085898 10668 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:12.098153 10668 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:28:12.099485  9912 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:12.100761 20836 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:12.101557 20836 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:12.102409 20836 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.148332 20752 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:12.149183 20752 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.155598 20752 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.158381 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:12.164081 10668 raft_consensus.cc:2368] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 31. Current term is 32.
I20260709 12:28:12.165505 10062 leader_election.cc:400] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 31. Current term is 32.
I20260709 12:28:12.165812 10062 leader_election.cc:403] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:28:12.171432 10065 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:12.174325 20792 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:28:12.192670 10023 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:28:12.205307 20789 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:28:12.251344 20792 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: Vote denied by peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979) with higher term. Message: Invalid argument: T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 for earlier term 31. Current term is 32.
I20260709 12:28:12.284768 20715 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.285421 20715 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.278103 20481 raft_consensus.cc:493] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:12.287170 20481 raft_consensus.cc:515] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.287968 20715 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.288945 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:12.289844 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0df211fae593420e88f685abc8d618ec" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:12.291532  9912 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:12.294998 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:12.295398 20837 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:12.295816 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:12.303963  9912 leader_election.cc:304] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:12.305053 20715 raft_consensus.cc:2749] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:12.305045 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "230693f05eb74e8095e27d669f16c8ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:12.307674 10668 raft_consensus.cc:1275] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:28:12.301604 20481 leader_election.cc:290] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.309340 20752 consensus_queue.cc:1048] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:12.315282 20715 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.315819 20715 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.317921 20715 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.319761 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:12.320631 10670 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 31.
I20260709 12:28:12.321312 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:12.321930 10096 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 30.
I20260709 12:28:12.322896  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:12.323648 20715 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:12.324043 20715 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.324352 20715 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:28:12.327952 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:12.333678 20715 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.336730 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:12.337406 10098 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:28:12.339365 20715 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.343549 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:12.344215 10675 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:12.345525 10098 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:28:12.347780  9914 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a; no voters: 
I20260709 12:28:12.348627 20715 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:12.349105 20715 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Becoming Leader. State: Replica: cd983415389444e292b2330ef630850a, State: Running, Role: LEADER
I20260709 12:28:12.350150 20715 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.352459 10675 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cd983415389444e292b2330ef630850a in term 32.
I20260709 12:28:12.393914 20836 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:12.394543 20836 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:12.398304 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:12.400081 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:12.400635 20836 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:12.401563  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:12.403142 20836 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:12.534451  9991 consensus_peers.cc:597] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:12.626711 20838 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.627249 20838 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:12.629365 20838 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:12.644658 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:12.647329 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "db738483c1444059b752648d302dc0d9" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:12.656620 20838 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.657218 20838 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.659614 20838 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.663255 20069 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:12.663798 20069 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.664588 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:12.665827 20069 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.666167  9914 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:12.669952 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:12.670137 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:12.670629 10675 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:28:12.670807 10096 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 31: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:12.671892  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:12.673139 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1892c18d4e1644109d95b077e989b10f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:12.673592 20069 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:12.673942 20069 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:12.674294 20069 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:12.674739 20836 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:12.679113  8743 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 9974)
I20260709 12:28:12.686488 20069 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.697283 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:12.698118 10096 raft_consensus.cc:2393] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 31: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:12.700093 20069 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Requested vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.701177 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:12.706738 10096 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:12.710248 10675 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:28:12.711440  9987 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7
I20260709 12:28:12.712687 20069 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:12.713402 20069 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 31 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:12.713966 20481 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:28:12.714313 20069 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:12.739289  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 59634.9us
I20260709 12:28:12.753963  9912 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:12.765293  9843 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 20 to 21, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 21 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:12.769681 10675 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:12.770974 20836 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:12.800515 10065 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:12.800151 20854 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.016s
I20260709 12:28:12.815697  9836 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44164 (request call id 329) took 3646 ms. Trace:
I20260709 12:28:12.830793 20752 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:12.831379 20752 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.833447 20752 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:12.835170 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:12.835821 10675 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 31.
I20260709 12:28:12.837414 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:12.838778 20752 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:12.839309 20752 raft_consensus.cc:493] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:12.844995 10025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:12.839865 20752 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:12.863112 20836 raft_consensus.cc:493] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:12.863816 20836 raft_consensus.cc:515] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } attrs { replace: true } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:12.844089 20715 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:28:12.870224  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 74448.2us
I20260709 12:28:12.824141  9836 rpcz_store.cc:276] 0709 12:28:09.167986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:11.029114 (+1861128us) service_pool.cc:224] Handling call
0709 12:28:12.813663 (+1784549us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:11.908765 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:11.909489 (+   724us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:11.909537 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:28:11.909573 (+    36us) tablet.cc:662] Decoding operations
0709 12:28:11.936648 (+ 27075us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:11.936759 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:11.942665 (+  5906us) tablet.cc:685] Acquiring locks for 32 operations
0709 12:28:11.959823 (+ 17158us) tablet.cc:701] Row locks acquired
0709 12:28:11.959870 (+    47us) write_op.cc:260] PREPARE: finished
0709 12:28:11.959997 (+   127us) write_op.cc:270] Start()
0709 12:28:11.960113 (+   116us) write_op.cc:276] Timestamp: P: 1783600091959948 usec, L: 0
0709 12:28:11.960141 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 12:28:11.960633 (+   492us) log.cc:844] Serialized 12709 byte log entry
0709 12:28:12.059626 (+ 98993us) op_driver.cc:464] REPLICATION: finished
0709 12:28:12.073377 (+ 13751us) write_op.cc:301] APPLY: starting
0709 12:28:12.073480 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:12.359459 (+285979us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:12.359592 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:12.489962 (+130370us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:12.490400 (+   438us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=64,key_file_lookups=64,delta_file_lookups=0,mrs_lookups=0
0709 12:28:12.490464 (+    64us) write_op.cc:312] APPLY: finished
0709 12:28:12.491052 (+   588us) log.cc:844] Serialized 276 byte log entry
0709 12:28:12.507785 (+ 16733us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:12.508033 (+   248us) write_op.cc:454] Released schema lock
0709 12:28:12.508113 (+    80us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:12.508535 (+   422us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:12.508563 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":90390,"reported_tablets":43,"child_traces":[["op",{"apply.queue_time_us":13411,"apply.run_cpu_time_us":41410,"apply.run_wall_time_us":435165,"cfile_cache_hit":130,"cfile_cache_hit_bytes":137951,"num_ops":32,"prepare.queue_time_us":71674,"prepare.run_cpu_time_us":39600,"prepare.run_wall_time_us":110571,"raft.queue_time_us":14801,"raft.run_cpu_time_us":1447,"raft.run_wall_time_us":12758,"replication_time_us":99323,"thread_start_us":155103,"threads_started":4,"wal-append.queue_time_us":58403}]]}
I20260709 12:28:12.877135 10675 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:12.870507 20715 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:12.895556 20852 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.068s
I20260709 12:28:12.994431 20715 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:12.992926  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 99978.1us
I20260709 12:28:13.015543 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:13.040962 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:13.046360 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:13.047747 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "208adf3e1124402cb1e6c43ba7306b7b" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:13.086350 20838 raft_consensus.cc:493] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:13.090343 20838 raft_consensus.cc:515] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:13.102808 10098 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Refusing update from remote peer cd983415389444e292b2330ef630850a: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:13.081992 20836 leader_election.cc:290] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:13.125710 10668 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:28:13.146802 20877 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:28:13.156462 20752 raft_consensus.cc:515] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:13.162650 10065 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:13.163947 10065 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.182529 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:13.183094 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:13.133025 20875 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.056s
W20260709 12:28:13.185452 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 158395us
W20260709 12:28:13.188377  9991 consensus_peers.cc:597] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.178499  9912 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:13.223030  9912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 44650.7us
I20260709 12:28:13.223454 20868 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:13.181478 20838 leader_election.cc:290] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:13.227353  9912 leader_election.cc:304] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:13.216524 20859 raft_consensus.cc:493] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.240268 20752 leader_election.cc:290] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:13.238369 20859 raft_consensus.cc:515] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:13.247350 20859 leader_election.cc:290] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:13.251801 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:13.249197 20838 raft_consensus.cc:2749] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:13.263522  9991 consensus_peers.cc:597] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:13.268762 10093 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 31. Current term is 32. Ops: []
I20260709 12:28:13.270568 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35eb6b5f8b8e4d2f8e0a6a08ded4d6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:13.276739 20875 consensus_queue.cc:1059] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.004s
I20260709 12:28:13.286755 20875 raft_consensus.cc:3055] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:28:13.287204 20875 raft_consensus.cc:740] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:13.288264 20875 consensus_queue.cc:260] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:13.289611 20875 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:13.296397 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4909dfc06b24d70bfd398607779f5ae" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:13.297055 10668 raft_consensus.cc:3060] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:13.301395 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "912ffd65ac564403ac65b7a5d005d367" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
W20260709 12:28:13.321923 10643 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.322687 20838 raft_consensus.cc:493] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:13.323369 20838 raft_consensus.cc:515] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:13.329247 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:13.324072 10668 raft_consensus.cc:2468] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 32.
I20260709 12:28:13.343879 10062 leader_election.cc:304] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:13.346992 20872 raft_consensus.cc:2804] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:13.347561 20872 raft_consensus.cc:697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:13.348423 20872 consensus_queue.cc:237] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:13.358795 20838 leader_election.cc:290] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:13.360391  9912 leader_election.cc:304] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:13.361263 20853 raft_consensus.cc:2749] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:13.362996 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.368743 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "181ca8c0fd384c8c909b095284d9e12c" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:13.401618  9912 leader_election.cc:304] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
W20260709 12:28:13.413806 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 183356us
I20260709 12:28:13.411478 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d80ed93cd18a4a1095bdd6f9fd0fca80" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:13.462606 10673 raft_consensus.cc:1240] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Rejecting Update request from peer 9aa345bcd43b44088114b7b6f2830b17 for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:28:13.461089  9912 leader_election.cc:304] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:13.475503 20869 raft_consensus.cc:2749] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:13.463064 20715 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.424402 20853 raft_consensus.cc:2749] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:13.477505 20715 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:13.483883 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:13.480572 20715 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:13.517655 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9bd550066b4447f8848484defc31b9d1" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:13.519137 20715 raft_consensus.cc:493] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:13.519680 20715 raft_consensus.cc:515] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:13.521732 20715 leader_election.cc:290] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:13.523367 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:13.526512 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9f126305b66248b59cd9ce87bc952694" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:13.530143  9912 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:13.531453 20715 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:13.538313  9912 leader_election.cc:304] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:13.539575 20859 raft_consensus.cc:2749] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:28:13.568177 20836 raft_consensus.cc:493] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:13.568810 20836 raft_consensus.cc:515] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:13.576757 20836 leader_election.cc:290] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:13.585264 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:13.609025 10065 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.612632 20878 raft_consensus.cc:493] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:13.613226 20878 raft_consensus.cc:515] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:13.615449 20878 leader_election.cc:290] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:13.616326 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:13.618278 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26e02c55bc5648e0baa2a4216f02f955" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 36 candidate_status { last_received { term: 34 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:13.620850  9914 leader_election.cc:304] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:13.621699 20859 raft_consensus.cc:2749] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:13.642952 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b5ca4a138b9345ab87fc4055395e0488" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:13.645015  9912 leader_election.cc:304] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:13.647095 20859 raft_consensus.cc:2749] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:28:13.666249 10643 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:13.679600  8743 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 9974)
W20260709 12:28:13.693779  9991 consensus_peers.cc:597] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:13.696832 10065 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:13.699139  9991 consensus_peers.cc:597] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.699409  9836 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 31 to 33. New cstate: current_term: 33 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:13.700964  9836 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:13.702138  9836 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 22 to 29, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 29 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:13.703498  9836 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 25 to 28, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 28 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:13.704854  9836 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 32 to 36, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 36 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:13.707165  9836 catalog_manager.cc:5697] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 24 to 28, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 28 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:13.708410  9836 catalog_manager.cc:5697] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 27, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 27 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:13.710287  9836 catalog_manager.cc:5697] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 33, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 33 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } }
I20260709 12:28:13.711675  9836 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 21 to 28, leader changed from 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 28 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:13.731712  9843 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 218) took 75262 ms (client timeout 15000 ms). Trace:
W20260709 12:28:13.732179  9843 rpcz_store.cc:269] 0709 12:26:58.476637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:26:58.649853 (+173216us) service_pool.cc:224] Handling call
0709 12:28:13.731627 (+75081774us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:12.953039 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:13.061308 (+108269us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:13.071959 (+ 10651us) write_op.cc:435] Acquired schema lock
0709 12:28:13.074940 (+  2981us) tablet.cc:662] Decoding operations
0709 12:28:13.121723 (+ 46783us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:13.128057 (+  6334us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:13.133552 (+  5495us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:28:13.149758 (+ 16206us) tablet.cc:701] Row locks acquired
0709 12:28:13.154294 (+  4536us) write_op.cc:260] PREPARE: finished
0709 12:28:13.181615 (+ 27321us) write_op.cc:270] Start()
0709 12:28:13.181739 (+   124us) write_op.cc:276] Timestamp: P: 1783600093181567 usec, L: 0
0709 12:28:13.181761 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:28:13.182424 (+   663us) log.cc:844] Serialized 536 byte log entry
0709 12:28:13.422046 (+239622us) op_driver.cc:464] REPLICATION: finished
0709 12:28:13.531849 (+109803us) write_op.cc:301] APPLY: starting
0709 12:28:13.531953 (+   104us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:13.533057 (+  1104us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:13.533150 (+    93us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:13.533919 (+   769us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:13.534099 (+   180us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=0,mrs_lookups=0
0709 12:28:13.534142 (+    43us) write_op.cc:312] APPLY: finished
0709 12:28:13.534322 (+   180us) log.cc:844] Serialized 25 byte log entry
0709 12:28:13.534477 (+   155us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:13.534562 (+    85us) write_op.cc:454] Released schema lock
0709 12:28:13.549924 (+ 15362us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:13.550369 (+   445us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:13.550398 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":152205,"reported_tablets":39,"child_traces":[["op",{"apply.queue_time_us":109535,"apply.run_cpu_time_us":3205,"apply.run_wall_time_us":18489,"cfile_cache_hit":6,"cfile_cache_hit_bytes":8309,"num_ops":1,"prepare.queue_time_us":122661,"prepare.run_cpu_time_us":3878,"prepare.run_wall_time_us":277172,"replication_time_us":237714,"thread_start_us":164612,"threads_started":2}]]}
I20260709 12:28:13.751684  9836 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 32. New cstate: current_term: 32 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:13.774693  9836 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:13.781754  9836 catalog_manager.cc:5697] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:13.783186  9836 catalog_manager.cc:5697] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 29, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 29 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:13.788893  9991 consensus_peers.cc:597] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:13.805519 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.806141 20481 raft_consensus.cc:493] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:13.806814 20481 raft_consensus.cc:515] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } attrs { replace: true } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:13.810271 20481 leader_election.cc:290] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:13.812407 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:13.820746 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:13.821488 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a7196ffc873e4e7e84c76068d5b95904" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:13.826220  9912 leader_election.cc:304] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:13.829094 20481 raft_consensus.cc:2749] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:13.829690 10643 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:13.831286  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 50954.8us
W20260709 12:28:13.832912 10643 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.866412 10673 consensus_queue.cc:237] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:13.873799 10643 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.875840 10670 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:28:13.881397 20789 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:28:13.887715 10018 raft_consensus.cc:1275] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
W20260709 12:28:13.888614 10065 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 264 similar messages]
I20260709 12:28:13.894655 20871 consensus_queue.cc:1048] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:28:13.904585 10065 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:13.919915 10023 raft_consensus.cc:1275] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 32 index: 22. (term mismatch)
I20260709 12:28:13.920390 10023 pending_rounds.cc:85] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Aborting all ops after (but not including) 21
I20260709 12:28:13.920609 10023 pending_rounds.cc:107] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: Aborting uncommitted NO_OP operation due to leader change: 31.22
I20260709 12:28:13.920830 10023 raft_consensus.cc:2889] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:28:13.923671  9991 consensus_peers.cc:597] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.923481 20789 consensus_queue.cc:1048] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:28:13.946615 20900 log.cc:927] Time spent T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 12:28:13.949314 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:13.972702  9991 consensus_peers.cc:597] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:13.985262 20864 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Committing config change with OpId 27.18: config changed from index 4 to 18, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:13.992218 10017 raft_consensus.cc:2955] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Committing config change with OpId 27.18: config changed from index 4 to 18, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New config: { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.032452  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 12:28:14.043247 20869 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:14.046221 20869 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:14.048936 20869 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:14.058233 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:14.067304  9834 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 22 to 26, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 26 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.069768 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93686c27d2cf4edea0d5da8d0c1d81ee" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:14.073662  9912 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:14.074779 20859 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:14.072237  9834 catalog_manager.cc:5697] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 22 to 29, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:14.089742  9834 catalog_manager.cc:5697] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:14.098493 10065 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:14.104449  9834 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 16 to 25. New cstate: current_term: 25 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:14.106928  9834 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 20 to 24, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 24 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.127759 20859 raft_consensus.cc:493] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.128384 20859 raft_consensus.cc:515] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:14.130563 20859 leader_election.cc:290] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:14.130743 20869 raft_consensus.cc:493] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.132611 20869 raft_consensus.cc:515] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:14.135157 10643 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:14.135357 20869 leader_election.cc:290] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:14.143357 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:14.145172 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d6726f0982834ecf8987472ed6c32a73" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:14.145744 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:14.150733 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a1e3e80f2cad43f08196a18ad2bee6a4" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:14.152292  9914 leader_election.cc:304] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:14.152598  9912 leader_election.cc:304] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:14.153934 20481 raft_consensus.cc:2749] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:14.154611 20481 raft_consensus.cc:2749] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:14.167848 20859 raft_consensus.cc:493] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:14.167701  9843 catalog_manager.cc:5697] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 27 to 32, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 32 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:14.168421 20859 raft_consensus.cc:515] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:14.169548  9843 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.170572 20859 leader_election.cc:290] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:14.173966 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:14.176565  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:14.179687  8743 tablet_replica.cc:333] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.181267  8743 raft_consensus.cc:2243] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.182399  8743 raft_consensus.cc:2272] T 8bab9d80dad949d8afde49b123fbac1a P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.186256  8743 tablet_replica.cc:333] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.188897 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "51528ebbf4884121ad7c7d564100498f" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:14.194633  8743 raft_consensus.cc:2243] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.195391  8743 raft_consensus.cc:2272] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.203532  9912 leader_election.cc:304] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:14.204823 20481 raft_consensus.cc:2749] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:14.208539  8743 tablet_replica.cc:333] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.210008  8743 raft_consensus.cc:2243] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.220278  8743 pending_rounds.cc:70] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Trying to abort 1 pending ops.
I20260709 12:28:14.220659  8743 pending_rounds.cc:77] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a: Aborting op as it isn't in flight: id { term: 32 index: 22 } timestamp: 7305625937592602624 op_type: NO_OP noop_request { }
I20260709 12:28:14.221120  8743 raft_consensus.cc:2889] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:14.221505  8743 raft_consensus.cc:2272] T 0df211fae593420e88f685abc8d618ec P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.230154  8743 tablet_replica.cc:333] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.235662  9840 catalog_manager.cc:5697] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 25 to 29. New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.238631  9840 catalog_manager.cc:5697] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 24 to 30, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 30 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.242323  8743 raft_consensus.cc:2243] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Raft consensus shutting down.
W20260709 12:28:14.247605  9834 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 220) took 30541 ms (client timeout 15000 ms). Trace:
I20260709 12:28:14.249118  8743 raft_consensus.cc:2272] T 3e4ebd0dc053435ea316f01d854592c9 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.256242  8743 tablet_replica.cc:333] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.257427  8743 raft_consensus.cc:2243] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.258028  8743 raft_consensus.cc:2272] T 8b34dc95d0414e70879dd34eaa8213de P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.261235  8743 tablet_replica.cc:333] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.241382  9840 catalog_manager.cc:5697] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 25 to 32. New cstate: current_term: 32 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:14.249509  9834 rpcz_store.cc:269] 0709 12:27:43.754697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:54.528798 (+10774101us) service_pool.cc:224] Handling call
0709 12:27:55.881609 (+1352811us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400329954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094c0090 00007f4e0955ef3a 00007f4e0f314e9c 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc 00007f4e0d8d3392 00007f4e0789148c 00007f4e078908c6
0709 12:28:14.247515 (+18365906us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:14.139428 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:14.139944 (+   516us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:14.139977 (+    33us) write_op.cc:435] Acquired schema lock
0709 12:28:14.139993 (+    16us) tablet.cc:662] Decoding operations
0709 12:28:14.140668 (+   675us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:14.140727 (+    59us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:14.140751 (+    24us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:28:14.141314 (+   563us) tablet.cc:701] Row locks acquired
0709 12:28:14.141332 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:28:14.141494 (+   162us) write_op.cc:270] Start()
0709 12:28:14.141607 (+   113us) write_op.cc:276] Timestamp: P: 1783600094141446 usec, L: 0
0709 12:28:14.141630 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:28:14.142281 (+   651us) log.cc:844] Serialized 2069 byte log entry
0709 12:28:14.146420 (+  4139us) op_driver.cc:464] REPLICATION: finished
0709 12:28:14.148407 (+  1987us) write_op.cc:301] APPLY: starting
0709 12:28:14.149076 (+   669us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:14.151714 (+  2638us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:14.151800 (+    86us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:14.158212 (+  6412us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:14.158390 (+   178us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:28:14.158430 (+    40us) write_op.cc:312] APPLY: finished
0709 12:28:14.158678 (+   248us) log.cc:844] Serialized 57 byte log entry
0709 12:28:14.159423 (+   745us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:14.159528 (+   105us) write_op.cc:454] Released schema lock
0709 12:28:14.159572 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:14.159995 (+   423us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:14.160022 (+    27us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":618755,"reported_tablets":39,"spinlock_wait_cycles":471137664,"child_traces":[["op",{"apply.queue_time_us":1745,"apply.run_cpu_time_us":7536,"apply.run_wall_time_us":11533,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20910,"num_ops":5,"prepare.queue_time_us":121,"prepare.run_cpu_time_us":3206,"prepare.run_wall_time_us":9383,"replication_time_us":4634}]]}
I20260709 12:28:14.270188  9840 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 25 to 29, leader changed from 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 29 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.273527  9840 catalog_manager.cc:5697] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 16 to 24. New cstate: current_term: 24 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.277710  9840 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 22 to 31, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to cd983415389444e292b2330ef630850a (127.8.137.193). New cstate: current_term: 31 leader_uuid: "cd983415389444e292b2330ef630850a" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.262247  8743 raft_consensus.cc:2243] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.280021  8743 raft_consensus.cc:2272] T b5ca4a138b9345ab87fc4055395e0488 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.279121  9840 catalog_manager.cc:5697] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 27 to 34, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 34 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:14.286077  9840 catalog_manager.cc:5697] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 24 to 29, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.288980  8743 tablet_replica.cc:333] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.291278  8743 raft_consensus.cc:2243] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.292456  8743 raft_consensus.cc:2272] T 6a2ff46f3f264308a58e49e2819b1d5b P cd983415389444e292b2330ef630850a [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.287472  9840 catalog_manager.cc:5697] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 20 to 24, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 24 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.298231  8743 tablet_replica.cc:333] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.299337  8743 raft_consensus.cc:2243] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.300652  9840 catalog_manager.cc:5697] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 26 to 30, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 30 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:14.304399  8743 raft_consensus.cc:2272] T 840d8356270745f492c7d2c7099794c1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.315809 20481 raft_consensus.cc:493] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:14.316381 20481 raft_consensus.cc:515] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:14.324594  8743 tablet_replica.cc:333] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.325711  8743 raft_consensus.cc:2243] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.326833  8743 raft_consensus.cc:2272] T fc2205f3947147b6ad9c3ac2b9bd5058 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.329672  8743 tablet_replica.cc:333] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.330837  8743 raft_consensus.cc:2243] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.333916  8743 raft_consensus.cc:2272] T a7196ffc873e4e7e84c76068d5b95904 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.338649  8743 tablet_replica.cc:333] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.340618  8743 raft_consensus.cc:2243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.341384  8743 raft_consensus.cc:2272] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.341912 20481 leader_election.cc:290] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:14.343035 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:14.345459  8743 tablet_replica.cc:333] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.346472 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "03a9b6ec47044f6d8c27ae5f07c64f3a" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:14.348143  8743 raft_consensus.cc:2243] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.349685  8743 raft_consensus.cc:2272] T 6dfe4ce70e5943448d384222d18a1215 P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.352608  9912 leader_election.cc:304] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:14.353312  8743 tablet_replica.cc:333] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a: stopping tablet replica
W20260709 12:28:14.353411  9912 raft_consensus.cc:2686] T 03a9b6ec47044f6d8c27ae5f07c64f3a P cd983415389444e292b2330ef630850a: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:28:14.354904  8743 raft_consensus.cc:2243] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.357223  8743 raft_consensus.cc:2272] T 208adf3e1124402cb1e6c43ba7306b7b P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.361426  8743 tablet_replica.cc:333] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.362404  8743 raft_consensus.cc:2243] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.363777  8743 raft_consensus.cc:2272] T 6c669b03539649e4a4164ca56748f662 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.367043  8743 tablet_replica.cc:333] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.369227  8743 raft_consensus.cc:2243] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.370347  8743 raft_consensus.cc:2272] T a78ccdb241a54c0398e2539a827e5d22 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:14.373787 10643 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:14.390793  8743 tablet_replica.cc:333] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.391974  8743 raft_consensus.cc:2243] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.392551  8743 raft_consensus.cc:2272] T d7b809cb22d54c31ad5985fb9f01e6ab P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.395591  8743 tablet_replica.cc:333] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.396487  8743 raft_consensus.cc:2243] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.397560  8743 raft_consensus.cc:2272] T 230693f05eb74e8095e27d669f16c8ee P cd983415389444e292b2330ef630850a [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.400604  8743 tablet_replica.cc:333] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.401506  8743 raft_consensus.cc:2243] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.402181  8743 pending_rounds.cc:70] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Trying to abort 1 pending ops.
I20260709 12:28:14.402436  8743 pending_rounds.cc:77] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a: Aborting op as it isn't in flight: id { term: 28 index: 18 } timestamp: 7305625921803612160 op_type: NO_OP noop_request { }
I20260709 12:28:14.414903  8743 raft_consensus.cc:2889] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:14.415337  8743 raft_consensus.cc:2272] T 7e7b0b17a4d646809d4dd46a0268f2eb P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.418309  8743 tablet_replica.cc:333] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.419103  8743 raft_consensus.cc:2243] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.420117  8743 raft_consensus.cc:2272] T 7253570248b8481791bf2187007dd5df P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:14.424995 10643 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:14.442788  8743 tablet_replica.cc:333] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.451649  8743 raft_consensus.cc:2243] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.453133  8743 raft_consensus.cc:2272] T 447b5d26fe15480dbb8fedf0f7a44777 P cd983415389444e292b2330ef630850a [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.456377  8743 tablet_replica.cc:333] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a: stopping tablet replica
W20260709 12:28:14.461388  9991 consensus_peers.cc:597] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:14.463505  8743 raft_consensus.cc:2243] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.464442  8743 raft_consensus.cc:2272] T 41bba71b483942ed97a9e1276a90514f P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:14.467404 10643 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:14.474519  8743 tablet_replica.cc:333] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.477118  9836 catalog_manager.cc:5697] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: config changed from index 4 to 18, VOTER 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195) evicted. New cstate: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 18 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:14.481024 20481 raft_consensus.cc:493] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:14.488488 20481 raft_consensus.cc:515] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:14.489300  8743 raft_consensus.cc:2243] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.490516  8743 raft_consensus.cc:2272] T bb6fd99043b344de885ff64fd4c81132 P cd983415389444e292b2330ef630850a [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.490727 20481 leader_election.cc:290] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:14.492705 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:14.496088 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa5a16c274db44a5aa641b51f71f7ec6" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:14.503952  9912 leader_election.cc:304] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:14.506193 20869 raft_consensus.cc:2749] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:28:14.506330  8743 tablet_replica.cc:333] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.532213 10089 tablet_service.cc:1558] Processing DeleteTablet for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with delete_type TABLET_DATA_TOMBSTONED (TS 4083ce96cefb47d0be1d8e99602b7ce7 not found in new config with opid_index 18) from {username='slave'} at 127.0.0.1:54288
I20260709 12:28:14.543322  8743 raft_consensus.cc:2243] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.543982  8743 pending_rounds.cc:70] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Trying to abort 2 pending ops.
I20260709 12:28:14.544236  8743 pending_rounds.cc:77] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Aborting op as it isn't in flight: id { term: 30 index: 18 } timestamp: 7305625936717221888 op_type: NO_OP noop_request { }
I20260709 12:28:14.544595  8743 raft_consensus.cc:2889] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:14.544906  8743 pending_rounds.cc:77] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a: Aborting op as it isn't in flight: id { term: 28 index: 17 } timestamp: 7305625915150954496 op_type: NO_OP noop_request { }
I20260709 12:28:14.545219  8743 raft_consensus.cc:2889] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:14.545572  8743 raft_consensus.cc:2272] T 93686c27d2cf4edea0d5da8d0c1d81ee P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.560423 20906 tablet_replica.cc:333] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:14.571636 20906 raft_consensus.cc:2243] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.590306 20906 raft_consensus.cc:2272] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:14.570200  9841 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 336) took 16310 ms (client timeout 15000 ms). Trace:
W20260709 12:28:14.594694  9841 rpcz_store.cc:269] 0709 12:27:58.255067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:02.092163 (+3837096us) service_pool.cc:224] Handling call
0709 12:28:14.565159 (+12472996us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14246,"reported_tablets":47}
I20260709 12:28:14.603045  8743 tablet_replica.cc:333] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.603991  8743 raft_consensus.cc:2243] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.604607  8743 raft_consensus.cc:2272] T 51528ebbf4884121ad7c7d564100498f P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.618886 20906 ts_tablet_manager.cc:1916] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:28:14.624112  9842 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 219) took 50301 ms (client timeout 15000 ms). Trace:
W20260709 12:28:14.624760  9842 rpcz_store.cc:269] 0709 12:27:24.322988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:31.006572 (+6683584us) service_pool.cc:224] Handling call
0709 12:28:14.623902 (+43617330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32831,"reported_tablets":39}
I20260709 12:28:14.636483  8743 tablet_replica.cc:333] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.640015  8743 raft_consensus.cc:2243] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.640800  8743 raft_consensus.cc:2272] T d80ed93cd18a4a1095bdd6f9fd0fca80 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.677647  8743 tablet_replica.cc:333] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.683362  8743 raft_consensus.cc:2243] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.693889  8743 raft_consensus.cc:2272] T 773b4422370c4cc6b9e7477bdaf29261 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.696973  8743 tablet_replica.cc:333] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.706808  8743 raft_consensus.cc:2243] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.715437  8743 raft_consensus.cc:2272] T d864b424b78d4fffb51785ea7d3b74ed P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:14.710070  9837 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44160 (request call id 335) took 42579 ms (client timeout 15000 ms). Trace:
I20260709 12:28:14.718941 20906 ts_tablet_manager.cc:1929] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 27.17
I20260709 12:28:14.719503 20906 log.cc:1199] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: Deleting WAL directory at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/wals/35eb6b5f8b8e4d2f8e0a6a08ded4d6a4
W20260709 12:28:14.718819  9837 rpcz_store.cc:269] 0709 12:27:32.130831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:27:43.546353 (+11415522us) service_pool.cc:224] Handling call
0709 12:28:14.709875 (+31163522us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17628,"reported_tablets":47}
I20260709 12:28:14.721858  9829 catalog_manager.cc:5028] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 (table test-workload [id=2485a8438f984c0a822a1b3a2b3cab6f]) successfully deleted
I20260709 12:28:14.718420  8743 tablet_replica.cc:333] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.726099  8743 raft_consensus.cc:2243] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.726711  8743 raft_consensus.cc:2272] T 181ca8c0fd384c8c909b095284d9e12c P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.730006  8743 tablet_replica.cc:333] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.752983  8743 raft_consensus.cc:2243] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.759055  8743 raft_consensus.cc:2272] T 062c203e13aa4527a094ec24339001e5 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.762286  8743 tablet_replica.cc:333] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.763224  8743 raft_consensus.cc:2243] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.763828  8743 raft_consensus.cc:2272] T 912ffd65ac564403ac65b7a5d005d367 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.788002  9835 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 with cas_config_opid_index 4: aborting the task: latest config opid_index 18; task opid_index 4
W20260709 12:28:14.789810  9835 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:42270 (request call id 221) took 15090 ms (client timeout 15000 ms). Trace:
W20260709 12:28:14.790225  9835 rpcz_store.cc:269] 0709 12:27:59.699715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:09.730744 (+10031029us) service_pool.cc:224] Handling call
0709 12:28:14.789711 (+5058967us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89232,"reported_tablets":39,"spinlock_wait_cycles":22912}
I20260709 12:28:14.837905  8743 tablet_replica.cc:333] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.838845  8743 raft_consensus.cc:2243] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.839510  8743 raft_consensus.cc:2272] T 9bd550066b4447f8848484defc31b9d1 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:14.877465 10643 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:14.842474  8743 tablet_replica.cc:333] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.881536  8743 raft_consensus.cc:2243] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.882138  8743 raft_consensus.cc:2272] T d6726f0982834ecf8987472ed6c32a73 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.885069  8743 tablet_replica.cc:333] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.886018  8743 raft_consensus.cc:2243] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Raft consensus shutting down.
W20260709 12:28:14.892700  9838 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:44152 (request call id 318) took 15713 ms (client timeout 15000 ms). Trace:
W20260709 12:28:14.893128  9838 rpcz_store.cc:269] 0709 12:27:59.179248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:06.358865 (+7179617us) service_pool.cc:224] Handling call
0709 12:28:14.892508 (+8533643us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19733,"reported_tablets":48}
I20260709 12:28:14.917733  8743 raft_consensus.cc:2272] T 32a9a1d2d5fc4243bf902da3519cd483 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:14.926450 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:14.933238  8743 tablet_replica.cc:333] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.934279  8743 raft_consensus.cc:2243] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.934916  8743 raft_consensus.cc:2272] T a1e3e80f2cad43f08196a18ad2bee6a4 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.937830  8743 tablet_replica.cc:333] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.957247  8743 raft_consensus.cc:2243] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.958642  8743 raft_consensus.cc:2272] T afb4bd9b5d6240b19d09014b985ea7f7 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.961541  8743 tablet_replica.cc:333] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.962344  8743 raft_consensus.cc:2243] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:14.979481  8743 raft_consensus.cc:2272] T 78df0ad0c1454828a838cd88472c8553 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.982427  8743 tablet_replica.cc:333] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:14.983281  8743 raft_consensus.cc:2243] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:14.983881  8743 raft_consensus.cc:2272] T 4cd31c96c4854a87a361975b579bc1c2 P cd983415389444e292b2330ef630850a [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:14.998644 20481 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:14.999151 20481 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.001223 20481 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.001333  8743 tablet_replica.cc:333] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.004323  8743 raft_consensus.cc:2243] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.004216 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:15.005282  8743 raft_consensus.cc:2272] T 1892c18d4e1644109d95b077e989b10f P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.005874 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "86c102903b3d4742a0b9ac1df1132361" candidate_uuid: "cd983415389444e292b2330ef630850a" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:15.008605  8743 tablet_replica.cc:333] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.009580  8743 raft_consensus.cc:2243] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.013573  9912 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd983415389444e292b2330ef630850a; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:15.019137 20481 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:15.022827  8743 raft_consensus.cc:2272] T eb955e753b7546bf8612b6ed1c3fc342 P cd983415389444e292b2330ef630850a [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.025938  8743 tablet_replica.cc:333] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.026832  8743 raft_consensus.cc:2243] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.027501  8743 raft_consensus.cc:2272] T 9f126305b66248b59cd9ce87bc952694 P cd983415389444e292b2330ef630850a [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.041316  8743 tablet_replica.cc:333] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.042161  8743 raft_consensus.cc:2243] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.042817  8743 raft_consensus.cc:2272] T 86c102903b3d4742a0b9ac1df1132361 P cd983415389444e292b2330ef630850a [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.059441  8743 tablet_replica.cc:333] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.060240  8743 raft_consensus.cc:2243] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.061370  8743 raft_consensus.cc:2272] T 59e61711ca80440e9c13392c6b6b25fa P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.064146  8743 tablet_replica.cc:333] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.064913  8743 raft_consensus.cc:2243] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.073298  8743 pending_rounds.cc:70] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Trying to abort 1 pending ops.
I20260709 12:28:15.073601  8743 pending_rounds.cc:77] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a: Aborting op as it isn't in flight: id { term: 27 index: 15 } timestamp: 7305625905791037440 op_type: NO_OP noop_request { }
I20260709 12:28:15.073966  8743 raft_consensus.cc:2889] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:15.074334  8743 raft_consensus.cc:2272] T db738483c1444059b752648d302dc0d9 P cd983415389444e292b2330ef630850a [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.086766  8743 tablet_replica.cc:333] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.087618  8743 raft_consensus.cc:2243] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.088719  8743 raft_consensus.cc:2272] T e346f82bc2694f9e968337cc4be9e53e P cd983415389444e292b2330ef630850a [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.091675  8743 tablet_replica.cc:333] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.092413  8743 raft_consensus.cc:2243] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.092989  8743 raft_consensus.cc:2272] T bf49747e9a8542e6b75e3545a18c0981 P cd983415389444e292b2330ef630850a [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.114717  8743 tablet_replica.cc:333] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.115504  8743 raft_consensus.cc:2243] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.116096  8743 raft_consensus.cc:2272] T 26e02c55bc5648e0baa2a4216f02f955 P cd983415389444e292b2330ef630850a [term 35 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:15.129384  9991 consensus_peers.cc:597] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:15.134990  8743 tablet_replica.cc:333] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.135878  8743 raft_consensus.cc:2243] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:15.136960  8743 raft_consensus.cc:2272] T d43a22dba58d423d907a568238e85284 P cd983415389444e292b2330ef630850a [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.140106  8743 tablet_replica.cc:333] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a: stopping tablet replica
I20260709 12:28:15.140758  8743 raft_consensus.cc:2243] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:15.142333 20789 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.142997 20789 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.146972 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:15.154817 10065 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.162973  8743 raft_consensus.cc:2272] T aa5a16c274db44a5aa641b51f71f7ec6 P cd983415389444e292b2330ef630850a [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:15.148486 20789 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:15.167243 20789 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.167794 20789 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.171142 20789 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:15.177266 10065 leader_election.cc:336] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.200955 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:15.201942 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:15.204216 10062 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:15.206416 20866 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:15.219508 20866 raft_consensus.cc:2749] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:28:15.269891 10065 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:15.270864 10065 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:15.326516 20894 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.327183 20894 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.332459 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 37 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:15.345463 10065 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:15.346174  9991 leader_election.cc:336] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.346692  9991 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:15.347823 20806 raft_consensus.cc:2749] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:28:15.352267 20894 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.363317 20894 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.363942 20894 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:15.370407  9991 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.370471 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:15.371137 20894 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.371230 10096 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:28:15.372828  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:15.376838 20806 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:15.377348 20806 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.377758 20806 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:15.381775 20871 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.382409 20871 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.389088 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:15.395471 10643 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:15.395979 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:15.399124 20871 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.418503 20789 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.419718 20789 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.424048 20789 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:15.436048 10065 connection.cc:573] client connection to 127.8.137.193:34909 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:28:15.436550 10065 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 12:28:15.400182 20871 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:15.452977 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:15.454154 20866 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.454712 20866 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.456972 20866 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:15.457446 20907 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.457988 20907 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.460789 20907 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:15.473866 20894 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.474707 20894 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.498283 20894 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:15.504985 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 113616us
I20260709 12:28:15.510483 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:15.512298 20907 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.512876 20907 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.517439 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:15.519426 10062 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:15.520799 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:15.520828 20789 raft_consensus.cc:2749] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:15.521070 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:15.523027 10674 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 32.
I20260709 12:28:15.525135 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:28:15.525394  9991 connection.cc:769] client connection to 127.8.137.193:34909 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 12:28:15.526012 20789 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:15.528087 20789 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.528520 20789 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:15.526098 20907 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:15.539428 20789 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.545663 20909 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.556421 20909 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:15.558797 10065 leader_election.cc:336] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.559329 10065 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:15.565122 20909 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:15.566493  9991 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.567015  9991 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:15.566812 20907 raft_consensus.cc:2749] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:15.568413 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:15.568869 10065 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.569041 10096 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
W20260709 12:28:15.569382 10643 leader_election.cc:336] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:15.569634 10065 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.570062 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:28:15.570570  9991 consensus_peers.cc:597] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:15.570789 20907 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:15.570758 20910 raft_consensus.cc:2749] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:28:15.570330 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:15.575124 20864 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:15.575555 20864 raft_consensus.cc:493] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.575943 20864 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:15.584281 20911 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.584983 20911 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.587497 20911 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.590252 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:15.590981 10096 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:28:15.592397 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:15.593137 20911 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:28:15.593181 10643 leader_election.cc:336] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.593605 20911 raft_consensus.cc:493] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.594051 20911 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:15.604918 20912 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.605600 20912 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.608816 20912 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:15.612025 20864 raft_consensus.cc:515] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.614933 20864 leader_election.cc:290] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:15.618028 10643 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:15.618552 10643 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.519428 20806 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.622048 20806 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:15.623426 10643 leader_election.cc:336] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.626432 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8bab9d80dad949d8afde49b123fbac1a" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:15.627040 10096 raft_consensus.cc:3060] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:15.625052 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:15.633450 20866 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:28:15.625881  9991 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.634073 20866 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.640489 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 37 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:15.641211 10017 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 36 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 36.
I20260709 12:28:15.642107 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "840d8356270745f492c7d2c7099794c1" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:15.642329 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:15.642803 10674 raft_consensus.cc:3060] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:15.643329 20907 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Leader pre-election won for term 37
I20260709 12:28:15.643764 20907 raft_consensus.cc:493] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.644167 20907 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:28:15.647997 20912 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.649050 20789 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:15.649845 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3e4ebd0dc053435ea316f01d854592c9" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:15.650411 10093 raft_consensus.cc:3060] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:15.650753 20912 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.653267 20912 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.655556 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:15.659771 20866 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:15.660243 10098 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 33.
W20260709 12:28:15.656891 10065 leader_election.cc:336] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.662390 20911 raft_consensus.cc:515] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:15.664310 10065 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.668226 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:15.670171 20910 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.670817 20910 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.673743 20909 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:28:15.678006  9991 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.679212 10674 raft_consensus.cc:2468] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 33.
I20260709 12:28:15.682166 20910 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:15.680552 10096 raft_consensus.cc:2468] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 33.
W20260709 12:28:15.659819 10643 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.689224 10062 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:15.690788 20866 raft_consensus.cc:2804] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:15.691336 20866 raft_consensus.cc:697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:15.692193 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:15.697170 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fc2205f3947147b6ad9c3ac2b9bd5058" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:15.697753 10098 raft_consensus.cc:3060] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:15.692273 20866 consensus_queue.cc:237] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.674993 10093 raft_consensus.cc:2468] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 27.
I20260709 12:28:15.693846  9990 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:15.694509 20911 leader_election.cc:290] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.673242 20907 raft_consensus.cc:515] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 37 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.708555 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:15.708782 10098 raft_consensus.cc:2468] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:28:15.709111 10674 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 31.
I20260709 12:28:15.710217  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:28:15.710510 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:15.713660 20916 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:28:15.714114 20916 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.714504 20916 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:15.715168 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6a2ff46f3f264308a58e49e2819b1d5b" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 37 candidate_status { last_received { term: 36 index: 22 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:15.715761 10017 raft_consensus.cc:3060] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 36 FOLLOWER]: Advancing to term 37
I20260709 12:28:15.723816 20907 leader_election.cc:290] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 37 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:15.724053 20916 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.712579 20806 raft_consensus.cc:2804] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:15.729945 20806 raft_consensus.cc:697] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:15.731019 10017 raft_consensus.cc:2468] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 37 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 37.
I20260709 12:28:15.730887 20806 consensus_queue.cc:237] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.732347 10062 leader_election.cc:304] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 37 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:15.733278 20907 raft_consensus.cc:2804] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 37 FOLLOWER]: Leader election won for term 37
I20260709 12:28:15.734272 20907 raft_consensus.cc:697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 37 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:28:15.735165  9991 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.735208 20907 consensus_queue.cc:237] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 36.22, Last appended by leader: 22, Current term: 37, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.708109 20864 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:28:15.737417 20864 raft_consensus.cc:493] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.737802 20864 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:15.740097 10642 leader_election.cc:304] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:15.738489 20915 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.743112 20915 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.745374 20915 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.756623 20866 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.756914 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:15.757189 20866 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:15.758919  9991 leader_election.cc:336] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.759375 20866 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:15.759516  9990 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:15.760365 20915 raft_consensus.cc:2749] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:15.763532 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:15.764232 10017 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 26.
I20260709 12:28:15.766319 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:28:15.767099 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 54153.5us
I20260709 12:28:15.767825 10642 leader_election.cc:304] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:15.773021 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7253570248b8481791bf2187007dd5df" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 32 candidate_status { last_received { term: 31 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:15.773609 10674 raft_consensus.cc:3060] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:28:15.789826 10674 raft_consensus.cc:2468] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:28:15.771341 20916 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:15.771486 20912 raft_consensus.cc:2804] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:15.770476 20789 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:28:15.775197 20911 raft_consensus.cc:2804] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:15.807158 20789 raft_consensus.cc:493] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.822986 20789 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:15.810326 20912 raft_consensus.cc:697] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:15.827708 20912 consensus_queue.cc:237] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.815685  9838 catalog_manager.cc:5697] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 32 to 33, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 33 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:15.811872 20911 raft_consensus.cc:697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:15.841336 20911 consensus_queue.cc:237] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.810879 20866 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:28:15.846473 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 75910.5us
I20260709 12:28:15.850894  9987 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:15.855172 20866 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.858075 20866 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:15.871946 20915 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.872603 20915 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.874683 20916 raft_consensus.cc:2804] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:28:15.874881 20915 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:15.875403 20916 raft_consensus.cc:697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:15.876492 20916 consensus_queue.cc:237] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 31.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:15.884402 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:15.885286 10096 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 27.
I20260709 12:28:15.899076  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:15.845300 20907 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.915400 20806 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:28:15.919040 20806 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.907104 20907 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:15.908061 20866 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:15.919514 20806 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:28:15.923373 10065 leader_election.cc:336] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 37 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.928467 20907 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:15.979807 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:15.980744 10023 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:28:16.017850 20789 raft_consensus.cc:515] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:16.021721 10065 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:15.950798 20866 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.071671 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:16.055848 20922 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.057150 20928 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.070966 20909 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.094096 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:16.093271 20922 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.101235 20922 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:16.095325 20907 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:16.103780 20907 raft_consensus.cc:493] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.082554 20864 raft_consensus.cc:515] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.104245 20907 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:16.105707 20864 leader_election.cc:290] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:16.110134 10065 leader_election.cc:336] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.111038 10065 leader_election.cc:336] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.111577 10065 leader_election.cc:336] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.088743 20789 leader_election.cc:290] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:16.123575 10643 leader_election.cc:336] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.137137 10065 leader_election.cc:336] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.137553 20806 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.143057 20806 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:16.143170 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71589.2us
I20260709 12:28:16.144948 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bb6fd99043b344de885ff64fd4c81132" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:16.145624 10093 raft_consensus.cc:3060] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:28:16.095781 20928 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.152721 20928 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:16.109851 20909 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.156771 20909 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.157516 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a78ccdb241a54c0398e2539a827e5d22" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:16.158141 10096 raft_consensus.cc:3060] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:28:16.159041 20866 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:16.160063 20910 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:28:16.160521  9991 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.167165 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6dfe4ce70e5943448d384222d18a1215" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:16.167843 10023 raft_consensus.cc:3060] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:28:16.168537 10017 raft_consensus.cc:1275] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 37 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 36 index: 22. Preceding OpId from leader: term: 37 index: 23. (index mismatch)
I20260709 12:28:16.167171 20927 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.170136 20927 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:16.172099  9991 consensus_peers.cc:597] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:16.172470 20864 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
W20260709 12:28:16.172717  9991 consensus_peers.cc:597] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:16.173187 20927 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:16.173060 20864 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.186486 20864 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:16.187784 10018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
W20260709 12:28:16.188329 10065 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.190272 10065 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:16.192178 10023 raft_consensus.cc:2468] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 27.
I20260709 12:28:16.175747 20942 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.192972 20942 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:16.173282  9991 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.195456 20942 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.179593 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:16.161653 20940 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.180029 20919 consensus_queue.cc:1048] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:28:16.196345 10094 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 32.
I20260709 12:28:16.196406 20940 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.182197 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:16.184033 20907 raft_consensus.cc:515] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.199357 20940 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.199944 20907 leader_election.cc:290] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:16.165904 10096 raft_consensus.cc:2468] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 34.
I20260709 12:28:16.160593 20910 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.202474 20910 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:16.192298 10065 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.204583 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:16.204032 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 62838us
I20260709 12:28:16.210759 10093 raft_consensus.cc:2468] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 28.
I20260709 12:28:16.212116  9990 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:28:16.212739  9991 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.213454  9991 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
W20260709 12:28:16.213976 10065 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.214426 20910 raft_consensus.cc:2804] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:28:16.214762 20806 raft_consensus.cc:2749] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:16.215224 20910 raft_consensus.cc:697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
W20260709 12:28:16.216684 10065 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:16.216307 20910 consensus_queue.cc:237] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.224076 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6c669b03539649e4a4164ca56748f662" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:16.224713 10017 raft_consensus.cc:3060] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:16.225572 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.226792 10642 leader_election.cc:304] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.228462 20942 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:16.228987 20942 raft_consensus.cc:493] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.232955 20942 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.233354 10017 raft_consensus.cc:2468] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
W20260709 12:28:16.233960 10065 leader_election.cc:336] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.229781 20940 raft_consensus.cc:2804] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:28:16.239080 20940 raft_consensus.cc:697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
W20260709 12:28:16.240419 10065 consensus_peers.cc:597] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.224664 10062 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:16.240360 20940 consensus_queue.cc:237] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.241608 20907 raft_consensus.cc:2749] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:16.242667 20941 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.243300 20941 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.244560  9837 catalog_manager.cc:5697] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 26 to 27, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 27 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:16.247840 20941 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:16.248317 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:16.248992  9835 catalog_manager.cc:5697] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 32 to 33, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 33 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:16.250316 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:16.251452 20907 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:16.251466 10670 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
I20260709 12:28:16.252746 10062 leader_election.cc:304] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:16.253728  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:16.254810 20907 raft_consensus.cc:2804] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:28:16.255959 10062 leader_election.cc:304] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:16.260262 20907 raft_consensus.cc:697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:16.261269 20907 consensus_queue.cc:237] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:16.264261  9991 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.265986 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:16.266759 10093 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 25.
I20260709 12:28:16.270076 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:16.270979 10094 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
I20260709 12:28:16.271657 10093 raft_consensus.cc:1275] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:28:16.268360 20927 raft_consensus.cc:2804] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:16.272504 20927 raft_consensus.cc:697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:16.255095 20941 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:16.273703 20927 consensus_queue.cc:237] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.283052 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.288509 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.289635 20871 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:28:16.290059 20871 raft_consensus.cc:493] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.290422 20871 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:16.290805 20864 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:16.291260 20864 raft_consensus.cc:493] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.291675 20864 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:16.293254 20909 consensus_queue.cc:1048] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:28:16.300413 20806 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.300976 20806 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.274889 20941 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.318512 20806 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.321871 10673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:16.322834 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:16.316704 20941 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:16.324100 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:16.325249 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:16.327236 10094 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 32.
I20260709 12:28:16.323762 20939 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.331175  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:16.329741  9838 catalog_manager.cc:5697] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 36 to 37, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 37 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:16.334779  9838 catalog_manager.cc:5697] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 26 to 27, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 27 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:16.340108 10675 raft_consensus.cc:1275] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:28:16.365185 20927 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.365806 20927 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.368310 20927 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:16.369872 10065 consensus_peers.cc:597] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.370626 20894 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.371217 20894 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.347136 20939 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:16.374533 10065 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.374815 20939 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:16.375030  9991 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.364034 20936 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:16.375911 20936 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.376312 20936 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.382953 20789 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.383543 20789 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.385710 20789 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:16.386897 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:16.354970 20871 raft_consensus.cc:515] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.387553 10023 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:28:16.391057 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:16.391825 10023 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:28:16.391916 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e346f82bc2694f9e968337cc4be9e53e" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 26 candidate_status { last_received { term: 25 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:16.392539 10096 raft_consensus.cc:3060] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:28:16.394680 10673 raft_consensus.cc:2393] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 26: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:16.397450 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:16.398234 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:16.398998 10094 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 32.
I20260709 12:28:16.399415 10091 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 29.
W20260709 12:28:16.404263  9991 consensus_peers.cc:597] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.404786 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
W20260709 12:28:16.404909  9991 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.405500  9991 consensus_peers.cc:597] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:16.408162 10098 raft_consensus.cc:1275] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
I20260709 12:28:16.408769 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:16.409842 10096 raft_consensus.cc:2468] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 26.
W20260709 12:28:16.396924 10643 leader_election.cc:336] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.408485  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:16.409878 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.412248 10642 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.389128 20871 leader_election.cc:290] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.408078 20866 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:16.429909 20942 raft_consensus.cc:515] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.430454 20866 raft_consensus.cc:493] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.430979 20866 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:16.431190 10674 raft_consensus.cc:2393] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 33: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:16.432180 20942 leader_election.cc:290] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.413141 20926 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:16.434834 20926 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.435323 20926 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:16.413684 20911 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:16.436161 20911 raft_consensus.cc:493] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.436470 20919 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.424832 20894 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.436609 20911 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.437106 20919 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.379926 20864 raft_consensus.cc:515] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.440028 10670 raft_consensus.cc:2393] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:16.441051 20864 leader_election.cc:290] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.446383 20919 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:16.452472  9991 consensus_peers.cc:597] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:16.456626 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "773b4422370c4cc6b9e7477bdaf29261" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:16.457441 10096 raft_consensus.cc:3060] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.457880 20864 raft_consensus.cc:2804] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:28:16.456905 10098 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "32a9a1d2d5fc4243bf902da3519cd483" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:16.458369 10091 raft_consensus.cc:1275] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:28:16.458493 10098 raft_consensus.cc:3060] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:16.458803 10065 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.461388 20866 raft_consensus.cc:515] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:16.463413  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 55006.6us
I20260709 12:28:16.458197 20911 raft_consensus.cc:515] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.463796 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:16.464494 10023 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:28:16.463256 20894 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:16.466769 20894 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.467209 20894 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:16.458638 20864 raft_consensus.cc:697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:16.469013 20864 consensus_queue.cc:237] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 25.20, Last appended by leader: 20, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:16.473382  9991 consensus_peers.cc:597] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:16.476634  9991 consensus_peers.cc:597] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.481127 10098 raft_consensus.cc:2468] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
I20260709 12:28:16.481356 20866 leader_election.cc:290] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:16.482854 10096 raft_consensus.cc:2468] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 33.
I20260709 12:28:16.484879 10674 raft_consensus.cc:1275] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 31 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:28:16.483218 20939 consensus_queue.cc:1048] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:28:16.485462 20894 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:16.486799 10643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 89901.1us
W20260709 12:28:16.487660 10643 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.487746 20910 consensus_queue.cc:1048] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:28:16.488498 10643 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:16.489071 10643 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.489470 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:16.490319 20871 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:16.465497 20925 consensus_queue.cc:1048] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20260709 12:28:16.498157 10643 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:16.498764 10643 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.499552 10643 leader_election.cc:336] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.501843 10096 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d864b424b78d4fffb51785ea7d3b74ed" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:16.502053 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:16.502545 10096 raft_consensus.cc:3060] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.505466 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afb4bd9b5d6240b19d09014b985ea7f7" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:16.505872 10642 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.506138 10023 raft_consensus.cc:3060] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:16.494436 20911 leader_election.cc:290] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.513742 10642 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.515004 10095 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:16.515277 20912 raft_consensus.cc:2804] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:28:16.520110  8743 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 9970)
I20260709 12:28:16.520788 10096 raft_consensus.cc:2468] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 33.
I20260709 12:28:16.515930 20912 raft_consensus.cc:697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:16.508625 20789 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:16.511899 20894 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:16.523110 10062 leader_election.cc:304] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:16.533329 10023 raft_consensus.cc:2468] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
I20260709 12:28:16.531056 20939 raft_consensus.cc:2804] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:16.536329 20939 raft_consensus.cc:697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:16.537418 20936 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.537298 20939 consensus_queue.cc:237] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.530582 20789 raft_consensus.cc:493] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader cd983415389444e292b2330ef630850a)
I20260709 12:28:16.546449 10062 leader_election.cc:304] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
W20260709 12:28:16.548354 10643 consensus_peers.cc:597] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.527001 20866 consensus_queue.cc:1048] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.024s
I20260709 12:28:16.549780 20936 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:16.552695 10643 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:16.551729 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d43a22dba58d423d907a568238e85284" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:16.554118 10093 raft_consensus.cc:3060] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:16.534488 20919 raft_consensus.cc:2749] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:28:16.561363 10062 leader_election.cc:304] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:16.532958 20941 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.556960 20789 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:16.573509 10642 leader_election.cc:304] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:16.563597 20927 raft_consensus.cc:2749] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:16.579602 20941 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:16.581342 10670 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59e61711ca80440e9c13392c6b6b25fa" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:16.575102 20912 consensus_queue.cc:237] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.592792 10670 raft_consensus.cc:3060] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:16.590443 10093 raft_consensus.cc:2468] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 33.
I20260709 12:28:16.591130 20962 raft_consensus.cc:2749] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:16.603161 20911 raft_consensus.cc:2804] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:16.603739 20911 raft_consensus.cc:697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:16.604549  9990 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
I20260709 12:28:16.604651 20911 consensus_queue.cc:237] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.608075 10670 raft_consensus.cc:2468] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9aa345bcd43b44088114b7b6f2830b17 in term 30.
I20260709 12:28:16.608672 20894 raft_consensus.cc:2804] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:16.616971  9835 catalog_manager.cc:5697] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 31 to 32, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 32 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:16.618180 20894 raft_consensus.cc:697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:16.618721  9835 catalog_manager.cc:5697] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 27 to 28, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 28 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:16.619936 20894 consensus_queue.cc:237] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.625283  9987 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:28:16.635144 10065 leader_election.cc:336] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.633124 20941 raft_consensus.cc:2804] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:28:16.637492 10065 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:16.635531 10062 leader_election.cc:304] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: 
W20260709 12:28:16.640602 10643 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:16.642418 10643 leader_election.cc:336] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.644394 20961 raft_consensus.cc:2804] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:16.645038 20961 raft_consensus.cc:697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
W20260709 12:28:16.646281 10643 leader_election.cc:336] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.647045 10643 leader_election.cc:336] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.647687 10643 leader_election.cc:336] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.648373 10643 leader_election.cc:336] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.651717 10096 raft_consensus.cc:1275] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:28:16.645901 20961 consensus_queue.cc:237] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:16.656947 10643 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:16.661579 10065 leader_election.cc:336] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.663290 20925 consensus_queue.cc:1048] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:16.642712 20941 raft_consensus.cc:697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Becoming Leader. State: Replica: 9aa345bcd43b44088114b7b6f2830b17, State: Running, Role: LEADER
I20260709 12:28:16.637571 20926 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:16.701032  9991 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.701897  9991 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:16.720558 10095 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 9aa345bcd43b44088114b7b6f2830b17 in current term 30: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:16.726192 10091 consensus_queue.cc:237] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:16.738068 10643 consensus_peers.cc:597] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.764525 10670 consensus_queue.cc:237] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.811059 20789 raft_consensus.cc:515] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:16.789747 20941 consensus_queue.cc:237] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:16.826793 20926 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:16.828486 20789 leader_election.cc:290] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:16.846635 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 124477us
I20260709 12:28:16.847016 10023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:16.847880 10023 raft_consensus.cc:2393] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 30: Already voted for candidate 9aa345bcd43b44088114b7b6f2830b17 in this term.
I20260709 12:28:16.851449 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb955e753b7546bf8612b6ed1c3fc342" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:16.852247 10017 raft_consensus.cc:3060] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:28:16.877203 10065 leader_election.cc:336] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.882477 10065 consensus_peers.cc:597] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:16.882993 10065 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.883478 10065 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:16.905725 10017 raft_consensus.cc:2468] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
W20260709 12:28:16.942330  9991 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:16.943961  9991 consensus_peers.cc:597] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:16.960356 20970 log.cc:927] Time spent T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 12:28:16.972472 10065 consensus_peers.cc:597] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:16.974210 10023 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:28:16.975931 20939 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:28:16.976581 10025 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:16.975952 10018 raft_consensus.cc:1275] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:28:16.982050 10643 consensus_peers.cc:597] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:16.982698 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:16.996488 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:17.001012  9835 catalog_manager.cc:5697] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 32 to 33, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 33 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:17.015985  9991 consensus_peers.cc:597] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.005616  9835 catalog_manager.cc:5697] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 28 to 29, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.029255 10062 leader_election.cc:304] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:17.044911  9990 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:17.031989  9835 catalog_manager.cc:5697] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 28 to 29, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.050547 20928 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.002s
I20260709 12:28:17.056025 20927 raft_consensus.cc:2804] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:28:17.056613 20927 raft_consensus.cc:697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:17.057545 20927 consensus_queue.cc:237] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:17.059221 10095 raft_consensus.cc:1275] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 34 index: 24. (index mismatch)
I20260709 12:28:17.058578 10096 raft_consensus.cc:1275] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:28:17.059250 10091 raft_consensus.cc:1275] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:28:17.052842 20922 consensus_queue.cc:1048] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.008s
I20260709 12:28:17.034921 20962 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:28:17.066478 10643 consensus_peers.cc:597] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.068432 10025 raft_consensus.cc:1275] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:28:17.050215  9835 catalog_manager.cc:5697] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 33 to 34, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 34 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.070719  9835 catalog_manager.cc:5697] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 25 to 26, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 26 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.073055 10674 raft_consensus.cc:1275] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:28:17.050098 20941 raft_consensus.cc:2749] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:28:17.097859 20937 consensus_queue.cc:1048] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.000s
I20260709 12:28:17.099321 20940 consensus_queue.cc:1048] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:28:17.104156 20939 consensus_queue.cc:1048] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:28:17.109798 10643 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:17.110436 10643 consensus_peers.cc:597] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:17.113024 10643 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.115948 20983 consensus_queue.cc:1048] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:28:17.128116 20911 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Committing config change with OpId 29.17: config changed from index 5 to 17, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:17.121069 20919 consensus_queue.cc:1048] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.015s
W20260709 12:28:17.138285 10643 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:17.126037 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 48776us
I20260709 12:28:17.145936 10025 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Committing config change with OpId 29.17: config changed from index 5 to 17, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:17.165311 10098 raft_consensus.cc:1275] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:28:17.203784  9827 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a7196ffc873e4e7e84c76068d5b95904 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 13)
I20260709 12:28:17.210819 10098 raft_consensus.cc:1275] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:28:17.223498 10099 raft_consensus.cc:1275] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 20. (index mismatch)
W20260709 12:28:17.224809  9991 consensus_peers.cc:597] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:17.225317  9991 consensus_peers.cc:597] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.188189 20939 consensus_queue.cc:1048] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.047s
I20260709 12:28:17.267444 10091 raft_consensus.cc:1275] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 25 index: 20. Preceding OpId from leader: term: 26 index: 21. (index mismatch)
I20260709 12:28:17.269315 10674 raft_consensus.cc:1275] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer 9aa345bcd43b44088114b7b6f2830b17: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:28:17.267073  9837 catalog_manager.cc:5697] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 29, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 29 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.282931  9837 catalog_manager.cc:5697] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 29, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 29 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.219914 20940 consensus_queue.cc:1048] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.061s
I20260709 12:28:17.280591 20916 consensus_queue.cc:1048] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.024s
W20260709 12:28:17.307396 10643 consensus_peers.cc:597] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.310258 10017 raft_consensus.cc:1275] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:28:17.313583 20922 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Committing config change with OpId 26.19: config changed from index 6 to 19, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:17.317953 10065 consensus_peers.cc:597] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.285697 20973 consensus_queue.cc:1048] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.050s
W20260709 12:28:17.372267  9991 consensus_peers.cc:597] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:17.360095 10095 raft_consensus.cc:2955] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Committing config change with OpId 29.17: config changed from index 5 to 17, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:17.381707  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 147794us
I20260709 12:28:17.329481 20961 consensus_queue.cc:1048] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
I20260709 12:28:17.415138 10018 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Committing config change with OpId 26.19: config changed from index 6 to 19, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
W20260709 12:28:17.471594 20989 log.cc:927] Time spent T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:28:17.474997  9829 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 230693f05eb74e8095e27d669f16c8ee with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 14)
I20260709 12:28:17.473683  9838 catalog_manager.cc:5697] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 29 to 30, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 30 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:17.494827  9838 catalog_manager.cc:5697] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 reported cstate change: term changed from 32 to 33, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194). New cstate: current_term: 33 leader_uuid: "9aa345bcd43b44088114b7b6f2830b17" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:17.520645  8743 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 9970)
I20260709 12:28:17.498920 10675 raft_consensus.cc:2955] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Committing config change with OpId 26.19: config changed from index 6 to 19, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New config: { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } } }
I20260709 12:28:17.487006 20994 consensus_queue.cc:1048] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.117s
W20260709 12:28:17.607100 10643 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:17.607534 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:17.608044 10065 consensus_peers.cc:597] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:17.611274  9990 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 59750.5us
W20260709 12:28:17.629796  9987 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 82204.2us
I20260709 12:28:17.655166 10025 raft_consensus.cc:1275] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:28:17.660816 20927 consensus_queue.cc:1048] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:28:17.673470 20998 log.cc:927] Time spent T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: Append to log took a long time: real 0.178s	user 0.003s	sys 0.000s
I20260709 12:28:17.685434  9835 catalog_manager.cc:5697] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 32 to 33, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 33 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.688210  9835 catalog_manager.cc:5697] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: config changed from index 5 to 17, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 29 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 17 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:17.689054 10065 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:17.776469  9991 consensus_peers.cc:597] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:17.770764  9838 catalog_manager.cc:5697] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 29, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 29 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:17.783987  9838 catalog_manager.cc:5697] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: config changed from index 6 to 19, VOTER cd983415389444e292b2330ef630850a (127.8.137.193) evicted. New cstate: current_term: 26 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:17.860037  9830 catalog_manager.cc:4755] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a7196ffc873e4e7e84c76068d5b95904: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:17.885004  9991 consensus_peers.cc:597] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:17.903012 10065 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:18.002714 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:18.580703 10065 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:18.645602  9991 consensus_peers.cc:597] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:18.689071 10643 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:18.703101  9991 consensus_peers.cc:597] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:18.745426 20919 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:18.746050 20919 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:18.750434 20919 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:18.753325 10065 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:18.767937  9991 consensus_peers.cc:597] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:18.796674 10643 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:18.807163 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17" is_pre_election: true
I20260709 12:28:18.807814 10017 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
W20260709 12:28:18.809875 10643 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:18.814796 10643 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:18.816377 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:18.819531 20983 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:18.821740 20983 raft_consensus.cc:493] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:18.822103 20983 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:18.830060 10065 consensus_peers.cc:597] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:18.831030 20983 raft_consensus.cc:515] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:18.843943 10065 leader_election.cc:336] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:18.846571 20983 leader_election.cc:290] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Requested vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:18.856980 10065 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:18.859623 10017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78df0ad0c1454828a838cd88472c8553" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "9aa345bcd43b44088114b7b6f2830b17"
I20260709 12:28:18.860262 10017 raft_consensus.cc:3060] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:28:18.885434 10017 raft_consensus.cc:2468] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 31.
W20260709 12:28:18.889659 10065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111) [suppressed 426 similar messages]
I20260709 12:28:18.890264 10062 leader_election.cc:304] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17; no voters: cd983415389444e292b2330ef630850a
I20260709 12:28:18.891141 20927 raft_consensus.cc:2804] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:18.891669 20927 raft_consensus.cc:697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:18.892598 20927 consensus_queue.cc:237] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:18.896400 10643 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:18.937601  9835 catalog_manager.cc:5697] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 29 to 31, leader changed from cd983415389444e292b2330ef630850a (127.8.137.193) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 31 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } }
W20260709 12:28:18.996430  9991 consensus_peers.cc:597] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:19.092687  8743 tablet_server.cc:196] TabletServer@127.8.137.193:0 shutdown complete.
W20260709 12:28:19.098745 10065 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:19.126726  9991 consensus_peers.cc:597] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:19.156821  9991 consensus_peers.cc:597] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.162375 10643 consensus_peers.cc:597] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.217263  9991 consensus_peers.cc:597] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.217947  9991 consensus_peers.cc:597] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:19.285940  8743 tablet_server.cc:179] TabletServer@127.8.137.194:0 shutting down...
W20260709 12:28:19.302441 10643 consensus_peers.cc:597] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.319000  9830 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 229811us
W20260709 12:28:19.316221 10065 consensus_peers.cc:597] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.328022 10065 consensus_peers.cc:597] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.341929  9830 catalog_manager.cc:4755] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 230693f05eb74e8095e27d669f16c8ee: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:19.353242 10065 consensus_peers.cc:597] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.357051 10062 consensus_peers.cc:597] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.357888 10062 consensus_peers.cc:597] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.357059 10640 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.362373 10062 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.368678 10640 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.377221 10062 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.381960 10062 consensus_peers.cc:597] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.387424 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:19.389612 10062 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.392036 10062 consensus_peers.cc:597] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.408505 10640 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.416061 10062 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.420508 10640 consensus_peers.cc:597] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.423763 10643 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:19.426268 10640 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.435917 10643 consensus_peers.cc:597] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.514216 10065 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:19.521991 10643 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:19.513769 10099 consensus_queue.cc:237] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
W20260709 12:28:19.536764  9991 consensus_peers.cc:597] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.537392 10062 consensus_peers.cc:597] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.539193 10640 consensus_peers.cc:597] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.543668 10062 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.541723 10065 consensus_peers.cc:597] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.555460 10640 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.581020 10640 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.593375 10640 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.617095 10062 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.618402 10065 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.619184 10062 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979): Couldn't send request to peer 6c7bd3fa1bfa431c89fc9c9096452a76. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8b34dc95d0414e70879dd34eaa8213de. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.640607 10062 consensus_peers.cc:597] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.641463 10062 consensus_peers.cc:597] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.642259 10062 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:19.655754 10643 consensus_peers.cc:597] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.656391 10643 consensus_peers.cc:597] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.662222  9991 consensus_peers.cc:597] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:19.663933 10643 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:19.677093  9991 consensus_peers.cc:597] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.739387 10643 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:19.746918 10640 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:19.754400  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:28:19.751389 10062 consensus_peers.cc:597] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:19.760941  8743 tablet_replica.cc:333] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:19.762573  8743 raft_consensus.cc:2243] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:19.771670  8743 raft_consensus.cc:2272] T ed7d787bc5ad4e8db7719c178ffbc59b P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:19.776994 10640 consensus_peers.cc:597] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:19.794142  8743 tablet_replica.cc:333] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:19.796425  8743 raft_consensus.cc:2243] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:28:19.801878  8743 raft_consensus.cc:2272] T 978cf11d4cd54b67a4fa39b2b089f3da P 9aa345bcd43b44088114b7b6f2830b17 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.816001  8743 tablet_replica.cc:333] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:19.828585  8743 raft_consensus.cc:2243] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:19.834558  8743 raft_consensus.cc:2272] T 8bab9d80dad949d8afde49b123fbac1a P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.838763  8743 tablet_replica.cc:333] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:19.851675  8743 raft_consensus.cc:2243] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Raft consensus shutting down.
W20260709 12:28:19.853132 10640 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:19.862977  8743 raft_consensus.cc:2272] T 0df211fae593420e88f685abc8d618ec P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:19.875331  9991 consensus_peers.cc:597] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.878410  8743 tablet_replica.cc:333] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:19.892272  8743 raft_consensus.cc:2243] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:19.893563  8743 raft_consensus.cc:2272] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.918081  8743 tablet_replica.cc:333] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
W20260709 12:28:19.920010 10643 consensus_peers.cc:597] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.927466  8743 raft_consensus.cc:2243] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:19.928682  8743 raft_consensus.cc:2272] T 8b34dc95d0414e70879dd34eaa8213de P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:19.929734 10065 consensus_peers.cc:597] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:19.945919  8743 tablet_replica.cc:333] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:19.955421  8743 raft_consensus.cc:2243] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:19.956357  8743 raft_consensus.cc:2272] T b5ca4a138b9345ab87fc4055395e0488 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.959800  8743 tablet_replica.cc:333] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:19.964406  8743 raft_consensus.cc:2243] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:19.975657  8743 raft_consensus.cc:2272] T 7e7b0b17a4d646809d4dd46a0268f2eb P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:19.987846  8743 tablet_replica.cc:333] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:19.988788  8743 raft_consensus.cc:2243] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:19.996367  8743 raft_consensus.cc:2272] T 230693f05eb74e8095e27d669f16c8ee P 9aa345bcd43b44088114b7b6f2830b17 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.001823  8743 tablet_replica.cc:333] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.003014  8743 raft_consensus.cc:2243] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 37 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.003997  8743 raft_consensus.cc:2272] T 6a2ff46f3f264308a58e49e2819b1d5b P 9aa345bcd43b44088114b7b6f2830b17 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.007798  8743 tablet_replica.cc:333] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.008904  8743 raft_consensus.cc:2243] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.012225  8743 raft_consensus.cc:2272] T a7196ffc873e4e7e84c76068d5b95904 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.017205  8743 tablet_replica.cc:333] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.018311  8743 raft_consensus.cc:2243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.019387  8743 raft_consensus.cc:2272] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.027707  8743 tablet_replica.cc:333] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
W20260709 12:28:20.028441 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:20.030043  8743 raft_consensus.cc:2243] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.031299  8743 raft_consensus.cc:2272] T 6dfe4ce70e5943448d384222d18a1215 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.035055  8743 tablet_replica.cc:333] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.036039  8743 raft_consensus.cc:2243] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.039139  8743 raft_consensus.cc:2272] T 0fe681129eb440b4925ce53e8c68aaeb P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.042670  8743 tablet_replica.cc:333] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.044193  8743 raft_consensus.cc:2243] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.045918  8743 raft_consensus.cc:2272] T d7b809cb22d54c31ad5985fb9f01e6ab P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.049145  8743 tablet_replica.cc:333] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.050221  8743 raft_consensus.cc:2243] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.051174  8743 raft_consensus.cc:2272] T 51528ebbf4884121ad7c7d564100498f P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.054350  8743 tablet_replica.cc:333] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.055335  8743 raft_consensus.cc:2243] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.056774  8743 raft_consensus.cc:2272] T 208adf3e1124402cb1e6c43ba7306b7b P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:20.063791 10065 consensus_peers.cc:597] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:20.064872  8743 tablet_replica.cc:333] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.065991  8743 raft_consensus.cc:2243] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.067281  8743 raft_consensus.cc:2272] T 6c669b03539649e4a4164ca56748f662 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.089445  8743 tablet_replica.cc:333] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.090320  8743 raft_consensus.cc:2243] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.091898  8743 raft_consensus.cc:2272] T 7253570248b8481791bf2187007dd5df P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.094787  8743 tablet_replica.cc:333] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.095479  8743 raft_consensus.cc:2243] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.096949  8743 raft_consensus.cc:2272] T 447b5d26fe15480dbb8fedf0f7a44777 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.099854  8743 tablet_replica.cc:333] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.100472  8743 raft_consensus.cc:2243] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.101663  8743 raft_consensus.cc:2272] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.104439  8743 tablet_replica.cc:333] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.105347  8743 raft_consensus.cc:2243] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.107916  8743 raft_consensus.cc:2272] T 93686c27d2cf4edea0d5da8d0c1d81ee P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.115424  8743 tablet_replica.cc:333] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.116385  8743 raft_consensus.cc:2243] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.120617  8743 raft_consensus.cc:2272] T 41bba71b483942ed97a9e1276a90514f P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.126765  8743 tablet_replica.cc:333] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.130967  8743 raft_consensus.cc:2243] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.132531  8743 raft_consensus.cc:2272] T bb6fd99043b344de885ff64fd4c81132 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.139330  8743 tablet_replica.cc:333] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.151477  8743 raft_consensus.cc:2243] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.152542  8743 raft_consensus.cc:2272] T d4909dfc06b24d70bfd398607779f5ae P 9aa345bcd43b44088114b7b6f2830b17 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.155826  8743 tablet_replica.cc:333] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.156637  8743 raft_consensus.cc:2243] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.157413  8743 raft_consensus.cc:2272] T afb4bd9b5d6240b19d09014b985ea7f7 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.160375  8743 tablet_replica.cc:333] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.161283  8743 raft_consensus.cc:2243] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.162204  8743 raft_consensus.cc:2272] T a1e3e80f2cad43f08196a18ad2bee6a4 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.169728  8743 tablet_replica.cc:333] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.175335  8743 raft_consensus.cc:2243] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.176656  8743 raft_consensus.cc:2272] T 9f126305b66248b59cd9ce87bc952694 P 9aa345bcd43b44088114b7b6f2830b17 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.182639  8743 tablet_replica.cc:333] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.183561  8743 raft_consensus.cc:2243] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.184954  8743 raft_consensus.cc:2272] T 86c102903b3d4742a0b9ac1df1132361 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.187942  8743 tablet_replica.cc:333] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.199270  8743 raft_consensus.cc:2243] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.200892  8743 raft_consensus.cc:2272] T 59e61711ca80440e9c13392c6b6b25fa P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.203981  8743 tablet_replica.cc:333] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.204871  8743 raft_consensus.cc:2243] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.206251  8743 raft_consensus.cc:2272] T db738483c1444059b752648d302dc0d9 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.209964  8743 tablet_replica.cc:333] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.219295  8743 raft_consensus.cc:2243] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.219848 21028 ts_tablet_manager.cc:933] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: Initiating tablet copy from peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:20.220777  8743 raft_consensus.cc:2272] T 15458ba5804c4c36b01b169af77b2691 P 9aa345bcd43b44088114b7b6f2830b17 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.221171 20930 raft_consensus.cc:493] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:20.221889 20930 raft_consensus.cc:515] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:20.224049  8743 tablet_replica.cc:333] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.224996  8743 raft_consensus.cc:2243] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.228760 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf49747e9a8542e6b75e3545a18c0981" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:20.233083  9991 leader_election.cc:336] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:20.228722  8743 raft_consensus.cc:2272] T 9bd550066b4447f8848484defc31b9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.233608  9991 leader_election.cc:304] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:20.234607 20930 leader_election.cc:290] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:20.235430 21013 raft_consensus.cc:2749] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:20.237423  8743 tablet_replica.cc:333] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.239426  8743 raft_consensus.cc:2243] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.246229 21028 tablet_copy_client.cc:323] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Beginning tablet copy session from remote peer at address 127.8.137.195:33983
I20260709 12:28:20.251600  8743 raft_consensus.cc:2272] T d6726f0982834ecf8987472ed6c32a73 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:20.251668 10643 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:20.253472 10109 tablet_copy_service.cc:140] P 4083ce96cefb47d0be1d8e99602b7ce7: Received BeginTabletCopySession request for tablet 8b34dc95d0414e70879dd34eaa8213de from peer 6c7bd3fa1bfa431c89fc9c9096452a76 ({username='slave'} at 127.0.0.1:57396)
I20260709 12:28:20.255182  8743 tablet_replica.cc:333] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.254558 10109 tablet_copy_service.cc:161] P 4083ce96cefb47d0be1d8e99602b7ce7: Beginning new tablet copy session on tablet 8b34dc95d0414e70879dd34eaa8213de from peer 6c7bd3fa1bfa431c89fc9c9096452a76 at {username='slave'} at 127.0.0.1:57396: session id = 6c7bd3fa1bfa431c89fc9c9096452a76-8b34dc95d0414e70879dd34eaa8213de
I20260709 12:28:20.256876  8743 raft_consensus.cc:2243] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.258293  8743 raft_consensus.cc:2272] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.258724 20930 raft_consensus.cc:493] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 6c7bd3fa1bfa431c89fc9c9096452a76)
I20260709 12:28:20.259485 20930 raft_consensus.cc:515] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:20.261509  8743 tablet_replica.cc:333] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.262377  8743 raft_consensus.cc:2243] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.264683 20930 leader_election.cc:290] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:20.266887  8743 raft_consensus.cc:2272] T 062c203e13aa4527a094ec24339001e5 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:20.270390  9991 leader_election.cc:336] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:20.271124 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4cd31c96c4854a87a361975b579bc1c2" candidate_uuid: "9aa345bcd43b44088114b7b6f2830b17" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:20.273131  9987 leader_election.cc:304] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9aa345bcd43b44088114b7b6f2830b17; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, cd983415389444e292b2330ef630850a
I20260709 12:28:20.270530  8743 tablet_replica.cc:333] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.275245 20930 raft_consensus.cc:2749] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:20.276044  8743 raft_consensus.cc:2243] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.287220  8743 raft_consensus.cc:2272] T eb955e753b7546bf8612b6ed1c3fc342 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.290370  8743 tablet_replica.cc:333] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.291374  8743 raft_consensus.cc:2243] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.292277  8743 raft_consensus.cc:2272] T 53edc906d1af41e0992f01434995f9d1 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.295892  8743 tablet_replica.cc:333] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.296923  8743 raft_consensus.cc:2243] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.302136 10109 tablet_copy_source_session.cc:215] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:28:20.302973  8743 raft_consensus.cc:2272] T d8453bea911c4756add721edebd5d124 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.327724  8743 tablet_replica.cc:333] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.328182 21028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b34dc95d0414e70879dd34eaa8213de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:20.328823  8743 raft_consensus.cc:2243] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.329715  8743 raft_consensus.cc:2272] T ab118912551248c1985fce40a2e815b0 P 9aa345bcd43b44088114b7b6f2830b17 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.333137  8743 tablet_replica.cc:333] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.334023  8743 raft_consensus.cc:2243] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.346724  8743 raft_consensus.cc:2272] T bf49747e9a8542e6b75e3545a18c0981 P 9aa345bcd43b44088114b7b6f2830b17 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.349984  8743 tablet_replica.cc:333] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.350872  8743 raft_consensus.cc:2243] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.351691  8743 raft_consensus.cc:2272] T 4cd31c96c4854a87a361975b579bc1c2 P 9aa345bcd43b44088114b7b6f2830b17 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:20.353523 10065 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:20.354516  8743 tablet_replica.cc:333] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.355372  8743 raft_consensus.cc:2243] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.356088  8743 raft_consensus.cc:2272] T 78df0ad0c1454828a838cd88472c8553 P 9aa345bcd43b44088114b7b6f2830b17 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.359429  8743 tablet_replica.cc:333] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.360208  8743 raft_consensus.cc:2243] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.360881  8743 raft_consensus.cc:2272] T 26e02c55bc5648e0baa2a4216f02f955 P 9aa345bcd43b44088114b7b6f2830b17 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.363924  8743 tablet_replica.cc:333] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.364730  8743 raft_consensus.cc:2243] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:20.366367  8743 raft_consensus.cc:2272] T d43a22dba58d423d907a568238e85284 P 9aa345bcd43b44088114b7b6f2830b17 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.370404  8743 tablet_replica.cc:333] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.380833  8743 raft_consensus.cc:2243] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.381682  8743 raft_consensus.cc:2272] T 886f3cb002374eae906fdbe2d392f3e7 P 9aa345bcd43b44088114b7b6f2830b17 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.384873  8743 tablet_replica.cc:333] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17: stopping tablet replica
I20260709 12:28:20.379315 21028 tablet_copy_client.cc:806] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 0 data blocks...
I20260709 12:28:20.395196 21028 tablet_copy_client.cc:670] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Starting download of 1 WAL segments...
I20260709 12:28:20.395268  8743 raft_consensus.cc:2243] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:20.396622  8743 raft_consensus.cc:2272] T aa5a16c274db44a5aa641b51f71f7ec6 P 9aa345bcd43b44088114b7b6f2830b17 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:20.401403 21028 tablet_copy_client.cc:538] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:28:20.420058 21028 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap starting.
W20260709 12:28:20.455433  9830 catalog_manager.cc:4755] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 230693f05eb74e8095e27d669f16c8ee: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:20.645011 21028 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=1 applied=18 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:28:20.646261 21028 tablet_bootstrap.cc:492] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: Bootstrap complete.
I20260709 12:28:20.647125 21028 ts_tablet_manager.cc:1403] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent bootstrapping tablet: real 0.227s	user 0.058s	sys 0.008s
I20260709 12:28:20.653956 21028 raft_consensus.cc:359] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:20.659474 21028 raft_consensus.cc:740] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 LEARNER]: Becoming Follower/Learner. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Initialized, Role: LEARNER
I20260709 12:28:20.660400 21028 consensus_queue.cc:260] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 18, Last appended: 31.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } }
I20260709 12:28:20.673506 21028 ts_tablet_manager.cc:1434] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:28:20.677654 10109 tablet_copy_service.cc:342] P 4083ce96cefb47d0be1d8e99602b7ce7: Request end of tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-8b34dc95d0414e70879dd34eaa8213de received from {username='slave'} at 127.0.0.1:57396
I20260709 12:28:20.678246 10109 tablet_copy_service.cc:434] P 4083ce96cefb47d0be1d8e99602b7ce7: ending tablet copy session 6c7bd3fa1bfa431c89fc9c9096452a76-8b34dc95d0414e70879dd34eaa8213de on tablet 8b34dc95d0414e70879dd34eaa8213de with peer 6c7bd3fa1bfa431c89fc9c9096452a76
I20260709 12:28:20.785410 10668 raft_consensus.cc:1217] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 LEARNER]: Deduplicated request from leader. Original: 31.18->[31.19-31.19]   Dedup: 31.19->[]
W20260709 12:28:20.846167 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:21.005405 10643 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:21.034965 20972 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.035607 20972 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.038472 20972 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:21.039149 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
W20260709 12:28:21.042834 10640 leader_election.cc:336] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.043429 10640 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:21.044322 20972 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:28:21.070421 21025 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.071053 21025 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.071424 21014 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.071956 21014 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:21.074100 21014 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:21.074313 21025 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:21.075961 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:21.076012 20972 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.079716 20972 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.084125 20972 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:21.086355 10065 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:21.092839 10062 leader_election.cc:336] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.094947 10062 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:21.096669 21025 raft_consensus.cc:2749] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:21.096778 10062 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:21.094223 10643 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:21.099030 10062 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:21.101807 21025 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:21.113716 10640 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.114543 21025 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.115350 21025 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:21.117652 21025 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:21.125208 10065 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:21.146778 10065 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:21.167775 10640 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:21.176544 21025 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.177215 21025 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:21.179706 20972 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:21.180506 21025 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:21.182246 21026 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.182818 21026 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:21.182863 10062 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.186002 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:21.186764 21026 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:21.185423 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 71734.3us
I20260709 12:28:21.193002 21025 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:21.200366 10065 leader_election.cc:336] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:21.203683 21025 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.207028 21025 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:21.209625 21025 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:21.214521 10643 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:21.216653 10640 leader_election.cc:336] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.219154 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:21.225273 10091 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 32.
I20260709 12:28:21.227566 21025 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.228252 21025 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:21.230574 21025 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:21.234544 10062 leader_election.cc:336] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.244733 10062 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:28:21.244879 10643 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:21.250519 21025 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:21.252120 20972 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:28:21.252900 10065 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:21.252665 20972 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.254889 20972 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:21.259569 10643 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:21.262202 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:21.262907 20972 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.263429 21026 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:28:21.263860 21026 raft_consensus.cc:493] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.264210 21026 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:28:21.269235 10062 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.269761 21025 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.278156 21025 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.282891 21025 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:21.283560 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:21.263451 20972 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.286862 20972 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:21.273813 21026 raft_consensus.cc:515] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.286972 21035 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.288386 21035 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.289903 21026 leader_election.cc:290] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:21.277307 10065 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:21.290791 10065 consensus_peers.cc:597] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.291563 21035 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:21.298887 10643 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:21.302327 10640 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.302985 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:21.299418 20972 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.300050 21035 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.304041 20929 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:21.305687 21035 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.307765 20972 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.311555 21035 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:21.312106 21025 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:21.314848 20972 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:21.314965 21014 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.320097 21014 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.322569 21014 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
W20260709 12:28:21.332841 10640 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.333395 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:21.333869 20972 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.334151 21035 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:21.334810 10640 leader_election.cc:336] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:21.335371 10640 leader_election.cc:336] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.337109 20972 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:21.339329 10640 leader_election.cc:336] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.583290 21035 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.584115 21035 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.590504 21035 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:21.721561  8743 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 10046)
I20260709 12:28:21.831346 21038 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:28:21.831774 10643 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:21.832304 10643 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:21.832032 21038 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.834414 21038 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:21.862926 21042 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.863592 21042 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.867879 21042 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:21.878509 20972 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:21.889576 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94aa8e92ea8d4f81a4ed4dbb1e99635d" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:21.890313 10091 raft_consensus.cc:3060] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:28:21.890451 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:21.891259 10093 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:28:21.893765 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:21.894402 10090 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
W20260709 12:28:21.896966 10065 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.003s	sys 0.002s
I20260709 12:28:21.899166 10674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:21.899832 10674 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 35.
W20260709 12:28:21.904613 10640 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 565279us
I20260709 12:28:21.904901 10091 raft_consensus.cc:2468] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 33.
W20260709 12:28:21.905714 10640 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.906260 10640 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:21.907112 20972 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:21.907392 10065 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.003s	sys 0.005s
I20260709 12:28:21.915721 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:21.916389 10668 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 28.
I20260709 12:28:21.917840 10094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:21.918478 10094 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 28.
W20260709 12:28:21.923919 10643 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:21.924903 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:21.926802 20972 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:28:21.927156 20972 raft_consensus.cc:493] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.927493 20972 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:21.930058 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:21.930466 10099 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" is_pre_election: true
I20260709 12:28:21.931098 10099 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 29.
W20260709 12:28:21.931186 10643 consensus_peers.cc:597] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:21.937813 10643 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:21.943841 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:21.946245 20972 raft_consensus.cc:515] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.951293 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:21.951655 10675 raft_consensus.cc:2393] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 31: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:21.952363 21040 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:21.952966 21040 raft_consensus.cc:493] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.953367 21040 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:21.958405 20972 leader_election.cc:290] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:21.959507 10642 leader_election.cc:304] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:21.963339 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:28:21.961932 10640 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.964079 20929 raft_consensus.cc:2804] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:28:21.965008 21039 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:21.965246 20929 raft_consensus.cc:697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:21.965438 21039 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.965940 21039 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:21.966427 20929 consensus_queue.cc:237] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:21.969458 10062 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.970033 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:21.970532 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:28:21.971722 10640 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:21.973248 10640 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:21.974020 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:21.972347 20972 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:28:21.960690 10643 consensus_peers.cc:597] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:21.974884 21038 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:21.975361 21038 raft_consensus.cc:493] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:21.975775 21038 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:21.974440 10640 leader_election.cc:336] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:21.978462 21039 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.975117 21045 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:21.982990 21039 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:21.980041 21040 raft_consensus.cc:515] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.985457 21040 leader_election.cc:290] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:21.990490 10090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fe681129eb440b4925ce53e8c68aaeb" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:21.985507 21038 raft_consensus.cc:515] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:21.991255 10090 raft_consensus.cc:3060] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:28:22.000571 10643 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:22.000665 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:22.001969 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
W20260709 12:28:22.003921 10062 leader_election.cc:336] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.004343 10062 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:21.990082 10640 consensus_peers.cc:597] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.005756 21014 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:28:22.005965 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:22.006271 21041 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:28:22.006345 21014 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:22.006773 21041 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:22.006879 21014 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:28:22.007205 21041 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:28:22.005893 10091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d8453bea911c4756add721edebd5d124" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:22.008196 10091 raft_consensus.cc:3060] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.016629 21025 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:28:22.024578 21038 leader_election.cc:290] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:22.034432 10090 raft_consensus.cc:2468] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 31.
I20260709 12:28:22.035264 21044 raft_consensus.cc:2749] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:22.039788 10091 raft_consensus.cc:2468] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:28:22.049821 21044 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:22.050351 21044 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:22.052496 21044 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:22.036845 21041 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:22.040308  9835 catalog_manager.cc:5697] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 32 to 33, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 33 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:28:22.060633 21041 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:22.038571 21014 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:22.062716 21045 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:22.063624 21014 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:22.063298 21045 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:22.064633 10093 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in current term 29: Already voted for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in this term.
I20260709 12:28:22.065737 21045 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:22.069233 21045 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:22.070075 21045 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:22.072592 21045 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:22.082880 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:35115, user_credentials={real_user=slave}} blocked reactor thread for 78984.7us
I20260709 12:28:22.099638 10642 leader_election.cc:304] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:22.102416 20929 raft_consensus.cc:2804] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:28:22.103055 20929 raft_consensus.cc:697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:22.104156 20929 consensus_queue.cc:237] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:22.104609 21041 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:22.105361 21041 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:22.107659 21041 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:22.114310 10640 leader_election.cc:336] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:22.115408 10640 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.120586 10642 leader_election.cc:304] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 
I20260709 12:28:22.122963 21038 raft_consensus.cc:2804] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:22.123661 21038 raft_consensus.cc:697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:22.124794 21038 consensus_queue.cc:237] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:22.126078 10065 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:22.128775 10643 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:22.126595 10065 consensus_peers.cc:597] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:22.132102 10065 leader_election.cc:336] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:22.133764 10065 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:22.134262 10065 leader_election.cc:336] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:22.134845 10065 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:22.137794 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:22.139626 21038 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:28:22.141168 10065 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:22.135334 10640 leader_election.cc:336] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:22.152767 10062 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.153168 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "978cf11d4cd54b67a4fa39b2b089f3da" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 36 candidate_status { last_received { term: 35 index: 21 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
W20260709 12:28:22.153841 10062 leader_election.cc:336] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.153859 10675 raft_consensus.cc:3060] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:28:22.160082 10093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1154fd85c1e440b7bb5ee236ac1e0a21" candidate_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "4083ce96cefb47d0be1d8e99602b7ce7"
I20260709 12:28:22.160676 10093 raft_consensus.cc:3060] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:22.162954 10675 raft_consensus.cc:2468] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 36.
I20260709 12:28:22.164146 10668 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 29 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:22.177412 10668 raft_consensus.cc:2393] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 4083ce96cefb47d0be1d8e99602b7ce7 in current term 29: Already voted for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in this term.
I20260709 12:28:22.169019 10093 raft_consensus.cc:2468] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c7bd3fa1bfa431c89fc9c9096452a76 in term 30.
I20260709 12:28:22.168581  9835 catalog_manager.cc:5697] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 30 to 31, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 31 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:22.174710 10062 leader_election.cc:336] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.178700 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:22.179620 21050 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:22.179411  9835 catalog_manager.cc:5697] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 29 to 30, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 30 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:22.180779 10062 leader_election.cc:336] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:22.181288 10062 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:22.181699 10062 leader_election.cc:336] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.182053 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:22.182731 21050 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:22.183632 10062 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.184068 10062 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.184417 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:22.186058 21050 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:22.188045 10640 leader_election.cc:336] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:22.188761 10062 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.191473 10642 leader_election.cc:304] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:22.192678 20929 raft_consensus.cc:2804] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:22.193152 20929 raft_consensus.cc:697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Becoming Leader. State: Replica: 6c7bd3fa1bfa431c89fc9c9096452a76, State: Running, Role: LEADER
I20260709 12:28:22.193986 20929 consensus_queue.cc:237] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:22.198755 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 6c7bd3fa1bfa431c89fc9c9096452a76, 9aa345bcd43b44088114b7b6f2830b17
W20260709 12:28:22.199576 10640 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.199719 21050 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:28:22.200578 10062 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:22.201306 21050 raft_consensus.cc:2804] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Leader election won for term 36
W20260709 12:28:22.201738 10062 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.201803 21050 raft_consensus.cc:697] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:22.202883 21050 consensus_queue.cc:237] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 35.21, Last appended by leader: 21, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:22.206725 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:22.207504 21041 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:22.270520 10643 consensus_peers.cc:597] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:22.274456  9835 catalog_manager.cc:5697] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 reported cstate change: term changed from 29 to 30, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196). New cstate: current_term: 30 leader_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:28:22.276494 10640 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.279624 10640 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.296454 10062 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.289510  9838 catalog_manager.cc:5697] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 35 to 36, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 36 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:22.307942 10640 consensus_peers.cc:597] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.311482 10640 consensus_peers.cc:597] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.319415 10065 consensus_peers.cc:597] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:22.330202 10062 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.339408 10640 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.351140 10065 consensus_peers.cc:597] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.353598 10062 consensus_peers.cc:597] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.356374 10643 consensus_peers.cc:597] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:22.357123  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800175700 after lost signal to thread 8746
I20260709 12:28:22.358109 10091 raft_consensus.cc:1275] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
W20260709 12:28:22.358295  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08003e9e80 after lost signal to thread 9879
W20260709 12:28:22.358724  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08004134a0 after lost signal to thread 9894
W20260709 12:28:22.359532 10065 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.359974  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800329480 after lost signal to thread 10123
I20260709 12:28:22.359589 20929 consensus_queue.cc:1048] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:28:22.361253  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08003c0f80 after lost signal to thread 10702
W20260709 12:28:22.374487 10062 consensus_peers.cc:597] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.375173 10643 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:22.377826 10062 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.383996 10640 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.395655 10640 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.401923 10643 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:22.410852 21050 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:28:22.437741 10640 consensus_peers.cc:597] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.438361 10640 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.451830 10640 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.475828 10065 consensus_peers.cc:597] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:22.490104 10062 consensus_peers.cc:597] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.500748 10062 consensus_peers.cc:597] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:22.501806 10091 raft_consensus.cc:1275] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 31 index: 21. (index mismatch)
I20260709 12:28:22.506572 21040 consensus_queue.cc:1048] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
W20260709 12:28:22.516739 10062 consensus_peers.cc:597] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.552476 10640 consensus_peers.cc:597] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.559839 10640 consensus_peers.cc:597] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.590936 10062 consensus_peers.cc:597] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.597981  9830 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.8.137.193:34909, user_credentials={real_user=slave}} blocked reactor thread for 44487.8us
W20260709 12:28:22.609495  9830 catalog_manager.cc:4755] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a7196ffc873e4e7e84c76068d5b95904: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:22.610713 10062 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.619647 10640 consensus_peers.cc:597] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.620421 10643 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:22.625692 10093 raft_consensus.cc:1275] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:28:22.628458 21040 consensus_queue.cc:1048] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 12:28:22.630882 10062 consensus_peers.cc:597] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:22.654336 10065 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:28:22.683701 10668 raft_consensus.cc:1275] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 36 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 35 index: 21. Preceding OpId from leader: term: 36 index: 22. (index mismatch)
I20260709 12:28:22.685308 21048 consensus_queue.cc:1048] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:28:22.708598 10062 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:22.722198  8743 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 10046)
I20260709 12:28:22.744899 10093 raft_consensus.cc:1275] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Refusing update from remote peer 6c7bd3fa1bfa431c89fc9c9096452a76: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:28:22.746647 21038 consensus_queue.cc:1048] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
W20260709 12:28:22.812371 10643 consensus_peers.cc:597] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:22.820928 21041 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:28:22.932137 21038 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:22.932848 21038 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:22.935356 21038 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:22.945079 10643 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:22.950388 10640 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:22.951659 10640 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:22.954932 21038 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:22.960229 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:22.972189 10062 consensus_peers.cc:597] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:23.009675 10643 consensus_peers.cc:597] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:23.085886 10065 consensus_peers.cc:597] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:23.102716 10062 consensus_peers.cc:597] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:23.122474 10065 consensus_peers.cc:597] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:23.148975 21041 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.149634 21041 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:23.151935 21041 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:23.163329 10062 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:23.168356 10065 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:23.170646 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:23.172020 21041 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:23.228168 21041 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.228701 21041 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:23.229461 21050 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:28:23.230887 21041 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:23.252575 10062 leader_election.cc:336] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:23.314646 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:23.315995 21040 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.316545 21040 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:23.318862 21040 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:23.324961 10640 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:23.330147 10643 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:23.330770 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:23.332602 21040 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:23.333799 10065 leader_election.cc:336] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:23.340232 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:23.341224 21068 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:23.402704 21048 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.403316 21048 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:23.405648 21048 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:23.421052 10065 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:23.429114 10062 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:23.431037 10062 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:23.432013 21048 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:28:23.450093 21038 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.450788 21038 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:23.453244 21038 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:23.464118 10643 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:23.482789 10640 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:23.490813 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:23.492641 21038 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:23.539557 10065 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:23.564421 21048 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.565094 21048 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:23.568691 21048 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:23.575839 10065 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:23.581720 21040 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.582314 21040 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:23.584555 21040 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:23.586802 10062 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:23.589207 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:28:23.592203 10640 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:23.596053 21068 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:28:23.602207 10643 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:23.602840 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:23.604075 21038 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:23.666520 10643 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:23.710263 21068 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.710985 21068 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
W20260709 12:28:23.721848 10065 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:23.722680  8743 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 10046)
I20260709 12:28:23.723498 21068 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:23.744356 10062 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:23.745882 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:23.748458 21068 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:23.756385 21068 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:28:23.820688 10065 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:23.830504 10643 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:23.841871 21069 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:23.842473 21069 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:23.844746 21069 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:23.849301 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" is_pre_election: true
I20260709 12:28:23.849985 10675 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 29.
W20260709 12:28:23.854799 10643 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:23.851709 10062 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:23.858404 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:23.859763 21069 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:28:23.860219 21069 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:23.860649 21069 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:28:23.889706 10640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111) [suppressed 644 similar messages]
I20260709 12:28:23.896759 21069 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:23.898936 21069 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Requested vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 6c7bd3fa1bfa431c89fc9c9096452a76 (127.8.137.196:36979)
I20260709 12:28:23.902204 10675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15458ba5804c4c36b01b169af77b2691" candidate_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" candidate_term: 30 candidate_status { last_received { term: 28 index: 22 } } ignore_live_leader: false dest_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76"
I20260709 12:28:23.902930 10675 raft_consensus.cc:3060] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:28:23.914041 10675 raft_consensus.cc:2468] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4083ce96cefb47d0be1d8e99602b7ce7 in term 30.
W20260709 12:28:23.942040 10062 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:23.943800 10062 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7, 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:23.944756 21048 raft_consensus.cc:2804] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:28:23.945302 21048 raft_consensus.cc:697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Becoming Leader. State: Replica: 4083ce96cefb47d0be1d8e99602b7ce7, State: Running, Role: LEADER
I20260709 12:28:23.946309 21048 consensus_queue.cc:237] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 28.22, Last appended by leader: 22, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:23.969067 10065 consensus_peers.cc:597] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:23.995414 21068 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:23.996184 21068 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:23.998826 21068 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:24.005116 10065 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:24.015690 10062 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36979, user_credentials={real_user=slave}} blocked reactor thread for 70380.5us
I20260709 12:28:24.019901 21069 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:24.020442 21069 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:24.020443  9835 catalog_manager.cc:5697] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 reported cstate change: term changed from 28 to 30, leader changed from 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194) to 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195). New cstate: current_term: 30 leader_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:24.031728 21069 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:24.033900 10065 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:24.049038 21071 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:24.049777 21071 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:24.052189 21071 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:24.053967 10062 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:24.058812 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
W20260709 12:28:24.060613 10643 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:24.060262 21069 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:24.069433 10643 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:24.070852 10640 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:24.073149 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:24.074146 21071 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:28:24.079301 10062 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:24.079888 10062 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:24.081815 21072 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:24.119488 21072 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:28:24.219141 10643 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:24.226927 21048 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:24.227519 21048 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:24.229559 21048 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:24.235881 10062 leader_election.cc:336] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:24.256903 10065 leader_election.cc:336] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:24.258244 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:24.257642 10065 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:24.259789 21048 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:24.324949 10062 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.391850 10065 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:24.392802 21048 raft_consensus.cc:493] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:24.393487 21048 raft_consensus.cc:515] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:24.395874 21048 leader_election.cc:290] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:24.401364 10065 leader_election.cc:336] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:24.436630 10640 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.438964 10062 leader_election.cc:336] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:24.442551 10062 leader_election.cc:304] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:24.460590 21072 raft_consensus.cc:2749] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:24.472900 10643 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:24.500617 10065 consensus_peers.cc:597] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.505394 10640 consensus_peers.cc:597] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.523811 10643 consensus_peers.cc:597] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.538902 10062 consensus_peers.cc:597] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:24.540934 10062 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.547253 10640 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.593449 10643 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:24.597975 10640 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.600739 10668 raft_consensus.cc:1275] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Refusing update from remote peer 4083ce96cefb47d0be1d8e99602b7ce7: Log matching property violated. Preceding OpId in replica: term: 28 index: 22. Preceding OpId from leader: term: 30 index: 23. (index mismatch)
I20260709 12:28:24.603420 21048 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:28:24.610821 10640 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.623263 10065 consensus_peers.cc:597] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:24.662145 21068 consensus_queue.cc:1048] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
W20260709 12:28:24.675289 10640 consensus_peers.cc:597] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.715577 10065 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:24.723160  8743 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 10046)
W20260709 12:28:24.736727 10643 consensus_peers.cc:597] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.763803 10065 consensus_peers.cc:597] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.808041 10640 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.814392 10643 consensus_peers.cc:597] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.826145 10640 consensus_peers.cc:597] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.847599 10062 consensus_peers.cc:597] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.855734 10643 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.872462 10643 consensus_peers.cc:597] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:24.873867 10065 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.902611 10062 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:24.914904 21068 raft_consensus.cc:493] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
W20260709 12:28:24.915172 10643 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:28:24.915612 21068 raft_consensus.cc:515] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:24.918089 21068 leader_election.cc:290] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:24.936116 10062 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:24.947520 10640 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.961578 10062 consensus_peers.cc:597] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:24.962293 10640 consensus_peers.cc:597] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.008747 10640 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.241225 10065 leader_election.cc:336] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:25.241798 10065 leader_election.cc:304] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:25.242647 21048 raft_consensus.cc:2749] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:25.248414 10062 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.249984 10062 consensus_peers.cc:597] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.250519 10062 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.251111 10062 consensus_peers.cc:597] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.251576 10062 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.252151 10062 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.252595 10062 consensus_peers.cc:597] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.253083 10062 consensus_peers.cc:597] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.253604 10062 consensus_peers.cc:597] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.281656 10065 consensus_peers.cc:597] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:25.300311 10643 consensus_peers.cc:597] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:25.311918 10643 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:25.318085 10640 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.324644 10640 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:25.325440 21048 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:28:25.325525 10640 consensus_peers.cc:597] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:25.326399 10640 consensus_peers.cc:597] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:25.345978 21090 raft_consensus.cc:493] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:25.346626 21090 raft_consensus.cc:515] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:25.349138 21090 leader_election.cc:290] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:25.354359 10643 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:25.356765 10640 leader_election.cc:336] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:25.357254 10640 leader_election.cc:304] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:25.358060 21091 raft_consensus.cc:2749] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:25.389851 21068 raft_consensus.cc:493] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:25.390569 21068 raft_consensus.cc:515] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:25.393141 21068 leader_election.cc:290] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:25.401400 10062 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:25.404814 10065 leader_election.cc:336] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:25.413367 10065 leader_election.cc:304] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:25.414309 21068 raft_consensus.cc:2749] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:28:25.475040 10640 consensus_peers.cc:597] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:28:25.490796 21068 raft_consensus.cc:493] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:25.492779 21068 raft_consensus.cc:515] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:25.495391 21068 leader_election.cc:290] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:25.501029 10065 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:25.502039 10062 leader_election.cc:336] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:25.502633 10062 leader_election.cc:304] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:25.503506 21068 raft_consensus.cc:2749] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:28:25.595476 21068 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:28:25.607776 10065 consensus_peers.cc:597] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:25.608378 10065 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:25.615088 10643 consensus_peers.cc:597] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:25.629791 10062 consensus_peers.cc:597] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.686808 10065 consensus_peers.cc:597] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:28:25.688642 10062 consensus_peers.cc:597] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:25.693193 10065 consensus_peers.cc:597] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:25.723737  8743 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 10046)
I20260709 12:28:25.823768 21048 raft_consensus.cc:493] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:25.824332 21048 raft_consensus.cc:515] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:25.827776 21048 leader_election.cc:290] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:25.835703 10065 leader_election.cc:336] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:25.840309 10062 leader_election.cc:336] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:25.840862 10062 leader_election.cc:304] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:25.841917 21048 raft_consensus.cc:2749] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:28:25.861092 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:25.970175 10065 consensus_peers.cc:597] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:26.019389 10120 debug-util.cc:398] Leaking SignalData structure 0x7b080032d520 after lost signal to thread 8746
W20260709 12:28:26.020514 10120 debug-util.cc:398] Leaking SignalData structure 0x7b0800325e00 after lost signal to thread 9879
W20260709 12:28:26.020864 10120 debug-util.cc:398] Leaking SignalData structure 0x7b080030e780 after lost signal to thread 9894
W20260709 12:28:26.022112 10120 debug-util.cc:398] Leaking SignalData structure 0x7b0800346900 after lost signal to thread 10123
W20260709 12:28:26.023248 10120 debug-util.cc:398] Leaking SignalData structure 0x7b08002b27e0 after lost signal to thread 10702
I20260709 12:28:26.145419 21048 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:28:26.148631 10643 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:26.189404 21072 raft_consensus.cc:493] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:26.190042 21072 raft_consensus.cc:515] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:26.194082 21072 leader_election.cc:290] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:26.215790 10062 leader_election.cc:336] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:26.221402 10065 leader_election.cc:336] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:26.222378 10065 leader_election.cc:304] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:26.224013 21072 raft_consensus.cc:2749] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:26.227502 21090 raft_consensus.cc:493] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:26.228235 21090 raft_consensus.cc:515] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:26.235996 10640 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:26.236840 21090 leader_election.cc:290] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:26.259428 10643 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:26.260058 10643 leader_election.cc:336] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:26.260535 10643 leader_election.cc:304] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:26.262638 21090 raft_consensus.cc:2749] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:28:26.331036 21072 raft_consensus.cc:493] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:26.331611 21072 raft_consensus.cc:515] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:26.335415 21072 leader_election.cc:290] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:26.353371 10065 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:26.390919 10062 leader_election.cc:336] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:26.391489 10062 leader_election.cc:304] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:26.392235 21072 raft_consensus.cc:2749] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:26.554536 21090 raft_consensus.cc:493] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:26.555295 21090 raft_consensus.cc:515] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:26.560109 21090 leader_election.cc:290] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:26.578774 10640 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:26.585408 10065 consensus_peers.cc:597] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:26.588703 10643 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:26.675956 10643 leader_election.cc:336] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:26.688601 10643 leader_election.cc:304] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:26.695874 21090 raft_consensus.cc:2749] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:28:26.712481 10643 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:26.722122 10643 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:26.730635 10065 consensus_peers.cc:597] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:26.777463 10643 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:26.809814 10640 consensus_peers.cc:597] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:26.810727 10643 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:28:26.844784  9830 catalog_manager.cc:4755] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a7196ffc873e4e7e84c76068d5b95904: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:26.847445 21048 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260709 12:28:26.859938 21095 raft_consensus.cc:493] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:26.860553 21095 raft_consensus.cc:515] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:26.862833 21095 leader_election.cc:290] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:26.864326 10640 consensus_peers.cc:597] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:26.867717 10640 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:26.869172  8743 tablet_server.cc:196] TabletServer@127.8.137.194:0 shutdown complete.
W20260709 12:28:26.893059 10062 consensus_peers.cc:597] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:26.897450 10062 consensus_peers.cc:597] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:26.906805 10643 leader_election.cc:336] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:26.907387 10643 leader_election.cc:304] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:26.908233 21095 raft_consensus.cc:2749] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:28:26.925966 10062 consensus_peers.cc:597] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:28:26.930368 10640 consensus_peers.cc:597] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:26.965576 21072 raft_consensus.cc:493] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:26.966387 21072 raft_consensus.cc:515] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:26.969388 21072 leader_election.cc:290] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:26.974654 10642 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:33983, user_credentials={real_user=slave}} blocked reactor thread for 78493.4us
W20260709 12:28:26.975251 10062 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:26.996490 10065 consensus_peers.cc:597] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.019641 10640 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.022706 10065 consensus_peers.cc:597] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:27.023204 10065 leader_election.cc:336] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:27.023588 10065 leader_election.cc:304] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:27.024268 21068 raft_consensus.cc:2749] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:27.058933 10643 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:28:27.068612  8743 tablet_server.cc:179] TabletServer@127.8.137.195:0 shutting down...
I20260709 12:28:27.078550 21072 raft_consensus.cc:1027] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: attempt to promote peer 6c7bd3fa1bfa431c89fc9c9096452a76: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
W20260709 12:28:27.090276 10642 consensus_peers.cc:597] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:27.096930 10640 consensus_peers.cc:597] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.122390 10642 consensus_peers.cc:597] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:27.128895 10640 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.128885 10062 consensus_peers.cc:597] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.133569 10640 consensus_peers.cc:597] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:27.152175 21097 raft_consensus.cc:493] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:27.152845 21097 raft_consensus.cc:515] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
W20260709 12:28:27.156129 10642 consensus_peers.cc:597] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.155207 21097 leader_election.cc:290] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
W20260709 12:28:27.169984 10643 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:27.171283 10642 consensus_peers.cc:597] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:27.172674 10640 leader_election.cc:336] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:27.173169 10640 leader_election.cc:304] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:27.174382 21097 raft_consensus.cc:2749] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:28:27.221127  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:27.224823  8743 tablet_replica.cc:333] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.225745  8743 raft_consensus.cc:2243] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.227326  8743 raft_consensus.cc:2272] T 978cf11d4cd54b67a4fa39b2b089f3da P 4083ce96cefb47d0be1d8e99602b7ce7 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.229264  8743 tablet_replica.cc:333] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.229887  8743 raft_consensus.cc:2243] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.230398  8743 raft_consensus.cc:2272] T ed7d787bc5ad4e8db7719c178ffbc59b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.232436  8743 tablet_replica.cc:333] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.233040  8743 raft_consensus.cc:2243] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.233821  8743 raft_consensus.cc:2272] T 8bab9d80dad949d8afde49b123fbac1a P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.235854  8743 tablet_replica.cc:333] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.236485  8743 raft_consensus.cc:2243] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.237032  8743 raft_consensus.cc:2272] T 3e4ebd0dc053435ea316f01d854592c9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.239105  8743 tablet_replica.cc:333] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.239777  8743 raft_consensus.cc:2243] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 37 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.240706  8743 raft_consensus.cc:2272] T 6a2ff46f3f264308a58e49e2819b1d5b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 37 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.242858  8743 tablet_replica.cc:333] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.243494  8743 raft_consensus.cc:2243] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.244656  8743 raft_consensus.cc:2272] T 7e7b0b17a4d646809d4dd46a0268f2eb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.246582  8743 tablet_replica.cc:333] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.247215  8743 raft_consensus.cc:2243] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.248077  8743 raft_consensus.cc:2272] T 230693f05eb74e8095e27d669f16c8ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.250326  8743 tablet_replica.cc:333] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.251150  8743 raft_consensus.cc:2243] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.253388  8743 pending_rounds.cc:70] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Trying to abort 1 pending ops.
I20260709 12:28:27.253613  8743 pending_rounds.cc:77] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting op as it isn't in flight: id { term: 31 index: 19 } timestamp: 7305626007597379584 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "8b34dc95d0414e70879dd34eaa8213de" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } } new_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } } }
I20260709 12:28:27.254660  8743 raft_consensus.cc:2917] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Aborting config change with OpId 31.19: Aborted: Op aborted
I20260709 12:28:27.255080  8743 raft_consensus.cc:2889] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:28:27.255522  8743 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:54752 (request call id 0) took 7752 ms. Trace:
I20260709 12:28:27.255878  8743 rpcz_store.cc:276] 0709 12:28:19.503788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:19.504046 (+   258us) service_pool.cc:224] Handling call
0709 12:28:19.537859 (+ 33813us) log.cc:844] Serialized 506 byte log entry
0709 12:28:27.255464 (+7717605us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13211,"raft.queue_time_us":13589,"raft.run_cpu_time_us":2197,"raft.run_wall_time_us":2196,"spinlock_wait_cycles":1792,"thread_start_us":60525,"threads_started":1,"wal-append.queue_time_us":59723,"wal-append.run_cpu_time_us":2400,"wal-append.run_wall_time_us":1029616}
W20260709 12:28:27.259368  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 8b34dc95d0414e70879dd34eaa8213de: Aborted: Op aborted
W20260709 12:28:27.261093 10640 consensus_peers.cc:597] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.267323 10642 consensus_peers.cc:597] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:27.268924 10642 consensus_peers.cc:597] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.274380  8743 raft_consensus.cc:2272] T 8b34dc95d0414e70879dd34eaa8213de P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.278409  8743 tablet_replica.cc:333] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.279650  8743 raft_consensus.cc:2243] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 LEADER]: Raft consensus shutting down.
W20260709 12:28:27.283234  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 41bba71b483942ed97a9e1276a90514f: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:27.286038 10642 consensus_peers.cc:597] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:27.288589  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 86c102903b3d4742a0b9ac1df1132361: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:27.292145 10642 consensus_peers.cc:597] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.292225 21068 raft_consensus.cc:493] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 9aa345bcd43b44088114b7b6f2830b17)
I20260709 12:28:27.292780 21068 raft_consensus.cc:515] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } }
I20260709 12:28:27.293545  8743 raft_consensus.cc:2272] T 840d8356270745f492c7d2c7099794c1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.297140  8743 tablet_replica.cc:333] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.298045  8743 raft_consensus.cc:2243] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.298763  8743 raft_consensus.cc:2272] T fc2205f3947147b6ad9c3ac2b9bd5058 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.300047  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 6a2ff46f3f264308a58e49e2819b1d5b: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:28:27.301887  8743 tablet_replica.cc:333] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.302850  8743 raft_consensus.cc:2243] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus shutting down.
W20260709 12:28:27.303037 10065 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:27.310091 10642 consensus_peers.cc:597] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.311440  8743 raft_consensus.cc:2272] T a7196ffc873e4e7e84c76068d5b95904 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.312032 10062 leader_election.cc:336] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:27.312531 10062 leader_election.cc:304] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4083ce96cefb47d0be1d8e99602b7ce7; no voters: 9aa345bcd43b44088114b7b6f2830b17, cd983415389444e292b2330ef630850a
I20260709 12:28:27.305859 21068 leader_election.cc:290] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115)
I20260709 12:28:27.318259 21048 raft_consensus.cc:2749] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:28:27.323964 10065 consensus_peers.cc:597] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.324433 10643 consensus_peers.cc:597] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:27.315735  8743 tablet_replica.cc:333] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.332084  8743 raft_consensus.cc:2243] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Raft consensus shutting down.
W20260709 12:28:27.332887 10642 consensus_peers.cc:597] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.334319  8743 raft_consensus.cc:2272] T 6dfe4ce70e5943448d384222d18a1215 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.334702 10065 consensus_peers.cc:597] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.335542 10062 consensus_peers.cc:597] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:27.338434  8743 tablet_replica.cc:333] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
W20260709 12:28:27.343433 10642 consensus_peers.cc:597] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.343454  8743 raft_consensus.cc:2243] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.344502  8743 raft_consensus.cc:2272] T 0fe681129eb440b4925ce53e8c68aaeb P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.345182 10642 consensus_peers.cc:597] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.347721  8743 tablet_replica.cc:333] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.348651  8743 raft_consensus.cc:2243] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.349399  8743 raft_consensus.cc:2272] T a78ccdb241a54c0398e2539a827e5d22 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.352274  8743 tablet_replica.cc:333] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.353116  8743 raft_consensus.cc:2243] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.354383  8743 raft_consensus.cc:2272] T 51528ebbf4884121ad7c7d564100498f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.357723  8743 tablet_replica.cc:333] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.358675  8743 raft_consensus.cc:2243] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.359272  8743 raft_consensus.cc:2272] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.361368 10642 consensus_peers.cc:597] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.361896  8743 tablet_replica.cc:333] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.362826  8743 raft_consensus.cc:2243] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.363536  8743 raft_consensus.cc:2272] T 93686c27d2cf4edea0d5da8d0c1d81ee P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.366305  8743 tablet_replica.cc:333] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.367337  8743 raft_consensus.cc:2243] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.368157  8743 raft_consensus.cc:2272] T 208adf3e1124402cb1e6c43ba7306b7b P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.370549 10643 consensus_peers.cc:597] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:27.371413  8743 tablet_replica.cc:333] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.372417  8743 raft_consensus.cc:2243] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.374063  8743 raft_consensus.cc:2272] T 6c669b03539649e4a4164ca56748f662 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.376417 10642 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.376960  8743 tablet_replica.cc:333] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.377877  8743 raft_consensus.cc:2243] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.378720  8743 raft_consensus.cc:2272] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.381460  8743 tablet_replica.cc:333] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.382273  8743 raft_consensus.cc:2243] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.383359  8743 raft_consensus.cc:2272] T 15458ba5804c4c36b01b169af77b2691 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.386018  8743 tablet_replica.cc:333] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.386828  8743 raft_consensus.cc:2243] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.387472  8743 raft_consensus.cc:2272] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.390089  8743 tablet_replica.cc:333] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.390945  8743 raft_consensus.cc:2243] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.391628  8743 raft_consensus.cc:2272] T 9bd550066b4447f8848484defc31b9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.394341  8743 tablet_replica.cc:333] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.395226  8743 raft_consensus.cc:2243] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.396481  8743 raft_consensus.cc:2272] T afb4bd9b5d6240b19d09014b985ea7f7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.399422  8743 tablet_replica.cc:333] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.400280  8743 raft_consensus.cc:2243] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.400923  8743 raft_consensus.cc:2272] T 32a9a1d2d5fc4243bf902da3519cd483 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.403815  8743 tablet_replica.cc:333] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.404685  8743 raft_consensus.cc:2243] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.405848  8743 raft_consensus.cc:2272] T a1e3e80f2cad43f08196a18ad2bee6a4 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.408694  8743 tablet_replica.cc:333] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.409601  8743 raft_consensus.cc:2243] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.410243  8743 raft_consensus.cc:2272] T db738483c1444059b752648d302dc0d9 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.413033  8743 tablet_replica.cc:333] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.413899  8743 raft_consensus.cc:2243] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.414718  8743 raft_consensus.cc:2272] T e346f82bc2694f9e968337cc4be9e53e P 4083ce96cefb47d0be1d8e99602b7ce7 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.417392  8743 tablet_replica.cc:333] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.418220  8743 raft_consensus.cc:2243] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.419224  8743 raft_consensus.cc:2272] T 86c102903b3d4742a0b9ac1df1132361 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.422096  8743 tablet_replica.cc:333] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.422976  8743 raft_consensus.cc:2243] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.424139  8743 pending_rounds.cc:70] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Trying to abort 1 pending ops.
I20260709 12:28:27.424463  8743 pending_rounds.cc:77] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7: Aborting op as it isn't in flight: id { term: 31 index: 17 } timestamp: 7305626005074350080 op_type: NO_OP noop_request { }
I20260709 12:28:27.424861  8743 raft_consensus.cc:2889] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:28:27.425259  8743 raft_consensus.cc:2272] T 78df0ad0c1454828a838cd88472c8553 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.428484  8743 tablet_replica.cc:333] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.429308  8743 raft_consensus.cc:2243] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.430101  8743 raft_consensus.cc:2272] T 53edc906d1af41e0992f01434995f9d1 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.432920  8743 tablet_replica.cc:333] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.433763  8743 raft_consensus.cc:2243] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.435003  8743 raft_consensus.cc:2272] T eb955e753b7546bf8612b6ed1c3fc342 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.437901  8743 tablet_replica.cc:333] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.438741  8743 raft_consensus.cc:2243] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.439975  8743 raft_consensus.cc:2272] T 1892c18d4e1644109d95b077e989b10f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.441468 10643 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:27.442078 10643 consensus_peers.cc:597] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:27.442793  8743 tablet_replica.cc:333] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
W20260709 12:28:27.446154 10642 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.446381  8743 raft_consensus.cc:2243] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.447206  8743 raft_consensus.cc:2272] T ab118912551248c1985fce40a2e815b0 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.450030  8743 tablet_replica.cc:333] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
W20260709 12:28:27.450572 10640 consensus_peers.cc:597] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:28:27.450911  8743 raft_consensus.cc:2243] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.451669  8743 raft_consensus.cc:2272] T 41bba71b483942ed97a9e1276a90514f P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.454586  8743 tablet_replica.cc:333] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.455456  8743 raft_consensus.cc:2243] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.456063  8743 raft_consensus.cc:2272] T bb6fd99043b344de885ff64fd4c81132 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.458796  8743 tablet_replica.cc:333] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.459653  8743 raft_consensus.cc:2243] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.461365  8743 raft_consensus.cc:2272] T bf49747e9a8542e6b75e3545a18c0981 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.464314  8743 tablet_replica.cc:333] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.465189  8743 raft_consensus.cc:2243] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.465862  8743 raft_consensus.cc:2272] T 773b4422370c4cc6b9e7477bdaf29261 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.468923  8743 tablet_replica.cc:333] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.469779  8743 raft_consensus.cc:2243] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.470643  8743 raft_consensus.cc:2272] T 181ca8c0fd384c8c909b095284d9e12c P 4083ce96cefb47d0be1d8e99602b7ce7 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.473686  8743 tablet_replica.cc:333] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.474498  8743 raft_consensus.cc:2243] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.475239  8743 raft_consensus.cc:2272] T d864b424b78d4fffb51785ea7d3b74ed P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.478120  8743 tablet_replica.cc:333] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.478956  8743 raft_consensus.cc:2243] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.479732  8743 raft_consensus.cc:2272] T 912ffd65ac564403ac65b7a5d005d367 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.482564  8743 tablet_replica.cc:333] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
W20260709 12:28:27.482616 10642 consensus_peers.cc:597] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:27.483569  8743 raft_consensus.cc:2243] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.484579  8743 raft_consensus.cc:2272] T d4909dfc06b24d70bfd398607779f5ae P 4083ce96cefb47d0be1d8e99602b7ce7 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.487510  8743 tablet_replica.cc:333] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.488502  8743 raft_consensus.cc:2243] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.489316  8743 raft_consensus.cc:2272] T d8453bea911c4756add721edebd5d124 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.492317  8743 tablet_replica.cc:333] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.493098  8743 raft_consensus.cc:2243] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:27.493743  8743 raft_consensus.cc:2272] T d43a22dba58d423d907a568238e85284 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.494720 10643 consensus_peers.cc:597] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:28:27.496577  8743 tablet_replica.cc:333] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.497417  8743 raft_consensus.cc:2243] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.498692  8743 raft_consensus.cc:2272] T 886f3cb002374eae906fdbe2d392f3e7 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:27.501508  8743 tablet_replica.cc:333] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7: stopping tablet replica
I20260709 12:28:27.502197  8743 raft_consensus.cc:2243] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:27.503376  8743 raft_consensus.cc:2272] T 26e02c55bc5648e0baa2a4216f02f955 P 4083ce96cefb47d0be1d8e99602b7ce7 [term 34 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:27.556317 10643 consensus_peers.cc:597] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.572726 10640 consensus_peers.cc:597] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.575089 10640 consensus_peers.cc:597] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.578449 10643 consensus_peers.cc:597] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:27.602772 10642 consensus_peers.cc:597] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:27.606112 10640 consensus_peers.cc:597] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.619966 10642 consensus_peers.cc:597] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Couldn't send request to peer 4083ce96cefb47d0be1d8e99602b7ce7. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:27.715637 10640 consensus_peers.cc:597] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:27.752413 10640 consensus_peers.cc:597] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:28:27.759042 10643 consensus_peers.cc:597] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:28:27.784772 10640 consensus_peers.cc:597] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:28:28.134979 10643 consensus_peers.cc:597] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:28:28.243067 10640 consensus_peers.cc:597] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Couldn't send request to peer 9aa345bcd43b44088114b7b6f2830b17. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:28:28.298851 21099 raft_consensus.cc:493] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 36 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:28.299474 21099 raft_consensus.cc:515] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:28.301743 21099 leader_election.cc:290] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:28.307204 10640 leader_election.cc:336] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:28.308068 10642 leader_election.cc:336] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 37 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
I20260709 12:28:28.309805 10642 leader_election.cc:304] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:28.311501 21099 raft_consensus.cc:2749] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:28:28.337473  8743 tablet_server.cc:196] TabletServer@127.8.137.195:0 shutdown complete.
W20260709 12:28:28.341977 10643 consensus_peers.cc:597] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 -> Peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Couldn't send request to peer cd983415389444e292b2330ef630850a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:28:28.411396  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 6c669b03539649e4a4164ca56748f662: Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
W20260709 12:28:28.414713  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 6a2ff46f3f264308a58e49e2819b1d5b: Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
W20260709 12:28:28.418962  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet bf49747e9a8542e6b75e3545a18c0981: Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
W20260709 12:28:28.434500  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 9bd550066b4447f8848484defc31b9d1: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
I20260709 12:28:28.472465  8743 tablet_server.cc:179] TabletServer@127.8.137.196:0 shutting down...
I20260709 12:28:28.499225 21099 raft_consensus.cc:493] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:28.499902 21099 raft_consensus.cc:515] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:28.502183 21099 leader_election.cc:290] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:28.508626 10643 leader_election.cc:336] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:28.510818 10642 leader_election.cc:336] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
I20260709 12:28:28.511368 10642 leader_election.cc:304] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:28.512238 21099 raft_consensus.cc:2749] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:28:28.520519 21099 raft_consensus.cc:493] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:28.521606 21099 raft_consensus.cc:515] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:28.524477 21099 leader_election.cc:290] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers cd983415389444e292b2330ef630850a (127.8.137.193:34909), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
W20260709 12:28:28.529788 10642 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
W20260709 12:28:28.531981 10643 leader_election.cc:336] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer cd983415389444e292b2330ef630850a (127.8.137.193:34909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
I20260709 12:28:28.532575 10643 leader_election.cc:304] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, cd983415389444e292b2330ef630850a
I20260709 12:28:28.533305 21099 raft_consensus.cc:2749] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:28:28.547209  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:28:28.548097  8743 tablet_replica.cc:333] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.548830  8743 raft_consensus.cc:2243] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 LEARNER]: Raft consensus shutting down.
I20260709 12:28:28.549331  8743 pending_rounds.cc:70] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: Trying to abort 1 pending ops.
I20260709 12:28:28.549583  8743 pending_rounds.cc:77] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76: Aborting op as it isn't in flight: id { term: 31 index: 19 } timestamp: 7305626007597379584 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "8b34dc95d0414e70879dd34eaa8213de" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } } new_config { opid_index: 19 OBSOLETE_local: false peers { permanent_uuid: "cd983415389444e292b2330ef630850a" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 34909 } } peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } attrs { replace: true } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: NON_VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: true } } } }
I20260709 12:28:28.550344  8743 raft_consensus.cc:2917] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 LEARNER]: Aborting config change with OpId 31.19: Aborted: Op aborted
I20260709 12:28:28.550745  8743 raft_consensus.cc:2889] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:28:28.551074  8743 raft_consensus.cc:2272] T 8b34dc95d0414e70879dd34eaa8213de P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 12:28:28.553246  8743 tablet_replica.cc:333] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.553956  8743 raft_consensus.cc:2243] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.554783  8743 raft_consensus.cc:2272] T 7e7b0b17a4d646809d4dd46a0268f2eb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.557618  8743 tablet_replica.cc:333] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.558645  8743 raft_consensus.cc:2243] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.559446  8743 raft_consensus.cc:2272] T 230693f05eb74e8095e27d669f16c8ee P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.562431  8743 tablet_replica.cc:333] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.563226  8743 raft_consensus.cc:2243] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.564288  8743 raft_consensus.cc:2272] T e346f82bc2694f9e968337cc4be9e53e P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.566855  8743 tablet_replica.cc:333] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.567596  8743 raft_consensus.cc:2243] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.568146  8743 raft_consensus.cc:2272] T 978cf11d4cd54b67a4fa39b2b089f3da P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.570387  8743 tablet_replica.cc:333] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.571053  8743 raft_consensus.cc:2243] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.572036  8743 raft_consensus.cc:2272] T b5ca4a138b9345ab87fc4055395e0488 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.574975  8743 tablet_replica.cc:333] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.575961  8743 raft_consensus.cc:2243] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.577037  8743 raft_consensus.cc:2272] T a7196ffc873e4e7e84c76068d5b95904 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.577282 21095 raft_consensus.cc:493] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader 4083ce96cefb47d0be1d8e99602b7ce7)
I20260709 12:28:28.577922 21095 raft_consensus.cc:515] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "9aa345bcd43b44088114b7b6f2830b17" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 35115 } } peers { permanent_uuid: "4083ce96cefb47d0be1d8e99602b7ce7" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33983 } } peers { permanent_uuid: "6c7bd3fa1bfa431c89fc9c9096452a76" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36979 } attrs { promote: false } }
I20260709 12:28:28.579492  8743 tablet_replica.cc:333] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.580298 21095 leader_election.cc:290] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115), 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983)
I20260709 12:28:28.580371  8743 raft_consensus.cc:2243] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.581928  8743 raft_consensus.cc:2272] T 181ca8c0fd384c8c909b095284d9e12c P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.584924  8743 tablet_replica.cc:333] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
W20260709 12:28:28.584985 10640 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 9aa345bcd43b44088114b7b6f2830b17 (127.8.137.194:35115): Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:28.585649 10642 leader_election.cc:336] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
I20260709 12:28:28.586006  8743 raft_consensus.cc:2243] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.586053 10642 leader_election.cc:304] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c7bd3fa1bfa431c89fc9c9096452a76; no voters: 4083ce96cefb47d0be1d8e99602b7ce7, 9aa345bcd43b44088114b7b6f2830b17
I20260709 12:28:28.587049 21095 raft_consensus.cc:2749] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:28:28.587529  8743 raft_consensus.cc:2272] T 0fe681129eb440b4925ce53e8c68aaeb P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.590327  8743 tablet_replica.cc:333] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.591254  8743 raft_consensus.cc:2243] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.592569  8743 raft_consensus.cc:2272] T 1154fd85c1e440b7bb5ee236ac1e0a21 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.595839  8743 tablet_replica.cc:333] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.596738  8743 raft_consensus.cc:2243] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.598019  8743 raft_consensus.cc:2272] T d6726f0982834ecf8987472ed6c32a73 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.601168  8743 tablet_replica.cc:333] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.602059  8743 raft_consensus.cc:2243] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.603178  8743 raft_consensus.cc:2272] T 9f126305b66248b59cd9ce87bc952694 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.605962  8743 tablet_replica.cc:333] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.606863  8743 raft_consensus.cc:2243] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.607549  8743 raft_consensus.cc:2272] T 59e61711ca80440e9c13392c6b6b25fa P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.610364  8743 tablet_replica.cc:333] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.611270  8743 raft_consensus.cc:2243] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.612126  8743 raft_consensus.cc:2272] T 447b5d26fe15480dbb8fedf0f7a44777 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.615013  8743 tablet_replica.cc:333] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.615864  8743 raft_consensus.cc:2243] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.616549  8743 raft_consensus.cc:2272] T 7253570248b8481791bf2187007dd5df P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.619585  8743 tablet_replica.cc:333] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.620446  8743 raft_consensus.cc:2243] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.621579  8743 raft_consensus.cc:2272] T ed7d787bc5ad4e8db7719c178ffbc59b P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.624543  8743 tablet_replica.cc:333] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.625373  8743 raft_consensus.cc:2243] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.626442  8743 raft_consensus.cc:2272] T aa5a16c274db44a5aa641b51f71f7ec6 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.629333  8743 tablet_replica.cc:333] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.630163  8743 raft_consensus.cc:2243] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.630872  8743 raft_consensus.cc:2272] T 886f3cb002374eae906fdbe2d392f3e7 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.633876  8743 tablet_replica.cc:333] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.634771  8743 raft_consensus.cc:2243] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.635457  8743 raft_consensus.cc:2272] T d7b809cb22d54c31ad5985fb9f01e6ab P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.638393  8743 tablet_replica.cc:333] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.639245  8743 raft_consensus.cc:2243] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.640417  8743 raft_consensus.cc:2272] T a78ccdb241a54c0398e2539a827e5d22 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.643275  8743 tablet_replica.cc:333] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.644086  8743 raft_consensus.cc:2243] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.645238  8743 raft_consensus.cc:2272] T 53edc906d1af41e0992f01434995f9d1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.648141  8743 tablet_replica.cc:333] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.648957  8743 raft_consensus.cc:2243] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.650134  8743 raft_consensus.cc:2272] T d8453bea911c4756add721edebd5d124 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.652948  8743 tablet_replica.cc:333] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.653782  8743 raft_consensus.cc:2243] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.654493  8743 raft_consensus.cc:2272] T 0df211fae593420e88f685abc8d618ec P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.657414  8743 tablet_replica.cc:333] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.658298  8743 raft_consensus.cc:2243] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.659006  8743 raft_consensus.cc:2272] T 840d8356270745f492c7d2c7099794c1 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.661966  8743 tablet_replica.cc:333] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.662807  8743 raft_consensus.cc:2243] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.664009  8743 raft_consensus.cc:2272] T 03a9b6ec47044f6d8c27ae5f07c64f3a P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.667002  8743 tablet_replica.cc:333] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.667923  8743 raft_consensus.cc:2243] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.669131  8743 raft_consensus.cc:2272] T 4cd31c96c4854a87a361975b579bc1c2 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.671924  8743 tablet_replica.cc:333] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.672730  8743 raft_consensus.cc:2243] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.673900  8743 raft_consensus.cc:2272] T ab118912551248c1985fce40a2e815b0 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.676919  8743 tablet_replica.cc:333] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.677829  8743 raft_consensus.cc:2243] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.679374  8743 raft_consensus.cc:2272] T fc2205f3947147b6ad9c3ac2b9bd5058 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.682528  8743 tablet_replica.cc:333] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.683365  8743 raft_consensus.cc:2243] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.684545  8743 raft_consensus.cc:2272] T 35eb6b5f8b8e4d2f8e0a6a08ded4d6a4 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.687377  8743 tablet_replica.cc:333] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.688236  8743 raft_consensus.cc:2243] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.689775  8743 raft_consensus.cc:2272] T 3e4ebd0dc053435ea316f01d854592c9 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.692977  8743 tablet_replica.cc:333] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.693773  8743 raft_consensus.cc:2243] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.694896  8743 raft_consensus.cc:2272] T 062c203e13aa4527a094ec24339001e5 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.697561  8743 tablet_replica.cc:333] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.698308  8743 raft_consensus.cc:2243] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.699556  8743 raft_consensus.cc:2272] T 773b4422370c4cc6b9e7477bdaf29261 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.702415  8743 tablet_replica.cc:333] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.703276  8743 raft_consensus.cc:2243] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.704408  8743 raft_consensus.cc:2272] T d864b424b78d4fffb51785ea7d3b74ed P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.707130  8743 tablet_replica.cc:333] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.707931  8743 raft_consensus.cc:2243] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.709044  8743 raft_consensus.cc:2272] T d80ed93cd18a4a1095bdd6f9fd0fca80 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.711879  8743 tablet_replica.cc:333] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.712635  8743 raft_consensus.cc:2243] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.713308  8743 raft_consensus.cc:2272] T 15458ba5804c4c36b01b169af77b2691 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.716042  8743 tablet_replica.cc:333] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.716962  8743 raft_consensus.cc:2243] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.718129  8743 raft_consensus.cc:2272] T 94aa8e92ea8d4f81a4ed4dbb1e99635d P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.720954  8743 tablet_replica.cc:333] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.721756  8743 raft_consensus.cc:2243] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.722882  8743 raft_consensus.cc:2272] T 912ffd65ac564403ac65b7a5d005d367 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.726001  8743 tablet_replica.cc:333] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.726771  8743 raft_consensus.cc:2243] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.727391  8743 raft_consensus.cc:2272] T d4909dfc06b24d70bfd398607779f5ae P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.730134  8743 tablet_replica.cc:333] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.730888  8743 raft_consensus.cc:2243] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:28:28.731591  8743 raft_consensus.cc:2272] T 1892c18d4e1644109d95b077e989b10f P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:28.734479  8743 tablet_replica.cc:333] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76: stopping tablet replica
I20260709 12:28:28.735193  8743 raft_consensus.cc:2243] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:28:28.736335  8743 raft_consensus.cc:2272] T 32a9a1d2d5fc4243bf902da3519cd483 P 6c7bd3fa1bfa431c89fc9c9096452a76 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:28:29.495890  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet afb4bd9b5d6240b19d09014b985ea7f7: Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
W20260709 12:28:29.499828  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 8bab9d80dad949d8afde49b123fbac1a: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:29.503125  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 86c102903b3d4742a0b9ac1df1132361: Network error: Client connection negotiation failed: client connection to 127.8.137.194:35115: connect: Connection refused (error 111)
W20260709 12:28:29.506387  9894 auto_rebalancer.cc:629] Failed to schedule move for tablet 26e02c55bc5648e0baa2a4216f02f955: Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
I20260709 12:28:29.624009  8743 tablet_server.cc:196] TabletServer@127.8.137.196:0 shutdown complete.
I20260709 12:28:29.719141  8743 master.cc:562] Master@127.8.137.254:38721 shutting down...
W20260709 12:28:35.075730  9830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111) [suppressed 634 similar messages]
W20260709 12:28:35.079370  9830 catalog_manager.cc:4755] TS cd983415389444e292b2330ef630850a (127.8.137.193:34909): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet a7196ffc873e4e7e84c76068d5b95904: Network error: Client connection negotiation failed: client connection to 127.8.137.193:34909: connect: Connection refused (error 111)
W20260709 12:28:36.880256  9876 debug-util.cc:398] Leaking SignalData structure 0x7b08002ea940 after lost signal to thread 8746
W20260709 12:28:36.880805  9876 debug-util.cc:398] Leaking SignalData structure 0x7b0800366680 after lost signal to thread 9879
W20260709 12:28:42.308333  9829 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20260709 12:28:42.311786  9829 catalog_manager.cc:4755] TS 4083ce96cefb47d0be1d8e99602b7ce7 (127.8.137.195:33983): ChangeConfig:REMOVE_PEER RPC failed for tablet 7e7b0b17a4d646809d4dd46a0268f2eb: Network error: Client connection negotiation failed: client connection to 127.8.137.195:33983: connect: Connection refused (error 111)
I20260709 12:28:43.379333  8743 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:28:43.380431  8743 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:28:43.380868  8743 tablet_replica.cc:333] T 00000000000000000000000000000000 P 2121bfede85b447c9f51ef23233606cc: stopping tablet replica
I20260709 12:28:43.652562  8743 master.cc:584] Master@127.8.137.254:38721 shutdown complete.
I20260709 12:28:43.705359  8743 test_util.cc:182] -----------------------------------------------
I20260709 12:28:43.705588  8743 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783599720900375-8743-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (342542 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:28:43.709187  8743 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.137.254:36393
I20260709 12:28:43.710384  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:43.716215 21110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.716943 21111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.717671 21113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:43.718526  8743 server_base.cc:1061] running on GCE node
I20260709 12:28:43.722275  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:43.722476  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:43.722652  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600123722585 us; error 0 us; skew 500 ppm
I20260709 12:28:43.723153  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:43.725561  8743 webserver.cc:533] Webserver started at http://127.8.137.254:44237/ using document root <none> and password file <none>
I20260709 12:28:43.726069  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:43.726227  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:43.726439  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:43.727834  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/master-0-root/instance:
uuid: "9f2356d6f7cb41419b59884869890214"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-kpwv"
I20260709 12:28:43.733455  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:28:43.737221 21118 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.738145  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:28:43.738482  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/master-0-root
uuid: "9f2356d6f7cb41419b59884869890214"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-kpwv"
I20260709 12:28:43.738839  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:43.760484  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:43.761786  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:43.801999  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.254:36393
I20260709 12:28:43.802119 21169 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.254:36393 every 8 connection(s)
I20260709 12:28:43.807439 21170 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:43.818838 21170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214: Bootstrap starting.
I20260709 12:28:43.823951 21170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:43.828464 21170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214: No bootstrap required, opened a new log
I20260709 12:28:43.830799 21170 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2356d6f7cb41419b59884869890214" member_type: VOTER }
I20260709 12:28:43.831233 21170 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:43.831413 21170 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f2356d6f7cb41419b59884869890214, State: Initialized, Role: FOLLOWER
I20260709 12:28:43.832034 21170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2356d6f7cb41419b59884869890214" member_type: VOTER }
I20260709 12:28:43.832485 21170 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:28:43.832693 21170 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:28:43.832909 21170 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:43.837553 21170 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2356d6f7cb41419b59884869890214" member_type: VOTER }
I20260709 12:28:43.838065 21170 leader_election.cc:304] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f2356d6f7cb41419b59884869890214; no voters: 
I20260709 12:28:43.839210 21170 leader_election.cc:290] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:28:43.839563 21173 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:43.841169 21173 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 1 LEADER]: Becoming Leader. State: Replica: 9f2356d6f7cb41419b59884869890214, State: Running, Role: LEADER
I20260709 12:28:43.841950 21173 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2356d6f7cb41419b59884869890214" member_type: VOTER }
I20260709 12:28:43.842523 21170 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:28:43.852689 21175 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f2356d6f7cb41419b59884869890214. Latest consensus state: current_term: 1 leader_uuid: "9f2356d6f7cb41419b59884869890214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2356d6f7cb41419b59884869890214" member_type: VOTER } }
I20260709 12:28:43.852602 21174 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f2356d6f7cb41419b59884869890214" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f2356d6f7cb41419b59884869890214" member_type: VOTER } }
I20260709 12:28:43.853451 21175 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:43.853603 21174 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [sys.catalog]: This master's current role is: LEADER
I20260709 12:28:43.855657 21184 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:28:43.860702 21184 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:28:43.861881  8743 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:28:43.870374 21184 catalog_manager.cc:1383] Generated new cluster ID: 2065d444b2c54f77972159d9e87da3eb
I20260709 12:28:43.870738 21184 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:28:43.898186 21184 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:28:43.899576 21184 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:28:43.915640 21184 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214: Generated new TSK 0
I20260709 12:28:43.916303 21184 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:28:43.929255  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:43.936503 21191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.937688 21192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:43.938421 21194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:43.941295  8743 server_base.cc:1061] running on GCE node
I20260709 12:28:43.944448  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:43.944672  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:43.945251  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600123945212 us; error 0 us; skew 500 ppm
I20260709 12:28:43.946085  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:43.949034  8743 webserver.cc:533] Webserver started at http://127.8.137.193:41199/ using document root <none> and password file <none>
I20260709 12:28:43.949658  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:43.949846  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:43.950214  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:43.952121  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/instance:
uuid: "482723f787a14005854e038688430a80"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-kpwv"
I20260709 12:28:43.958416  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:43.962312 21199 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:43.963349  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:28:43.963745  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "482723f787a14005854e038688430a80"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-kpwv"
I20260709 12:28:43.964085  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:44.003047  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:44.004550  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:44.006174  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:44.008982  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:44.009251  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.009526  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:44.009728  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.057039  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:33017
I20260709 12:28:44.057170 21261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:33017 every 8 connection(s)
I20260709 12:28:44.062579  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:44.072070 21266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:44.073419 21267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:44.077666 21269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:44.078581  8743 server_base.cc:1061] running on GCE node
I20260709 12:28:44.082005  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:28:44.082175 21262 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36393
W20260709 12:28:44.082275  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:44.082739  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600124082720 us; error 0 us; skew 500 ppm
I20260709 12:28:44.082792 21262 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:44.083455  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:44.083798 21262 heartbeater.cc:507] Master 127.8.137.254:36393 requested a full tablet report, sending...
I20260709 12:28:44.086377  8743 webserver.cc:533] Webserver started at http://127.8.137.194:39805/ using document root <none> and password file <none>
I20260709 12:28:44.086566 21135 ts_manager.cc:194] Registered new tserver with Master: 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:44.087069  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:44.087342  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:44.087592  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:44.088991  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/instance:
uuid: "b565aef1b93143c78c5c3dc4a861a68a"
format_stamp: "Formatted at 2026-07-09 12:28:44 on dist-test-slave-kpwv"
I20260709 12:28:44.089589 21135 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60562
I20260709 12:28:44.094813  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.000s
I20260709 12:28:44.099071 21274 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.099913  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:28:44.100247  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-1-root
uuid: "b565aef1b93143c78c5c3dc4a861a68a"
format_stamp: "Formatted at 2026-07-09 12:28:44 on dist-test-slave-kpwv"
I20260709 12:28:44.100574  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:44.126278  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:44.127652  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:44.130247  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:44.132925  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:44.133208  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.133503  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:44.133698  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.183408  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.194:41173
I20260709 12:28:44.183527 21336 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.194:41173 every 8 connection(s)
I20260709 12:28:44.188535  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:44.197388 21341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:44.198187 21340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:44.200747 21343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:44.201625  8743 server_base.cc:1061] running on GCE node
I20260709 12:28:44.202890  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:44.203142  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:44.203361  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600124203342 us; error 0 us; skew 500 ppm
I20260709 12:28:44.203599 21337 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36393
I20260709 12:28:44.204007 21337 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:44.204032  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:44.204967 21337 heartbeater.cc:507] Master 127.8.137.254:36393 requested a full tablet report, sending...
I20260709 12:28:44.206977  8743 webserver.cc:533] Webserver started at http://127.8.137.195:40409/ using document root <none> and password file <none>
I20260709 12:28:44.207397 21135 ts_manager.cc:194] Registered new tserver with Master: b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:44.207679  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:44.207957  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:44.208282  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:44.209283 21135 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60570
I20260709 12:28:44.209914  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/instance:
uuid: "a3220692a27d44e49348c63d58fcf666"
format_stamp: "Formatted at 2026-07-09 12:28:44 on dist-test-slave-kpwv"
I20260709 12:28:44.214711  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:28:44.218349 21348 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.219218  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 12:28:44.219551  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-2-root
uuid: "a3220692a27d44e49348c63d58fcf666"
format_stamp: "Formatted at 2026-07-09 12:28:44 on dist-test-slave-kpwv"
I20260709 12:28:44.219866  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:44.234813  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:44.236337  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:44.238541  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:44.241232  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:44.241447  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.241693  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:44.241844  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.291087  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.195:44499
I20260709 12:28:44.291285 21410 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.195:44499 every 8 connection(s)
I20260709 12:28:44.296657  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:28:44.305316 21414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:28:44.307986 21415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:44.312346 21411 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36393
I20260709 12:28:44.312871 21411 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:44.313249  8743 server_base.cc:1061] running on GCE node
W20260709 12:28:44.313884 21417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:28:44.313956 21411 heartbeater.cc:507] Master 127.8.137.254:36393 requested a full tablet report, sending...
I20260709 12:28:44.315057  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:28:44.315348  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:28:44.315600  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600124315579 us; error 0 us; skew 500 ppm
I20260709 12:28:44.316257  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:28:44.316589 21135 ts_manager.cc:194] Registered new tserver with Master: a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:44.318759 21135 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60574
I20260709 12:28:44.319195  8743 webserver.cc:533] Webserver started at http://127.8.137.196:39903/ using document root <none> and password file <none>
I20260709 12:28:44.319836  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:28:44.320088  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:28:44.320381  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:28:44.321650  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-3-root/instance:
uuid: "1095bdbabc944624b91311b39a5a02ce"
format_stamp: "Formatted at 2026-07-09 12:28:44 on dist-test-slave-kpwv"
I20260709 12:28:44.326335  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.003s
I20260709 12:28:44.331099 21422 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.332283  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:28:44.332837  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-3-root
uuid: "1095bdbabc944624b91311b39a5a02ce"
format_stamp: "Formatted at 2026-07-09 12:28:44 on dist-test-slave-kpwv"
I20260709 12:28:44.333282  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:28:44.345243  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:28:44.346936  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:28:44.348660  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:28:44.351114  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:28:44.351368  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.351629  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:28:44.351810  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:28:44.396713  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.196:36263
I20260709 12:28:44.396801 21484 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.196:36263 every 8 connection(s)
I20260709 12:28:44.412061 21485 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36393
I20260709 12:28:44.412606 21485 heartbeater.cc:461] Registering TS with master...
I20260709 12:28:44.413834 21485 heartbeater.cc:507] Master 127.8.137.254:36393 requested a full tablet report, sending...
I20260709 12:28:44.416687 21135 ts_manager.cc:194] Registered new tserver with Master: 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:44.417008  8743 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016697921s
I20260709 12:28:44.418188 21135 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60588
I20260709 12:28:44.664099 21135 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60602:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:28:44.854089 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:44.934455 21376 tablet_service.cc:1511] Processing CreateTablet for tablet 4d4f5802ffb348c78eaca62ca6fac2fc (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:44.936125 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4f5802ffb348c78eaca62ca6fac2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.936462 21450 tablet_service.cc:1511] Processing CreateTablet for tablet 4d4f5802ffb348c78eaca62ca6fac2fc (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:44.937902 21450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4f5802ffb348c78eaca62ca6fac2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.940315 21375 tablet_service.cc:1511] Processing CreateTablet for tablet 31b54df203a347c993ed917ad39bbf1c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:44.941075 21224 tablet_service.cc:1511] Processing CreateTablet for tablet 14c3f29aa4ff473db2aec020d298d279 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:44.941792 21375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b54df203a347c993ed917ad39bbf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.942446 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c3f29aa4ff473db2aec020d298d279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.942343 21227 tablet_service.cc:1511] Processing CreateTablet for tablet 4d4f5802ffb348c78eaca62ca6fac2fc (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:28:44.943746 21227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4d4f5802ffb348c78eaca62ca6fac2fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.945577 21226 tablet_service.cc:1511] Processing CreateTablet for tablet 31b54df203a347c993ed917ad39bbf1c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:44.947273 21374 tablet_service.cc:1511] Processing CreateTablet for tablet c956710bf64e4bb5b40784ba8060147c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:44.947742 21225 tablet_service.cc:1511] Processing CreateTablet for tablet d49430f4da5e4c108a5cf967c8844659 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:44.948784 21374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c956710bf64e4bb5b40784ba8060147c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.949070 21225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d49430f4da5e4c108a5cf967c8844659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.951921 21223 tablet_service.cc:1511] Processing CreateTablet for tablet d22f7d658c5f48e7badd08e41f3e6e0d (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:44.953507 21223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22f7d658c5f48e7badd08e41f3e6e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.954001 21373 tablet_service.cc:1511] Processing CreateTablet for tablet 14c3f29aa4ff473db2aec020d298d279 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:44.955495 21373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c3f29aa4ff473db2aec020d298d279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.955778 21222 tablet_service.cc:1511] Processing CreateTablet for tablet f44f7e65b47e48649e1e7ab852e0fc84 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:44.957250 21222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f44f7e65b47e48649e1e7ab852e0fc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.959887 21221 tablet_service.cc:1511] Processing CreateTablet for tablet 54443d308bbc43359e598ace1951bd70 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:44.960937 21297 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac0d0feb45a42ccaa8859902754853c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:44.961068 21447 tablet_service.cc:1511] Processing CreateTablet for tablet d22f7d658c5f48e7badd08e41f3e6e0d (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:44.961452 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 54443d308bbc43359e598ace1951bd70 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:44.962431 21297 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac0d0feb45a42ccaa8859902754853c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.962572 21447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22f7d658c5f48e7badd08e41f3e6e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.962728 21301 tablet_service.cc:1511] Processing CreateTablet for tablet d49430f4da5e4c108a5cf967c8844659 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:44.963240 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54443d308bbc43359e598ace1951bd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.963840 21302 tablet_service.cc:1511] Processing CreateTablet for tablet 31b54df203a347c993ed917ad39bbf1c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:28:44.988897 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b54df203a347c993ed917ad39bbf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.989264 21295 tablet_service.cc:1511] Processing CreateTablet for tablet a8f34093df9d4e6db1b428ae3b532fec (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:44.963910 21298 tablet_service.cc:1511] Processing CreateTablet for tablet d22f7d658c5f48e7badd08e41f3e6e0d (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:28:44.965024 21446 tablet_service.cc:1511] Processing CreateTablet for tablet f44f7e65b47e48649e1e7ab852e0fc84 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:44.994489 21446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f44f7e65b47e48649e1e7ab852e0fc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.994716 21369 tablet_service.cc:1511] Processing CreateTablet for tablet 27cdc0acc5734dd6ac5fb17c5805ce11 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:44.996170 21369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27cdc0acc5734dd6ac5fb17c5805ce11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.964965 21372 tablet_service.cc:1511] Processing CreateTablet for tablet f44f7e65b47e48649e1e7ab852e0fc84 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:28:45.000775 21372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f44f7e65b47e48649e1e7ab852e0fc84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.966648 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac0d0feb45a42ccaa8859902754853c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:45.005779 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac0d0feb45a42ccaa8859902754853c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.967409 21300 tablet_service.cc:1511] Processing CreateTablet for tablet c956710bf64e4bb5b40784ba8060147c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:44.961480 21221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54443d308bbc43359e598ace1951bd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.968227 21370 tablet_service.cc:1511] Processing CreateTablet for tablet 54443d308bbc43359e598ace1951bd70 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:28:45.012581 21370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 54443d308bbc43359e598ace1951bd70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.014956 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31b54df203a347c993ed917ad39bbf1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.966228 21445 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac0d0feb45a42ccaa8859902754853c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:28:45.016705 21445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac0d0feb45a42ccaa8859902754853c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.968822 21220 tablet_service.cc:1511] Processing CreateTablet for tablet a8f34093df9d4e6db1b428ae3b532fec (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:45.020673 21220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8f34093df9d4e6db1b428ae3b532fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.968616 21444 tablet_service.cc:1511] Processing CreateTablet for tablet a8f34093df9d4e6db1b428ae3b532fec (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:28:45.023270 21219 tablet_service.cc:1511] Processing CreateTablet for tablet 27cdc0acc5734dd6ac5fb17c5805ce11 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:45.024701 21219 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27cdc0acc5734dd6ac5fb17c5805ce11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.024736 21444 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8f34093df9d4e6db1b428ae3b532fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.969831 21299 tablet_service.cc:1511] Processing CreateTablet for tablet 14c3f29aa4ff473db2aec020d298d279 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:28:44.970646 21449 tablet_service.cc:1511] Processing CreateTablet for tablet d49430f4da5e4c108a5cf967c8844659 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:28:45.030720 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d49430f4da5e4c108a5cf967c8844659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:44.990792 21295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8f34093df9d4e6db1b428ae3b532fec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.038848 21294 tablet_service.cc:1511] Processing CreateTablet for tablet 7ac1f07547154488aa8d69f3b35d29bf (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:44.959873 21448 tablet_service.cc:1511] Processing CreateTablet for tablet c956710bf64e4bb5b40784ba8060147c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:28:45.040685 21294 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac1f07547154488aa8d69f3b35d29bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.041034 21448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c956710bf64e4bb5b40784ba8060147c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.018913 21443 tablet_service.cc:1511] Processing CreateTablet for tablet 27cdc0acc5734dd6ac5fb17c5805ce11 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:28:45.044785 21443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27cdc0acc5734dd6ac5fb17c5805ce11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.049146 21298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d22f7d658c5f48e7badd08e41f3e6e0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.051443 21368 tablet_service.cc:1511] Processing CreateTablet for tablet 7ac1f07547154488aa8d69f3b35d29bf (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:45.052951 21368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac1f07547154488aa8d69f3b35d29bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.059137 21300 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c956710bf64e4bb5b40784ba8060147c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.063081 21299 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14c3f29aa4ff473db2aec020d298d279. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.066715 21301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d49430f4da5e4c108a5cf967c8844659. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.077124 21442 tablet_service.cc:1511] Processing CreateTablet for tablet 7ac1f07547154488aa8d69f3b35d29bf (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:28:45.078766 21442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ac1f07547154488aa8d69f3b35d29bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.092983 21262 heartbeater.cc:499] Master 127.8.137.254:36393 was elected leader, sending a full tablet report...
I20260709 12:28:45.097867 21218 tablet_service.cc:1511] Processing CreateTablet for tablet dbd17ab2b7d04cc19cf64cdd44c442a7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:45.099327 21218 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd17ab2b7d04cc19cf64cdd44c442a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.139775 21504 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:45.145000 21293 tablet_service.cc:1511] Processing CreateTablet for tablet dbd17ab2b7d04cc19cf64cdd44c442a7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:45.146574 21293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd17ab2b7d04cc19cf64cdd44c442a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.148409 21441 tablet_service.cc:1511] Processing CreateTablet for tablet a8c8433d1e274f879bb2b84ad59f5c3f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:45.154271 21441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c8433d1e274f879bb2b84ad59f5c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.155666 21376 tablet_service.cc:1511] Processing CreateTablet for tablet dbd17ab2b7d04cc19cf64cdd44c442a7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:28:45.165743 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbd17ab2b7d04cc19cf64cdd44c442a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.205096 21504 tablet_bootstrap.cc:654] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.216533 21506 tablet_bootstrap.cc:492] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:45.217075 21337 heartbeater.cc:499] Master 127.8.137.254:36393 was elected leader, sending a full tablet report...
I20260709 12:28:45.221782 21505 tablet_bootstrap.cc:492] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:45.227607 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 9e36b9296b6e408da863f2bd3bb7b36b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:45.250500 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e36b9296b6e408da863f2bd3bb7b36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.226023 21295 tablet_service.cc:1511] Processing CreateTablet for tablet a8c8433d1e274f879bb2b84ad59f5c3f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:45.253938 21295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c8433d1e274f879bb2b84ad59f5c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.266199 21220 tablet_service.cc:1511] Processing CreateTablet for tablet 9e36b9296b6e408da863f2bd3bb7b36b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:45.278710 21220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e36b9296b6e408da863f2bd3bb7b36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.268323 21224 tablet_service.cc:1511] Processing CreateTablet for tablet a8c8433d1e274f879bb2b84ad59f5c3f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:28:45.273339 21506 tablet_bootstrap.cc:654] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.276876 21505 tablet_bootstrap.cc:654] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.267627 21446 tablet_service.cc:1511] Processing CreateTablet for tablet 9e36b9296b6e408da863f2bd3bb7b36b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:28:45.284886 21507 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:45.289266 21445 tablet_service.cc:1511] Processing CreateTablet for tablet b17439cec0b24d57853f995ce7c8e494 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:45.290853 21445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17439cec0b24d57853f995ce7c8e494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.290864 21222 tablet_service.cc:1511] Processing CreateTablet for tablet 93df7ba57f144211b8429c8c773bd74e (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:45.295029 21446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e36b9296b6e408da863f2bd3bb7b36b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.287165 21372 tablet_service.cc:1511] Processing CreateTablet for tablet b17439cec0b24d57853f995ce7c8e494 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:45.298956 21293 tablet_service.cc:1511] Processing CreateTablet for tablet b17439cec0b24d57853f995ce7c8e494 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:28:45.303618 21293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17439cec0b24d57853f995ce7c8e494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.304754 21507 tablet_bootstrap.cc:654] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.305677 21376 tablet_service.cc:1511] Processing CreateTablet for tablet 93df7ba57f144211b8429c8c773bd74e (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:45.306674 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a8c8433d1e274f879bb2b84ad59f5c3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.294481 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 93df7ba57f144211b8429c8c773bd74e (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:28:45.312114 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93df7ba57f144211b8429c8c773bd74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.311074 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93df7ba57f144211b8429c8c773bd74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.312543 21505 tablet_bootstrap.cc:492] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:45.312907 21372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17439cec0b24d57853f995ce7c8e494. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.313061 21505 ts_tablet_manager.cc:1403] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.095s	user 0.016s	sys 0.000s
I20260709 12:28:45.316550 21505 raft_consensus.cc:359] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:45.317478 21505 raft_consensus.cc:385] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.317867 21505 raft_consensus.cc:740] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.319347 21371 tablet_service.cc:1511] Processing CreateTablet for tablet eaea00780b02433fa3a95bf6e01cb78d (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:45.320716 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaea00780b02433fa3a95bf6e01cb78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.316633 21221 tablet_service.cc:1511] Processing CreateTablet for tablet d4da9543a3474f228769c7d28609a702 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:45.321712 21411 heartbeater.cc:499] Master 127.8.137.254:36393 was elected leader, sending a full tablet report...
I20260709 12:28:45.322791 21447 tablet_service.cc:1511] Processing CreateTablet for tablet eaea00780b02433fa3a95bf6e01cb78d (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:45.320338 21302 tablet_service.cc:1511] Processing CreateTablet for tablet d4da9543a3474f228769c7d28609a702 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:45.325724 21447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaea00780b02433fa3a95bf6e01cb78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.325958 21226 tablet_service.cc:1511] Processing CreateTablet for tablet eaea00780b02433fa3a95bf6e01cb78d (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:28:45.327435 21504 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:45.327986 21504 ts_tablet_manager.cc:1403] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.189s	user 0.009s	sys 0.010s
I20260709 12:28:45.327984 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaea00780b02433fa3a95bf6e01cb78d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.329006 21222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93df7ba57f144211b8429c8c773bd74e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.331372 21504 raft_consensus.cc:359] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:45.331872 21449 tablet_service.cc:1511] Processing CreateTablet for tablet d4da9543a3474f228769c7d28609a702 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:28:45.338847 21504 raft_consensus.cc:385] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.334919 21221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4da9543a3474f228769c7d28609a702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.339625 21504 raft_consensus.cc:740] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.341274 21504 consensus_queue.cc:260] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:45.341893 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4da9543a3474f228769c7d28609a702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.345104 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4da9543a3474f228769c7d28609a702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.368842 21301 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbb15b421ee44b48cdf0afeed6430e9 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:45.370026 21370 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbb15b421ee44b48cdf0afeed6430e9 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:45.370471 21301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbb15b421ee44b48cdf0afeed6430e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.371346 21370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbb15b421ee44b48cdf0afeed6430e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.382198 21485 heartbeater.cc:499] Master 127.8.137.254:36393 was elected leader, sending a full tablet report...
I20260709 12:28:45.382829 21504 ts_tablet_manager.cc:1434] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.054s	user 0.011s	sys 0.002s
I20260709 12:28:45.383944 21504 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:45.392815 21505 consensus_queue.cc:260] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:28:45.401809 21486 tablet.cc:2406] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:28:45.421011 21506 tablet_bootstrap.cc:492] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:45.421643 21506 ts_tablet_manager.cc:1403] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.206s	user 0.015s	sys 0.026s
I20260709 12:28:45.425416 21506 raft_consensus.cc:359] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:45.426416 21506 raft_consensus.cc:385] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.426831 21506 raft_consensus.cc:740] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.427637 21506 consensus_queue.cc:260] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:45.454370 21445 tablet_service.cc:1511] Processing CreateTablet for tablet 3dbb15b421ee44b48cdf0afeed6430e9 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:28:45.455873 21445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3dbb15b421ee44b48cdf0afeed6430e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.456840 21504 tablet_bootstrap.cc:654] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.459816 21505 ts_tablet_manager.cc:1434] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.146s	user 0.000s	sys 0.019s
I20260709 12:28:45.461122 21505 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:45.463307 21506 ts_tablet_manager.cc:1434] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.041s	user 0.015s	sys 0.006s
I20260709 12:28:45.464561 21506 tablet_bootstrap.cc:492] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:45.471822 21224 tablet_service.cc:1511] Processing CreateTablet for tablet 5de0c984317b4a16bd865892a98794b7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:28:45.473208 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de0c984317b4a16bd865892a98794b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.481691 21505 tablet_bootstrap.cc:654] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.488407 21507 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:45.488927 21507 ts_tablet_manager.cc:1403] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.205s	user 0.020s	sys 0.011s
I20260709 12:28:45.492094 21507 raft_consensus.cc:359] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:45.492874 21507 raft_consensus.cc:385] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.493186 21507 raft_consensus.cc:740] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.497347 21511 raft_consensus.cc:493] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.497646 21506 tablet_bootstrap.cc:654] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.497941 21511 raft_consensus.cc:515] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:45.503365 21511 leader_election.cc:290] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:45.505223 21507 consensus_queue.cc:260] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:45.517501 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 5de0c984317b4a16bd865892a98794b7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:28:45.519749 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de0c984317b4a16bd865892a98794b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.521404 21504 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:45.523900 21445 tablet_service.cc:1511] Processing CreateTablet for tablet 27e40fdd4a714c729e2bcc297a44ec4f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:45.524044 21301 tablet_service.cc:1511] Processing CreateTablet for tablet 27e40fdd4a714c729e2bcc297a44ec4f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:45.522140 21504 ts_tablet_manager.cc:1403] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.138s	user 0.014s	sys 0.012s
I20260709 12:28:45.527760 21505 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:45.528151 21301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e40fdd4a714c729e2bcc297a44ec4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.528376 21505 ts_tablet_manager.cc:1403] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.067s	user 0.013s	sys 0.005s
I20260709 12:28:45.531821 21505 raft_consensus.cc:359] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:45.533141 21505 raft_consensus.cc:385] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.535492 21505 raft_consensus.cc:740] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.535517 21222 tablet_service.cc:1511] Processing CreateTablet for tablet 27e40fdd4a714c729e2bcc297a44ec4f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:28:45.537186 21222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e40fdd4a714c729e2bcc297a44ec4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.536707 21505 consensus_queue.cc:260] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:45.533671 21445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27e40fdd4a714c729e2bcc297a44ec4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.545722 21507 ts_tablet_manager.cc:1434] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.056s	user 0.009s	sys 0.005s
I20260709 12:28:45.548686 21449 tablet_service.cc:1511] Processing CreateTablet for tablet 63656d4d714a4ce89de05b1e5ba3fe43 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:45.549813 21370 tablet_service.cc:1511] Processing CreateTablet for tablet 5de0c984317b4a16bd865892a98794b7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:28:45.550402 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63656d4d714a4ce89de05b1e5ba3fe43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.551192 21370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5de0c984317b4a16bd865892a98794b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.551522 21221 tablet_service.cc:1511] Processing CreateTablet for tablet 63656d4d714a4ce89de05b1e5ba3fe43 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:45.553066 21221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63656d4d714a4ce89de05b1e5ba3fe43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.560555 21507 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:45.560158 21376 tablet_service.cc:1511] Processing CreateTablet for tablet 63656d4d714a4ce89de05b1e5ba3fe43 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:28:45.560335 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:45.575184 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63656d4d714a4ce89de05b1e5ba3fe43. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:45.576061 21423 leader_election.cc:343] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.579909 21507 tablet_bootstrap.cc:654] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.569582 21506 tablet_bootstrap.cc:492] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:45.561815 21504 raft_consensus.cc:359] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:45.580549 21504 raft_consensus.cc:385] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.580705 21506 ts_tablet_manager.cc:1403] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.116s	user 0.010s	sys 0.006s
I20260709 12:28:45.580916 21504 raft_consensus.cc:740] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.581975 21504 consensus_queue.cc:260] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:45.584205 21504 ts_tablet_manager.cc:1434] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.002s
I20260709 12:28:45.585330 21504 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:45.562549 21505 ts_tablet_manager.cc:1434] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.005s
I20260709 12:28:45.573498 21293 tablet_service.cc:1511] Processing CreateTablet for tablet 7d3bfcb7836d457caed630ee21058a19 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:45.587536 21293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d3bfcb7836d457caed630ee21058a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.589774 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 7d3bfcb7836d457caed630ee21058a19 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:45.591202 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d3bfcb7836d457caed630ee21058a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.592868 21224 tablet_service.cc:1511] Processing CreateTablet for tablet fb7aa63607864dce845a979faebae247 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:45.594295 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7aa63607864dce845a979faebae247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.595232 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:45.591329 21506 raft_consensus.cc:359] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:28:45.596975 21426 leader_election.cc:343] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.602027 21506 raft_consensus.cc:385] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.602130 21426 leader_election.cc:304] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:45.599505 21505 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:45.606081 21511 raft_consensus.cc:2749] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:45.599730 21449 tablet_service.cc:1511] Processing CreateTablet for tablet 7d3bfcb7836d457caed630ee21058a19 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:28:45.608158 21506 raft_consensus.cc:740] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.615361 21506 consensus_queue.cc:260] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:45.619496 21513 raft_consensus.cc:493] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.620077 21513 raft_consensus.cc:515] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:45.621037 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d3bfcb7836d457caed630ee21058a19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.629458 21514 raft_consensus.cc:493] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.630188 21514 raft_consensus.cc:515] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:45.650244 21504 tablet_bootstrap.cc:654] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.655800 21505 tablet_bootstrap.cc:654] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.662881 21296 tablet_service.cc:1511] Processing CreateTablet for tablet fb7aa63607864dce845a979faebae247 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:45.664294 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7aa63607864dce845a979faebae247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.664027 21372 tablet_service.cc:1511] Processing CreateTablet for tablet fb7aa63607864dce845a979faebae247 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:28:45.665088 21514 leader_election.cc:290] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:28:45.669112 21481 debug-util.cc:398] Leaking SignalData structure 0x7b0800134a20 after lost signal to thread 8746
W20260709 12:28:45.670288 21481 debug-util.cc:398] Leaking SignalData structure 0x7b08001380e0 after lost signal to thread 21169
I20260709 12:28:45.666410 21506 ts_tablet_manager.cc:1434] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.085s	user 0.009s	sys 0.000s
I20260709 12:28:45.687762 21513 leader_election.cc:290] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:45.689167 21507 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:45.689788 21507 ts_tablet_manager.cc:1403] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.130s	user 0.012s	sys 0.006s
I20260709 12:28:45.691231 21506 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:45.693641 21507 raft_consensus.cc:359] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:45.697252 21507 raft_consensus.cc:385] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.697598 21507 raft_consensus.cc:740] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.698573 21507 consensus_queue.cc:260] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:45.700461 21372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7aa63607864dce845a979faebae247. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.701994 21507 ts_tablet_manager.cc:1434] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 12:28:45.703235 21507 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:45.707588 21504 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:45.708209 21504 ts_tablet_manager.cc:1403] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.123s	user 0.015s	sys 0.012s
I20260709 12:28:45.710824 21224 tablet_service.cc:1511] Processing CreateTablet for tablet 0d0bfa2f2d224261803bbc1ee6978296 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:45.712261 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d0bfa2f2d224261803bbc1ee6978296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.716230 21293 tablet_service.cc:1511] Processing CreateTablet for tablet 0d0bfa2f2d224261803bbc1ee6978296 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:45.717693 21293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d0bfa2f2d224261803bbc1ee6978296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.722193 21445 tablet_service.cc:1511] Processing CreateTablet for tablet 0d0bfa2f2d224261803bbc1ee6978296 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:28:45.723846 21445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d0bfa2f2d224261803bbc1ee6978296. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.733544 21505 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:45.734220 21505 ts_tablet_manager.cc:1403] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.135s	user 0.018s	sys 0.012s
I20260709 12:28:45.727658 21506 tablet_bootstrap.cc:654] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.740219 21221 tablet_service.cc:1511] Processing CreateTablet for tablet cc850ba2d5144b66bf78197d972b70d8 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:45.741732 21221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc850ba2d5144b66bf78197d972b70d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.742388 21504 raft_consensus.cc:359] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:45.743222 21504 raft_consensus.cc:385] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.743528 21504 raft_consensus.cc:740] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.750432 21220 tablet_service.cc:1511] Processing CreateTablet for tablet 13c370d75f794be68da9f93a6a876a55 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:45.751895 21220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13c370d75f794be68da9f93a6a876a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.754844 21507 tablet_bootstrap.cc:654] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.769119 21505 raft_consensus.cc:359] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:45.769976 21505 raft_consensus.cc:385] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.751394 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 81d20a4b91864a4787df9fdd31977f44 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:45.770314 21505 raft_consensus.cc:740] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.774719 21449 tablet_service.cc:1511] Processing CreateTablet for tablet cc850ba2d5144b66bf78197d972b70d8 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:45.775563 21301 tablet_service.cc:1511] Processing CreateTablet for tablet 13c370d75f794be68da9f93a6a876a55 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:45.776213 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc850ba2d5144b66bf78197d972b70d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.776998 21301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13c370d75f794be68da9f93a6a876a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.778569 21447 tablet_service.cc:1511] Processing CreateTablet for tablet 81d20a4b91864a4787df9fdd31977f44 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:45.780504 21447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d20a4b91864a4787df9fdd31977f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.789489 21505 consensus_queue.cc:260] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:45.790216 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:45.792482 21351 leader_election.cc:343] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.792753 21505 ts_tablet_manager.cc:1434] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.058s	user 0.005s	sys 0.005s
I20260709 12:28:45.794001 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 81d20a4b91864a4787df9fdd31977f44 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:28:45.795454 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d20a4b91864a4787df9fdd31977f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.800335 21376 tablet_service.cc:1511] Processing CreateTablet for tablet cc850ba2d5144b66bf78197d972b70d8 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:28:45.804592 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:45.806489 21275 leader_election.cc:343] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.744302 21504 consensus_queue.cc:260] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:45.807260 21505 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:45.809183 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81d20a4b91864a4787df9fdd31977f44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.810685 21511 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.834167 21511 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:45.834635 21446 tablet_service.cc:1511] Processing CreateTablet for tablet 79a82803d22242a991a35ece42323ea7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:45.817180 21370 tablet_service.cc:1511] Processing CreateTablet for tablet 13c370d75f794be68da9f93a6a876a55 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:28:45.836051 21446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a82803d22242a991a35ece42323ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.840682 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:45.842664 21426 leader_election.cc:343] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.845780 21511 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:45.846894 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc850ba2d5144b66bf78197d972b70d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.850992 21370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13c370d75f794be68da9f93a6a876a55. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.853657 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:45.853956 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:45.855424 21351 leader_election.cc:343] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.855911 21351 leader_election.cc:304] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:28:45.856900 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:45.857491 21513 raft_consensus.cc:2749] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:45.858408 21513 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.858893 21513 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:45.862471 21513 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:45.866384 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
W20260709 12:28:45.867980 21351 leader_election.cc:343] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:45.869248 21277 leader_election.cc:343] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.874761 21506 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:45.875378 21506 ts_tablet_manager.cc:1403] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.184s	user 0.022s	sys 0.019s
I20260709 12:28:45.876192 21521 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.876784 21521 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:45.879472 21277 leader_election.cc:304] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:45.885073 21521 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:45.889194 21514 raft_consensus.cc:2749] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:45.890985 21504 ts_tablet_manager.cc:1434] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.182s	user 0.012s	sys 0.005s
I20260709 12:28:45.891943 21220 tablet_service.cc:1511] Processing CreateTablet for tablet 79a82803d22242a991a35ece42323ea7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:45.892549 21504 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:45.905283 21372 tablet_service.cc:1511] Processing CreateTablet for tablet de84b253ee104e63be65f2ef9e9173dd (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:45.906046 21220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a82803d22242a991a35ece42323ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.906903 21372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de84b253ee104e63be65f2ef9e9173dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.907764 21293 tablet_service.cc:1511] Processing CreateTablet for tablet 79a82803d22242a991a35ece42323ea7 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:28:45.909087 21293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 79a82803d22242a991a35ece42323ea7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.911065 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:45.914040 21352 leader_election.cc:343] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.914531 21352 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:45.915751 21513 raft_consensus.cc:2749] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:45.878871 21506 raft_consensus.cc:359] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:45.922166 21506 raft_consensus.cc:385] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.922554 21506 raft_consensus.cc:740] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:45.923383 21506 consensus_queue.cc:260] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:45.932780 21505 tablet_bootstrap.cc:654] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.933055 21507 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:45.935148 21506 ts_tablet_manager.cc:1434] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.000s
I20260709 12:28:45.937794 21504 tablet_bootstrap.cc:654] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.941808 21507 ts_tablet_manager.cc:1403] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.239s	user 0.003s	sys 0.018s
I20260709 12:28:45.942914 21446 tablet_service.cc:1511] Processing CreateTablet for tablet de84b253ee104e63be65f2ef9e9173dd (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:45.944489 21446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de84b253ee104e63be65f2ef9e9173dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.946553 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:45.946818 21507 raft_consensus.cc:359] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:45.947540 21507 raft_consensus.cc:385] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:45.947877 21507 raft_consensus.cc:740] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
W20260709 12:28:45.948426 21425 leader_election.cc:343] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.948961 21425 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:45.948761 21507 consensus_queue.cc:260] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:45.951797 21534 raft_consensus.cc:2749] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:45.953495 21506 tablet_bootstrap.cc:492] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:45.958027 21221 tablet_service.cc:1511] Processing CreateTablet for tablet de84b253ee104e63be65f2ef9e9173dd (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:28:45.959632 21221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de84b253ee104e63be65f2ef9e9173dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.962764 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:45.964337 21507 ts_tablet_manager.cc:1434] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:28:45.970388 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 1f486fb8a1874281ab5022d0aea60379 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
W20260709 12:28:45.971750 21200 leader_election.cc:343] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.975127 21506 tablet_bootstrap.cc:654] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:45.974980 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 1f486fb8a1874281ab5022d0aea60379 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:28:45.976518 21507 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:45.976480 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f486fb8a1874281ab5022d0aea60379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.980310 21447 tablet_service.cc:1511] Processing CreateTablet for tablet 1f486fb8a1874281ab5022d0aea60379 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:28:45.984741 21224 tablet_service.cc:1511] Processing CreateTablet for tablet d0ceb29ef8444e9d96f0276fe5fe93fd (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:45.987797 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:45.990554 21202 leader_election.cc:343] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:45.991087 21202 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:45.992233 21521 raft_consensus.cc:2749] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:45.987154 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f486fb8a1874281ab5022d0aea60379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.994750 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ceb29ef8444e9d96f0276fe5fe93fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:45.994453 21534 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:45.995345 21534 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:45.997690 21534 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:45.998438 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:45.998842 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:45.999516 21312 raft_consensus.cc:2468] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
W20260709 12:28:45.999761 21426 leader_election.cc:343] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.001087 21447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f486fb8a1874281ab5022d0aea60379. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.003834 21425 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:46.005291 21506 tablet_bootstrap.cc:492] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:46.006196 21301 tablet_service.cc:1511] Processing CreateTablet for tablet d0ceb29ef8444e9d96f0276fe5fe93fd (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:46.009284 21507 tablet_bootstrap.cc:654] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.010213 21372 tablet_service.cc:1511] Processing CreateTablet for tablet d0ceb29ef8444e9d96f0276fe5fe93fd (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:28:46.015491 21446 tablet_service.cc:1511] Processing CreateTablet for tablet b1620754a7ca4c3ca00eff22b9493e9c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:46.016659 21221 tablet_service.cc:1511] Processing CreateTablet for tablet b1620754a7ca4c3ca00eff22b9493e9c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:46.018756 21534 raft_consensus.cc:2804] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:46.020499 21293 tablet_service.cc:1511] Processing CreateTablet for tablet b1620754a7ca4c3ca00eff22b9493e9c (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:28:46.021721 21370 tablet_service.cc:1511] Processing CreateTablet for tablet 2f6dbe944bf742148d2688474712308b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:28:46.024436 21506 ts_tablet_manager.cc:1403] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.071s	user 0.014s	sys 0.003s
I20260709 12:28:46.028645 21534 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.029831 21534 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.032488 21506 raft_consensus.cc:359] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.033279 21506 raft_consensus.cc:385] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.023280 21445 tablet_service.cc:1511] Processing CreateTablet for tablet 2f6dbe944bf742148d2688474712308b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:28:46.025847 21221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1620754a7ca4c3ca00eff22b9493e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.025923 21372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ceb29ef8444e9d96f0276fe5fe93fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.039000 21521 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.039616 21521 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.026793 21446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1620754a7ca4c3ca00eff22b9493e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.042250 21521 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:46.023334 21301 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0ceb29ef8444e9d96f0276fe5fe93fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.068876 21295 tablet_service.cc:1511] Processing CreateTablet for tablet 85044ca33b9947dface5200a544f79a2 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:28:46.070401 21295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85044ca33b9947dface5200a544f79a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.043167 21445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f6dbe944bf742148d2688474712308b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.074764 21521 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.075348 21521 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.077834 21514 raft_consensus.cc:493] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.027769 21302 tablet_service.cc:1511] Processing CreateTablet for tablet 96593726ac324bc78be4e7c2bdb7d850 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:46.033600 21506 raft_consensus.cc:740] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.078541 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:46.079480 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96593726ac324bc78be4e7c2bdb7d850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.079326 21506 consensus_queue.cc:260] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:28:46.080197 21200 leader_election.cc:343] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.046963 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.054093 21504 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:46.081301 21504 ts_tablet_manager.cc:1403] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.189s	user 0.007s	sys 0.016s
W20260709 12:28:46.082321 21202 leader_election.cc:343] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.050861 21370 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f6dbe944bf742148d2688474712308b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.084597 21504 raft_consensus.cc:359] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.059893 21505 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:46.085454 21504 raft_consensus.cc:385] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.085847 21504 raft_consensus.cc:740] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.085902 21505 ts_tablet_manager.cc:1403] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.279s	user 0.016s	sys 0.008s
I20260709 12:28:46.058367 21376 tablet_service.cc:1511] Processing CreateTablet for tablet 96593726ac324bc78be4e7c2bdb7d850 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:46.071681 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:46.068879 21222 tablet_service.cc:1511] Processing CreateTablet for tablet 85044ca33b9947dface5200a544f79a2 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:28:46.087585 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96593726ac324bc78be4e7c2bdb7d850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.088297 21222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85044ca33b9947dface5200a544f79a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.089522 21505 raft_consensus.cc:359] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.090294 21505 raft_consensus.cc:385] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.078369 21514 raft_consensus.cc:515] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.047451 21534 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:46.060670 21449 tablet_service.cc:1511] Processing CreateTablet for tablet 96593726ac324bc78be4e7c2bdb7d850 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:28:46.092399 21514 leader_election.cc:290] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:46.032642 21220 tablet_service.cc:1511] Processing CreateTablet for tablet 2f6dbe944bf742148d2688474712308b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:28:46.093758 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.093881 21534 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:46.090575 21505 raft_consensus.cc:740] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
W20260709 12:28:46.121899 21277 leader_election.cc:343] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.095362 21504 consensus_queue.cc:260] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.123796 21293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b1620754a7ca4c3ca00eff22b9493e9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.125025 21504 ts_tablet_manager.cc:1434] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.043s	user 0.003s	sys 0.003s
I20260709 12:28:46.123297 21220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f6dbe944bf742148d2688474712308b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.126099 21504 tablet_bootstrap.cc:492] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:46.096802 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96593726ac324bc78be4e7c2bdb7d850. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.102384 21443 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc226a03f0e4040aabb242f51269a5b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:46.130066 21443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc226a03f0e4040aabb242f51269a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.134929 21505 consensus_queue.cc:260] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.140601 21505 ts_tablet_manager.cc:1434] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.054s	user 0.000s	sys 0.008s
I20260709 12:28:46.141716 21505 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:46.141927 21442 tablet_service.cc:1511] Processing CreateTablet for tablet e5192ed91b2d4c14905f4a0a7410da9f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:46.143316 21442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5192ed91b2d4c14905f4a0a7410da9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.145450 21302 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2b7f6dfa92472f8a79868af4c7d106 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:46.146776 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2b7f6dfa92472f8a79868af4c7d106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.151065 21505 tablet_bootstrap.cc:654] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.155964 21446 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2b7f6dfa92472f8a79868af4c7d106 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:46.159596 21295 tablet_service.cc:1511] Processing CreateTablet for tablet 3cd8e0c4a39c4a85bc4a2663a8a39c5f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:46.161262 21295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd8e0c4a39c4a85bc4a2663a8a39c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.161079 21369 tablet_service.cc:1511] Processing CreateTablet for tablet 3cd8e0c4a39c4a85bc4a2663a8a39c5f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:46.162468 21369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd8e0c4a39c4a85bc4a2663a8a39c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.166182 21446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2b7f6dfa92472f8a79868af4c7d106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.159677 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 9e2b7f6dfa92472f8a79868af4c7d106 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:28:46.167418 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9e2b7f6dfa92472f8a79868af4c7d106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.103778 21507 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:46.174180 21447 tablet_service.cc:1511] Processing CreateTablet for tablet f78e89ba4d49471aaf6eb57ea8f738bf (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:46.175233 21507 ts_tablet_manager.cc:1403] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.199s	user 0.012s	sys 0.007s
I20260709 12:28:46.107909 21521 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:46.178925 21447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78e89ba4d49471aaf6eb57ea8f738bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.111454 21368 tablet_service.cc:1511] Processing CreateTablet for tablet e5192ed91b2d4c14905f4a0a7410da9f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:46.180665 21368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5192ed91b2d4c14905f4a0a7410da9f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.093936 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:46.095595 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:46.095952 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:46.185668 21312 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.192422 21504 tablet_bootstrap.cc:654] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:46.192819 21275 leader_election.cc:343] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:46.192914 21426 leader_election.cc:343] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:46.103046 21202 leader_election.cc:343] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.198910 21202 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:28:46.199908 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 96903.2us
I20260709 12:28:46.110278 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc226a03f0e4040aabb242f51269a5b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:46.203444 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.203819 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc226a03f0e4040aabb242f51269a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.203625 21226 tablet_service.cc:1511] Processing CreateTablet for tablet f78e89ba4d49471aaf6eb57ea8f738bf (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:46.204638 21521 raft_consensus.cc:2749] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:46.204998 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78e89ba4d49471aaf6eb57ea8f738bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.099097 21372 tablet_service.cc:1511] Processing CreateTablet for tablet 85044ca33b9947dface5200a544f79a2 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:28:46.206207 21372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85044ca33b9947dface5200a544f79a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.207152 21224 tablet_service.cc:1511] Processing CreateTablet for tablet 3cd8e0c4a39c4a85bc4a2663a8a39c5f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:28:46.208638 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3cd8e0c4a39c4a85bc4a2663a8a39c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.104862 21221 tablet_service.cc:1511] Processing CreateTablet for tablet 5dc226a03f0e4040aabb242f51269a5b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:28:46.198513 21275 leader_election.cc:304] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:46.211742 21221 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dc226a03f0e4040aabb242f51269a5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.212157 21542 raft_consensus.cc:2749] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:46.214627 21222 tablet_service.cc:1511] Processing CreateTablet for tablet e5192ed91b2d4c14905f4a0a7410da9f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:28:46.217079 21222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5192ed91b2d4c14905f4a0a7410da9f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:46.215586 21202 leader_election.cc:343] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.222123 21202 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:46.223238 21521 raft_consensus.cc:2749] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:46.193746 21506 ts_tablet_manager.cc:1434] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.164s	user 0.016s	sys 0.004s
I20260709 12:28:46.191145 21507 raft_consensus.cc:359] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:46.225749 21507 raft_consensus.cc:385] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.226118 21507 raft_consensus.cc:740] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.226923 21507 consensus_queue.cc:260] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:46.229557 21507 ts_tablet_manager.cc:1434] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 12:28:46.231411 21506 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:46.243125 21295 tablet_service.cc:1511] Processing CreateTablet for tablet f78e89ba4d49471aaf6eb57ea8f738bf (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:28:46.244683 21295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f78e89ba4d49471aaf6eb57ea8f738bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.234572 21507 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:46.280439 21505 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:46.281116 21505 ts_tablet_manager.cc:1403] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.140s	user 0.018s	sys 0.012s
I20260709 12:28:46.282861 21513 raft_consensus.cc:493] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.283429 21513 raft_consensus.cc:515] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.285604 21513 leader_election.cc:290] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:46.286559 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.289604 21460 raft_consensus.cc:2468] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:46.289748 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:46.291610 21351 leader_election.cc:304] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
W20260709 12:28:46.292297 21352 leader_election.cc:343] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.292467 21513 raft_consensus.cc:2804] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:46.292940 21513 raft_consensus.cc:493] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.293535 21513 raft_consensus.cc:3060] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.294124 21376 tablet_service.cc:1511] Processing CreateTablet for tablet cdd6e626d31d4da8a2e4a407e1710ba2 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:46.301743 21442 tablet_service.cc:1511] Processing CreateTablet for tablet cdd6e626d31d4da8a2e4a407e1710ba2 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:46.303257 21442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd6e626d31d4da8a2e4a407e1710ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.303933 21312 raft_consensus.cc:2468] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:46.305445 21425 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:46.306320 21534 raft_consensus.cc:2804] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:46.306857 21534 raft_consensus.cc:697] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:46.307695 21534 consensus_queue.cc:237] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:46.316721 21505 raft_consensus.cc:359] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.317569 21505 raft_consensus.cc:385] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.317883 21505 raft_consensus.cc:740] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.323323 21293 tablet_service.cc:1511] Processing CreateTablet for tablet c0cc4e1a190343f19f100f757eab0c65 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:46.324700 21293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0cc4e1a190343f19f100f757eab0c65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.326820 21507 tablet_bootstrap.cc:654] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.327142 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd6e626d31d4da8a2e4a407e1710ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.332371 21220 tablet_service.cc:1511] Processing CreateTablet for tablet cdd6e626d31d4da8a2e4a407e1710ba2 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:28:46.335085 21506 tablet_bootstrap.cc:654] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.336066 21369 tablet_service.cc:1511] Processing CreateTablet for tablet c0cc4e1a190343f19f100f757eab0c65 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:46.337517 21369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0cc4e1a190343f19f100f757eab0c65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.353294 21542 raft_consensus.cc:493] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.354084 21542 raft_consensus.cc:515] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.356181 21133 catalog_manager.cc:5697] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:46.357646 21542 leader_election.cc:290] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:46.357791 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:46.358686 21386 raft_consensus.cc:2468] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:46.359076 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.360085 21275 leader_election.cc:304] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:46.360989 21542 raft_consensus.cc:2804] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:28:46.361141 21277 leader_election.cc:343] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.366667 21226 tablet_service.cc:1511] Processing CreateTablet for tablet f08ca19d90734deca4d113875d7def85 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:46.367064 21443 tablet_service.cc:1511] Processing CreateTablet for tablet c0cc4e1a190343f19f100f757eab0c65 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:28:46.368528 21443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0cc4e1a190343f19f100f757eab0c65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.363615 21542 raft_consensus.cc:493] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.371135 21542 raft_consensus.cc:3060] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.383322 21513 raft_consensus.cc:515] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.338887 21505 consensus_queue.cc:260] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.386561 21513 leader_election.cc:290] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:46.388339 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:46.409057 21295 tablet_service.cc:1511] Processing CreateTablet for tablet f08ca19d90734deca4d113875d7def85 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:46.395946 21504 tablet_bootstrap.cc:492] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:46.411234 21504 ts_tablet_manager.cc:1403] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.285s	user 0.014s	sys 0.023s
I20260709 12:28:46.415609 21224 tablet_service.cc:1511] Processing CreateTablet for tablet c4c805fca65a4771bf0e2270955c14c0 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:46.416643 21220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdd6e626d31d4da8a2e4a407e1710ba2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.417452 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c805fca65a4771bf0e2270955c14c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.388672 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:46.419481 21460 raft_consensus.cc:3060] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:46.422079 21352 leader_election.cc:343] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.423913 21295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f08ca19d90734deca4d113875d7def85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.426306 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f08ca19d90734deca4d113875d7def85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.414398 21504 raft_consensus.cc:359] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.441371 21302 tablet_service.cc:1511] Processing CreateTablet for tablet c4c805fca65a4771bf0e2270955c14c0 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:46.442802 21534 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.443488 21534 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.445523 21460 raft_consensus.cc:2468] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:46.446511 21368 tablet_service.cc:1511] Processing CreateTablet for tablet f08ca19d90734deca4d113875d7def85 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:28:46.447247 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c805fca65a4771bf0e2270955c14c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.447659 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:46.449674 21506 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:46.450083 21504 raft_consensus.cc:385] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.452270 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:46.446539 21534 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:46.466280 21504 raft_consensus.cc:740] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.469102 21506 ts_tablet_manager.cc:1403] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.238s	user 0.028s	sys 0.040s
W20260709 12:28:46.473656 21423 leader_election.cc:343] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.474318 21507 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:46.468952 21237 raft_consensus.cc:2468] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:46.474999 21507 ts_tablet_manager.cc:1403] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.245s	user 0.022s	sys 0.031s
I20260709 12:28:46.475847 21426 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:46.477612 21371 tablet_service.cc:1511] Processing CreateTablet for tablet f6048d81937c4eaab03874e7a6577982 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:46.478798 21534 raft_consensus.cc:2804] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:46.479130 21368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f08ca19d90734deca4d113875d7def85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.474387 21351 leader_election.cc:304] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:46.479450 21504 consensus_queue.cc:260] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.470124 21442 tablet_service.cc:1511] Processing CreateTablet for tablet c4c805fca65a4771bf0e2270955c14c0 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:28:46.481458 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6048d81937c4eaab03874e7a6577982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.482285 21504 ts_tablet_manager.cc:1434] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.002s
I20260709 12:28:46.479218 21534 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.482961 21548 raft_consensus.cc:2804] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:46.475715 21506 raft_consensus.cc:359] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.482450 21442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c4c805fca65a4771bf0e2270955c14c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.493821 21506 raft_consensus.cc:385] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.494310 21506 raft_consensus.cc:740] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.496315 21506 consensus_queue.cc:260] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.488222 21548 raft_consensus.cc:697] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:46.479171 21443 tablet_service.cc:1511] Processing CreateTablet for tablet f6048d81937c4eaab03874e7a6577982 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:46.498167 21443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6048d81937c4eaab03874e7a6577982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.497921 21548 consensus_queue.cc:237] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.478227 21507 raft_consensus.cc:359] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.502930 21507 raft_consensus.cc:385] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.503337 21507 raft_consensus.cc:740] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.491289 21542 raft_consensus.cc:515] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.486797 21534 raft_consensus.cc:3060] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.495488 21504 tablet_bootstrap.cc:492] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:46.512277 21513 raft_consensus.cc:493] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.512862 21513 raft_consensus.cc:515] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.514182 21542 leader_election.cc:290] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:46.515084 21513 leader_election.cc:290] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:46.505137 21507 consensus_queue.cc:260] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.505752 21505 ts_tablet_manager.cc:1434] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.224s	user 0.010s	sys 0.015s
I20260709 12:28:46.517481 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:46.515481 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:46.518671 21506 ts_tablet_manager.cc:1434] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.046s	user 0.008s	sys 0.000s
I20260709 12:28:46.516640 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:46.518812 21505 tablet_bootstrap.cc:492] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:46.520040 21506 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
W20260709 12:28:46.520378 21352 leader_election.cc:343] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:46.521917 21351 leader_election.cc:343] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.522436 21351 leader_election.cc:304] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:46.523293 21521 raft_consensus.cc:493] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.517481 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:46.523414 21513 raft_consensus.cc:2749] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:46.523986 21460 raft_consensus.cc:3060] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.523794 21521 raft_consensus.cc:515] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.496742 21224 tablet_service.cc:1511] Processing CreateTablet for tablet f6048d81937c4eaab03874e7a6577982 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:28:46.527798 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6048d81937c4eaab03874e7a6577982. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.518924 21386 raft_consensus.cc:3060] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.529305 21521 leader_election.cc:290] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:46.532123 21505 tablet_bootstrap.cc:654] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.531920 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.533242 21459 raft_consensus.cc:2393] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:46.539526 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb329d7c3274501a5c1c9f03eebd18f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:46.540953 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb329d7c3274501a5c1c9f03eebd18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.541463 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:46.543220 21200 leader_election.cc:304] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:46.543641 21369 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb329d7c3274501a5c1c9f03eebd18f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:46.545758 21521 raft_consensus.cc:3060] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.546710 21369 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb329d7c3274501a5c1c9f03eebd18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.554068 21504 tablet_bootstrap.cc:654] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.562330 21386 raft_consensus.cc:2468] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:46.563767 21275 leader_election.cc:304] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:46.554124 21521 raft_consensus.cc:2749] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:46.564611 21542 raft_consensus.cc:2804] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:46.565189 21542 raft_consensus.cc:697] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:46.555531 21506 tablet_bootstrap.cc:654] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.565917 21521 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.566020 21542 consensus_queue.cc:237] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.570138 21460 raft_consensus.cc:2468] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:46.565886 21449 tablet_service.cc:1511] Processing CreateTablet for tablet 1cb329d7c3274501a5c1c9f03eebd18f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:28:46.571427 21513 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.590817 21513 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.591549 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1cb329d7c3274501a5c1c9f03eebd18f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.593173 21513 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:46.579116 21226 tablet_service.cc:1511] Processing CreateTablet for tablet d3c29a71139f45fc849c307af6259abb (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:46.594039 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.594437 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c29a71139f45fc849c307af6259abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.580833 21376 tablet_service.cc:1511] Processing CreateTablet for tablet d3c29a71139f45fc849c307af6259abb (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:46.582618 21521 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:46.566572 21534 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.601610 21447 tablet_service.cc:1511] Processing CreateTablet for tablet 482afeed6d904bb9a5133bff3cb22b8b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:28:46.607053 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:46.607402 21521 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:46.607837 21237 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:46.609280 21447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482afeed6d904bb9a5133bff3cb22b8b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:46.611141 21351 leader_election.cc:343] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.612462 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.613126 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:46.614174 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c29a71139f45fc849c307af6259abb. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:46.614531 21202 leader_election.cc:343] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.614500 21220 tablet_service.cc:1511] Processing CreateTablet for tablet 482afeed6d904bb9a5133bff3cb22b8b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
W20260709 12:28:46.615454 21202 leader_election.cc:343] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.615996 21220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482afeed6d904bb9a5133bff3cb22b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.615861 21202 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:46.616580 21352 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:46.617059 21521 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:46.611591 21302 tablet_service.cc:1511] Processing CreateTablet for tablet d3c29a71139f45fc849c307af6259abb (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:28:46.623668 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:46.625245 21423 leader_election.cc:343] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.624927 21134 catalog_manager.cc:5697] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:46.626638 21534 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:46.628530 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:46.629222 21237 raft_consensus.cc:3060] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.629087 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 482afeed6d904bb9a5133bff3cb22b8b (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:28:46.628389 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3c29a71139f45fc849c307af6259abb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.638351 21548 raft_consensus.cc:2804] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:46.640519 21548 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.640916 21548 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.638051 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 1b2bb57292a3448bba84e7e4ade90382 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:46.644832 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2bb57292a3448bba84e7e4ade90382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.639684 21442 tablet_service.cc:1511] Processing CreateTablet for tablet 1b2bb57292a3448bba84e7e4ade90382 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:46.637897 21507 ts_tablet_manager.cc:1434] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.163s	user 0.008s	sys 0.018s
I20260709 12:28:46.651839 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 482afeed6d904bb9a5133bff3cb22b8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.657191 21237 raft_consensus.cc:2468] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:46.658545 21426 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:46.659736 21534 raft_consensus.cc:2804] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:46.660229 21534 raft_consensus.cc:697] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:46.661010 21534 consensus_queue.cc:237] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.663635 21507 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:46.666435 21442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2bb57292a3448bba84e7e4ade90382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.709297 21507 tablet_bootstrap.cc:654] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:46.717111 21426 consensus_peers.cc:597] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:46.726639 21224 tablet_service.cc:1511] Processing CreateTablet for tablet 1b2bb57292a3448bba84e7e4ade90382 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:28:46.727541 21505 tablet_bootstrap.cc:492] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:46.727914 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b2bb57292a3448bba84e7e4ade90382. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.728071 21505 ts_tablet_manager.cc:1403] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.210s	user 0.011s	sys 0.011s
I20260709 12:28:46.736999 21504 tablet_bootstrap.cc:492] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:46.736681 21295 tablet_service.cc:1511] Processing CreateTablet for tablet bc7d870ec3c54968ba293f0a1e2afab5 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:46.737537 21504 ts_tablet_manager.cc:1403] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.242s	user 0.024s	sys 0.031s
I20260709 12:28:46.741709 21506 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:46.742547 21506 ts_tablet_manager.cc:1403] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.223s	user 0.026s	sys 0.005s
I20260709 12:28:46.743813 21295 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc7d870ec3c54968ba293f0a1e2afab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.747182 21135 catalog_manager.cc:5697] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:46.756398 21542 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.756932 21542 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.759446 21542 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:46.761567 21371 tablet_service.cc:1511] Processing CreateTablet for tablet bc7d870ec3c54968ba293f0a1e2afab5 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:46.763360 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc7d870ec3c54968ba293f0a1e2afab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.775285 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:46.775741 21449 tablet_service.cc:1511] Processing CreateTablet for tablet bc7d870ec3c54968ba293f0a1e2afab5 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:28:46.776078 21386 raft_consensus.cc:2468] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:46.776895 21504 raft_consensus.cc:359] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.777678 21504 raft_consensus.cc:385] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.777992 21504 raft_consensus.cc:740] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.778908 21275 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:46.780212 21542 raft_consensus.cc:2804] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:46.780644 21542 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:46.781003 21542 raft_consensus.cc:3060] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.782277 21534 consensus_queue.cc:1048] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:46.778921 21504 consensus_queue.cc:260] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.785079 21504 ts_tablet_manager.cc:1434] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.047s	user 0.009s	sys 0.000s
I20260709 12:28:46.778025 21224 tablet_service.cc:1511] Processing CreateTablet for tablet f7e6811f72d0447781abab4178b3bdd4 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:46.787134 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e6811f72d0447781abab4178b3bdd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.790366 21376 tablet_service.cc:1511] Processing CreateTablet for tablet f7e6811f72d0447781abab4178b3bdd4 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:46.790979 21302 tablet_service.cc:1511] Processing CreateTablet for tablet f7e6811f72d0447781abab4178b3bdd4 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:28:46.791622 21376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e6811f72d0447781abab4178b3bdd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.792258 21302 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f7e6811f72d0447781abab4178b3bdd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.791957 21296 tablet_service.cc:1511] Processing CreateTablet for tablet e8a5b040cb6442da9a9cfca14c12225a (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:46.793193 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8a5b040cb6442da9a9cfca14c12225a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.794860 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc7d870ec3c54968ba293f0a1e2afab5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.794536 21220 tablet_service.cc:1511] Processing CreateTablet for tablet e8a5b040cb6442da9a9cfca14c12225a (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:46.795933 21220 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8a5b040cb6442da9a9cfca14c12225a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.795621 21447 tablet_service.cc:1511] Processing CreateTablet for tablet e8a5b040cb6442da9a9cfca14c12225a (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:28:46.796897 21447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8a5b040cb6442da9a9cfca14c12225a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.796563 21505 raft_consensus.cc:359] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.797344 21505 raft_consensus.cc:385] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.797655 21505 raft_consensus.cc:740] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.798722 21505 consensus_queue.cc:260] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.787209 21504 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:46.813730 21368 tablet_service.cc:1511] Processing CreateTablet for tablet f52db4db2acf4abfa01aa7604963edbc (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:46.814482 21507 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:46.815135 21507 ts_tablet_manager.cc:1403] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.152s	user 0.016s	sys 0.014s
I20260709 12:28:46.815253 21368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52db4db2acf4abfa01aa7604963edbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.822229 21443 tablet_service.cc:1511] Processing CreateTablet for tablet f52db4db2acf4abfa01aa7604963edbc (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:46.835599 21443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52db4db2acf4abfa01aa7604963edbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.836997 21505 ts_tablet_manager.cc:1434] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.109s	user 0.008s	sys 0.000s
I20260709 12:28:46.838172 21505 tablet_bootstrap.cc:492] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:46.835186 21226 tablet_service.cc:1511] Processing CreateTablet for tablet f52db4db2acf4abfa01aa7604963edbc (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:28:46.838886 21548 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.839392 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f52db4db2acf4abfa01aa7604963edbc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.841099 21548 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:46.842748 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:46.843382 21237 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.843240 21293 tablet_service.cc:1511] Processing CreateTablet for tablet 315529067cd1423da537bde53365bbbe (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:46.850488 21293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 315529067cd1423da537bde53365bbbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:46.862346 21548 raft_consensus.cc:493] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:46.868290 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
W20260709 12:28:46.869918 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:46.874119 21548 raft_consensus.cc:515] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:46.871018 21507 raft_consensus.cc:359] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.876650 21507 raft_consensus.cc:385] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.876981 21507 raft_consensus.cc:740] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.877802 21371 tablet_service.cc:1511] Processing CreateTablet for tablet 315529067cd1423da537bde53365bbbe (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:46.878031 21507 consensus_queue.cc:260] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.879387 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 315529067cd1423da537bde53365bbbe. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:46.881606 21351 leader_election.cc:343] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:46.873581 21504 tablet_bootstrap.cc:654] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:46.870358 21506 raft_consensus.cc:359] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.884680 21506 raft_consensus.cc:385] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:46.885090 21506 raft_consensus.cc:740] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:46.948815 21548 leader_election.cc:290] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:46.950106 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:46.950313 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:46.950870 21460 raft_consensus.cc:2468] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:46.951045 21311 raft_consensus.cc:2468] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:46.952461 21351 leader_election.cc:304] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:46.952708 21237 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:46.950498 21506 consensus_queue.cc:260] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.953995 21352 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:46.954932 21513 raft_consensus.cc:2804] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:46.955973 21513 raft_consensus.cc:493] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:28:46.966459 21121 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 47129.4us
I20260709 12:28:46.956337 21513 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:46.962085 21548 raft_consensus.cc:2804] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:46.988530 21548 raft_consensus.cc:697] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:46.989562 21548 consensus_queue.cc:237] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:46.999258 21507 ts_tablet_manager.cc:1434] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.184s	user 0.011s	sys 0.005s
I20260709 12:28:47.000196 21506 ts_tablet_manager.cc:1434] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.249s	user 0.013s	sys 0.009s
I20260709 12:28:47.000486 21507 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:47.001267 21506 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:47.000389 21542 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:46.973023 21514 consensus_queue.cc:1048] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
I20260709 12:28:47.009338 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:47.011211 21278 leader_election.cc:343] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.020697 21513 raft_consensus.cc:515] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.023972 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:47.024657 21312 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:47.028043 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 71928.6us
I20260709 12:28:47.030737 21506 tablet_bootstrap.cc:654] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.035151 21449 tablet_service.cc:1511] Processing CreateTablet for tablet 315529067cd1423da537bde53365bbbe (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:28:47.036731 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 315529067cd1423da537bde53365bbbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.048079 21569 consensus_queue.cc:1048] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:47.051676 21542 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:47.058959 21513 leader_election.cc:290] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:47.060230 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:47.060912 21459 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.061162 21296 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed20ce36b0e46b6a043a3d1b558fb8f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 12:28:47.062691 21122 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 54968.8us
I20260709 12:28:47.062999 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
W20260709 12:28:47.064575 21278 leader_election.cc:343] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.065305 21505 tablet_bootstrap.cc:654] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.065062 21224 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed20ce36b0e46b6a043a3d1b558fb8f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:28:47.066279 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed20ce36b0e46b6a043a3d1b558fb8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.062438 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed20ce36b0e46b6a043a3d1b558fb8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.072079 21134 catalog_manager.cc:5697] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.076453 21507 tablet_bootstrap.cc:654] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.077792 21459 raft_consensus.cc:2468] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:47.077896 21571 consensus_queue.cc:1048] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 12:28:47.097400 21351 leader_election.cc:304] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:47.098304 21571 raft_consensus.cc:2804] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:47.098834 21571 raft_consensus.cc:697] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:47.099730 21571 consensus_queue.cc:237] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.105027 21504 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:47.105633 21504 ts_tablet_manager.cc:1403] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.320s	user 0.017s	sys 0.019s
I20260709 12:28:47.111338 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:47.100940 21368 tablet_service.cc:1511] Processing CreateTablet for tablet 5ed20ce36b0e46b6a043a3d1b558fb8f (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:28:47.116079 21368 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ed20ce36b0e46b6a043a3d1b558fb8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.112023 21385 raft_consensus.cc:3060] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.118654 21449 tablet_service.cc:1511] Processing CreateTablet for tablet a2384bff30cd4d3f9801e52ba3c28c08 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:47.120808 21224 tablet_service.cc:1511] Processing CreateTablet for tablet a2384bff30cd4d3f9801e52ba3c28c08 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:47.121457 21449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2384bff30cd4d3f9801e52ba3c28c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.121788 21224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2384bff30cd4d3f9801e52ba3c28c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.124734 21296 tablet_service.cc:1511] Processing CreateTablet for tablet a2384bff30cd4d3f9801e52ba3c28c08 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:28:47.125895 21296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2384bff30cd4d3f9801e52ba3c28c08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.125695 21513 consensus_queue.cc:1048] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:47.128763 21504 raft_consensus.cc:359] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.148819 21552 raft_consensus.cc:493] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.150355 21552 raft_consensus.cc:515] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.152691 21552 leader_election.cc:290] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:47.154085 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:47.154125 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:47.149843 21504 raft_consensus.cc:385] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.136703 21312 raft_consensus.cc:2468] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
W20260709 12:28:47.159825 21202 leader_election.cc:343] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.161574 21570 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.162155 21570 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.162976 21371 tablet_service.cc:1511] Processing CreateTablet for tablet f68cddadeb0443b28b03d76d5195fc45 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:28:47.164278 21371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68cddadeb0443b28b03d76d5195fc45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.164664 21570 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:28:47.166119 21200 leader_election.cc:343] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.166647 21200 leader_election.cc:304] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:47.158366 21504 raft_consensus.cc:740] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.181859 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:47.171986 21552 raft_consensus.cc:2749] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.199142 21443 tablet_service.cc:1511] Processing CreateTablet for tablet f68cddadeb0443b28b03d76d5195fc45 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:28:47.200523 21443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68cddadeb0443b28b03d76d5195fc45. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:28:47.187059 21277 leader_election.cc:343] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.205272 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.205749 21511 consensus_queue.cc:1048] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:47.206001 21236 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:47.216296 21226 tablet_service.cc:1511] Processing CreateTablet for tablet f68cddadeb0443b28b03d76d5195fc45 (DEFAULT_TABLE table=test-workload [id=4ac9e3851b394b65a6f52ea1032ec6a2]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:28:47.217742 21226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f68cddadeb0443b28b03d76d5195fc45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:28:47.201824 21504 consensus_queue.cc:260] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.222064 21278 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:47.224865 21570 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.229427 21570 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.235126 21570 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.237715 21506 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:47.238260 21506 ts_tablet_manager.cc:1403] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.237s	user 0.003s	sys 0.024s
I20260709 12:28:47.231468 21133 catalog_manager.cc:5697] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.243825 21133 catalog_manager.cc:5697] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.241907 21506 raft_consensus.cc:359] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.245321 21506 raft_consensus.cc:385] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.245715 21506 raft_consensus.cc:740] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.244977 21505 tablet_bootstrap.cc:492] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:47.276027 21505 ts_tablet_manager.cc:1403] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.438s	user 0.033s	sys 0.016s
I20260709 12:28:47.271173 21385 raft_consensus.cc:2468] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:47.276609 21506 consensus_queue.cc:260] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.278872 21275 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:47.279886 21568 raft_consensus.cc:2804] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:47.280407 21568 raft_consensus.cc:697] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:47.281344 21568 consensus_queue.cc:237] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:47.286661 21423 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:47.288292 21504 ts_tablet_manager.cc:1434] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.182s	user 0.011s	sys 0.000s
I20260709 12:28:47.307534 21504 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:47.308326 21506 ts_tablet_manager.cc:1434] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.070s	user 0.008s	sys 0.000s
I20260709 12:28:47.309446 21506 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:47.327886 21505 raft_consensus.cc:359] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.328649 21505 raft_consensus.cc:385] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.328967 21505 raft_consensus.cc:740] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.329718 21505 consensus_queue.cc:260] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.329496 21135 catalog_manager.cc:5697] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.332729 21507 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:47.333241 21507 ts_tablet_manager.cc:1403] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.333s	user 0.021s	sys 0.012s
I20260709 12:28:47.335642 21507 raft_consensus.cc:359] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.336239 21507 raft_consensus.cc:385] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.336429 21507 raft_consensus.cc:740] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.336925 21507 consensus_queue.cc:260] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.337345 21570 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.340104 21570 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:47.340884 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
W20260709 12:28:47.342226 21277 leader_election.cc:343] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.342263 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:47.343017 21234 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.344383 21504 tablet_bootstrap.cc:654] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.339061 21507 ts_tablet_manager.cc:1434] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:28:47.347833 21507 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:47.350709 21505 ts_tablet_manager.cc:1434] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.001s
I20260709 12:28:47.351852 21505 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:47.354979 21507 tablet_bootstrap.cc:654] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.358229 21505 tablet_bootstrap.cc:654] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.359912 21506 tablet_bootstrap.cc:654] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.364981 21504 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:47.365510 21504 ts_tablet_manager.cc:1403] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.058s	user 0.007s	sys 0.007s
I20260709 12:28:47.368088 21504 raft_consensus.cc:359] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.368790 21504 raft_consensus.cc:385] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.369046 21504 raft_consensus.cc:740] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.369637 21571 raft_consensus.cc:493] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.369719 21504 consensus_queue.cc:260] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.370272 21571 raft_consensus.cc:515] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.372150 21505 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:47.372471 21571 leader_election.cc:290] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:47.372668 21505 ts_tablet_manager.cc:1403] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.005s
I20260709 12:28:47.373529 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:47.373781 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:47.374440 21312 raft_consensus.cc:2468] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
W20260709 12:28:47.375073 21351 leader_election.cc:343] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.375417 21505 raft_consensus.cc:359] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.375906 21351 leader_election.cc:304] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:47.379005 21571 raft_consensus.cc:2804] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.379117 21504 ts_tablet_manager.cc:1434] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:28:47.379540 21571 raft_consensus.cc:493] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.379582 21234 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:47.380296 21571 raft_consensus.cc:3060] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.381157 21278 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:47.380443 21504 tablet_bootstrap.cc:492] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:47.382120 21570 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:47.382628 21570 raft_consensus.cc:697] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:47.383464 21570 consensus_queue.cc:237] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.376155 21505 raft_consensus.cc:385] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.386749 21505 raft_consensus.cc:740] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.387606 21505 consensus_queue.cc:260] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.404919 21505 ts_tablet_manager.cc:1434] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.032s	user 0.009s	sys 0.003s
I20260709 12:28:47.405543 21504 tablet_bootstrap.cc:654] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.409600 21507 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:47.410205 21507 ts_tablet_manager.cc:1403] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.063s	user 0.009s	sys 0.007s
I20260709 12:28:47.409482 21506 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:47.411183 21506 ts_tablet_manager.cc:1403] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.102s	user 0.008s	sys 0.005s
I20260709 12:28:47.411149 21571 raft_consensus.cc:515] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.413453 21571 leader_election.cc:290] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:47.414319 21506 raft_consensus.cc:359] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.414649 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:47.415055 21506 raft_consensus.cc:385] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.415416 21506 raft_consensus.cc:740] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
W20260709 12:28:47.416304 21351 leader_election.cc:343] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.416165 21506 consensus_queue.cc:260] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.417168 21507 raft_consensus.cc:359] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.417941 21507 raft_consensus.cc:385] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.418263 21507 raft_consensus.cc:740] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.419497 21505 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:47.419153 21507 consensus_queue.cc:260] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.419618 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:47.420207 21312 raft_consensus.cc:3060] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.418467 21506 ts_tablet_manager.cc:1434] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:28:47.422019 21507 ts_tablet_manager.cc:1434] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:28:47.422335 21506 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:47.423224 21507 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:47.437567 21571 consensus_queue.cc:1048] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:47.437247 21132 catalog_manager.cc:5697] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.449405 21534 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.449972 21534 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.454838 21312 raft_consensus.cc:2468] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:47.456480 21351 leader_election.cc:304] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:47.457588 21571 raft_consensus.cc:2804] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:47.458173 21571 raft_consensus.cc:697] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:47.459316 21571 consensus_queue.cc:237] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.485569 21507 tablet_bootstrap.cc:654] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.488090 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.488152 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:47.485997 21505 tablet_bootstrap.cc:654] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.491047 21511 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.491636 21511 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.493006 21534 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:47.494217 21511 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:47.494988 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.491045 21131 catalog_manager.cc:5697] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:47.496505 21426 leader_election.cc:343] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.496404 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:47.497030 21423 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:47.498239 21511 raft_consensus.cc:2749] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:28:47.498660 21425 leader_election.cc:343] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.499316 21425 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:47.503600 21506 tablet_bootstrap.cc:654] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.504231 21511 raft_consensus.cc:2749] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.509001 21504 tablet_bootstrap.cc:492] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:47.509342 21507 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:47.509694 21504 ts_tablet_manager.cc:1403] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.129s	user 0.016s	sys 0.021s
I20260709 12:28:47.509855 21507 ts_tablet_manager.cc:1403] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.087s	user 0.032s	sys 0.014s
I20260709 12:28:47.512913 21507 raft_consensus.cc:359] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.513604 21507 raft_consensus.cc:385] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.514249 21507 raft_consensus.cc:740] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.513581 21504 raft_consensus.cc:359] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.514794 21504 raft_consensus.cc:385] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.515164 21504 raft_consensus.cc:740] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.515077 21507 consensus_queue.cc:260] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.516247 21504 consensus_queue.cc:260] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.517598 21507 ts_tablet_manager.cc:1434] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:28:47.518622 21507 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:47.524669 21507 tablet_bootstrap.cc:654] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.533568 21504 ts_tablet_manager.cc:1434] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.005s
I20260709 12:28:47.538252 21458 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.545670 21504 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:47.548246 21513 consensus_queue.cc:1048] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:47.551455 21507 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:47.551985 21507 ts_tablet_manager.cc:1403] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.034s	user 0.013s	sys 0.004s
I20260709 12:28:47.554785 21507 raft_consensus.cc:359] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.555445 21507 raft_consensus.cc:385] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.555712 21507 raft_consensus.cc:740] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.556665 21507 consensus_queue.cc:260] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.558732 21504 tablet_bootstrap.cc:654] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.559103 21507 ts_tablet_manager.cc:1434] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:47.560232 21505 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:47.561381 21505 ts_tablet_manager.cc:1403] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.142s	user 0.021s	sys 0.023s
I20260709 12:28:47.563128 21513 raft_consensus.cc:493] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.563323 21507 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:47.563506 21234 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.563728 21513 raft_consensus.cc:515] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.567001 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.567701 21236 raft_consensus.cc:2468] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:47.569128 21352 leader_election.cc:304] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:47.570247 21548 raft_consensus.cc:2804] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.570662 21548 raft_consensus.cc:493] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.571017 21548 raft_consensus.cc:3060] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.571954 21513 leader_election.cc:290] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:47.572134 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:47.572888 21312 raft_consensus.cc:2468] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:47.576378 21505 raft_consensus.cc:359] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.578655 21548 raft_consensus.cc:515] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.589293 21504 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:47.589841 21504 ts_tablet_manager.cc:1403] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.005s
I20260709 12:28:47.590955 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:47.581369 21506 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:47.592790 21234 raft_consensus.cc:3060] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.591640 21548 leader_election.cc:290] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:47.593026 21506 ts_tablet_manager.cc:1403] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.171s	user 0.017s	sys 0.007s
I20260709 12:28:47.596367 21506 raft_consensus.cc:359] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.597086 21506 raft_consensus.cc:385] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.597415 21506 raft_consensus.cc:740] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.598223 21506 consensus_queue.cc:260] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.600338 21506 ts_tablet_manager.cc:1434] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:47.601795 21506 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:47.579433 21507 tablet_bootstrap.cc:654] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.592000 21505 raft_consensus.cc:385] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.592186 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:47.592579 21504 raft_consensus.cc:359] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.605515 21505 raft_consensus.cc:740] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.605712 21312 raft_consensus.cc:3060] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.605736 21504 raft_consensus.cc:385] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.606308 21504 raft_consensus.cc:740] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.606498 21505 consensus_queue.cc:260] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.607183 21504 consensus_queue.cc:260] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.609082 21506 tablet_bootstrap.cc:654] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.609270 21505 ts_tablet_manager.cc:1434] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 12:28:47.610625 21505 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:47.613628 21504 ts_tablet_manager.cc:1434] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:28:47.614629 21504 tablet_bootstrap.cc:492] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:47.619374 21505 tablet_bootstrap.cc:654] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.626799 21506 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:47.627375 21506 ts_tablet_manager.cc:1403] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.002s
I20260709 12:28:47.630234 21506 raft_consensus.cc:359] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.631145 21506 raft_consensus.cc:385] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.631448 21506 raft_consensus.cc:740] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.632292 21506 consensus_queue.cc:260] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.634910 21506 ts_tablet_manager.cc:1434] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:47.636358 21506 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:47.638198 21504 tablet_bootstrap.cc:654] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.642055 21312 raft_consensus.cc:2468] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:47.640206 21505 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:47.643090 21505 ts_tablet_manager.cc:1403] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.033s	user 0.012s	sys 0.000s
I20260709 12:28:47.643813 21351 leader_election.cc:304] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:47.645141 21548 raft_consensus.cc:2804] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:47.645839 21548 raft_consensus.cc:697] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:47.647006 21505 raft_consensus.cc:359] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.647074 21548 consensus_queue.cc:237] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.653446 21505 raft_consensus.cc:385] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.653981 21505 raft_consensus.cc:740] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.657635 21505 consensus_queue.cc:260] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.659106 21234 raft_consensus.cc:2468] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:47.660159 21506 tablet_bootstrap.cc:654] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.662873 21505 ts_tablet_manager.cc:1434] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:28:47.664453 21505 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:47.667233 21511 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.669286 21511 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.671527 21511 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:47.673870 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:47.674621 21505 tablet_bootstrap.cc:654] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.676262 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.676931 21234 raft_consensus.cc:2468] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:47.674649 21312 raft_consensus.cc:2468] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:47.678399 21426 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:47.679464 21511 raft_consensus.cc:2804] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.679963 21511 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.680354 21511 raft_consensus.cc:3060] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.689302 21571 raft_consensus.cc:493] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.689903 21571 raft_consensus.cc:515] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.690869 21513 consensus_queue.cc:1048] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:47.692126 21571 leader_election.cc:290] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:47.693960 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:47.695626 21352 leader_election.cc:343] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.694393 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:47.701181 21351 leader_election.cc:343] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:47.706581 21504 tablet_bootstrap.cc:492] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:47.708000 21504 ts_tablet_manager.cc:1403] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.094s	user 0.018s	sys 0.003s
I20260709 12:28:47.703847 21351 leader_election.cc:304] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:47.710237 21571 raft_consensus.cc:2749] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.711453 21511 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.714046 21511 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:47.710830 21507 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:47.715103 21507 ts_tablet_manager.cc:1403] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.152s	user 0.039s	sys 0.055s
I20260709 12:28:47.715497 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:47.715273 21504 raft_consensus.cc:359] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.716014 21504 raft_consensus.cc:385] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.716061 21311 raft_consensus.cc:3060] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.716395 21504 raft_consensus.cc:740] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.717108 21504 consensus_queue.cc:260] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:47.717859 21507 raft_consensus.cc:359] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.718559 21507 raft_consensus.cc:385] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.718845 21507 raft_consensus.cc:740] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.719285 21504 ts_tablet_manager.cc:1434] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:28:47.719636 21507 consensus_queue.cc:260] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.720252 21504 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:47.718860 21570 consensus_queue.cc:1048] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:47.724835 21507 ts_tablet_manager.cc:1434] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 12:28:47.724838 21571 consensus_queue.cc:1048] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:47.726022 21507 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:47.726953 21504 tablet_bootstrap.cc:654] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.719928 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:47.731818 21234 raft_consensus.cc:3060] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.733428 21505 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:47.733948 21505 ts_tablet_manager.cc:1403] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.070s	user 0.007s	sys 0.003s
I20260709 12:28:47.713511 21131 catalog_manager.cc:5697] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:47.735886 21507 tablet_bootstrap.cc:654] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.747700 21505 raft_consensus.cc:359] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.748479 21505 raft_consensus.cc:385] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.749388 21552 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.749925 21552 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.752080 21552 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:47.752485 21236 raft_consensus.cc:3060] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.748740 21505 raft_consensus.cc:740] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.753695 21505 consensus_queue.cc:260] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.754633 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:47.755847 21505 ts_tablet_manager.cc:1434] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:28:47.756749 21234 raft_consensus.cc:2468] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:47.758214 21426 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:47.759210 21511 raft_consensus.cc:2804] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:47.759711 21511 raft_consensus.cc:697] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:47.760957 21511 consensus_queue.cc:237] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.763697 21506 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:47.764457 21506 ts_tablet_manager.cc:1403] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.128s	user 0.018s	sys 0.002s
I20260709 12:28:47.767002 21505 tablet_bootstrap.cc:492] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:47.767015 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:47.769557 21200 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:47.796928 21506 raft_consensus.cc:359] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.798064 21506 raft_consensus.cc:385] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.801131 21311 raft_consensus.cc:2468] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:47.798415 21506 raft_consensus.cc:740] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.808624 21506 consensus_queue.cc:260] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.815127 21506 ts_tablet_manager.cc:1434] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.007s
I20260709 12:28:47.816659 21507 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:47.817348 21507 ts_tablet_manager.cc:1403] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.092s	user 0.018s	sys 0.015s
I20260709 12:28:47.818217 21506 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:47.820947 21570 raft_consensus.cc:493] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.824502 21570 raft_consensus.cc:515] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.826368 21570 leader_election.cc:290] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:47.832885 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.840955 21234 raft_consensus.cc:2468] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:47.844077 21278 leader_election.cc:304] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:47.848017 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:47.852402 21384 raft_consensus.cc:2468] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:47.848213 21568 raft_consensus.cc:2804] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.853668 21568 raft_consensus.cc:493] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.854012 21568 raft_consensus.cc:3060] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.849337 21615 raft_consensus.cc:493] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.855369 21504 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:47.855861 21504 ts_tablet_manager.cc:1403] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.136s	user 0.022s	sys 0.046s
I20260709 12:28:47.856995 21513 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.857517 21513 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.859746 21513 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:47.860738 21552 raft_consensus.cc:2749] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.862151 21570 consensus_queue.cc:1048] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:47.866279 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:47.867408 21507 raft_consensus.cc:359] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.868224 21507 raft_consensus.cc:385] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.868498 21507 raft_consensus.cc:740] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.869221 21507 consensus_queue.cc:260] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.872453 21505 tablet_bootstrap.cc:654] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.877255 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.875986 21385 raft_consensus.cc:3060] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.878907 21352 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:47.883625 21568 raft_consensus.cc:515] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:47.880043 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:47.883684 21132 catalog_manager.cc:5697] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:47.887727 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:47.888550 21384 raft_consensus.cc:3060] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.889810 21568 leader_election.cc:290] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:47.891368 21504 raft_consensus.cc:359] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.898916 21504 raft_consensus.cc:385] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.892079 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:47.854673 21615 raft_consensus.cc:515] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.899261 21504 raft_consensus.cc:740] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.909585 21507 ts_tablet_manager.cc:1434] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.092s	user 0.006s	sys 0.017s
I20260709 12:28:47.910746 21507 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:47.911298 21506 tablet_bootstrap.cc:654] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.905400 21568 raft_consensus.cc:493] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.908042 21504 consensus_queue.cc:260] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.917495 21236 raft_consensus.cc:3060] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.931311 21459 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.931944 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:47.932849 21311 raft_consensus.cc:2393] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:28:47.934461 21507 tablet_bootstrap.cc:654] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:47.934823 21615 leader_election.cc:290] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:47.920015 21568 raft_consensus.cc:515] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.937140 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:47.938722 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:47.939353 21386 raft_consensus.cc:2468] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:47.940713 21275 leader_election.cc:304] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:47.941154 21200 leader_election.cc:304] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:47.941601 21570 raft_consensus.cc:2804] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.941980 21570 raft_consensus.cc:493] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.942312 21570 raft_consensus.cc:3060] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.942576 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.943248 21234 raft_consensus.cc:2468] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:47.944957 21506 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:47.945501 21506 ts_tablet_manager.cc:1403] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.130s	user 0.010s	sys 0.003s
I20260709 12:28:47.947432 21534 raft_consensus.cc:493] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.947970 21534 raft_consensus.cc:515] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:47.952044 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:47.952800 21386 raft_consensus.cc:2468] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:47.950302 21534 leader_election.cc:290] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:47.953986 21423 leader_election.cc:304] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:47.954905 21534 raft_consensus.cc:2804] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:47.914490 21384 raft_consensus.cc:2468] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:47.955291 21534 raft_consensus.cc:493] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:47.958841 21534 raft_consensus.cc:3060] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:47.956878 21506 raft_consensus.cc:359] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.959762 21506 raft_consensus.cc:385] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:47.960150 21506 raft_consensus.cc:740] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:47.961064 21506 consensus_queue.cc:260] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:47.956048 21511 raft_consensus.cc:493] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.955390 21568 leader_election.cc:290] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:47.929112 21236 raft_consensus.cc:2468] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:47.982383 21505 tablet_bootstrap.cc:492] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:47.984210 21505 ts_tablet_manager.cc:1403] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.217s	user 0.042s	sys 0.023s
I20260709 12:28:47.966249 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:47.964347 21570 raft_consensus.cc:515] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:47.989778 21504 ts_tablet_manager.cc:1434] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.134s	user 0.007s	sys 0.002s
I20260709 12:28:47.982807 21615 raft_consensus.cc:2749] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.993355 21504 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:47.983832 21275 leader_election.cc:304] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:47.994593 21572 raft_consensus.cc:2749] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:47.998912 21619 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:47.999415 21619 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.000936 21568 consensus_queue.cc:1048] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.001551 21619 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:48.003286 21568 raft_consensus.cc:2804] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.003021 21623 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.004410 21623 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.006960 21507 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:48.006762 21623 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:47.975996 21511 raft_consensus.cc:515] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.007738 21507 ts_tablet_manager.cc:1403] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.097s	user 0.029s	sys 0.023s
I20260709 12:28:48.008235 21506 ts_tablet_manager.cc:1434] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.062s	user 0.009s	sys 0.003s
W20260709 12:28:47.998369 21426 leader_election.cc:343] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.012970 21506 tablet_bootstrap.cc:492] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:48.018549 21625 consensus_queue.cc:1048] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.018962 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:48.019629 21311 raft_consensus.cc:2468] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:48.020535 21506 tablet_bootstrap.cc:654] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.024241 21202 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:48.024619 21507 raft_consensus.cc:359] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.025312 21507 raft_consensus.cc:385] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.025430 21552 raft_consensus.cc:2804] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.025619 21507 raft_consensus.cc:740] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.025867 21552 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.026239 21552 raft_consensus.cc:3060] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.026484 21507 consensus_queue.cc:260] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.027120 21572 consensus_queue.cc:1048] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.029031 21507 ts_tablet_manager.cc:1434] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.017s	user 0.006s	sys 0.000s
I20260709 12:28:48.030049 21507 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:48.041965 21506 tablet_bootstrap.cc:492] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:48.042505 21506 ts_tablet_manager.cc:1403] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.031s	user 0.013s	sys 0.000s
I20260709 12:28:48.042759 21504 tablet_bootstrap.cc:654] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.045217 21570 leader_election.cc:290] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:48.045934 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:48.046535 21234 raft_consensus.cc:3060] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.047387 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:48.047793 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 49463.1us
I20260709 12:28:48.048077 21459 raft_consensus.cc:2468] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:48.047868 21534 raft_consensus.cc:515] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.050321 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:48.051348 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:48.051963 21385 raft_consensus.cc:3060] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.052265 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:48.051393 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:48.055087 21534 leader_election.cc:290] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:48.055419 21234 raft_consensus.cc:2468] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:48.046379 21511 leader_election.cc:290] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:48.068542 21425 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:48.074133 21615 raft_consensus.cc:493] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.074797 21615 raft_consensus.cc:515] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.061498 21570 raft_consensus.cc:493] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.083109 21570 raft_consensus.cc:515] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.090790 21507 tablet_bootstrap.cc:654] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.091605 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:48.091748 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:48.092460 21234 raft_consensus.cc:3060] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.092867 21278 leader_election.cc:304] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:48.093926 21626 raft_consensus.cc:2804] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.094447 21626 raft_consensus.cc:697] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:48.095362 21626 consensus_queue.cc:237] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.095927 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:48.090052 21506 raft_consensus.cc:359] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.096009 21423 leader_election.cc:304] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:48.096697 21506 raft_consensus.cc:385] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.097048 21506 raft_consensus.cc:740] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.097842 21506 consensus_queue.cc:260] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.100220 21506 ts_tablet_manager.cc:1434] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.000s
I20260709 12:28:48.100343 21552 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.101254 21506 tablet_bootstrap.cc:492] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:48.101336 21384 raft_consensus.cc:3060] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.107329 21534 raft_consensus.cc:2749] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.108198 21570 leader_election.cc:290] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:48.108760 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:48.115468 21458 raft_consensus.cc:3060] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.109283 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:48.116145 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:48.118438 21457 raft_consensus.cc:3060] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.110407 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:48.111223 21506 tablet_bootstrap.cc:654] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.111994 21571 consensus_queue.cc:1048] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.123509 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:48.114229 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:48.115494 21615 leader_election.cc:290] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:28:48.122555 21277 leader_election.cc:343] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.067396 21568 raft_consensus.cc:697] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:48.129189 21460 raft_consensus.cc:2393] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:48.131271 21504 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:48.131786 21504 ts_tablet_manager.cc:1403] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.141s	user 0.010s	sys 0.008s
I20260709 12:28:48.121647 21552 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:48.127218 21310 raft_consensus.cc:3060] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.124759 21385 raft_consensus.cc:2468] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:48.153092 21504 raft_consensus.cc:359] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.153823 21504 raft_consensus.cc:385] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.154079 21504 raft_consensus.cc:740] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.155242 21504 consensus_queue.cc:260] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.157590 21504 ts_tablet_manager.cc:1434] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:28:48.158742 21504 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:48.162873 21568 consensus_queue.cc:237] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:48.151350 21278 leader_election.cc:343] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.168188 21278 leader_election.cc:304] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:48.166953 21202 leader_election.cc:304] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:48.169052 21625 raft_consensus.cc:2749] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.184131 21505 raft_consensus.cc:359] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.185505 21505 raft_consensus.cc:385] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.185998 21505 raft_consensus.cc:740] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.187911 21505 consensus_queue.cc:260] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.188589 21623 raft_consensus.cc:2749] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:48.195451 21234 raft_consensus.cc:2468] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:48.197086 21426 leader_election.cc:304] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:48.197100 21552 raft_consensus.cc:2749] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.201493 21632 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.202041 21632 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.204489 21632 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:48.205063 21632 raft_consensus.cc:2804] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.203784 21135 catalog_manager.cc:5697] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.205888 21552 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.209430 21552 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.211191 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:48.211951 21311 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:48.212399 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:48.212651 21458 raft_consensus.cc:2468] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
W20260709 12:28:48.214349 21200 leader_election.cc:343] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.214555 21310 raft_consensus.cc:2468] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:48.214841 21425 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:48.215871 21632 raft_consensus.cc:697] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:48.216269 21202 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:48.216842 21632 consensus_queue.cc:237] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.218811 21505 ts_tablet_manager.cc:1434] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.232s	user 0.011s	sys 0.005s
I20260709 12:28:48.219584 21505 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:48.220201 21534 raft_consensus.cc:2804] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.221462 21534 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.221845 21534 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.206708 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:48.226795 21385 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:48.193361 21384 raft_consensus.cc:2468] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:48.234436 21552 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:48.235849 21572 consensus_queue.cc:1048] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.237207 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:48.237883 21457 raft_consensus.cc:2468] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:48.238526 21504 tablet_bootstrap.cc:654] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.238897 21202 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:48.240682 21507 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:48.241377 21507 ts_tablet_manager.cc:1403] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.212s	user 0.023s	sys 0.010s
I20260709 12:28:48.243854 21534 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.243889 21552 raft_consensus.cc:2804] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.246830 21552 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.247196 21552 raft_consensus.cc:3060] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.232462 21615 raft_consensus.cc:2804] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.248939 21615 raft_consensus.cc:697] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:48.259375 21534 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:48.260579 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:48.261382 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:48.267753 21505 tablet_bootstrap.cc:654] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.270885 21310 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.271292 21571 consensus_queue.cc:1048] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.274803 21384 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.249819 21615 consensus_queue.cc:237] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.292155 21507 raft_consensus.cc:359] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.292364 21511 consensus_queue.cc:1048] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.299814 21507 raft_consensus.cc:385] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.300202 21507 raft_consensus.cc:740] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.301115 21507 consensus_queue.cc:260] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.298308 21384 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:48.315513 21423 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:48.316538 21632 raft_consensus.cc:2804] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.317301 21632 raft_consensus.cc:697] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:48.319453 21632 consensus_queue.cc:237] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.321044 21506 tablet_bootstrap.cc:492] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:48.322065 21506 ts_tablet_manager.cc:1403] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.221s	user 0.028s	sys 0.021s
I20260709 12:28:48.322175 21310 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:48.320775 21552 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.329571 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:48.329668 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:48.330480 21459 raft_consensus.cc:3060] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.330902 21506 raft_consensus.cc:359] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.331615 21506 raft_consensus.cc:385] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.331873 21506 raft_consensus.cc:740] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.332579 21506 consensus_queue.cc:260] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.335417 21506 ts_tablet_manager.cc:1434] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:28:48.336411 21552 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:48.336673 21506 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:48.337515 21504 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:48.338193 21504 ts_tablet_manager.cc:1403] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.180s	user 0.011s	sys 0.003s
I20260709 12:28:48.352612 21506 tablet_bootstrap.cc:654] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.357050 21459 raft_consensus.cc:2468] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:48.361039 21507 ts_tablet_manager.cc:1434] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.119s	user 0.006s	sys 0.005s
I20260709 12:28:48.362545 21507 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:48.361613 21202 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:48.366221 21135 catalog_manager.cc:5697] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:48.368116 21200 leader_election.cc:343] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.368458 21645 raft_consensus.cc:2804] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.369084 21645 raft_consensus.cc:697] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:48.370086 21645 consensus_queue.cc:237] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.386509 21504 raft_consensus.cc:359] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.387751 21504 raft_consensus.cc:385] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.387272 21131 catalog_manager.cc:5697] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.399166 21504 raft_consensus.cc:740] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.400071 21506 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:48.400609 21506 ts_tablet_manager.cc:1403] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.064s	user 0.030s	sys 0.017s
I20260709 12:28:48.400278 21504 consensus_queue.cc:260] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.404484 21506 raft_consensus.cc:359] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.405164 21506 raft_consensus.cc:385] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.405453 21506 raft_consensus.cc:740] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.406121 21506 consensus_queue.cc:260] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.408375 21506 ts_tablet_manager.cc:1434] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:48.409325 21506 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:48.408380 21504 ts_tablet_manager.cc:1434] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.000s
I20260709 12:28:48.411396 21504 tablet_bootstrap.cc:492] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:48.390218 21507 tablet_bootstrap.cc:654] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.390185 21505 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:48.413164 21505 ts_tablet_manager.cc:1403] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.194s	user 0.015s	sys 0.009s
I20260709 12:28:48.413314 21623 consensus_queue.cc:1048] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.415441 21506 tablet_bootstrap.cc:654] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.420467 21504 tablet_bootstrap.cc:654] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.421967 21132 catalog_manager.cc:5697] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:48.427297 21506 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:48.427482 21505 raft_consensus.cc:359] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.427999 21506 ts_tablet_manager.cc:1403] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.005s
I20260709 12:28:48.428335 21505 raft_consensus.cc:385] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.428763 21505 raft_consensus.cc:740] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.430012 21505 consensus_queue.cc:260] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.432134 21506 raft_consensus.cc:359] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.433140 21506 raft_consensus.cc:385] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.433501 21506 raft_consensus.cc:740] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.425443 21135 catalog_manager.cc:5697] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.434790 21506 consensus_queue.cc:260] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.438773 21504 tablet_bootstrap.cc:492] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:48.439477 21506 ts_tablet_manager.cc:1434] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 12:28:48.439539 21504 ts_tablet_manager.cc:1403] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.028s	user 0.024s	sys 0.000s
I20260709 12:28:48.440682 21506 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:48.442716 21504 raft_consensus.cc:359] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.443461 21504 raft_consensus.cc:385] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.443823 21504 raft_consensus.cc:740] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.444752 21504 consensus_queue.cc:260] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.447508 21504 ts_tablet_manager.cc:1434] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:48.447531 21506 tablet_bootstrap.cc:654] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.448717 21504 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:48.458118 21504 tablet_bootstrap.cc:654] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.459556 21505 ts_tablet_manager.cc:1434] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.046s	user 0.007s	sys 0.000s
I20260709 12:28:48.459574 21506 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:48.460400 21506 ts_tablet_manager.cc:1403] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.008s
I20260709 12:28:48.464133 21506 raft_consensus.cc:359] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.464852 21506 raft_consensus.cc:385] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.465113 21506 raft_consensus.cc:740] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.465847 21506 consensus_queue.cc:260] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.468380 21506 ts_tablet_manager.cc:1434] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:28:48.469390 21506 tablet_bootstrap.cc:492] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:48.472232 21505 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:48.482954 21506 tablet_bootstrap.cc:654] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.488461 21645 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.489029 21645 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.492127 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:48.491722 21131 catalog_manager.cc:5697] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:48.492822 21312 raft_consensus.cc:2468] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:48.495090 21202 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:48.496074 21552 raft_consensus.cc:2804] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.496420 21552 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.496706 21552 raft_consensus.cc:3060] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.497184 21504 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:48.497712 21504 ts_tablet_manager.cc:1403] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.008s
I20260709 12:28:48.500193 21507 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:48.500716 21504 raft_consensus.cc:359] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.502730 21504 raft_consensus.cc:385] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.502795 21507 ts_tablet_manager.cc:1403] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.140s	user 0.009s	sys 0.006s
I20260709 12:28:48.503103 21504 raft_consensus.cc:740] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.503885 21504 consensus_queue.cc:260] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.504809 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:48.505892 21507 raft_consensus.cc:359] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.506313 21504 ts_tablet_manager.cc:1434] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:28:48.506636 21507 raft_consensus.cc:385] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.506858 21645 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:48.507056 21507 raft_consensus.cc:740] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
W20260709 12:28:48.506413 21200 leader_election.cc:343] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.507500 21504 tablet_bootstrap.cc:492] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:48.508015 21507 consensus_queue.cc:260] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.510331 21507 ts_tablet_manager.cc:1434] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:28:48.512194 21552 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.513422 21504 tablet_bootstrap.cc:654] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.515251 21626 raft_consensus.cc:493] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.515805 21626 raft_consensus.cc:515] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.516911 21552 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:48.515002 21507 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:48.518111 21626 leader_election.cc:290] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:48.520253 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:48.520493 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:48.521059 21458 raft_consensus.cc:2468] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:48.520287 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:48.522632 21277 leader_election.cc:304] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:48.523159 21505 tablet_bootstrap.cc:654] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.523605 21626 raft_consensus.cc:2804] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:28:48.523741 21275 leader_election.cc:343] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.520255 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:48.525961 21312 raft_consensus.cc:3060] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:48.532450 21200 leader_election.cc:343] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.533860 21571 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.524056 21626 raft_consensus.cc:493] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.535285 21507 tablet_bootstrap.cc:654] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.534519 21571 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.536013 21626 raft_consensus.cc:3060] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.539614 21632 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.541487 21632 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:48.544097 21504 tablet_bootstrap.cc:492] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:48.544832 21632 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:48.546047 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:48.546185 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:48.548734 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:48.550218 21506 tablet_bootstrap.cc:492] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:48.550791 21506 ts_tablet_manager.cc:1403] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.007s
I20260709 12:28:48.546661 21571 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:48.544804 21504 ts_tablet_manager.cc:1403] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.038s	user 0.014s	sys 0.013s
I20260709 12:28:48.554087 21506 raft_consensus.cc:359] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.554924 21506 raft_consensus.cc:385] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.555253 21506 raft_consensus.cc:740] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.556139 21506 consensus_queue.cc:260] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.557822 21506 ts_tablet_manager.cc:1434] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:48.558838 21506 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:48.558753 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:48.560667 21458 raft_consensus.cc:2393] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:48.562057 21351 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:48.563380 21571 raft_consensus.cc:3060] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:48.563763 21423 leader_election.cc:343] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:48.564623 21426 leader_election.cc:343] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.565091 21426 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:48.566268 21632 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.570873 21571 raft_consensus.cc:2749] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.582566 21506 tablet_bootstrap.cc:654] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.583400 21312 raft_consensus.cc:2468] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:48.585580 21202 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:48.586043 21507 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:48.586498 21507 ts_tablet_manager.cc:1403] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.072s	user 0.025s	sys 0.020s
I20260709 12:28:48.586798 21552 raft_consensus.cc:2804] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.587329 21552 raft_consensus.cc:697] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:48.588136 21552 consensus_queue.cc:237] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.590570 21507 raft_consensus.cc:359] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.591292 21507 raft_consensus.cc:385] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.591207 21504 raft_consensus.cc:359] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.591730 21507 raft_consensus.cc:740] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.591995 21504 raft_consensus.cc:385] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.592314 21504 raft_consensus.cc:740] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.592429 21507 consensus_queue.cc:260] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.593127 21504 consensus_queue.cc:260] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.595866 21504 ts_tablet_manager.cc:1434] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:28:48.596843 21504 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:48.597532 21626 raft_consensus.cc:515] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.599300 21505 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:48.600142 21505 ts_tablet_manager.cc:1403] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.128s	user 0.022s	sys 0.001s
I20260709 12:28:48.600847 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:48.606652 21275 leader_election.cc:343] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.607963 21505 raft_consensus.cc:359] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.608795 21505 raft_consensus.cc:385] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.611042 21505 raft_consensus.cc:740] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.610894 21625 consensus_queue.cc:1048] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.612011 21505 consensus_queue.cc:260] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.612604 21507 ts_tablet_manager.cc:1434] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:28:48.618790 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:48.619266 21615 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.619493 21458 raft_consensus.cc:3060] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.619680 21505 ts_tablet_manager.cc:1434] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.001s
I20260709 12:28:48.619823 21615 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.620769 21505 tablet_bootstrap.cc:492] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:48.621445 21507 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:48.624356 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:48.625088 21384 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:48.626397 21504 tablet_bootstrap.cc:654] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.622334 21615 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:48.626946 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:48.626966 21200 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:48.631392 21312 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:48.632325 21615 raft_consensus.cc:2804] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.632753 21615 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.633080 21615 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.636299 21568 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.636844 21568 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.639128 21568 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:48.641983 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:48.642640 21568 raft_consensus.cc:493] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.642831 21632 raft_consensus.cc:493] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:48.642401 21625 consensus_queue.cc:1048] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:28:48.634473 21458 raft_consensus.cc:2468] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
W20260709 12:28:48.643961 21277 leader_election.cc:343] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.642035 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:48.643703 21626 leader_election.cc:290] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:48.643208 21568 raft_consensus.cc:515] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.643429 21632 raft_consensus.cc:515] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.660497 21277 leader_election.cc:304] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:48.661569 21506 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:48.661897 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:48.662266 21506 ts_tablet_manager.cc:1403] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.104s	user 0.014s	sys 0.002s
I20260709 12:28:48.659238 21131 catalog_manager.cc:5697] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.665457 21312 raft_consensus.cc:2468] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:48.659186 21632 leader_election.cc:290] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:48.666735 21625 raft_consensus.cc:2804] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.667325 21625 raft_consensus.cc:697] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:48.668179 21625 consensus_queue.cc:237] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.668999 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:48.669723 21236 raft_consensus.cc:2468] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:48.670087 21568 leader_election.cc:290] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:48.670827 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:48.673261 21275 leader_election.cc:343] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:48.670882 21278 leader_election.cc:343] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.671831 21425 leader_election.cc:304] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:48.674032 21506 raft_consensus.cc:359] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.674809 21506 raft_consensus.cc:385] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.673964 21278 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:48.674556 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:48.675321 21506 raft_consensus.cc:740] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.676124 21626 raft_consensus.cc:2749] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:48.674722 21632 raft_consensus.cc:2804] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.677681 21632 raft_consensus.cc:493] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.678162 21632 raft_consensus.cc:3060] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.680748 21623 consensus_queue.cc:1048] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.682753 21458 raft_consensus.cc:2468] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:48.693230 21277 leader_election.cc:304] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:48.694487 21568 raft_consensus.cc:2804] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:48.694922 21568 raft_consensus.cc:493] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:48.695273 21568 raft_consensus.cc:3060] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.696655 21615 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.700835 21506 consensus_queue.cc:260] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.703524 21504 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:48.704051 21504 ts_tablet_manager.cc:1403] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.107s	user 0.018s	sys 0.019s
I20260709 12:28:48.705911 21505 tablet_bootstrap.cc:654] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.707293 21507 tablet_bootstrap.cc:654] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.711593 21506 ts_tablet_manager.cc:1434] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.000s
I20260709 12:28:48.711400 21570 consensus_queue.cc:1048] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.712779 21506 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:48.728295 21570 consensus_queue.cc:1048] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.728933 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:48.729614 21386 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.725108 21615 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:48.728212 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:48.748378 21312 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.755787 21504 raft_consensus.cc:359] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.762646 21504 raft_consensus.cc:385] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.763854 21504 raft_consensus.cc:740] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.751950 21568 raft_consensus.cc:515] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.793208 21507 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:48.793780 21507 ts_tablet_manager.cc:1403] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.173s	user 0.021s	sys 0.022s
I20260709 12:28:48.796463 21386 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:48.799052 21200 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:48.801628 21552 raft_consensus.cc:2804] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.802225 21552 raft_consensus.cc:697] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:48.803200 21552 consensus_queue.cc:237] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.805953 21623 consensus_queue.cc:1048] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:48.808694 21632 raft_consensus.cc:515] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.810964 21632 leader_election.cc:290] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:48.813793 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:48.814514 21310 raft_consensus.cc:3060] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.816584 21506 tablet_bootstrap.cc:654] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:48.818500 21131 catalog_manager.cc:5697] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:48.825837 21615 consensus_queue.cc:1048] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.834388 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:48.840677 21507 raft_consensus.cc:359] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.842625 21507 raft_consensus.cc:385] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.843051 21507 raft_consensus.cc:740] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.843992 21507 consensus_queue.cc:260] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:48.852664 21505 tablet_bootstrap.cc:492] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:48.853271 21505 ts_tablet_manager.cc:1403] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.233s	user 0.009s	sys 0.024s
I20260709 12:28:48.854036 21568 leader_election.cc:290] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:48.840298 21312 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:48.859879 21504 consensus_queue.cc:260] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:28:48.847466 21275 leader_election.cc:343] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:48.862053 21507 ts_tablet_manager.cc:1434] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.068s	user 0.010s	sys 0.001s
I20260709 12:28:48.871392 21507 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:48.872681 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:48.876166 21505 raft_consensus.cc:359] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.876995 21505 raft_consensus.cc:385] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.877312 21505 raft_consensus.cc:740] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.878013 21505 consensus_queue.cc:260] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:48.879874 21504 ts_tablet_manager.cc:1434] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.176s	user 0.011s	sys 0.002s
W20260709 12:28:48.856945 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 80586.7us
I20260709 12:28:48.882021 21504 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:48.873378 21235 raft_consensus.cc:3060] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.892010 21506 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:48.892613 21506 ts_tablet_manager.cc:1403] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.180s	user 0.003s	sys 0.013s
I20260709 12:28:48.895656 21505 ts_tablet_manager.cc:1434] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.042s	user 0.007s	sys 0.002s
I20260709 12:28:48.899243 21505 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:48.909508 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:48.910290 21459 raft_consensus.cc:3060] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:48.893844 21511 consensus_queue.cc:1048] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:28:48.912583 21619 consensus_queue.cc:1048] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.030s
I20260709 12:28:48.921815 21310 raft_consensus.cc:2468] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:48.942826 21615 consensus_queue.cc:1048] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:48.946288 21235 raft_consensus.cc:2468] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:48.968410 21425 leader_election.cc:304] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:48.974903 21511 raft_consensus.cc:2804] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:48.981917 21459 raft_consensus.cc:2468] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:48.962862 21632 consensus_queue.cc:1048] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
I20260709 12:28:48.994130 21506 raft_consensus.cc:359] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.995733 21506 raft_consensus.cc:385] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:48.996232 21506 raft_consensus.cc:740] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:48.997778 21506 consensus_queue.cc:260] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:48.994537 21277 leader_election.cc:304] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:49.000746 21626 raft_consensus.cc:2804] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.001253 21626 raft_consensus.cc:697] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:49.002075 21626 consensus_queue.cc:237] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:49.019178 21506 ts_tablet_manager.cc:1434] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.126s	user 0.006s	sys 0.003s
I20260709 12:28:49.020666 21506 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:49.025269 21507 tablet_bootstrap.cc:654] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.032502 21511 raft_consensus.cc:697] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:49.037525 21511 consensus_queue.cc:237] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.039098 21552 consensus_queue.cc:1048] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.045174 21506 tablet_bootstrap.cc:654] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:49.050784 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:49.070006 21505 tablet_bootstrap.cc:654] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.075228 21504 tablet_bootstrap.cc:654] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.075985 21645 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.076536 21645 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.150099 21507 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:49.150537 21507 ts_tablet_manager.cc:1403] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.279s	user 0.013s	sys 0.019s
I20260709 12:28:49.157869 21645 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:49.160868 21507 raft_consensus.cc:359] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.161608 21507 raft_consensus.cc:385] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.161899 21507 raft_consensus.cc:740] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.163486 21135 catalog_manager.cc:5697] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.171391 21626 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.171485 21507 consensus_queue.cc:260] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.171967 21694 consensus_queue.cc:1048] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.172114 21626 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.173864 21507 ts_tablet_manager.cc:1434] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
W20260709 12:28:49.174399 21200 consensus_peers.cc:597] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:49.174873 21507 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:49.175042 21626 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:49.187119 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:49.187177 21504 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:49.189008 21504 ts_tablet_manager.cc:1403] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.308s	user 0.016s	sys 0.006s
I20260709 12:28:49.187139 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:49.188464 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:49.189821 21310 raft_consensus.cc:2468] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:49.191618 21507 tablet_bootstrap.cc:654] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.192082 21504 raft_consensus.cc:359] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.192793 21504 raft_consensus.cc:385] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.193073 21504 raft_consensus.cc:740] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.193850 21504 consensus_queue.cc:260] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:28:49.196973 21275 leader_election.cc:343] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.196936 21504 ts_tablet_manager.cc:1434] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:49.198544 21504 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
W20260709 12:28:49.199502 21200 leader_election.cc:343] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.204221 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:49.205026 21234 raft_consensus.cc:2468] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:49.208184 21504 tablet_bootstrap.cc:654] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.215029 21278 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:49.214258 21131 catalog_manager.cc:5697] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.218021 21671 raft_consensus.cc:2804] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.218758 21671 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.219260 21671 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.218235 21202 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:49.223493 21645 raft_consensus.cc:2804] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.223928 21645 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.224295 21645 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.230407 21134 catalog_manager.cc:5697] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.245894 21671 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.249358 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:49.250454 21673 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.250643 21507 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:49.251313 21507 ts_tablet_manager.cc:1403] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.077s	user 0.014s	sys 0.001s
I20260709 12:28:49.251050 21673 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.253302 21673 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:49.254483 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:49.254948 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:49.256455 21277 leader_election.cc:343] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:49.257061 21278 leader_election.cc:343] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.257131 21673 consensus_queue.cc:1048] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.257602 21278 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:49.258436 21507 raft_consensus.cc:359] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.259397 21507 raft_consensus.cc:385] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.259802 21507 raft_consensus.cc:740] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.260774 21507 consensus_queue.cc:260] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.233475 21645 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.238643 21506 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:49.263187 21506 ts_tablet_manager.cc:1403] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.243s	user 0.017s	sys 0.003s
I20260709 12:28:49.262928 21626 raft_consensus.cc:2749] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.266697 21383 raft_consensus.cc:3060] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.267858 21506 raft_consensus.cc:359] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.270465 21506 raft_consensus.cc:385] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.270912 21506 raft_consensus.cc:740] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.271837 21507 ts_tablet_manager.cc:1434] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:28:49.271862 21506 consensus_queue.cc:260] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.274055 21507 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:49.284245 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:49.285331 21312 raft_consensus.cc:2393] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:28:49.284248 21671 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:49.292590 21534 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.293133 21507 tablet_bootstrap.cc:654] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.293453 21534 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.274412 21234 raft_consensus.cc:2393] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:49.276650 21505 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:49.295431 21505 ts_tablet_manager.cc:1403] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.396s	user 0.015s	sys 0.028s
I20260709 12:28:49.304840 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:49.313385 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:49.314191 21275 leader_election.cc:343] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.314989 21275 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:49.314205 21552 consensus_queue.cc:1048] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:49.316636 21570 raft_consensus.cc:2749] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:28:49.321033 21200 leader_election.cc:343] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.321296 21534 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:49.321595 21200 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:49.323200 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:49.324162 21235 raft_consensus.cc:2468] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:49.327862 21505 raft_consensus.cc:359] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.328601 21506 ts_tablet_manager.cc:1434] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.065s	user 0.009s	sys 0.000s
I20260709 12:28:49.328758 21505 raft_consensus.cc:385] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.329077 21505 raft_consensus.cc:740] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.329816 21506 tablet_bootstrap.cc:492] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:49.331670 21505 consensus_queue.cc:260] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.339226 21505 ts_tablet_manager.cc:1434] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.002s
I20260709 12:28:49.339572 21504 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:49.340057 21504 ts_tablet_manager.cc:1403] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.142s	user 0.023s	sys 0.031s
I20260709 12:28:49.319617 21645 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:49.331219 21552 raft_consensus.cc:2749] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.334803 21626 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.345489 21626 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.347859 21626 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:49.349237 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:49.349980 21311 raft_consensus.cc:2468] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:49.351119 21426 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:49.351984 21511 raft_consensus.cc:2804] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.352416 21511 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.352769 21511 raft_consensus.cc:3060] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.354439 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:49.355540 21504 raft_consensus.cc:359] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.356240 21504 raft_consensus.cc:385] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.356510 21504 raft_consensus.cc:740] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.357200 21504 consensus_queue.cc:260] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.361733 21511 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:28:49.365099 21278 leader_election.cc:343] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.366744 21505 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:49.366549 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:49.366856 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:49.367358 21511 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:49.367532 21311 raft_consensus.cc:3060] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.371587 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:49.372450 21236 raft_consensus.cc:3060] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:49.368517 21275 leader_election.cc:343] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.379025 21275 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:49.379912 21619 consensus_queue.cc:1048] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.381036 21626 raft_consensus.cc:2749] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.394217 21504 ts_tablet_manager.cc:1434] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 12:28:49.395411 21504 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:49.402380 21571 raft_consensus.cc:493] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.402745 21504 tablet_bootstrap.cc:654] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.402961 21571 raft_consensus.cc:515] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:49.422065 21673 consensus_queue.cc:1048] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:49.436695 21311 raft_consensus.cc:2468] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:49.438071 21425 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:49.439036 21632 raft_consensus.cc:2804] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.439545 21632 raft_consensus.cc:697] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:49.441656 21571 leader_election.cc:290] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:49.441346 21632 consensus_queue.cc:237] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.420627 21236 raft_consensus.cc:2468] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:49.443945 21507 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:49.445289 21507 ts_tablet_manager.cc:1403] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.171s	user 0.010s	sys 0.018s
I20260709 12:28:49.446312 21505 tablet_bootstrap.cc:654] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.453655 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:49.449167 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:49.488574 21351 leader_election.cc:304] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:49.487018 21504 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:49.490871 21504 ts_tablet_manager.cc:1403] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.096s	user 0.028s	sys 0.001s
I20260709 12:28:49.493427 21671 consensus_queue.cc:1048] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.496780 21506 tablet_bootstrap.cc:654] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.497087 21507 raft_consensus.cc:359] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.498284 21507 raft_consensus.cc:385] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.498566 21507 raft_consensus.cc:740] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.508253 21507 consensus_queue.cc:260] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.511422 21511 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.511992 21511 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.514205 21511 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:49.515934 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:49.518515 21423 leader_election.cc:343] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.520833 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:49.524734 21507 ts_tablet_manager.cc:1434] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.079s	user 0.000s	sys 0.007s
I20260709 12:28:49.537261 21507 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Bootstrap starting.
W20260709 12:28:49.520202 21275 consensus_peers.cc:597] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:49.549775 21426 leader_election.cc:343] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.550704 21426 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:49.551937 21511 raft_consensus.cc:2749] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:49.562798 21645 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.567052 21645 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.568228 21505 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:49.568774 21505 ts_tablet_manager.cc:1403] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.202s	user 0.016s	sys 0.015s
I20260709 12:28:49.572001 21505 raft_consensus.cc:359] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.574509 21571 raft_consensus.cc:2749] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:49.576874 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:49.579314 21507 tablet_bootstrap.cc:654] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.579253 21632 consensus_queue.cc:1048] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.572861 21505 raft_consensus.cc:385] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.575109 21504 raft_consensus.cc:359] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.582332 21505 raft_consensus.cc:740] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.576961 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:49.585403 21505 consensus_queue.cc:260] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.582772 21504 raft_consensus.cc:385] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.569945 21645 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:49.586406 21504 raft_consensus.cc:740] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.588492 21504 consensus_queue.cc:260] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.585422 21312 raft_consensus.cc:2468] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:49.582201 21459 raft_consensus.cc:2468] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:49.608825 21202 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:49.617522 21504 ts_tablet_manager.cc:1434] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.126s	user 0.005s	sys 0.002s
I20260709 12:28:49.618674 21504 tablet_bootstrap.cc:492] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:49.623219 21511 consensus_queue.cc:1048] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.625773 21505 ts_tablet_manager.cc:1434] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.001s
I20260709 12:28:49.628710 21505 tablet_bootstrap.cc:492] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:49.630481 21506 tablet_bootstrap.cc:492] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:49.631662 21506 ts_tablet_manager.cc:1403] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.302s	user 0.022s	sys 0.002s
I20260709 12:28:49.627942 21645 raft_consensus.cc:2804] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.635017 21645 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.635506 21645 raft_consensus.cc:3060] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.638465 21615 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.639086 21615 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.642086 21615 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:49.652411 21384 raft_consensus.cc:3060] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.653095 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:49.653909 21385 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:49.656239 21200 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:49.657227 21552 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.657624 21552 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.657989 21552 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.665019 21506 raft_consensus.cc:359] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:49.665906 21506 raft_consensus.cc:385] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.666198 21506 raft_consensus.cc:740] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.666409 21505 tablet_bootstrap.cc:654] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.652210 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:49.667610 21459 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:49.668432 21504 tablet_bootstrap.cc:654] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.670933 21645 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.673400 21645 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:49.674965 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:49.675792 21312 raft_consensus.cc:3060] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.677953 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:49.678637 21457 raft_consensus.cc:3060] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.680788 21506 consensus_queue.cc:260] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:49.684350 21506 ts_tablet_manager.cc:1434] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 12:28:49.688828 21552 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.694453 21552 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:49.694762 21312 raft_consensus.cc:2468] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:49.695364 21507 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:49.695900 21507 ts_tablet_manager.cc:1403] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.170s	user 0.010s	sys 0.009s
I20260709 12:28:49.696897 21506 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:49.701591 21202 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:49.703647 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:49.704331 21459 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.704073 21507 raft_consensus.cc:359] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.704824 21507 raft_consensus.cc:385] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.705101 21507 raft_consensus.cc:740] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.705850 21552 raft_consensus.cc:2804] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.706555 21552 raft_consensus.cc:697] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:49.707477 21552 consensus_queue.cc:237] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.710883 21507 consensus_queue.cc:260] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.711457 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:49.712610 21385 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.719987 21507 ts_tablet_manager.cc:1434] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:28:49.712437 21457 raft_consensus.cc:2468] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:49.721334 21507 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:49.723585 21459 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:49.729198 21385 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:49.732980 21507 tablet_bootstrap.cc:654] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.735121 21506 tablet_bootstrap.cc:654] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.784422 21200 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:49.795536 21615 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:49.796700 21615 raft_consensus.cc:697] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:49.823372 21505 tablet_bootstrap.cc:492] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:49.827421 21505 ts_tablet_manager.cc:1403] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.199s	user 0.007s	sys 0.015s
I20260709 12:28:49.819288 21615 consensus_queue.cc:237] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.860066 21505 raft_consensus.cc:359] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:49.861451 21505 raft_consensus.cc:385] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.861835 21505 raft_consensus.cc:740] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.863005 21131 catalog_manager.cc:5697] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:49.873872 21507 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:49.876401 21504 tablet_bootstrap.cc:492] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:49.877337 21504 ts_tablet_manager.cc:1403] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.259s	user 0.015s	sys 0.000s
I20260709 12:28:49.879916 21506 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:49.880801 21506 ts_tablet_manager.cc:1403] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.185s	user 0.013s	sys 0.006s
I20260709 12:28:49.883502 21507 ts_tablet_manager.cc:1403] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.162s	user 0.003s	sys 0.018s
I20260709 12:28:49.894773 21505 consensus_queue.cc:260] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:49.882349 21504 raft_consensus.cc:359] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:49.896140 21504 raft_consensus.cc:385] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.895694 21506 raft_consensus.cc:359] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.896986 21506 raft_consensus.cc:385] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.897387 21506 raft_consensus.cc:740] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.898259 21506 consensus_queue.cc:260] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:49.896571 21504 raft_consensus.cc:740] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.904958 21506 ts_tablet_manager.cc:1434] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.024s	user 0.013s	sys 0.000s
I20260709 12:28:49.904932 21504 consensus_queue.cc:260] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:49.907984 21506 tablet_bootstrap.cc:492] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:49.909097 21504 ts_tablet_manager.cc:1434] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.031s	user 0.011s	sys 0.000s
I20260709 12:28:49.913038 21504 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:49.913638 21507 raft_consensus.cc:359] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.914765 21507 raft_consensus.cc:385] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.915148 21507 raft_consensus.cc:740] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.916101 21505 ts_tablet_manager.cc:1434] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.085s	user 0.008s	sys 0.003s
I20260709 12:28:49.916136 21507 consensus_queue.cc:260] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.918071 21505 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:49.927857 21505 tablet_bootstrap.cc:654] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.936872 21632 consensus_queue.cc:1048] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:49.941190 21507 ts_tablet_manager.cc:1434] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.057s	user 0.001s	sys 0.008s
I20260709 12:28:49.942463 21615 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.942869 21507 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:49.943315 21615 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.949754 21615 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:49.950217 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:49.952148 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:49.952821 21507 tablet_bootstrap.cc:654] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.952952 21312 raft_consensus.cc:2468] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
W20260709 12:28:49.953039 21200 leader_election.cc:343] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.956202 21505 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:49.956844 21505 ts_tablet_manager.cc:1403] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.039s	user 0.019s	sys 0.003s
I20260709 12:28:49.956765 21202 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:49.958802 21615 raft_consensus.cc:2804] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:49.959558 21615 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:49.960021 21615 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:49.972471 21552 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:49.973220 21552 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:49.977627 21552 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:49.978112 21632 consensus_queue.cc:1048] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:49.979575 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:49.981600 21200 leader_election.cc:343] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:49.983089 21505 raft_consensus.cc:359] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.983896 21505 raft_consensus.cc:385] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:49.984218 21505 raft_consensus.cc:740] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:49.985031 21505 consensus_queue.cc:260] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:49.987960 21504 tablet_bootstrap.cc:654] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.988767 21506 tablet_bootstrap.cc:654] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:49.989259 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:49.990172 21457 raft_consensus.cc:2468] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:49.997803 21505 ts_tablet_manager.cc:1434] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.002s
I20260709 12:28:49.999343 21505 tablet_bootstrap.cc:492] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:49.996959 21202 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:50.001428 21552 raft_consensus.cc:2804] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.001822 21552 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.002199 21552 raft_consensus.cc:3060] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.029471 21507 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:50.030184 21507 ts_tablet_manager.cc:1403] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.088s	user 0.015s	sys 0.006s
I20260709 12:28:50.039885 21506 tablet_bootstrap.cc:492] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:50.040453 21506 ts_tablet_manager.cc:1403] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.134s	user 0.007s	sys 0.017s
I20260709 12:28:50.041716 21507 raft_consensus.cc:359] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.042541 21507 raft_consensus.cc:385] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.042922 21507 raft_consensus.cc:740] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.044121 21552 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.044788 21615 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.045127 21507 consensus_queue.cc:260] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.046576 21552 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:50.051258 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:50.051892 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:50.052613 21312 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:50.053408 21200 leader_election.cc:343] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.047406 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:50.047612 21505 tablet_bootstrap.cc:654] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.058647 21507 ts_tablet_manager.cc:1434] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.001s
I20260709 12:28:50.049667 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:50.059877 21457 raft_consensus.cc:3060] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.060317 21507 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Bootstrap starting.
W20260709 12:28:50.062748 21200 leader_election.cc:343] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.047367 21615 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:50.049541 21506 raft_consensus.cc:359] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.067294 21506 raft_consensus.cc:385] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.068632 21506 raft_consensus.cc:740] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.069756 21504 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:50.070338 21504 ts_tablet_manager.cc:1403] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.158s	user 0.022s	sys 0.002s
I20260709 12:28:50.071821 21506 consensus_queue.cc:260] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.073712 21504 raft_consensus.cc:359] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.076082 21507 tablet_bootstrap.cc:654] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.076136 21504 raft_consensus.cc:385] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.076943 21504 raft_consensus.cc:740] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.086661 21534 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.087428 21534 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.090128 21534 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:50.094729 21504 consensus_queue.cc:260] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.099956 21506 ts_tablet_manager.cc:1434] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.059s	user 0.009s	sys 0.003s
I20260709 12:28:50.101096 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:50.102967 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:50.103735 21311 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:50.105913 21425 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:50.107082 21687 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.107503 21687 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.107882 21687 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.109556 21505 tablet_bootstrap.cc:492] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:50.111140 21505 ts_tablet_manager.cc:1403] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.112s	user 0.017s	sys 0.007s
W20260709 12:28:50.111196 21426 leader_election.cc:343] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.121510 21312 raft_consensus.cc:2468] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:50.126013 21202 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:50.127372 21552 raft_consensus.cc:2804] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.127863 21552 raft_consensus.cc:697] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:50.128963 21552 consensus_queue.cc:237] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.130672 21457 raft_consensus.cc:2468] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:50.111730 21506 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:50.124153 21505 raft_consensus.cc:359] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.133271 21505 raft_consensus.cc:385] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.133420 21570 raft_consensus.cc:493] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.133643 21505 raft_consensus.cc:740] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.134003 21570 raft_consensus.cc:515] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.134912 21504 ts_tablet_manager.cc:1434] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.064s	user 0.009s	sys 0.000s
I20260709 12:28:50.136065 21615 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.137799 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:50.133428 21202 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:50.135808 21505 consensus_queue.cc:260] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.147527 21504 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:50.148409 21687 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.150358 21385 raft_consensus.cc:2468] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:50.145872 21570 leader_election.cc:290] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:50.151218 21645 raft_consensus.cc:2804] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.155398 21645 raft_consensus.cc:697] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:50.156404 21645 consensus_queue.cc:237] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.158183 21275 leader_election.cc:304] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:50.159252 21570 raft_consensus.cc:2804] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.149463 21615 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.159766 21570 raft_consensus.cc:493] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.160384 21570 raft_consensus.cc:3060] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.161854 21615 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:50.163450 21505 ts_tablet_manager.cc:1434] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.050s	user 0.008s	sys 0.002s
I20260709 12:28:50.164731 21505 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:50.165864 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:50.166627 21459 raft_consensus.cc:2468] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:50.154345 21506 tablet_bootstrap.cc:654] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.167280 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:50.168251 21202 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
W20260709 12:28:50.169390 21200 leader_election.cc:343] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.170477 21615 raft_consensus.cc:2804] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.171002 21615 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.171473 21615 raft_consensus.cc:3060] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.174638 21570 raft_consensus.cc:515] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.176314 21505 tablet_bootstrap.cc:654] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.177654 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:50.178296 21383 raft_consensus.cc:3060] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.178695 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:50.179350 21459 raft_consensus.cc:3060] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.182571 21507 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:50.183169 21507 ts_tablet_manager.cc:1403] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.123s	user 0.016s	sys 0.009s
I20260709 12:28:50.188884 21687 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:50.190141 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:50.190922 21312 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.191351 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
W20260709 12:28:50.193126 21426 leader_election.cc:343] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.206120 21504 tablet_bootstrap.cc:654] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.210835 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:50.212088 21132 catalog_manager.cc:5697] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.213097 21383 raft_consensus.cc:2468] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:50.214629 21275 leader_election.cc:304] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:50.215713 21625 raft_consensus.cc:2804] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.214890 21132 catalog_manager.cc:5697] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:50.229835 21507 raft_consensus.cc:359] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:28:50.230788 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:50.230937 21507 raft_consensus.cc:385] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.187443 21570 leader_election.cc:290] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:50.231345 21507 raft_consensus.cc:740] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.232219 21507 consensus_queue.cc:260] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.212167 21312 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:50.232762 21615 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.237368 21425 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:50.238667 21687 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.239508 21507 ts_tablet_manager.cc:1434] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.056s	user 0.008s	sys 0.001s
I20260709 12:28:50.238670 21506 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:50.241704 21506 ts_tablet_manager.cc:1403] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.130s	user 0.010s	sys 0.006s
I20260709 12:28:50.240814 21687 raft_consensus.cc:697] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:50.234862 21459 raft_consensus.cc:2468] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:50.248046 21506 raft_consensus.cc:359] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.248939 21506 raft_consensus.cc:385] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.249219 21506 raft_consensus.cc:740] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.250381 21687 consensus_queue.cc:237] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.252727 21625 raft_consensus.cc:697] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:50.255607 21507 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:50.255507 21552 consensus_queue.cc:1048] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:50.257789 21625 consensus_queue.cc:237] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.259442 21505 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:50.259948 21505 ts_tablet_manager.cc:1403] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.096s	user 0.017s	sys 0.017s
I20260709 12:28:50.263165 21505 raft_consensus.cc:359] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.264146 21505 raft_consensus.cc:385] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.264448 21505 raft_consensus.cc:740] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.265292 21505 consensus_queue.cc:260] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.273029 21505 ts_tablet_manager.cc:1434] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:28:50.274325 21505 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:50.275660 21571 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.275838 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:50.276257 21571 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:50.277657 21200 leader_election.cc:343] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.286151 21552 consensus_queue.cc:1048] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.288841 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:50.289855 21311 raft_consensus.cc:2393] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:28:50.292693 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:50.293987 21236 raft_consensus.cc:2393] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:50.294783 21571 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:50.299212 21507 tablet_bootstrap.cc:654] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.304035 21615 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:50.304131 21570 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.305318 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:50.305996 21458 raft_consensus.cc:3060] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.305738 21570 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.308043 21570 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:50.328068 21351 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:50.330041 21571 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.319763 21506 consensus_queue.cc:260] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.337172 21132 catalog_manager.cc:5697] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:50.351215 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:50.355630 21132 catalog_manager.cc:5697] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.364008 21458 raft_consensus.cc:2468] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:50.372819 21507 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:50.373592 21507 ts_tablet_manager.cc:1403] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.118s	user 0.020s	sys 0.006s
I20260709 12:28:50.386382 21504 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:50.386934 21504 ts_tablet_manager.cc:1403] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.240s	user 0.022s	sys 0.014s
I20260709 12:28:50.387267 21506 ts_tablet_manager.cc:1434] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.145s	user 0.009s	sys 0.000s
I20260709 12:28:50.392243 21506 tablet_bootstrap.cc:492] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:50.392726 21134 catalog_manager.cc:5697] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:50.395941 21277 leader_election.cc:343] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.385756 21552 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:28:50.406471 21202 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:50.408427 21552 raft_consensus.cc:2804] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.408939 21552 raft_consensus.cc:697] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:50.409780 21552 consensus_queue.cc:237] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.410946 21504 raft_consensus.cc:359] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.415735 21504 raft_consensus.cc:385] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.416635 21504 raft_consensus.cc:740] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.415369 21505 tablet_bootstrap.cc:654] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.420555 21506 tablet_bootstrap.cc:654] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.423905 21131 catalog_manager.cc:5697] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:50.438066 21741 raft_consensus.cc:493] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.439473 21741 raft_consensus.cc:515] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.467199 21571 raft_consensus.cc:2749] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.456620 21504 consensus_queue.cc:260] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.479107 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:50.486147 21385 raft_consensus.cc:3060] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.478143 21507 raft_consensus.cc:359] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.487156 21507 raft_consensus.cc:385] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.487558 21507 raft_consensus.cc:740] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.491236 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:50.492003 21507 consensus_queue.cc:260] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.495596 21507 ts_tablet_manager.cc:1434] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.122s	user 0.008s	sys 0.000s
I20260709 12:28:50.496833 21507 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:50.497455 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:50.500129 21615 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:50.502033 21275 leader_election.cc:343] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.502537 21275 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:50.502959 21351 leader_election.cc:304] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:50.503489 21737 raft_consensus.cc:2749] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.506464 21504 ts_tablet_manager.cc:1434] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.119s	user 0.006s	sys 0.000s
I20260709 12:28:50.507808 21504 tablet_bootstrap.cc:492] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:50.509088 21506 tablet_bootstrap.cc:492] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:50.535280 21506 ts_tablet_manager.cc:1403] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.143s	user 0.025s	sys 0.011s
I20260709 12:28:50.518491 21132 catalog_manager.cc:5697] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:50.485286 21741 leader_election.cc:290] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:50.532954 21505 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:50.541465 21505 ts_tablet_manager.cc:1403] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.267s	user 0.008s	sys 0.004s
I20260709 12:28:50.543828 21506 raft_consensus.cc:359] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.544632 21506 raft_consensus.cc:385] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.544903 21506 raft_consensus.cc:740] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.545620 21506 consensus_queue.cc:260] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.546813 21571 raft_consensus.cc:2749] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.551368 21504 tablet_bootstrap.cc:654] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.552742 21505 raft_consensus.cc:359] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.553650 21505 raft_consensus.cc:385] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.554109 21505 raft_consensus.cc:740] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.555083 21506 ts_tablet_manager.cc:1434] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.019s	user 0.004s	sys 0.002s
I20260709 12:28:50.555053 21505 consensus_queue.cc:260] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.558879 21385 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.558900 21507 tablet_bootstrap.cc:654] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.564863 21506 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:50.578459 21615 consensus_queue.cc:1048] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.589583 21507 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:50.590209 21507 ts_tablet_manager.cc:1403] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.094s	user 0.011s	sys 0.000s
I20260709 12:28:50.592160 21505 ts_tablet_manager.cc:1434] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.050s	user 0.003s	sys 0.003s
I20260709 12:28:50.603171 21385 raft_consensus.cc:3060] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.603539 21505 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:50.605703 21741 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.606222 21741 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.594271 21506 tablet_bootstrap.cc:654] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.598935 21507 raft_consensus.cc:359] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.607481 21507 raft_consensus.cc:385] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.607765 21507 raft_consensus.cc:740] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.601948 21615 consensus_queue.cc:1048] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.608412 21741 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:50.608430 21507 consensus_queue.cc:260] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.610468 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:50.611063 21312 raft_consensus.cc:2468] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:50.611927 21507 ts_tablet_manager.cc:1434] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:28:50.611961 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:50.613742 21507 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:50.614300 21351 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:50.623265 21505 tablet_bootstrap.cc:654] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.625799 21504 tablet_bootstrap.cc:492] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:50.626549 21504 ts_tablet_manager.cc:1403] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.119s	user 0.012s	sys 0.007s
I20260709 12:28:50.627382 21741 raft_consensus.cc:2804] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:50.629710 21504 raft_consensus.cc:359] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.630515 21504 raft_consensus.cc:385] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.630839 21504 raft_consensus.cc:740] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.631636 21504 consensus_queue.cc:260] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.633786 21741 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:50.634259 21504 ts_tablet_manager.cc:1434] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:50.634343 21741 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:50.635624 21504 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:50.642102 21619 consensus_queue.cc:1048] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.645701 21741 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.651883 21384 raft_consensus.cc:1240] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:50.654546 21504 tablet_bootstrap.cc:654] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.655289 21507 tablet_bootstrap.cc:654] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.656543 21552 consensus_queue.cc:1059] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:50.658422 21552 raft_consensus.cc:3055] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:50.658784 21552 raft_consensus.cc:740] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:50.659473 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:50.659626 21552 consensus_queue.cc:260] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.660845 21552 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:50.664703 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:50.672295 21506 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:50.672881 21506 ts_tablet_manager.cc:1403] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.108s	user 0.010s	sys 0.004s
I20260709 12:28:50.676092 21506 raft_consensus.cc:359] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.676815 21506 raft_consensus.cc:385] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.677070 21506 raft_consensus.cc:740] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.677739 21741 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:50.678530 21506 consensus_queue.cc:260] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.681349 21615 consensus_queue.cc:1048] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:50.682233 21619 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.684358 21619 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.693634 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:50.697966 21310 raft_consensus.cc:2393] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:50.691553 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:50.700984 21202 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:50.702319 21552 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.707887 21236 raft_consensus.cc:2468] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:28:50.709283 21505 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:50.709860 21505 ts_tablet_manager.cc:1403] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.107s	user 0.006s	sys 0.007s
I20260709 12:28:50.691325 21619 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:50.711630 21506 ts_tablet_manager.cc:1434] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.000s
I20260709 12:28:50.713111 21506 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:50.713176 21505 raft_consensus.cc:359] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.713972 21505 raft_consensus.cc:385] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.714251 21505 raft_consensus.cc:740] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.715334 21505 consensus_queue.cc:260] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.718072 21352 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:50.718349 21552 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.719497 21737 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.720038 21737 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.722443 21632 consensus_queue.cc:1048] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.722528 21737 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:50.725709 21741 raft_consensus.cc:2804] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:50.727063 21507 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:50.727689 21507 ts_tablet_manager.cc:1403] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.114s	user 0.025s	sys 0.000s
I20260709 12:28:50.723063 21506 tablet_bootstrap.cc:654] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.726509 21741 raft_consensus.cc:697] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:50.730743 21505 ts_tablet_manager.cc:1434] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:28:50.724315 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:50.724745 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:50.732918 21741 consensus_queue.cc:237] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:50.735962 21275 leader_election.cc:343] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.743562 21570 consensus_queue.cc:1048] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.747668 21507 raft_consensus.cc:359] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.748442 21507 raft_consensus.cc:385] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.748703 21507 raft_consensus.cc:740] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
W20260709 12:28:50.750905 21278 leader_election.cc:343] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.751545 21278 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:50.749346 21507 consensus_queue.cc:260] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.753039 21552 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.753664 21504 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:50.753592 21552 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.754675 21504 ts_tablet_manager.cc:1403] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.119s	user 0.009s	sys 0.021s
I20260709 12:28:50.756057 21552 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:50.757524 21570 raft_consensus.cc:2749] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.757759 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:50.759299 21202 leader_election.cc:343] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.759676 21504 raft_consensus.cc:359] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.760362 21504 raft_consensus.cc:385] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.760608 21504 raft_consensus.cc:740] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.761376 21504 consensus_queue.cc:260] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.764791 21506 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:50.765321 21506 ts_tablet_manager.cc:1403] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.001s
I20260709 12:28:50.767004 21625 consensus_queue.cc:1048] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:50.774117 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:50.774401 21571 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.775161 21571 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:28:50.775568 21202 leader_election.cc:343] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.776077 21202 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:50.777237 21552 raft_consensus.cc:2749] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.777297 21571 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:50.779022 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:50.779870 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:50.780364 21457 raft_consensus.cc:2393] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:50.781962 21351 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:50.783383 21505 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:50.787176 21504 ts_tablet_manager.cc:1434] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.002s
I20260709 12:28:50.788292 21504 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:50.788537 21741 raft_consensus.cc:3060] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.779372 21506 raft_consensus.cc:359] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.793210 21506 raft_consensus.cc:385] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.793515 21506 raft_consensus.cc:740] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.761765 21507 ts_tablet_manager.cc:1434] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.000s
I20260709 12:28:50.795701 21507 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:50.797578 21506 consensus_queue.cc:260] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.800102 21534 consensus_queue.cc:1048] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:50.823064 21506 ts_tablet_manager.cc:1434] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 12:28:50.822366 21552 consensus_queue.cc:1048] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:50.827314 21504 tablet_bootstrap.cc:654] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.829083 21534 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.829639 21534 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.832980 21570 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.833501 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:50.834638 21534 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:50.834997 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:50.833564 21570 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:28:50.836365 21426 leader_election.cc:343] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.837481 21570 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:50.839068 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:50.839499 21423 leader_election.cc:343] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.839643 21505 tablet_bootstrap.cc:654] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.839762 21236 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:50.839895 21423 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:50.840606 21534 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.841986 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:50.842561 21459 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:50.842694 21741 raft_consensus.cc:2749] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.844782 21277 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:50.847842 21570 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.848284 21570 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.848621 21570 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.848699 21506 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:50.857663 21570 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.868036 21507 tablet_bootstrap.cc:654] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.872741 21506 tablet_bootstrap.cc:654] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.880110 21570 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:50.881682 21135 catalog_manager.cc:5697] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:50.877767 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:50.927953 21504 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:50.898674 21552 consensus_queue.cc:1048] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:50.924099 21505 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:50.885398 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:50.935395 21505 ts_tablet_manager.cc:1403] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.152s	user 0.010s	sys 0.004s
I20260709 12:28:50.935434 21459 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.935570 21236 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.938612 21505 raft_consensus.cc:359] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.939376 21505 raft_consensus.cc:385] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.939659 21505 raft_consensus.cc:740] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.900508 21737 raft_consensus.cc:493] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.935039 21504 ts_tablet_manager.cc:1403] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.147s	user 0.015s	sys 0.024s
I20260709 12:28:50.941370 21507 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:50.941901 21507 ts_tablet_manager.cc:1403] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.146s	user 0.012s	sys 0.000s
I20260709 12:28:50.943123 21504 raft_consensus.cc:359] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.943912 21504 raft_consensus.cc:385] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.944167 21504 raft_consensus.cc:740] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.944893 21504 consensus_queue.cc:260] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.945640 21505 consensus_queue.cc:260] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.947233 21504 ts_tablet_manager.cc:1434] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:28:50.948458 21504 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:50.948175 21737 raft_consensus.cc:515] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.949342 21459 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:50.950563 21737 leader_election.cc:290] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:50.951799 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:50.951987 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:50.952486 21235 raft_consensus.cc:2468] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:50.952678 21384 raft_consensus.cc:2468] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:50.954290 21277 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:50.957461 21275 leader_election.cc:304] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:50.958151 21505 ts_tablet_manager.cc:1434] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:28:50.958638 21570 raft_consensus.cc:2804] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:50.959067 21570 raft_consensus.cc:493] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:50.959332 21505 tablet_bootstrap.cc:492] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:50.959431 21570 raft_consensus.cc:3060] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:50.960846 21236 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:50.957280 21507 raft_consensus.cc:359] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.965418 21507 raft_consensus.cc:385] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.965786 21507 raft_consensus.cc:740] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:50.967079 21507 consensus_queue.cc:260] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:50.969731 21507 ts_tablet_manager.cc:1434] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:28:50.966555 21737 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:50.970541 21737 raft_consensus.cc:697] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:50.973156 21507 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:50.975327 21625 raft_consensus.cc:493] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:50.979959 21506 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:50.980670 21506 ts_tablet_manager.cc:1403] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.152s	user 0.012s	sys 0.002s
I20260709 12:28:50.982242 21504 tablet_bootstrap.cc:654] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:50.977932 21625 raft_consensus.cc:515] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:50.987015 21625 leader_election.cc:290] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:50.987778 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:50.988592 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:50.989289 21275 leader_election.cc:343] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.990412 21506 raft_consensus.cc:359] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.991272 21506 raft_consensus.cc:385] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:50.991539 21506 raft_consensus.cc:740] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
W20260709 12:28:50.992492 21277 leader_election.cc:343] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:50.992233 21506 consensus_queue.cc:260] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.993005 21277 leader_election.cc:304] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:50.995158 21625 raft_consensus.cc:2749] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:50.971429 21737 consensus_queue.cc:237] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:50.973878 21570 raft_consensus.cc:515] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:50.984110 21505 tablet_bootstrap.cc:654] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.006771 21506 ts_tablet_manager.cc:1434] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:28:51.007653 21552 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.008138 21552 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.011962 21552 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.013552 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:51.013715 21570 leader_election.cc:290] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.014889 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:51.016237 21200 leader_election.cc:343] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.016590 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:51.021173 21383 raft_consensus.cc:3060] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.021392 21310 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:51.014953 21506 tablet_bootstrap.cc:492] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:51.015643 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:51.023052 21202 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.024247 21237 raft_consensus.cc:3060] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.016623 21534 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.025643 21534 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.027570 21552 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.027976 21552 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.028277 21552 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.028609 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:51.029366 21310 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:51.030021 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:51.031498 21423 leader_election.cc:343] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.033006 21534 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.041839 21425 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.052865 21552 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.055348 21552 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.056579 21383 raft_consensus.cc:2468] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:51.060123 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:51.060665 21504 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:51.061196 21504 ts_tablet_manager.cc:1403] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.113s	user 0.011s	sys 0.007s
W20260709 12:28:51.061671 21200 leader_election.cc:343] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.061820 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:51.062433 21312 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.048563 21632 raft_consensus.cc:2804] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.062978 21632 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.063329 21632 raft_consensus.cc:3060] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.064108 21504 raft_consensus.cc:359] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.064848 21504 raft_consensus.cc:385] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.065099 21504 raft_consensus.cc:740] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.065733 21504 consensus_queue.cc:260] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.060115 21132 catalog_manager.cc:5697] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.068063 21275 leader_election.cc:304] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:51.071821 21504 ts_tablet_manager.cc:1434] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:28:51.072798 21737 raft_consensus.cc:2804] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.072993 21504 tablet_bootstrap.cc:492] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:51.077466 21507 tablet_bootstrap.cc:654] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.083976 21312 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:51.091200 21202 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.092664 21237 raft_consensus.cc:2468] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:51.093034 21552 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.095383 21552 raft_consensus.cc:697] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:51.096509 21552 consensus_queue.cc:237] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.100983 21625 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.103152 21505 tablet_bootstrap.cc:492] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:51.103854 21505 ts_tablet_manager.cc:1403] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.145s	user 0.012s	sys 0.005s
I20260709 12:28:51.104468 21737 raft_consensus.cc:697] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:51.105506 21737 consensus_queue.cc:237] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.101701 21625 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.117590 21506 tablet_bootstrap.cc:654] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.119537 21632 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.124866 21504 tablet_bootstrap.cc:654] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.124549 21507 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:51.125941 21507 ts_tablet_manager.cc:1403] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.153s	user 0.013s	sys 0.014s
I20260709 12:28:51.132081 21505 raft_consensus.cc:359] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.132874 21505 raft_consensus.cc:385] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.133136 21505 raft_consensus.cc:740] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.133435 21534 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.133846 21505 consensus_queue.cc:260] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.133965 21534 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.136782 21534 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:51.137518 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:51.138989 21423 leader_election.cc:343] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.139992 21625 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.145404 21505 ts_tablet_manager.cc:1434] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 12:28:51.146133 21506 tablet_bootstrap.cc:492] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:51.146456 21505 tablet_bootstrap.cc:492] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:51.146665 21506 ts_tablet_manager.cc:1403] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.132s	user 0.014s	sys 0.000s
I20260709 12:28:51.150666 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:51.152393 21426 leader_election.cc:343] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.152886 21426 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.155864 21687 raft_consensus.cc:2749] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.155339 21506 raft_consensus.cc:359] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.155397 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:51.156745 21506 raft_consensus.cc:385] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.158008 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:51.157472 21506 raft_consensus.cc:740] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.159319 21132 catalog_manager.cc:5697] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
W20260709 12:28:51.160764 21278 leader_election.cc:343] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.160614 21506 consensus_queue.cc:260] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.162804 21506 ts_tablet_manager.cc:1434] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
W20260709 12:28:51.164261 21275 leader_election.cc:343] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.164820 21275 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.166396 21507 raft_consensus.cc:359] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.167619 21781 raft_consensus.cc:2749] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.172494 21507 raft_consensus.cc:385] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.172992 21507 raft_consensus.cc:740] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.168638 21505 tablet_bootstrap.cc:654] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.175367 21507 consensus_queue.cc:260] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.178216 21506 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:51.178193 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:51.179100 21507 ts_tablet_manager.cc:1434] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.052s	user 0.007s	sys 0.001s
I20260709 12:28:51.179062 21310 raft_consensus.cc:3060] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.180178 21632 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.183494 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:51.184216 21384 raft_consensus.cc:3060] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.185449 21506 tablet_bootstrap.cc:654] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.195374 21504 tablet_bootstrap.cc:492] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:51.195936 21504 ts_tablet_manager.cc:1403] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.123s	user 0.011s	sys 0.008s
I20260709 12:28:51.199000 21619 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.199606 21619 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.201720 21619 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.203375 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:51.207432 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
W20260709 12:28:51.207621 21200 leader_election.cc:343] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.208065 21312 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:51.210704 21619 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.214095 21202 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.214195 21619 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.212158 21507 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:51.217000 21619 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.217309 21312 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.209270 21504 raft_consensus.cc:359] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.213101 21384 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:51.217823 21504 raft_consensus.cc:385] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.218410 21552 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.219004 21552 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.218375 21504 raft_consensus.cc:740] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.219297 21552 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.224057 21507 tablet_bootstrap.cc:654] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.211614 21310 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:51.227207 21312 raft_consensus.cc:1275] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:51.227072 21552 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.228622 21423 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:51.229172 21571 consensus_queue.cc:1048] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:51.232803 21687 raft_consensus.cc:2804] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.227680 21504 consensus_queue.cc:260] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.234360 21687 raft_consensus.cc:697] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:51.235351 21687 consensus_queue.cc:237] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.259159 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:51.259974 21310 raft_consensus.cc:2468] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:51.262022 21202 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:51.263397 21132 catalog_manager.cc:5697] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.264240 21619 raft_consensus.cc:2804] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:51.264735 21619 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.265115 21619 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.262480 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:51.266795 21383 raft_consensus.cc:2468] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:51.274142 21552 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.275389 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:51.283879 21200 leader_election.cc:343] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.280493 21505 tablet_bootstrap.cc:492] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:51.286509 21505 ts_tablet_manager.cc:1403] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.140s	user 0.021s	sys 0.017s
I20260709 12:28:51.289475 21505 raft_consensus.cc:359] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.290199 21505 raft_consensus.cc:385] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.281291 21236 raft_consensus.cc:1275] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:51.295459 21781 raft_consensus.cc:493] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.296123 21781 raft_consensus.cc:515] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.298309 21504 ts_tablet_manager.cc:1434] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.102s	user 0.012s	sys 0.004s
I20260709 12:28:51.298454 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:51.299494 21312 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.304823 21781 leader_election.cc:290] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:51.299612 21504 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:51.290457 21505 raft_consensus.cc:740] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.305930 21505 consensus_queue.cc:260] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.306519 21747 consensus_queue.cc:1048] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:51.309505 21505 ts_tablet_manager.cc:1434] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:28:51.310042 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:51.311565 21275 leader_election.cc:343] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.311830 21619 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.315032 21619 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:51.315122 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:51.315810 21311 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:51.317920 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:51.320976 21135 catalog_manager.cc:5697] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.323743 21507 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:51.318989 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:51.324275 21507 ts_tablet_manager.cc:1403] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.112s	user 0.011s	sys 0.002s
I20260709 12:28:51.324710 21459 raft_consensus.cc:2468] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:51.327378 21507 raft_consensus.cc:359] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.328153 21507 raft_consensus.cc:385] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.328437 21507 raft_consensus.cc:740] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.329342 21277 leader_election.cc:304] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.330572 21781 raft_consensus.cc:2804] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.330982 21781 raft_consensus.cc:493] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.331341 21781 raft_consensus.cc:3060] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.332360 21534 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.332880 21534 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.335131 21534 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:51.337023 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:51.339242 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:51.339612 21426 leader_election.cc:343] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.340101 21383 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:51.341619 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:51.338042 21507 consensus_queue.cc:260] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.346740 21507 ts_tablet_manager.cc:1434] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
W20260709 12:28:51.347440 21425 leader_election.cc:343] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.349409 21425 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:51.350701 21687 raft_consensus.cc:2749] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.352345 21507 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:51.363651 21619 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.364455 21619 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.365239 21505 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:51.369086 21504 tablet_bootstrap.cc:654] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.375305 21507 tablet_bootstrap.cc:654] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.378340 21619 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:51.379009 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:51.379660 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:51.379717 21459 raft_consensus.cc:2468] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
W20260709 12:28:51.381014 21200 leader_election.cc:343] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.381536 21202 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.382170 21383 raft_consensus.cc:2468] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:51.382468 21506 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:51.383527 21619 raft_consensus.cc:2804] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.388556 21619 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.388607 21506 ts_tablet_manager.cc:1403] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.211s	user 0.016s	sys 0.010s
I20260709 12:28:51.389134 21619 raft_consensus.cc:3060] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.390082 21312 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:51.391650 21200 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:51.401196 21552 raft_consensus.cc:2804] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:51.401897 21552 raft_consensus.cc:697] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:51.408413 21202 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.409386 21615 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.409884 21615 raft_consensus.cc:697] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:51.410753 21615 consensus_queue.cc:237] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.416040 21552 consensus_queue.cc:237] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.431444 21781 raft_consensus.cc:515] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.433403 21311 raft_consensus.cc:2468] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:51.433909 21781 leader_election.cc:290] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:51.416747 21506 raft_consensus.cc:359] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.438380 21506 raft_consensus.cc:385] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.438766 21506 raft_consensus.cc:740] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.439230 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:51.438021 21505 tablet_bootstrap.cc:654] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.440076 21459 raft_consensus.cc:3060] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.440707 21506 consensus_queue.cc:260] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.453227 21506 ts_tablet_manager.cc:1434] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.064s	user 0.003s	sys 0.003s
I20260709 12:28:51.455351 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:51.458983 21507 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:51.456864 21619 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.459596 21507 ts_tablet_manager.cc:1403] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.108s	user 0.018s	sys 0.006s
W20260709 12:28:51.459585 21275 leader_election.cc:343] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.464316 21619 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:51.467856 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:51.468526 21457 raft_consensus.cc:3060] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.469234 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:51.470583 21200 leader_election.cc:343] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.472818 21504 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:51.473398 21504 ts_tablet_manager.cc:1403] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.174s	user 0.018s	sys 0.006s
I20260709 12:28:51.474288 21459 raft_consensus.cc:2468] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:51.475507 21506 tablet_bootstrap.cc:492] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:51.478396 21277 leader_election.cc:304] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.480098 21781 raft_consensus.cc:2804] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.481235 21781 raft_consensus.cc:697] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:51.482084 21781 consensus_queue.cc:237] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:28:51.484591 21200 consensus_peers.cc:597] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:51.492295 21504 raft_consensus.cc:359] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.466324 21507 raft_consensus.cc:359] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.496276 21505 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:51.496840 21505 ts_tablet_manager.cc:1403] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.132s	user 0.012s	sys 0.000s
I20260709 12:28:51.493178 21504 raft_consensus.cc:385] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.497529 21504 raft_consensus.cc:740] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.497840 21506 tablet_bootstrap.cc:654] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.498818 21504 consensus_queue.cc:260] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.504503 21505 raft_consensus.cc:359] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.505553 21505 raft_consensus.cc:385] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.505939 21505 raft_consensus.cc:740] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.495777 21507 raft_consensus.cc:385] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.510547 21507 raft_consensus.cc:740] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.511389 21507 consensus_queue.cc:260] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.492971 21132 catalog_manager.cc:5697] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 2, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.515203 21505 consensus_queue.cc:260] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.522137 21504 ts_tablet_manager.cc:1434] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:28:51.533318 21504 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:51.523574 21505 ts_tablet_manager.cc:1434] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 12:28:51.524627 21737 consensus_queue.cc:1048] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.526423 21506 tablet_bootstrap.cc:492] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:51.538093 21506 ts_tablet_manager.cc:1403] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.063s	user 0.016s	sys 0.013s
I20260709 12:28:51.527408 21457 raft_consensus.cc:2468] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:51.541455 21506 raft_consensus.cc:359] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.542289 21506 raft_consensus.cc:385] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.542563 21506 raft_consensus.cc:740] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.542906 21202 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.543329 21506 consensus_queue.cc:260] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.544611 21552 raft_consensus.cc:2804] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.545261 21552 raft_consensus.cc:697] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:51.546255 21552 consensus_queue.cc:237] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.549847 21505 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:51.513568 21507 ts_tablet_manager.cc:1434] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.000s
I20260709 12:28:51.551939 21507 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:51.565542 21737 consensus_queue.cc:1048] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.566553 21784 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.567093 21784 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.569224 21784 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:51.571300 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:51.571985 21457 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:51.572578 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:51.573185 21277 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:51.574210 21784 raft_consensus.cc:2804] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.574577 21784 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.573235 21236 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:51.574936 21784 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.575381 21505 tablet_bootstrap.cc:654] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.579115 21506 ts_tablet_manager.cc:1434] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:28:51.580199 21506 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:51.579974 21619 consensus_queue.cc:1048] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.583741 21784 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.587369 21506 tablet_bootstrap.cc:654] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.588320 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:51.589133 21457 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.593326 21505 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:51.593905 21505 ts_tablet_manager.cc:1403] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.044s	user 0.012s	sys 0.000s
I20260709 12:28:51.597293 21505 raft_consensus.cc:359] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.598018 21505 raft_consensus.cc:385] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.599030 21505 raft_consensus.cc:740] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.599907 21505 consensus_queue.cc:260] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.605494 21504 tablet_bootstrap.cc:654] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.608989 21505 ts_tablet_manager.cc:1434] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 12:28:51.609717 21506 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:51.613747 21507 tablet_bootstrap.cc:654] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.616873 21784 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:51.617489 21625 consensus_queue.cc:1048] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:51.618213 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:51.638255 21236 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.645749 21134 catalog_manager.cc:5697] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.657357 21781 consensus_queue.cc:1048] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:51.657720 21131 catalog_manager.cc:5697] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:51.663496 21131 catalog_manager.cc:5697] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:51.678709 21236 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:51.624410 21457 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:51.629261 21506 ts_tablet_manager.cc:1403] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.000s
I20260709 12:28:51.635175 21505 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:51.698786 21505 tablet_bootstrap.cc:654] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.699592 21506 raft_consensus.cc:359] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.700357 21506 raft_consensus.cc:385] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.700634 21506 raft_consensus.cc:740] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.701299 21506 consensus_queue.cc:260] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.680828 21278 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:51.703418 21781 raft_consensus.cc:2804] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:51.703958 21781 raft_consensus.cc:697] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:51.704874 21781 consensus_queue.cc:237] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.717744 21506 ts_tablet_manager.cc:1434] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 12:28:51.744946 21506 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:51.754966 21504 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:51.755548 21504 ts_tablet_manager.cc:1403] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.223s	user 0.046s	sys 0.023s
I20260709 12:28:51.792017 21747 raft_consensus.cc:493] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.792790 21747 raft_consensus.cc:515] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.803920 21747 leader_election.cc:290] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:51.806743 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:51.810541 21351 leader_election.cc:343] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.814116 21570 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.814996 21570 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.823058 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:51.824061 21309 raft_consensus.cc:2468] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:51.827003 21504 raft_consensus.cc:359] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.831193 21504 raft_consensus.cc:385] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.831606 21504 raft_consensus.cc:740] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.837781 21504 consensus_queue.cc:260] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.853200 21504 ts_tablet_manager.cc:1434] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.097s	user 0.009s	sys 0.000s
I20260709 12:28:51.863785 21632 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.864305 21632 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.866099 21506 tablet_bootstrap.cc:654] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.877755 21504 tablet_bootstrap.cc:492] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:51.882649 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:51.884186 21625 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.884994 21625 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:51.872720 21570 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:51.903992 21386 raft_consensus.cc:1275] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:28:51.904189 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:51.866184 21632 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:51.910643 21795 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.911252 21795 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.913110 21505 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:51.913339 21795 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:51.913661 21505 ts_tablet_manager.cc:1403] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.279s	user 0.031s	sys 0.055s
I20260709 12:28:51.916911 21507 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
W20260709 12:28:51.917105 21426 leader_election.cc:343] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.917469 21507 ts_tablet_manager.cc:1403] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.366s	user 0.022s	sys 0.008s
W20260709 12:28:51.918689 21423 leader_election.cc:343] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:51.919143 21423 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:51.928663 21632 raft_consensus.cc:2749] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:51.941284 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:51.942042 21232 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:51.942148 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:51.942786 21457 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:51.943375 21311 raft_consensus.cc:1275] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:28:51.935612 21687 consensus_queue.cc:1048] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:28:51.932296 21615 consensus_queue.cc:1048] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260709 12:28:51.948477 21505 raft_consensus.cc:359] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.949275 21505 raft_consensus.cc:385] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.949527 21505 raft_consensus.cc:740] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.950317 21505 consensus_queue.cc:260] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.953986 21505 ts_tablet_manager.cc:1434] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.040s	user 0.009s	sys 0.000s
I20260709 12:28:51.960213 21747 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.960784 21747 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:51.963027 21747 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:51.970685 21278 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:51.971791 21784 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:51.972055 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:51.972385 21784 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:51.972862 21784 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:51.971967 21505 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:51.982395 21505 tablet_bootstrap.cc:654] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:51.983491 21784 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.987856 21311 raft_consensus.cc:2393] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:28:51.988837 21571 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:51.989338 21571 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:51.990015 21625 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:51.991474 21571 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:51.988564 21507 raft_consensus.cc:359] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.992585 21507 raft_consensus.cc:385] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:51.992938 21507 raft_consensus.cc:740] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:51.993660 21507 consensus_queue.cc:260] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:51.995326 21507 ts_tablet_manager.cc:1434] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.078s	user 0.006s	sys 0.000s
I20260709 12:28:51.996629 21507 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:52.002883 21507 tablet_bootstrap.cc:654] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.009891 21784 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:52.012804 21385 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.041096 21504 tablet_bootstrap.cc:654] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.043455 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:52.046962 21232 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:52.051348 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 194439us
I20260709 12:28:52.061043 21134 catalog_manager.cc:5697] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.054610 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:52.066481 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 128494us
I20260709 12:28:52.070324 21457 raft_consensus.cc:2393] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:52.079366 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:52.078444 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:52.081880 21232 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
W20260709 12:28:52.064637 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 204383us
I20260709 12:28:52.096942 21278 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:52.098901 21781 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.099597 21781 raft_consensus.cc:697] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:52.100739 21505 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:52.101466 21505 ts_tablet_manager.cc:1403] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.130s	user 0.009s	sys 0.005s
I20260709 12:28:52.103452 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:52.104458 21459 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:52.121279 21571 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.100811 21781 consensus_queue.cc:237] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:52.122095 21571 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:52.128747 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:52.128913 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:52.133526 21351 leader_election.cc:304] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:52.134824 21741 raft_consensus.cc:2804] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:52.135263 21741 raft_consensus.cc:493] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.135607 21741 raft_consensus.cc:3060] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.126370 21571 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:52.139556 21385 raft_consensus.cc:3060] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.140172 21384 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.143215 21506 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:52.143985 21506 ts_tablet_manager.cc:1403] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.399s	user 0.013s	sys 0.024s
I20260709 12:28:52.147392 21741 raft_consensus.cc:515] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.149224 21505 raft_consensus.cc:359] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.152297 21505 raft_consensus.cc:385] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.152737 21505 raft_consensus.cc:740] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.169054 21615 consensus_queue.cc:1048] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.172851 21795 consensus_queue.cc:1048] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.173167 21202 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:52.174350 21821 raft_consensus.cc:2764] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:28:52.176601 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:52.178431 21275 consensus_peers.cc:597] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:52.179085 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:52.179915 21457 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:52.180022 21275 leader_election.cc:343] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.192625 21741 leader_election.cc:290] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:52.194630 21820 consensus_queue.cc:1048] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.204066 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:52.211295 21505 consensus_queue.cc:260] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.218287 21506 raft_consensus.cc:359] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:52.219103 21506 raft_consensus.cc:385] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.219388 21506 raft_consensus.cc:740] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.222745 21457 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
W20260709 12:28:52.226253 21277 leader_election.cc:343] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.220059 21506 consensus_queue.cc:260] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:52.227691 21507 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:52.228223 21507 ts_tablet_manager.cc:1403] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.232s	user 0.049s	sys 0.053s
I20260709 12:28:52.228454 21506 ts_tablet_manager.cc:1434] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.084s	user 0.001s	sys 0.006s
I20260709 12:28:52.229525 21506 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:52.230249 21504 tablet_bootstrap.cc:492] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:52.226778 21277 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:52.231645 21784 raft_consensus.cc:2749] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:52.232182 21505 ts_tablet_manager.cc:1434] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.130s	user 0.010s	sys 0.001s
I20260709 12:28:52.233309 21505 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:52.230732 21504 ts_tablet_manager.cc:1403] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.353s	user 0.027s	sys 0.046s
I20260709 12:28:52.234143 21507 raft_consensus.cc:359] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.236105 21507 raft_consensus.cc:385] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.238253 21507 raft_consensus.cc:740] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.238874 21504 raft_consensus.cc:359] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.239267 21507 consensus_queue.cc:260] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.240423 21784 consensus_queue.cc:1048] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.243644 21351 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:52.245131 21534 consensus_queue.cc:1048] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.246685 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:52.253310 21507 ts_tablet_manager.cc:1434] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:28:52.242758 21504 raft_consensus.cc:385] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.257834 21505 tablet_bootstrap.cc:654] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.254140 21312 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:52.258021 21134 catalog_manager.cc:5697] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:52.254142 21504 raft_consensus.cc:740] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.261201 21818 consensus_queue.cc:1048] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.261551 21504 consensus_queue.cc:260] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.263906 21504 ts_tablet_manager.cc:1434] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.030s	user 0.006s	sys 0.000s
I20260709 12:28:52.264997 21504 tablet_bootstrap.cc:492] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:52.269021 21784 consensus_queue.cc:1048] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.270738 21571 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:52.280442 21507 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:52.285327 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:52.286074 21456 raft_consensus.cc:3060] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.286896 21821 consensus_queue.cc:1048] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.296703 21506 tablet_bootstrap.cc:654] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.296705 21456 raft_consensus.cc:2468] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:52.311101 21351 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:52.302333 21504 tablet_bootstrap.cc:654] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.301970 21822 consensus_queue.cc:1048] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.332316 21507 tablet_bootstrap.cc:654] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.337956 21571 raft_consensus.cc:2749] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:52.330080 21351 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:52.378197 21747 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:52.379722 21747 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:52.380313 21747 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.393579 21505 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:52.394354 21505 ts_tablet_manager.cc:1403] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.161s	user 0.015s	sys 0.011s
I20260709 12:28:52.399632 21351 leader_election.cc:304] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:52.429832 21571 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.430469 21571 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.428508 21741 raft_consensus.cc:2804] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.455112 21822 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.479730 21504 tablet_bootstrap.cc:492] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:52.509505 21504 ts_tablet_manager.cc:1403] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.245s	user 0.029s	sys 0.014s
I20260709 12:28:52.502636 21781 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.509330 21571 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:52.486085 21505 raft_consensus.cc:359] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.517993 21505 raft_consensus.cc:385] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.518824 21505 raft_consensus.cc:740] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.515002 21822 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:52.525612 21822 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:52.527593 21504 raft_consensus.cc:359] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.528352 21504 raft_consensus.cc:385] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.528606 21504 raft_consensus.cc:740] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.529237 21504 consensus_queue.cc:260] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.542279 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:52.543064 21456 raft_consensus.cc:2468] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:52.514830 21781 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:52.521270 21505 consensus_queue.cc:260] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.548224 21781 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:52.552583 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:52.553987 21504 ts_tablet_manager.cc:1434] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:28:52.566715 21505 ts_tablet_manager.cc:1434] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.137s	user 0.005s	sys 0.003s
I20260709 12:28:52.575541 21504 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:52.576043 21506 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:52.576529 21506 ts_tablet_manager.cc:1403] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.347s	user 0.024s	sys 0.017s
I20260709 12:28:52.572795 21741 raft_consensus.cc:697] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:52.577870 21741 consensus_queue.cc:237] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.578850 21505 tablet_bootstrap.cc:492] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
W20260709 12:28:52.581138 21275 leader_election.cc:343] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.578969 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:52.585279 21747 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:52.589335 21822 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.589895 21822 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.592020 21822 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:28:52.593703 21202 leader_election.cc:343] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.599458 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:52.600390 21456 raft_consensus.cc:2468] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:52.602299 21202 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:52.603297 21822 raft_consensus.cc:2804] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:52.603736 21822 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.603472 21506 raft_consensus.cc:359] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:52.604308 21822 raft_consensus.cc:3060] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.604565 21506 raft_consensus.cc:385] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.604983 21506 raft_consensus.cc:740] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
W20260709 12:28:52.562124 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 59807.1us
I20260709 12:28:52.606864 21505 tablet_bootstrap.cc:654] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.607879 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:52.608536 21308 raft_consensus.cc:3060] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.610816 21747 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:52.605823 21506 consensus_queue.cc:260] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:52.628866 21504 tablet_bootstrap.cc:654] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:52.637887 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 190688us
I20260709 12:28:52.652191 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:52.655082 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:52.660152 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:52.660948 21309 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:52.664449 21506 ts_tablet_manager.cc:1434] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.088s	user 0.009s	sys 0.000s
W20260709 12:28:52.671728 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:52.679071 21202 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:52.681986 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:52.683063 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:52.683795 21384 raft_consensus.cc:2468] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:52.683979 21507 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:52.684572 21507 ts_tablet_manager.cc:1403] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.406s	user 0.011s	sys 0.020s
I20260709 12:28:52.684826 21380 raft_consensus.cc:1240] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:52.689954 21507 raft_consensus.cc:359] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:52.690726 21507 raft_consensus.cc:385] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.690999 21507 raft_consensus.cc:740] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.694147 21351 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:52.694890 21507 consensus_queue.cc:260] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:52.696079 21552 consensus_queue.cc:1059] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:52.700241 21506 tablet_bootstrap.cc:492] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:52.680531 21822 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.704267 21822 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:52.705655 21232 raft_consensus.cc:2393] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:52.696452 21820 raft_consensus.cc:2804] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:52.714118 21820 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.714614 21820 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.718634 21308 raft_consensus.cc:2468] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:52.727584 21507 ts_tablet_manager.cc:1434] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.002s
I20260709 12:28:52.733445 21843 raft_consensus.cc:2804] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:28:52.751395 21278 leader_election.cc:343] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.751955 21278 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:52.753408 21814 consensus_queue.cc:1048] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.753800 21818 raft_consensus.cc:2749] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:52.761794 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:52.762485 21459 raft_consensus.cc:3060] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.738924 21843 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:52.766767 21843 raft_consensus.cc:3060] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.770880 21459 raft_consensus.cc:2468] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:52.746583 21135 catalog_manager.cc:5697] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:52.794106 21507 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:52.805578 21506 tablet_bootstrap.cc:654] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.797660 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:52.813395 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 48125.7us
I20260709 12:28:52.814424 21505 tablet_bootstrap.cc:492] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:52.815016 21505 ts_tablet_manager.cc:1403] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.244s	user 0.018s	sys 0.024s
I20260709 12:28:52.817015 21202 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:52.821244 21855 raft_consensus.cc:2804] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:52.824607 21818 consensus_queue.cc:1048] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:52.850363 21843 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.852164 21385 raft_consensus.cc:2393] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:52.866442 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:52.869524 21843 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:52.873409 21504 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:52.874142 21504 ts_tablet_manager.cc:1403] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.301s	user 0.021s	sys 0.000s
I20260709 12:28:52.885120 21855 raft_consensus.cc:697] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:52.875711 21821 raft_consensus.cc:3055] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:52.889420 21507 tablet_bootstrap.cc:654] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:52.891031 21821 raft_consensus.cc:740] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:52.901307 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:52.902166 21459 raft_consensus.cc:2393] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:52.906577 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:52.905782 21632 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:52.893134 21821 consensus_queue.cc:260] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:52.898905 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 73433.1us
I20260709 12:28:52.919147 21855 consensus_queue.cc:237] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:52.932996 21820 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:52.935693 21821 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:52.922343 21632 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.937409 21504 raft_consensus.cc:359] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:52.938122 21504 raft_consensus.cc:385] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.938545 21504 raft_consensus.cc:740] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.939625 21632 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:52.943892 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:52.951622 21423 leader_election.cc:343] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:52.939368 21504 consensus_queue.cc:260] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:52.955282 21820 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:52.959600 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:52.959779 21505 raft_consensus.cc:359] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.960522 21505 raft_consensus.cc:385] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:52.960969 21505 raft_consensus.cc:740] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:52.962023 21504 ts_tablet_manager.cc:1434] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.087s	user 0.006s	sys 0.003s
I20260709 12:28:52.961669 21505 consensus_queue.cc:260] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:52.963469 21505 ts_tablet_manager.cc:1434] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.148s	user 0.006s	sys 0.000s
I20260709 12:28:52.964416 21505 tablet_bootstrap.cc:492] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:52.967060 21351 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:52.960441 21309 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:52.969404 21504 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:52.984658 21741 raft_consensus.cc:2749] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:53.010761 21507 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:53.011623 21507 ts_tablet_manager.cc:1403] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.220s	user 0.014s	sys 0.015s
W20260709 12:28:53.012519 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 64992.1us
I20260709 12:28:53.024973 21858 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.029470 21507 raft_consensus.cc:359] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.035434 21507 raft_consensus.cc:385] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.035751 21507 raft_consensus.cc:740] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.036511 21507 consensus_queue.cc:260] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.026448 21309 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:53.040046 21202 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:53.040066 21505 tablet_bootstrap.cc:654] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.029551 21858 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.051156 21861 raft_consensus.cc:2804] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.051764 21861 raft_consensus.cc:697] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:53.052603 21861 consensus_queue.cc:237] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.054339 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:53.055318 21309 raft_consensus.cc:2468] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:53.043104 21741 consensus_queue.cc:1048] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:28:53.059680 21352 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:53.060913 21741 raft_consensus.cc:2749] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.063879 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:53.064771 21459 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.071702 21507 ts_tablet_manager.cc:1434] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.059s	user 0.006s	sys 0.001s
I20260709 12:28:53.073654 21507 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:53.077034 21741 consensus_queue.cc:1048] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:53.080507 21858 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:53.087500 21504 tablet_bootstrap.cc:654] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.090191 21506 tablet_bootstrap.cc:492] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:53.090746 21506 ts_tablet_manager.cc:1403] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.412s	user 0.032s	sys 0.024s
I20260709 12:28:53.090432 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:53.093118 21310 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:53.094370 21202 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:53.095940 21506 raft_consensus.cc:359] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.096107 21425 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:53.097086 21858 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.097340 21815 raft_consensus.cc:2804] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.097523 21858 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.097800 21815 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.097980 21858 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.098232 21815 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.101117 21747 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.101650 21747 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.103346 21506 raft_consensus.cc:385] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.103813 21506 raft_consensus.cc:740] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.104489 21506 consensus_queue.cc:260] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.107715 21747 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:53.107870 21506 ts_tablet_manager.cc:1434] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.001s
I20260709 12:28:53.110183 21506 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:53.112696 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:53.114105 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:53.105348 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:53.122741 21459 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:53.121857 21232 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
W20260709 12:28:53.129367 21200 leader_election.cc:343] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.136881 21352 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:53.130072 21460 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:53.140621 21741 raft_consensus.cc:2804] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.141014 21741 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.141342 21741 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.147653 21507 tablet_bootstrap.cc:654] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.157241 21815 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.168737 21506 tablet_bootstrap.cc:654] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.181836 21815 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:53.190466 21505 tablet_bootstrap.cc:492] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:53.191320 21505 ts_tablet_manager.cc:1403] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.227s	user 0.012s	sys 0.011s
I20260709 12:28:53.190914 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:53.192044 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:53.196352 21309 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.196800 21505 raft_consensus.cc:359] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.197623 21505 raft_consensus.cc:385] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.197878 21505 raft_consensus.cc:740] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
W20260709 12:28:53.199365 21423 leader_election.cc:343] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.206634 21858 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.211099 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:53.212009 21308 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.211690 21741 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.219380 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:53.225289 21504 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:53.224896 21505 consensus_queue.cc:260] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:28:53.230139 21200 leader_election.cc:343] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.231120 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:53.231813 21457 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.232532 21741 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:53.233706 21504 ts_tablet_manager.cc:1403] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.265s	user 0.022s	sys 0.002s
I20260709 12:28:53.237007 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:53.237768 21232 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.240602 21383 raft_consensus.cc:3060] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.213971 21858 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:53.245097 21309 raft_consensus.cc:2468] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:53.246860 21505 ts_tablet_manager.cc:1434] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.055s	user 0.008s	sys 0.001s
I20260709 12:28:53.248018 21505 tablet_bootstrap.cc:492] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:53.249792 21308 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:53.254715 21131 catalog_manager.cc:5697] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:53.271806 21202 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:53.273330 21131 catalog_manager.cc:5697] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.248693 21425 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:53.275579 21687 raft_consensus.cc:2804] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.276104 21687 raft_consensus.cc:697] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:53.273541 21858 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.276975 21687 consensus_queue.cc:237] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.277446 21858 raft_consensus.cc:697] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:53.278785 21858 consensus_queue.cc:237] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.292640 21232 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:53.296219 21505 tablet_bootstrap.cc:654] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.296314 21457 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:53.288424 21504 raft_consensus.cc:359] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.299680 21507 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:53.303488 21504 raft_consensus.cc:385] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.300246 21506 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:53.303717 21507 ts_tablet_manager.cc:1403] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.231s	user 0.017s	sys 0.015s
I20260709 12:28:53.302881 21352 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:53.304697 21741 raft_consensus.cc:2804] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.305255 21741 raft_consensus.cc:697] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:53.306409 21741 consensus_queue.cc:237] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.307027 21507 raft_consensus.cc:359] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.307890 21507 raft_consensus.cc:385] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.308163 21507 raft_consensus.cc:740] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.308920 21507 consensus_queue.cc:260] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.311144 21507 ts_tablet_manager.cc:1434] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:28:53.312264 21507 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:53.315178 21506 ts_tablet_manager.cc:1403] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.205s	user 0.027s	sys 0.045s
I20260709 12:28:53.318542 21504 raft_consensus.cc:740] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.321408 21131 catalog_manager.cc:5697] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:53.331533 21507 tablet_bootstrap.cc:654] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.331413 21506 raft_consensus.cc:359] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.336684 21506 raft_consensus.cc:385] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.336971 21506 raft_consensus.cc:740] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.337746 21507 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:53.337666 21506 consensus_queue.cc:260] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.338440 21507 ts_tablet_manager.cc:1403] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.026s	user 0.005s	sys 0.007s
I20260709 12:28:53.337826 21132 catalog_manager.cc:5697] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.341418 21507 raft_consensus.cc:359] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.342088 21507 raft_consensus.cc:385] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.342334 21507 raft_consensus.cc:740] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.343842 21507 consensus_queue.cc:260] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.346045 21507 ts_tablet_manager.cc:1434] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:28:53.347502 21507 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:53.351620 21506 ts_tablet_manager.cc:1434] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.000s
I20260709 12:28:53.352813 21506 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:53.354229 21507 tablet_bootstrap.cc:654] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.362056 21504 consensus_queue.cc:260] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.362967 21687 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.363437 21687 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.367872 21687 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:53.368980 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:53.369705 21232 raft_consensus.cc:2468] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:53.371057 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:53.372522 21426 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
W20260709 12:28:53.373160 21423 leader_election.cc:343] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.375203 21815 raft_consensus.cc:2804] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.375669 21815 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.376021 21815 raft_consensus.cc:3060] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.380275 21505 tablet_bootstrap.cc:492] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:53.381078 21505 ts_tablet_manager.cc:1403] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.016s
I20260709 12:28:53.388891 21504 ts_tablet_manager.cc:1434] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.155s	user 0.007s	sys 0.000s
I20260709 12:28:53.389312 21507 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:53.389838 21507 ts_tablet_manager.cc:1403] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.043s	user 0.013s	sys 0.018s
I20260709 12:28:53.386953 21815 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.392503 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:53.392756 21507 raft_consensus.cc:359] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.393481 21507 raft_consensus.cc:385] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.393759 21507 raft_consensus.cc:740] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.394075 21505 raft_consensus.cc:359] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.394850 21505 raft_consensus.cc:385] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.394549 21507 consensus_queue.cc:260] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:28:53.394888 21423 leader_election.cc:343] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.395162 21505 raft_consensus.cc:740] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.395877 21505 consensus_queue.cc:260] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.396279 21843 raft_consensus.cc:493] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:53.396661 21507 ts_tablet_manager.cc:1434] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 12:28:53.396839 21843 raft_consensus.cc:515] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.398047 21505 ts_tablet_manager.cc:1434] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.000s
I20260709 12:28:53.399034 21505 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:53.399787 21843 leader_election.cc:290] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:53.397661 21507 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:53.403239 21504 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:53.404268 21815 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:53.410029 21505 tablet_bootstrap.cc:654] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.413757 21507 tablet_bootstrap.cc:654] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.416335 21131 catalog_manager.cc:5697] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.418208 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:53.417392 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:53.421135 21506 tablet_bootstrap.cc:654] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.430047 21855 consensus_queue.cc:1048] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:53.433341 21351 leader_election.cc:304] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:53.435312 21843 raft_consensus.cc:2749] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:53.446120 21504 tablet_bootstrap.cc:654] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.446842 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:53.447566 21235 raft_consensus.cc:3060] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.479542 21505 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:53.480104 21505 ts_tablet_manager.cc:1403] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.081s	user 0.007s	sys 0.006s
I20260709 12:28:53.492841 21507 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:53.495294 21507 ts_tablet_manager.cc:1403] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.098s	user 0.030s	sys 0.025s
I20260709 12:28:53.496014 21505 raft_consensus.cc:359] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.496742 21505 raft_consensus.cc:385] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.497006 21505 raft_consensus.cc:740] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.499202 21505 consensus_queue.cc:260] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.499466 21855 consensus_queue.cc:1048] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:53.503540 21235 raft_consensus.cc:2468] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:53.504596 21505 ts_tablet_manager.cc:1434] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.003s
I20260709 12:28:53.506816 21504 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:53.499133 21861 consensus_queue.cc:1048] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:28:53.516865 21505 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:53.512465 21504 ts_tablet_manager.cc:1403] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.109s	user 0.016s	sys 0.006s
I20260709 12:28:53.520084 21507 raft_consensus.cc:359] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.522553 21507 raft_consensus.cc:385] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.522953 21507 raft_consensus.cc:740] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.523370 21843 raft_consensus.cc:493] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.523976 21507 consensus_queue.cc:260] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.526422 21506 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:53.526968 21506 ts_tablet_manager.cc:1403] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.174s	user 0.021s	sys 0.014s
I20260709 12:28:53.529923 21505 tablet_bootstrap.cc:654] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.529770 21506 raft_consensus.cc:359] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.530514 21506 raft_consensus.cc:385] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.530860 21506 raft_consensus.cc:740] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.524322 21843 raft_consensus.cc:515] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.536650 21506 consensus_queue.cc:260] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.540419 21426 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:53.542987 21741 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.543447 21687 raft_consensus.cc:2804] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:53.543638 21741 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.544198 21687 raft_consensus.cc:697] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:53.545001 21843 leader_election.cc:290] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:53.545197 21687 consensus_queue.cc:237] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.546641 21506 ts_tablet_manager.cc:1434] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:28:53.542256 21781 raft_consensus.cc:493] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.546499 21741 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:53.547325 21781 raft_consensus.cc:515] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.547832 21506 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:53.553205 21505 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:53.553721 21507 ts_tablet_manager.cc:1434] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.058s	user 0.002s	sys 0.006s
I20260709 12:28:53.559412 21505 ts_tablet_manager.cc:1403] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.043s	user 0.017s	sys 0.001s
I20260709 12:28:53.554641 21504 raft_consensus.cc:359] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.559844 21504 raft_consensus.cc:385] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.560132 21504 raft_consensus.cc:740] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.561532 21504 consensus_queue.cc:260] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.579538 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:53.580389 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:53.581173 21309 raft_consensus.cc:2468] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:53.581440 21504 ts_tablet_manager.cc:1434] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.000s
I20260709 12:28:53.582582 21504 tablet_bootstrap.cc:492] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:53.587713 21781 leader_election.cc:290] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:53.588337 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:53.589249 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:28:53.590538 21277 leader_election.cc:343] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.584025 21351 leader_election.cc:304] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:28:53.596084 21351 leader_election.cc:343] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.597886 21507 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:53.597782 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:53.599676 21747 raft_consensus.cc:2804] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:53.600274 21747 raft_consensus.cc:493] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:53.600703 21747 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.572438 21858 raft_consensus.cc:493] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.613638 21507 tablet_bootstrap.cc:654] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.615417 21506 tablet_bootstrap.cc:654] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.630867 21861 consensus_queue.cc:1048] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.633237 21505 raft_consensus.cc:359] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.634007 21505 raft_consensus.cc:385] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.634268 21505 raft_consensus.cc:740] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.636746 21131 catalog_manager.cc:5697] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:53.643923 21504 tablet_bootstrap.cc:654] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.649290 21505 consensus_queue.cc:260] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.654801 21507 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:53.655381 21507 ts_tablet_manager.cc:1403] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.083s	user 0.011s	sys 0.006s
I20260709 12:28:53.660506 21505 ts_tablet_manager.cc:1434] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.101s	user 0.007s	sys 0.000s
I20260709 12:28:53.606041 21858 raft_consensus.cc:515] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.675884 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:53.677151 21307 raft_consensus.cc:2393] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:53.692255 21505 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:53.693251 21351 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:53.706449 21507 raft_consensus.cc:359] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.698271 21843 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.709034 21507 raft_consensus.cc:385] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.720145 21507 raft_consensus.cc:740] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.714269 21506 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:53.720973 21506 ts_tablet_manager.cc:1403] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.173s	user 0.022s	sys 0.018s
I20260709 12:28:53.726833 21507 consensus_queue.cc:260] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.725993 21814 raft_consensus.cc:493] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.742893 21847 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.743482 21847 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.752288 21506 raft_consensus.cc:359] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.753564 21506 raft_consensus.cc:385] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.753871 21506 raft_consensus.cc:740] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.755059 21506 consensus_queue.cc:260] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:28:53.768067 21200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 97287us
I20260709 12:28:53.775749 21507 ts_tablet_manager.cc:1434] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.120s	user 0.017s	sys 0.000s
I20260709 12:28:53.782872 21506 ts_tablet_manager.cc:1434] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.001s
I20260709 12:28:53.739667 21814 raft_consensus.cc:515] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.807142 21506 tablet_bootstrap.cc:492] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:53.810354 21507 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:53.811174 21814 leader_election.cc:290] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:53.812136 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:53.812865 21308 raft_consensus.cc:2468] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:53.814311 21505 tablet_bootstrap.cc:654] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.797250 21741 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.814930 21741 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.815179 21747 raft_consensus.cc:515] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.817070 21741 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:53.819080 21385 raft_consensus.cc:2393] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 1: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:53.820205 21506 tablet_bootstrap.cc:654] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.821633 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:53.822479 21232 raft_consensus.cc:2393] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:53.795859 21858 leader_election.cc:290] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:53.809012 21896 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.835690 21896 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:53.836477 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:53.842583 21311 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.817086 21747 leader_election.cc:290] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:53.842494 21504 tablet_bootstrap.cc:492] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:53.845626 21504 ts_tablet_manager.cc:1403] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.263s	user 0.025s	sys 0.009s
I20260709 12:28:53.847559 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:53.884356 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:53.876410 21843 raft_consensus.cc:2749] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.887995 21351 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:53.891860 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:53.907001 21847 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:53.911681 21233 raft_consensus.cc:2468] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:53.914395 21896 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:53.916100 21506 tablet_bootstrap.cc:492] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:53.900665 21747 raft_consensus.cc:3060] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.917582 21275 leader_election.cc:304] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:53.920359 21505 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:53.921981 21507 tablet_bootstrap.cc:654] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:53.923133 21505 ts_tablet_manager.cc:1403] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.254s	user 0.020s	sys 0.002s
I20260709 12:28:53.923067 21504 raft_consensus.cc:359] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.923945 21504 raft_consensus.cc:385] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.924153 21632 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:53.924816 21632 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:53.916675 21506 ts_tablet_manager.cc:1403] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.118s	user 0.013s	sys 0.023s
I20260709 12:28:53.927011 21632 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:53.924244 21504 raft_consensus.cc:740] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.927980 21504 consensus_queue.cc:260] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.929301 21506 raft_consensus.cc:359] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.930044 21506 raft_consensus.cc:385] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.930258 21504 ts_tablet_manager.cc:1434] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.000s
I20260709 12:28:53.926086 21505 raft_consensus.cc:359] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.932585 21505 raft_consensus.cc:385] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:53.930899 21506 raft_consensus.cc:740] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.932797 21505 raft_consensus.cc:740] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:53.933631 21506 consensus_queue.cc:260] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:53.941679 21504 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:53.943907 21278 leader_election.cc:304] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:53.945465 21506 ts_tablet_manager.cc:1434] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 12:28:53.945565 21381 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.947794 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:53.941084 21202 leader_election.cc:304] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:53.933619 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:53.953128 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:53.953872 21233 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:53.953943 21741 consensus_queue.cc:1048] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:53.957620 21506 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:53.954150 21311 raft_consensus.cc:2468] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:53.968853 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:53.970018 21904 raft_consensus.cc:2749] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.970562 21505 consensus_queue.cc:260] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:53.973467 21426 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
W20260709 12:28:53.973627 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:28:53.965123 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 45842.5us
W20260709 12:28:53.974025 21200 leader_election.cc:343] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:53.960767 21747 raft_consensus.cc:2749] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.975118 21898 raft_consensus.cc:2804] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:54.003212 21898 raft_consensus.cc:493] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.003746 21898 raft_consensus.cc:3060] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.003786 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:53.989220 21505 ts_tablet_manager.cc:1434] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.066s	user 0.007s	sys 0.000s
W20260709 12:28:53.986653 21202 leader_election.cc:343] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.006892 21202 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:54.007652 21505 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:53.972271 21741 consensus_queue.cc:1048] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:28:54.007786 21862 consensus_queue.cc:1048] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:53.991525 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:54.009624 21235 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:53.979429 21632 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:54.011137 21632 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.011545 21632 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.013986 21815 consensus_queue.cc:1048] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:53.981926 21781 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.014991 21781 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:54.017127 21781 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:54.021143 21901 raft_consensus.cc:2749] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:53.979121 21384 raft_consensus.cc:2468] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:54.028028 21351 leader_election.cc:304] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:54.029728 21741 raft_consensus.cc:2804] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:54.030664 21741 raft_consensus.cc:697] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:54.031059 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:54.032016 21900 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.031725 21741 consensus_queue.cc:237] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:54.032511 21900 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:54.035027 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:54.035763 21307 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 0.
I20260709 12:28:53.996520 21459 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:53.986106 21899 consensus_queue.cc:1048] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.051s
I20260709 12:28:54.001871 21737 raft_consensus.cc:493] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.038537 21737 raft_consensus.cc:515] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:54.040799 21737 leader_election.cc:290] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:53.979737 21847 raft_consensus.cc:2804] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:54.041328 21847 raft_consensus.cc:493] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.041682 21847 raft_consensus.cc:3060] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.049546 21687 consensus_queue.cc:1048] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:54.054231 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:54.056149 21900 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:54.054790 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:54.069417 21275 leader_election.cc:343] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:28:54.069969 21202 leader_election.cc:343] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.076195 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:54.083112 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:54.091872 21459 raft_consensus.cc:2468] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
W20260709 12:28:54.097867 21277 leader_election.cc:343] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.101282 21504 tablet_bootstrap.cc:654] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.102458 21505 tablet_bootstrap.cc:654] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:54.124128 21277 leader_election.cc:343] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.124747 21277 leader_election.cc:304] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:54.134009 21507 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:54.134788 21507 ts_tablet_manager.cc:1403] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.325s	user 0.011s	sys 0.007s
W20260709 12:28:54.176637 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 77531us
I20260709 12:28:54.204574 21913 raft_consensus.cc:2749] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:54.205379 21913 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:54.205848 21913 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.206893 21848 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.207389 21848 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.209064 21632 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:54.233227 21896 consensus_queue.cc:1048] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:28:54.257550 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 67752us
I20260709 12:28:54.261870 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:54.269443 21506 tablet_bootstrap.cc:654] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.266144 21455 raft_consensus.cc:2393] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 1: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:54.258106 21632 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:54.273137 21307 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.307246 21913 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:28:54.308526 21917 log.cc:927] Time spent T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:28:54.310382 21307 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:54.327016 21507 raft_consensus.cc:359] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.328217 21507 raft_consensus.cc:385] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.328900 21507 raft_consensus.cc:740] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
W20260709 12:28:54.331108 21922 log.cc:927] Time spent T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:28:54.351755 21507 consensus_queue.cc:260] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.361076 21505 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:54.361904 21505 ts_tablet_manager.cc:1403] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.355s	user 0.016s	sys 0.013s
I20260709 12:28:54.367238 21507 ts_tablet_manager.cc:1434] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.232s	user 0.005s	sys 0.004s
W20260709 12:28:54.368055 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 104151us
I20260709 12:28:54.319926 21847 raft_consensus.cc:515] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.318293 21898 raft_consensus.cc:515] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.372493 21425 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:28:54.380893 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 78542us
I20260709 12:28:54.383417 21504 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:54.383947 21504 ts_tablet_manager.cc:1403] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.447s	user 0.016s	sys 0.009s
I20260709 12:28:54.385542 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:54.386409 21383 raft_consensus.cc:2393] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 2: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:54.393857 21924 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:54.398190 21924 raft_consensus.cc:697] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:54.399112 21924 consensus_queue.cc:237] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:54.401793 21848 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:54.403584 21506 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:54.404114 21506 ts_tablet_manager.cc:1403] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.447s	user 0.011s	sys 0.011s
I20260709 12:28:54.411743 21506 raft_consensus.cc:359] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.412472 21506 raft_consensus.cc:385] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.412729 21506 raft_consensus.cc:740] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.413344 21506 consensus_queue.cc:260] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.413775 21505 raft_consensus.cc:359] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:54.414503 21507 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Bootstrap starting.
W20260709 12:28:54.130390 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 145088us
I20260709 12:28:54.416684 21504 raft_consensus.cc:359] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:54.417351 21504 raft_consensus.cc:385] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.417613 21504 raft_consensus.cc:740] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
W20260709 12:28:54.420095 21921 log.cc:927] Time spent T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Append to log took a long time: real 0.178s	user 0.003s	sys 0.001s
I20260709 12:28:54.414522 21505 raft_consensus.cc:385] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.422114 21505 raft_consensus.cc:740] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.422856 21505 consensus_queue.cc:260] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:54.424367 21505 ts_tablet_manager.cc:1434] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 12:28:54.425388 21505 tablet_bootstrap.cc:492] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:54.427120 21847 leader_election.cc:290] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:54.429414 21815 consensus_queue.cc:1048] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:54.431125 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:54.433863 21507 tablet_bootstrap.cc:654] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.439422 21506 ts_tablet_manager.cc:1434] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.001s
I20260709 12:28:54.440241 21505 tablet_bootstrap.cc:654] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.441998 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:54.446972 21504 consensus_queue.cc:260] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:28:54.460907 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 75525.5us
I20260709 12:28:54.462396 21504 ts_tablet_manager.cc:1434] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.078s	user 0.007s	sys 0.001s
I20260709 12:28:54.468163 21504 tablet_bootstrap.cc:492] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:54.470170 21278 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:54.484375 21506 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:54.472808 21384 raft_consensus.cc:3060] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:54.487351 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 99468.4us
I20260709 12:28:54.478178 21781 raft_consensus.cc:2764] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:28:54.498624 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:54.486073 21741 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.499372 21303 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:54.499471 21741 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:54.501814 21741 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:54.503895 21903 raft_consensus.cc:493] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.504546 21903 raft_consensus.cc:515] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.507534 21903 leader_election.cc:290] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:54.483358 21898 leader_election.cc:290] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:54.513659 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:54.517455 21504 tablet_bootstrap.cc:654] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:54.623905 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 81039.2us
I20260709 12:28:54.660422 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:54.661538 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:54.661713 21237 raft_consensus.cc:2393] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:54.642059 21900 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:54.667814 21202 leader_election.cc:343] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:54.697273 21384 raft_consensus.cc:2468] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:54.672926 21234 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:54.678110 21900 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.703733 21505 tablet_bootstrap.cc:492] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:54.704658 21505 ts_tablet_manager.cc:1403] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.279s	user 0.010s	sys 0.013s
I20260709 12:28:54.707402 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:54.716352 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:54.717139 21228 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.720180 21507 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:54.720546 21900 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:54.720794 21507 ts_tablet_manager.cc:1403] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.307s	user 0.027s	sys 0.020s
I20260709 12:28:54.728371 21507 raft_consensus.cc:359] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.730163 21507 raft_consensus.cc:385] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.730477 21507 raft_consensus.cc:740] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.731215 21351 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:54.731151 21507 consensus_queue.cc:260] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.732584 21741 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:54.732792 21507 ts_tablet_manager.cc:1434] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:28:54.729290 21506 tablet_bootstrap.cc:654] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:54.733013 21741 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:54.733556 21741 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:54.735862 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 70125us
I20260709 12:28:54.760934 21931 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:54.761838 21931 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:54.764838 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:54.770327 21505 raft_consensus.cc:359] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:54.772177 21505 raft_consensus.cc:385] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:54.772464 21505 raft_consensus.cc:740] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:54.773214 21505 consensus_queue.cc:260] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:54.774333 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:54.775313 21309 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:54.781735 21202 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:54.783741 21900 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:54.784162 21900 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:54.784529 21900 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:54.785310 21452 raft_consensus.cc:2468] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:54.798281 21505 ts_tablet_manager.cc:1434] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.093s	user 0.001s	sys 0.006s
W20260709 12:28:54.960075 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 220012us
I20260709 12:28:54.984604 21505 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:55.016587 21933 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.017339 21933 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.019779 21933 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:54.992864 21848 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.020948 21848 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:55.023679 21507 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:54.995556 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:55.033057 21848 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:55.076745 21504 tablet_bootstrap.cc:492] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:55.077749 21504 ts_tablet_manager.cc:1403] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.610s	user 0.044s	sys 0.072s
I20260709 12:28:55.074193 21228 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:55.039674 21309 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:55.103793 21505 tablet_bootstrap.cc:654] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.110620 21916 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.110924 21960 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:55.111289 21916 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.111534 21960 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.113780 21916 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:55.113898 21960 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.090456 21931 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:55.125996 21506 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:55.144227 21506 ts_tablet_manager.cc:1403] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.668s	user 0.020s	sys 0.010s
W20260709 12:28:55.115152 21423 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.027s	sys 0.036s
W20260709 12:28:55.142988 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 179428us
I20260709 12:28:55.171242 21504 raft_consensus.cc:359] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:55.178161 21504 raft_consensus.cc:385] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.178572 21504 raft_consensus.cc:740] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.183435 21504 consensus_queue.cc:260] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:55.191781 21504 ts_tablet_manager.cc:1434] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.114s	user 0.005s	sys 0.002s
I20260709 12:28:55.197012 21507 tablet_bootstrap.cc:654] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:55.185180 21423 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.028s	sys 0.036s
I20260709 12:28:55.174580 21900 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.203617 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:55.219815 21949 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:55.222873 21504 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:55.224068 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.194466 21741 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.225028 21453 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:55.227864 21741 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.220180 21310 raft_consensus.cc:2393] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:28:55.237123 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:55.238270 21228 raft_consensus.cc:2468] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:55.243207 21949 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.252470 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:55.253654 21234 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:28:55.254287 21351 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:55.256755 21949 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:55.257467 21932 raft_consensus.cc:2804] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.258106 21932 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.258575 21932 raft_consensus.cc:3060] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.260103 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:55.263525 21741 consensus_queue.cc:1048] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:55.224942 21900 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.274039 21949 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.274849 21949 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.283738 21949 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:55.285358 21506 raft_consensus.cc:359] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.292460 21506 raft_consensus.cc:385] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.293756 21506 raft_consensus.cc:740] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.298230 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:55.299396 21275 leader_election.cc:304] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:55.299292 21932 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.300300 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:55.301868 21737 raft_consensus.cc:2804] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.290307 21504 tablet_bootstrap.cc:654] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.289420 21505 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:55.305119 21505 ts_tablet_manager.cc:1403] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.326s	user 0.019s	sys 0.006s
W20260709 12:28:55.311084 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 54445.4us
I20260709 12:28:55.312152 21351 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:55.303417 21916 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.326875 21506 consensus_queue.cc:260] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:55.380892 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 80014.5us
W20260709 12:28:55.394227 21275 leader_election.cc:343] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.323838 21916 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.547099 21506 ts_tablet_manager.cc:1434] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.403s	user 0.008s	sys 0.002s
I20260709 12:28:55.544585 21202 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:55.573062 21967 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:55.573933 21967 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.375434 21352 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:55.408010 21904 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.590472 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:55.551111 21505 raft_consensus.cc:359] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.543314 21843 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.587352 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:55.595307 21505 raft_consensus.cc:385] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.416688 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:55.618224 21505 raft_consensus.cc:740] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.627864 21505 consensus_queue.cc:260] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.593214 21960 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:55.646950 21305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47032 (request call id 102) took 1184 ms. Trace:
I20260709 12:28:55.629367 21310 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.647536 21305 rpcz_store.cc:276] 0709 12:28:54.462071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.467935 (+  5864us) service_pool.cc:224] Handling call
0709 12:28:55.643283 (+1175348us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b5800174098. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:55.646157 (+  2874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.646370 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.646402 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:55.646448 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:55.646488 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.646554 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.646738 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9790848}
I20260709 12:28:55.649472 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:55.650126 21305 raft_consensus.cc:2468] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:55.625371 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.583956 21967 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:28:55.584055 21350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.544s	user 0.012s	sys 0.017s
I20260709 12:28:55.612174 21843 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.413216 21737 raft_consensus.cc:697] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:55.624470 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.667073 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:55.668057 21454 raft_consensus.cc:3060] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.668301 21959 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:55.669139 21959 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.594825 21904 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:28:55.676208 21961 log.cc:927] Time spent T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 12:28:55.677024 21504 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:55.677552 21504 ts_tablet_manager.cc:1403] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.474s	user 0.041s	sys 0.056s
I20260709 12:28:55.678210 21311 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47032 (request call id 95) took 1218 ms. Trace:
I20260709 12:28:55.678977 21312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47032 (request call id 98) took 1218 ms. Trace:
I20260709 12:28:55.679109 21912 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:55.680562 21972 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.679395 21312 rpcz_store.cc:276] 0709 12:28:54.460887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.465025 (+  4138us) service_pool.cc:224] Handling call
0709 12:28:54.919003 (+453978us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b5800174098. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:54.919280 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:28:54.919749 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:54.919790 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:28:54.920378 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:28:55.294910 (+374532us) raft_consensus.cc:1645] Marking committed up to 0
0709 12:28:55.294949 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.295022 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:28:55.678668 (+383646us) raft_consensus.cc:1685] finished
0709 12:28:55.678689 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.678922 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6207,"raft.queue_time_us":1207,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":48248,"spinlock_wait_cycles":5912960,"thread_start_us":374156,"threads_started":1,"wal-append.queue_time_us":389476}
I20260709 12:28:55.681119 21972 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:28:55.681980 21350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.013s	sys 0.017s
I20260709 12:28:55.683367 21858 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:55.689308 21507 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:55.689530 21938 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.689743 21229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:55.694641 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 150174us
I20260709 12:28:55.689918 21507 ts_tablet_manager.cc:1403] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.681s	user 0.040s	sys 0.057s
I20260709 12:28:55.642982 21964 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.680008 21386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39610 (request call id 125) took 1154 ms. Trace:
I20260709 12:28:55.683846 21858 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.684245 21932 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:55.685259 21901 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:55.679917 21912 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.712515 21505 ts_tablet_manager.cc:1434] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.407s	user 0.013s	sys 0.000s
I20260709 12:28:55.716436 21504 raft_consensus.cc:359] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.717417 21504 raft_consensus.cc:385] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.717760 21504 raft_consensus.cc:740] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
W20260709 12:28:55.719022 21202 leader_election.cc:343] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:55.654541 21960 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.690039 21938 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.712042 21815 raft_consensus.cc:493] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:55.724493 21815 raft_consensus.cc:515] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:55.726059 21948 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:55.726575 21948 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.728708 21948 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.728818 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:55.731886 21504 consensus_queue.cc:260] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.733796 21383 raft_consensus.cc:2393] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:28:55.734107 21306 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47032 (request call id 100) took 1269 ms. Trace:
I20260709 12:28:55.734381 21383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39610 (request call id 127) took 1207 ms. Trace:
I20260709 12:28:55.734879 21383 rpcz_store.cc:276] 0709 12:28:54.526875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.531337 (+  4462us) service_pool.cc:224] Handling call
0709 12:28:55.728631 (+1197294us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b5800181898. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:28:55.734290 (+  5659us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7791872}
I20260709 12:28:55.734630 21306 rpcz_store.cc:276] 0709 12:28:54.461499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.465516 (+  4017us) service_pool.cc:224] Handling call
0709 12:28:55.728736 (+1263220us) spinlock_profiling.cc:243] Waited 924 us on lock 0x7b5800174098. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:55.728936 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.729135 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.729164 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:55.729229 (+    65us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:55.729260 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.729349 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.731150 (+  1801us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2126080}
I20260709 12:28:55.706794 21386 rpcz_store.cc:276] 0709 12:28:54.526103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.533993 (+  7890us) service_pool.cc:224] Handling call
0709 12:28:55.679231 (+1145238us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b5800181898. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:55.679460 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.679659 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.679688 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:55.679736 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:55.679768 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.679825 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.679929 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14633344}
I20260709 12:28:55.737278 21841 raft_consensus.cc:493] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:55.737880 21841 raft_consensus.cc:515] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:55.738879 21975 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:55.739431 21975 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.740262 21841 leader_election.cc:290] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:55.741725 21975 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.742918 21310 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:55.743521 21454 raft_consensus.cc:2468] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:55.744874 21843 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:55.707698 21964 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.746186 21964 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.712700 21858 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:55.747247 21815 leader_election.cc:290] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:55.748453 21504 ts_tablet_manager.cc:1434] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.071s	user 0.007s	sys 0.002s
I20260709 12:28:55.747978 21737 consensus_queue.cc:237] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.757123 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.757527 21351 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:55.758270 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.760794 21938 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:55.722019 21912 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:28:55.763013 21968 log.cc:927] Time spent T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:28:55.723631 21901 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.764664 21977 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.766361 21972 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:55.767323 21926 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:28:55.701233 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 379330us
I20260709 12:28:55.779284 21505 tablet_bootstrap.cc:492] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:55.854192 21814 raft_consensus.cc:493] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.678681 21311 rpcz_store.cc:276] 0709 12:28:54.460008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.461103 (+  1095us) service_pool.cc:224] Handling call
0709 12:28:55.677443 (+1216340us) spinlock_profiling.cc:243] Waited 921 us on lock 0x7b5800174098. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:55.677641 (+   198us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.677834 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.677863 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:55.677906 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:55.677941 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.678014 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.678142 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2118528}
I20260709 12:28:55.754375 21960 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.783310 21843 raft_consensus.cc:2804] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:55.886503 21923 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.887120 21923 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.879667 21814 raft_consensus.cc:515] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:55.889577 21923 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.890187 21960 raft_consensus.cc:493] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:55.890764 21960 raft_consensus.cc:515] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.891335 21814 leader_election.cc:290] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:55.891633 21901 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.894609 21916 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:55.886194 21308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47032 (request call id 99) took 1425 ms. Trace:
I20260709 12:28:55.895687 21308 rpcz_store.cc:276] 0709 12:28:54.461178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.465271 (+  4093us) service_pool.cc:224] Handling call
0709 12:28:55.881050 (+1415779us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b5800174098. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431e9c 00007f4e06431a5b 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:55.882787 (+  1737us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.885778 (+  2991us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.885814 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:55.885863 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:55.885908 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.885986 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.886127 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5838080}
I20260709 12:28:55.892774 21960 leader_election.cc:290] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:55.883002 21926 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:55.900846 21926 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.905432 21959 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:55.879954 21977 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.906705 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:55.907372 21385 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:55.908041 21977 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:28:55.909772 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 189337us
I20260709 12:28:55.909438 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:55.915828 21307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47032 (request call id 96) took 1455 ms. Trace:
I20260709 12:28:55.917940 21506 tablet_bootstrap.cc:492] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:55.910871 21843 raft_consensus.cc:697] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:55.919453 21843 consensus_queue.cc:237] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.941751 21504 tablet_bootstrap.cc:492] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:55.948709 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:55.949129 21202 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:55.950366 21896 raft_consensus.cc:2804] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:55.917349 21307 rpcz_store.cc:276] 0709 12:28:54.460319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.461385 (+  1066us) service_pool.cc:224] Handling call
0709 12:28:55.915102 (+1453717us) spinlock_profiling.cc:243] Waited 5.66 ms on lock 0x7b5800174098. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431eee 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:55.915278 (+   176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:55.915482 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:55.915508 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:55.915551 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:55.915593 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:55.915654 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:55.915771 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14921344}
I20260709 12:28:55.955569 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:55.956399 21308 raft_consensus.cc:2468] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:55.956974 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.957499 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.959031 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.959749 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:55.960177 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:55.960413 21310 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:55.960717 21862 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:55.947182 21974 raft_consensus.cc:493] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:55.961299 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.961805 21974 raft_consensus.cc:515] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.963632 21896 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:55.964097 21896 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.964339 21975 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:55.964886 21975 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.968150 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:55.968466 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.968602 21959 raft_consensus.cc:493] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:28:55.969187 21959 raft_consensus.cc:515] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:55.969401 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:55.970954 21231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:55.972167 21231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:55.973272 21231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:55.973474 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:55.974004 21231 raft_consensus.cc:2393] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:55.975003 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:55.975607 21233 raft_consensus.cc:3060] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:55.976118 21930 raft_consensus.cc:493] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:55.976624 21930 raft_consensus.cc:515] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:55.979417 21948 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.979935 21948 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.981331 21923 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:55.981850 21923 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.982231 21506 tablet_bootstrap.cc:654] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.982838 21507 raft_consensus.cc:359] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:55.983525 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.983031 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:55.985073 21457 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:55.988883 21975 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:55.992442 21505 tablet_bootstrap.cc:654] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:55.983549 21507 raft_consensus.cc:385] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:55.953069 21202 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:28:56.007638 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 54678.6us
I20260709 12:28:55.994382 21507 raft_consensus.cc:740] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:55.992486 21937 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.009749 21275 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:56.010263 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.010701 21975 raft_consensus.cc:2749] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.012192 21814 raft_consensus.cc:2804] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.012660 21814 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:56.013052 21814 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:55.961314 21862 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.026991 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.009895 21937 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:56.041249 21504 tablet_bootstrap.cc:654] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.042140 21351 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:56.039703 21202 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:56.039727 21352 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:55.990497 21959 leader_election.cc:290] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:55.964110 21974 leader_election.cc:290] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.039326 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.067266 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.067698 21958 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:56.068281 21958 raft_consensus.cc:697] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:56.068925 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.069144 21958 consensus_queue.cc:237] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.070211 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:55.994921 21904 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:56.059302 21507 consensus_queue.cc:260] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:56.040132 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.079146 21352 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:56.040062 21231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.080766 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.071537 21818 raft_consensus.cc:493] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:28:56.076622 21912 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.085052 21912 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.085490 21896 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.086166 21383 raft_consensus.cc:2393] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 3: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
W20260709 12:28:56.064817 21200 leader_election.cc:343] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.050688 21843 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:56.039841 21975 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
W20260709 12:28:56.087409 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 76769.3us
I20260709 12:28:56.087132 21200 leader_election.cc:304] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:28:56.092247 21275 leader_election.cc:343] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.086797 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:56.095343 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 86397.2us
I20260709 12:28:56.076149 21974 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.102617 21200 leader_election.cc:304] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.103255 21202 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:56.104486 21507 ts_tablet_manager.cc:1434] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.408s	user 0.013s	sys 0.000s
I20260709 12:28:56.093868 21971 raft_consensus.cc:493] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.091285 21841 raft_consensus.cc:2749] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.087062 21930 leader_election.cc:290] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.076220 21858 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.102353 21974 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:56.111733 21234 raft_consensus.cc:2393] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:56.082502 21818 raft_consensus.cc:515] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.120683 21897 raft_consensus.cc:2804] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.121196 21897 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.110850 21923 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:56.137414 21897 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.110072 21233 raft_consensus.cc:2468] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:56.122638 21507 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:56.101776 21843 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:56.117920 21974 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.121282 21818 leader_election.cc:290] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.094899 21964 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.148005 21981 raft_consensus.cc:2749] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:56.148998 21979 raft_consensus.cc:2749] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:56.112509 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.098995 21975 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.142445 21948 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:28:56.184741 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 81546.1us
I20260709 12:28:56.210067 21202 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:56.185964 21229 raft_consensus.cc:2368] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:56.187875 21937 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:28:56.225777 21200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 95370.6us
I20260709 12:28:56.188978 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:56.227207 21200 leader_election.cc:343] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.228633 21200 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:56.115763 21971 raft_consensus.cc:515] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.221025 21948 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.207096 21814 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.261655 21937 raft_consensus.cc:493] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.261838 21958 raft_consensus.cc:2749] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.262394 21937 raft_consensus.cc:515] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.265395 21937 leader_election.cc:290] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.266187 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.267349 21971 leader_election.cc:290] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.267827 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.235810 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.266240 21507 tablet_bootstrap.cc:654] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.275136 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:56.267215 21843 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:56.276223 21378 raft_consensus.cc:1240] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:56.261765 21990 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.276986 21990 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.277374 21990 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.245500 21980 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.278947 21989 raft_consensus.cc:493] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.279507 21989 raft_consensus.cc:515] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.254791 21230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.281152 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.281867 21989 leader_election.cc:290] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.241489 21896 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.242048 21938 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:56.289959 21964 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.292335 21938 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.294082 21504 tablet_bootstrap.cc:492] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:56.295425 21818 raft_consensus.cc:493] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:28:56.296042 21818 raft_consensus.cc:515] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.296363 21975 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.298310 21818 leader_election.cc:290] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.298704 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.299085 21858 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.299460 21459 raft_consensus.cc:2468] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:56.285247 21979 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.305079 21979 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:56.307749 21979 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:28:56.311508 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:56.310386 21131 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60588 (request call id 78) took 1977 ms. Trace:
I20260709 12:28:56.319932 21131 rpcz_store.cc:276] 0709 12:28:54.333511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.333686 (+   175us) service_pool.cc:224] Handling call
0709 12:28:56.310289 (+1976603us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105263,"reported_tablets":3}
I20260709 12:28:56.320940 21814 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.323836 21814 raft_consensus.cc:493] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.324409 21814 raft_consensus.cc:515] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.327440 21938 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.327575 21814 leader_election.cc:290] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.301131 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.331255 21988 raft_consensus.cc:493] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:28:56.331902 21988 raft_consensus.cc:515] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.333674 21897 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.270888 21948 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.334270 21988 leader_election.cc:290] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.335397 21988 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.336527 21948 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.336194 21988 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.337646 21232 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55834 (request call id 83) took 1113 ms. Trace:
I20260709 12:28:56.338132 21232 rpcz_store.cc:276] 0709 12:28:55.228878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.229017 (+   139us) service_pool.cc:224] Handling call
0709 12:28:56.336774 (+1107757us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b58001a2298. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:56.337012 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.337227 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.337264 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.337315 (+    51us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:28:56.337348 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.337414 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:56.337554 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4726656}
I20260709 12:28:56.339596 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:56.339610 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.340474 21303 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:56.342402 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.342824 21814 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.343931 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.343587 21814 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.346707 21814 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:56.282543 21989 raft_consensus.cc:493] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.349045 21989 raft_consensus.cc:515] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.350847 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:56.351191 21989 leader_election.cc:290] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.351511 21453 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.353683 21132 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60562 (request call id 72) took 1713 ms. Trace:
I20260709 12:28:56.354913 21974 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:56.355664 21862 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.357067 21974 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:28:56.362403 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 290663us
I20260709 12:28:56.365968 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:56.366672 21278 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:56.354100 21132 rpcz_store.cc:276] 0709 12:28:54.656646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.663398 (+  6752us) service_pool.cc:224] Handling call
0709 12:28:56.353619 (+1690221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":358945,"reported_tablets":3}
I20260709 12:28:56.364805 21202 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.365509 21505 tablet_bootstrap.cc:492] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:56.301144 21504 ts_tablet_manager.cc:1403] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.379s	user 0.029s	sys 0.013s
I20260709 12:28:56.368829 21505 ts_tablet_manager.cc:1403] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.590s	user 0.012s	sys 0.027s
I20260709 12:28:56.371197 21351 leader_election.cc:304] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:56.373106 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.373426 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.375401 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:56.375764 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:56.455672 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 91647.8us
W20260709 12:28:56.494419 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 130385us
W20260709 12:28:56.494874 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 128310us
I20260709 12:28:56.342417 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.496505 21202 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.501338 21202 leader_election.cc:304] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:56.503906 21988 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.506367 21958 raft_consensus.cc:2749] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.506803 21858 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.507602 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.508951 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.510192 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.510830 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:56.512164 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.371363 21896 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.523603 21231 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.523885 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.524152 21896 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.525693 21897 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.526896 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.527086 21453 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:56.527252 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.371121 21841 consensus_queue.cc:1048] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.143s
I20260709 12:28:56.529098 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.528959 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.529783 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.513095 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.519764 22000 raft_consensus.cc:2764] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:28:56.520201 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.520429 21230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.530686 21311 raft_consensus.cc:2468] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:56.513037 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.531270 21230 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:56.493108 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.477571 21964 raft_consensus.cc:2804] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.522346 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.522791 21308 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:28:56.523348 21991 raft_consensus.cc:2749] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.352335 21990 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.512989 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.533885 21200 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:56.477345 21980 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
W20260709 12:28:56.546432 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 50683us
I20260709 12:28:56.561904 21278 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:56.529023 21231 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:56.378026 21943 consensus_queue.cc:1048] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.160s
I20260709 12:28:56.564044 21425 leader_election.cc:304] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.555372 21505 raft_consensus.cc:359] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.571374 21505 raft_consensus.cc:385] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.571849 21505 raft_consensus.cc:740] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.526484 21896 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.567415 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.547246 21938 raft_consensus.cc:493] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.575078 21505 consensus_queue.cc:260] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.548851 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.577800 21278 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
W20260709 12:28:56.594918 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 118020us
I20260709 12:28:56.541251 21948 raft_consensus.cc:493] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.571465 21980 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:56.581817 21923 raft_consensus.cc:2749] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.582099 21507 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:56.599291 21507 ts_tablet_manager.cc:1403] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.490s	user 0.040s	sys 0.039s
I20260709 12:28:56.596015 21351 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:56.577029 21988 raft_consensus.cc:2749] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.558470 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.583635 21230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.584173 21506 tablet_bootstrap.cc:492] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:56.602560 21506 ts_tablet_manager.cc:1403] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.685s	user 0.024s	sys 0.010s
I20260709 12:28:56.584082 21990 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.558470 22000 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:28:56.586117 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.605818 21505 ts_tablet_manager.cc:1434] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.237s	user 0.006s	sys 0.003s
I20260709 12:28:56.587450 21235 raft_consensus.cc:1240] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:56.588605 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:56.589790 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.545977 21964 raft_consensus.cc:493] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.592590 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.577035 21304 raft_consensus.cc:2393] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:56.547971 21913 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:56.585003 21504 raft_consensus.cc:359] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.604439 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.551064 21900 consensus_queue.cc:1059] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.050s
I20260709 12:28:56.608568 21964 raft_consensus.cc:3060] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.602371 21967 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.608718 22000 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.617408 21900 raft_consensus.cc:3055] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.630030 21900 raft_consensus.cc:740] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:56.612252 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.606545 21234 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:28:56.610705 21351 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.610961 21913 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.631758 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:56.632095 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:56.611800 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:56.630944 21900 consensus_queue.cc:260] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.634138 21900 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.598261 21938 raft_consensus.cc:515] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.616189 21980 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:56.640829 21938 leader_election.cc:290] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.603860 21916 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.610512 21504 raft_consensus.cc:385] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.611299 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:56.611063 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:56.641636 21504 raft_consensus.cc:740] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.641593 21916 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.613313 21233 raft_consensus.cc:2468] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:56.628664 21230 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.642518 21504 consensus_queue.cc:260] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.613415 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.613559 22007 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:56.603437 21948 raft_consensus.cc:515] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.645201 21916 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:56.645115 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.646229 21948 leader_election.cc:290] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.612004 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.646971 21504 ts_tablet_manager.cc:1434] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.278s	user 0.007s	sys 0.000s
I20260709 12:28:56.613443 21506 raft_consensus.cc:359] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.638363 21507 raft_consensus.cc:359] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.634974 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.621470 21426 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.647847 21506 raft_consensus.cc:385] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.637265 21308 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:28:56.648473 21507 raft_consensus.cc:385] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:56.696020 21507 raft_consensus.cc:740] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.659863 21506 raft_consensus.cc:740] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:56.633913 21913 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.733076 22005 raft_consensus.cc:2749] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:28:56.736039 21201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:56.735884 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.736244 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.737295 21967 raft_consensus.cc:2749] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:56.737637 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.738332 21234 raft_consensus.cc:2468] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:56.738917 21964 raft_consensus.cc:515] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.739126 21507 consensus_queue.cc:260] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.741022 21964 leader_election.cc:290] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.741835 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.742270 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:56.742929 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.743510 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:56.743880 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.744772 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.745126 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:56.745478 21916 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.746210 21352 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:56.746019 21916 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.758916 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.760617 21916 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.761427 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.762367 21383 raft_consensus.cc:2393] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 2: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:56.763103 21504 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:56.765533 21946 raft_consensus.cc:2749] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.746929 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.748020 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.743541 21311 raft_consensus.cc:2468] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:56.750231 21506 consensus_queue.cc:260] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:28:56.749328 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 138769us
I20260709 12:28:56.776861 21972 raft_consensus.cc:493] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.777117 21916 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.777472 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:56.630892 22000 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:28:56.788283 21351 leader_election.cc:343] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:56.766201 21451 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.748512 21949 raft_consensus.cc:2749] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.737238 22007 raft_consensus.cc:697] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:56.794806 21507 ts_tablet_manager.cc:1434] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.195s	user 0.014s	sys 0.012s
I20260709 12:28:56.789188 21351 leader_election.cc:304] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:56.799249 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.800014 21385 raft_consensus.cc:2393] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 2: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:56.800055 21202 leader_election.cc:304] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.800953 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.801100 21901 raft_consensus.cc:2749] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.802462 21202 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.803047 21383 raft_consensus.cc:1240] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:56.803835 21202 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.804466 21959 raft_consensus.cc:2749] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.804595 21981 raft_consensus.cc:2749] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.780390 21352 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:56.805776 21202 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.787614 21972 raft_consensus.cc:515] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.806191 22007 consensus_queue.cc:237] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.807032 21959 raft_consensus.cc:493] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.807565 21959 raft_consensus.cc:515] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.808215 21972 leader_election.cc:290] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.808281 21841 raft_consensus.cc:2749] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.695920 21971 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:56.791858 22005 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.809613 21959 leader_election.cc:290] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.809950 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.810935 21507 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:28:56.811233 21994 raft_consensus.cc:2749] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.812654 21237 raft_consensus.cc:1240] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:56.809363 21971 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.792011 21896 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.815760 21896 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:56.816180 21971 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:56.696240 21235 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:55826 (request call id 118) took 2004 ms. Trace:
I20260709 12:28:56.815364 21200 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.819550 21423 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:56.820271 21960 raft_consensus.cc:2749] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:28:56.821012 21423 leader_election.cc:304] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.817822 21896 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:56.817997 21506 ts_tablet_manager.cc:1434] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.215s	user 0.006s	sys 0.001s
I20260709 12:28:56.816802 21235 rpcz_store.cc:276] 0709 12:28:54.691602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:55.869949 (+1178347us) service_pool.cc:224] Handling call
0709 12:28:56.586886 (+716937us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b58001a2298. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:56.587182 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.609184 (+ 22002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:56.650195 (+ 41011us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49824640}
I20260709 12:28:56.821491 21200 leader_election.cc:304] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.791348 21916 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.825501 21899 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.793123 21426 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.784217 21980 raft_consensus.cc:2749] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.805819 22004 raft_consensus.cc:2804] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:56.809888 22005 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:56.810369 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:56.836161 21931 consensus_queue.cc:1059] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:28:56.829245 21352 leader_election.cc:304] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:56.831172 21504 tablet_bootstrap.cc:654] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
W20260709 12:28:56.843135 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 50141us
I20260709 12:28:56.838317 21202 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:56.847355 21202 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.845580 21426 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:56.850361 21426 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:56.851233 21506 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a: Bootstrap starting.
I20260709 12:28:56.835781 21351 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:56.858194 21946 raft_consensus.cc:2804] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.858728 21946 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.859318 21946 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:28:56.845077 21278 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.193:33017 timed out after 1.844s (SENT)
I20260709 12:28:56.839157 21924 raft_consensus.cc:2749] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.836385 22000 consensus_queue.cc:1048] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.031s
I20260709 12:28:56.838420 21899 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:56.854748 21994 raft_consensus.cc:2804] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:56.878417 21994 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:28:56.878885 21994 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:56.879992 21899 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.856187 21938 raft_consensus.cc:2804] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.882068 21938 raft_consensus.cc:493] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.841470 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:56.882550 21938 raft_consensus.cc:3060] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.842453 22004 raft_consensus.cc:493] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:56.883765 22004 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:56.841391 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.860123 21901 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.841807 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.887298 21236 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:56.864568 21980 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.838856 21737 raft_consensus.cc:493] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.888983 21737 raft_consensus.cc:515] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:56.858457 21947 raft_consensus.cc:493] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:28:56.925945 21506 tablet_bootstrap.cc:654] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.926055 21947 raft_consensus.cc:515] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.926749 22000 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:56.927277 22000 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:56.928377 21947 leader_election.cc:290] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.928975 21925 raft_consensus.cc:2749] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.929371 22000 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:56.872606 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.930160 21228 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:28:56.930214 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:56.930528 21303 raft_consensus.cc:2468] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
W20260709 12:28:56.932686 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 56319.1us
I20260709 12:28:56.833983 21971 consensus_queue.cc:1048] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.028s
I20260709 12:28:56.842047 21931 raft_consensus.cc:3055] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:56.869639 21505 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:56.843139 21896 raft_consensus.cc:2749] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:56.871528 21937 consensus_queue.cc:1059] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:56.872128 21926 consensus_queue.cc:1059] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:56.840559 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.859325 21955 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:56.876884 21954 consensus_queue.cc:1048] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:56.849025 21943 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:28:56.905826 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 70114.5us
I20260709 12:28:56.872151 21916 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:56.924145 21737 leader_election.cc:290] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:56.925330 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:56.950984 21425 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:28:56.958146 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 52226.9us
I20260709 12:28:56.974282 21351 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:56.953051 21931 raft_consensus.cc:740] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:56.975459 21507 tablet_bootstrap.cc:654] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:56.973582 21979 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:56.993814 21351 leader_election.cc:304] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.974747 21955 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:56.997340 21351 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:56.974962 21943 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:57.005730 21912 raft_consensus.cc:2749] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.006191 21974 raft_consensus.cc:2749] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.007442 21943 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.011188 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:56.987819 21964 raft_consensus.cc:2749] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.015622 21931 consensus_queue.cc:260] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:57.016901 21931 raft_consensus.cc:3060] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.038018 21351 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:57.002894 21955 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:56.997592 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:57.086880 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 103387us
I20260709 12:28:57.002650 21926 raft_consensus.cc:2749] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.093441 21277 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:56.990717 21980 consensus_queue.cc:1048] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.076s
I20260709 12:28:57.020411 21979 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.121264 21277 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:57.084614 21964 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:57.155750 21964 raft_consensus.cc:697] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:57.156085 21943 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.157150 21964 consensus_queue.cc:237] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.159132 21943 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:57.124462 21988 raft_consensus.cc:2749] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.127974 21938 raft_consensus.cc:515] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:57.120947 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 187287us
I20260709 12:28:57.129597 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.132084 21277 leader_election.cc:304] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.132331 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:57.132658 22000 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.178720 21947 raft_consensus.cc:2749] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.178916 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:57.180933 22005 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:57.100073 21979 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:57.150612 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:57.153108 21504 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:57.100611 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.175789 21277 leader_election.cc:304] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.195647 21277 leader_election.cc:304] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
W20260709 12:28:57.182636 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 316877us
I20260709 12:28:57.186331 21505 tablet_bootstrap.cc:654] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.186334 21504 ts_tablet_manager.cc:1403] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.539s	user 0.010s	sys 0.009s
I20260709 12:28:57.187825 21200 leader_election.cc:304] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.200519 21277 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.166234 21955 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:28:57.210199 21200 leader_election.cc:343] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:57.177259 22004 raft_consensus.cc:515] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.214985 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:57.203902 21958 raft_consensus.cc:2749] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.218676 21381 raft_consensus.cc:1240] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:28:57.220427 21504 raft_consensus.cc:359] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.235363 21504 raft_consensus.cc:385] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.235204 21994 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.235757 21504 raft_consensus.cc:740] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.236577 21504 consensus_queue.cc:260] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.237561 21994 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:57.239045 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:57.239787 22004 leader_election.cc:290] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:57.241647 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:57.199030 21988 raft_consensus.cc:2749] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.242229 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:57.201372 21423 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
W20260709 12:28:57.243685 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 57604.4us
I20260709 12:28:57.244608 21423 leader_election.cc:304] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.244801 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.242283 21228 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.201599 21460 raft_consensus.cc:2468] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:57.246300 21943 raft_consensus.cc:2749] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.245898 22012 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.201226 22000 raft_consensus.cc:2749] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.255092 21938 leader_election.cc:290] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:57.258615 21946 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:28:57.265520 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 65098.5us
I20260709 12:28:57.234153 21955 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:57.265679 21351 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:57.267503 22004 raft_consensus.cc:2749] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.266062 21228 raft_consensus.cc:2468] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:57.269472 21505 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:57.270112 22012 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.270394 21505 ts_tablet_manager.cc:1403] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.410s	user 0.013s	sys 0.007s
W20260709 12:28:57.259953 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 414919us
I20260709 12:28:57.273165 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:57.273828 21308 raft_consensus.cc:1240] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:57.281225 21351 leader_election.cc:304] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:57.267506 21504 ts_tablet_manager.cc:1434] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.001s
I20260709 12:28:57.287968 21278 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.289592 21278 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:57.291121 21278 leader_election.cc:304] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.277602 21988 raft_consensus.cc:2749] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.286733 21847 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.301982 21504 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:57.307493 21278 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.293363 22006 raft_consensus.cc:2804] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:57.293882 21913 raft_consensus.cc:2749] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.295712 21967 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:57.303524 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:57.295346 21916 consensus_queue.cc:1048] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:28:57.310899 21988 raft_consensus.cc:2749] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.311416 22006 raft_consensus.cc:493] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.318528 21967 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.329792 21506 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a: No bootstrap required, opened a new log
I20260709 12:28:57.308578 21847 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:57.331221 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:57.363427 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:57.332031 21305 raft_consensus.cc:2468] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:57.363670 21232 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:57.308717 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:57.364916 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:57.365780 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:57.366130 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:57.367039 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:57.367677 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:57.370024 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:57.371208 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:57.342051 21504 tablet_bootstrap.cc:654] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.374891 21946 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:57.361832 21506 ts_tablet_manager.cc:1403] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a: Time spent bootstrapping tablet: real 0.524s	user 0.028s	sys 0.050s
I20260709 12:28:57.363379 21380 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39610 (request call id 137) took 1271 ms. Trace:
I20260709 12:28:57.376963 21380 rpcz_store.cc:276] 0709 12:28:56.091818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.093225 (+  1407us) service_pool.cc:224] Handling call
0709 12:28:56.093577 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.150821 (+ 57244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:56.178908 (+ 28087us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:56.186688 (+  7780us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:56.186752 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.363135 (+1176383us) spinlock_profiling.cc:243] Waited 44.4 ms on lock 0x7b54005497d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:28:57.363183 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.363313 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102195072}
I20260709 12:28:57.315598 21818 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:57.348238 22006 raft_consensus.cc:3060] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.378639 21818 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:57.379098 21818 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.311832 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:57.382179 21425 leader_election.cc:304] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.345201 21505 raft_consensus.cc:359] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.382917 21505 raft_consensus.cc:385] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.383189 21505 raft_consensus.cc:740] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.383445 21946 raft_consensus.cc:2749] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.383843 21505 consensus_queue.cc:260] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.385438 21505 ts_tablet_manager.cc:1434] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.115s	user 0.004s	sys 0.000s
I20260709 12:28:57.371341 21847 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:28:57.377781 21201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.367674 21308 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.358848 21967 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.391942 22006 raft_consensus.cc:515] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:57.396109 22006 leader_election.cc:290] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:57.400063 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:57.400831 21235 raft_consensus.cc:3060] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.401413 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:57.402268 21460 raft_consensus.cc:3060] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.403713 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.404522 21453 raft_consensus.cc:2468] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 0.
I20260709 12:28:57.407752 21351 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.335012 21507 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: No bootstrap required, opened a new log
I20260709 12:28:57.414664 21507 ts_tablet_manager.cc:1403] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.604s	user 0.012s	sys 0.014s
I20260709 12:28:57.420747 21507 raft_consensus.cc:359] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.421588 21507 raft_consensus.cc:385] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.421926 21507 raft_consensus.cc:740] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.429804 21235 raft_consensus.cc:2468] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:28:57.430263 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:57.431830 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:57.433455 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:57.434190 21309 raft_consensus.cc:3060] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.440086 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:57.449447 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 139846us
I20260709 12:28:57.349584 21352 leader_election.cc:304] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
W20260709 12:28:57.451227 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 101712us
I20260709 12:28:57.448526 21507 consensus_queue.cc:260] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.454085 21454 raft_consensus.cc:1240] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:57.455080 21980 raft_consensus.cc:2804] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:57.455641 21980 raft_consensus.cc:697] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:57.456589 21278 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.456481 21980 consensus_queue.cc:237] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.457461 21202 leader_election.cc:304] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.459626 21901 raft_consensus.cc:2749] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:28:57.459662 21229 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51230 (request call id 109) took 2812 ms (client timeout 1844 ms). Trace:
W20260709 12:28:57.460282 21229 rpcz_store.cc:269] 0709 12:28:54.646758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:54.660080 (+ 13322us) service_pool.cc:224] Handling call
0709 12:28:55.689345 (+1029265us) spinlock_profiling.cc:243] Waited 6.13 ms on lock 0x7b58001a2298. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:28:57.459438 (+1770093us) spinlock_profiling.cc:243] Waited 448 ms on lock 0x7b5400024ed4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:57.459573 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1045559424}
I20260709 12:28:57.461077 21507 ts_tablet_manager.cc:1434] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.001s
I20260709 12:28:57.462440 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:57.463138 21960 consensus_queue.cc:1059] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.464267 21960 consensus_queue.cc:1048] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:57.466329 21278 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
W20260709 12:28:57.468297 21202 leader_election.cc:343] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:57.468662 21352 leader_election.cc:304] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.467267 21990 raft_consensus.cc:3055] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:57.470382 21202 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.442874 21964 raft_consensus.cc:2749] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.469235 21988 raft_consensus.cc:2749] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:57.487520 21506 raft_consensus.cc:359] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.488849 21506 raft_consensus.cc:385] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.489217 21506 raft_consensus.cc:740] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.469866 21847 raft_consensus.cc:2749] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.491938 21235 raft_consensus.cc:1240] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:28:57.493377 21506 consensus_queue.cc:260] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.493889 21901 consensus_queue.cc:1059] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:57.495601 21506 ts_tablet_manager.cc:1434] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a: Time spent starting tablet: real 0.101s	user 0.006s	sys 0.001s
I20260709 12:28:57.496337 21818 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.478478 21974 raft_consensus.cc:2804] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:57.497371 21931 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:57.497932 21931 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.500531 21916 consensus_queue.cc:1059] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:57.481513 21896 raft_consensus.cc:2804] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:57.501533 22000 raft_consensus.cc:3055] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:57.501803 21896 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.501999 22000 raft_consensus.cc:740] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:57.502280 21896 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.481453 21308 raft_consensus.cc:2468] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:57.502976 22000 consensus_queue.cc:260] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.503700 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:57.504098 21924 raft_consensus.cc:3055] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:57.504552 21924 raft_consensus.cc:740] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:57.505353 21924 consensus_queue.cc:260] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.506299 21924 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.508041 21949 raft_consensus.cc:3055] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:57.508459 21949 raft_consensus.cc:740] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:57.509428 21304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47030 (request call id 115) took 1001 ms. Trace:
I20260709 12:28:57.509387 21949 consensus_queue.cc:260] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.509919 21304 rpcz_store.cc:276] 0709 12:28:56.507920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.511893 (+  3973us) service_pool.cc:224] Handling call
0709 12:28:57.509183 (+997290us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b5400152554. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:28:57.509359 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18928384}
I20260709 12:28:57.469393 21990 raft_consensus.cc:740] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:57.510838 21818 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:57.511442 21990 consensus_queue.cc:260] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.512482 21460 raft_consensus.cc:2468] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:57.512655 21990 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.497419 21974 raft_consensus.cc:697] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:57.515723 21974 consensus_queue.cc:237] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.518919 21967 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.521237 21967 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:57.521488 21310 raft_consensus.cc:1240] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260709 12:28:57.530573 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:57.536388 21425 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.538838 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.540479 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.541857 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.544770 21946 raft_consensus.cc:2749] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:57.546765 21931 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:57.547595 21312 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.549364 21351 leader_election.cc:304] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
W20260709 12:28:57.553514 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 80523us
I20260709 12:28:57.554025 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:57.555142 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.555827 21459 raft_consensus.cc:2393] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 1: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:57.555637 21277 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.557438 21426 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:28:57.557940 21202 leader_election.cc:343] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:57.559588 21202 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.563490 21202 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.565785 21202 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:57.566572 21277 leader_election.cc:304] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.572281 21277 leader_election.cc:304] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:57.574156 21457 raft_consensus.cc:1240] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:57.575413 21312 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:57.558452 22005 raft_consensus.cc:2804] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:57.589314 21312 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47022 (request call id 92) took 1085 ms. Trace:
I20260709 12:28:57.589740 21312 rpcz_store.cc:276] 0709 12:28:56.491233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.491516 (+   283us) service_pool.cc:224] Handling call
0709 12:28:56.588407 (+ 96891us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b5800174098. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:28:57.547425 (+959018us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b54004737a8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f3335f4 00007f4e0e51c227 00007f4e0e51bab8 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6
0709 12:28:57.576531 (+ 29106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41966720}
I20260709 12:28:57.564211 21946 raft_consensus.cc:2749] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.564393 21979 raft_consensus.cc:2749] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:57.573803 21958 raft_consensus.cc:2749] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.573071 21967 raft_consensus.cc:2749] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.574157 22025 raft_consensus.cc:2749] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.599691 21202 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:57.573312 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.583827 21925 raft_consensus.cc:3055] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:28:57.603636 21200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 104067us
I20260709 12:28:57.589447 21904 raft_consensus.cc:2749] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:57.591243 21981 raft_consensus.cc:2749] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:28:57.582901 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.598627 22005 raft_consensus.cc:697] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:57.578485 21309 raft_consensus.cc:2468] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:57.627614 21925 raft_consensus.cc:740] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:57.635062 21456 raft_consensus.cc:1240] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: []
W20260709 12:28:57.690682 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 64359.3us
I20260709 12:28:57.638085 21981 raft_consensus.cc:2804] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:57.739534 21933 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.742779 21504 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:57.752641 21504 ts_tablet_manager.cc:1403] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.456s	user 0.014s	sys 0.019s
W20260709 12:28:57.762847 21200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 134830us
W20260709 12:28:57.756690 21201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.687091 21896 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.761480 21981 raft_consensus.cc:697] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:57.759832 21933 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.789306 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:28:57.794533 21377 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39610 (request call id 143) took 1457 ms (client timeout 1930 ms). Trace:
W20260709 12:28:57.795010 21377 rpcz_store.cc:269] 0709 12:28:56.337574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.364138 (+ 26564us) service_pool.cc:224] Handling call
0709 12:28:57.789048 (+1424910us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x7b5800181898. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:28:57.794468 (+  5420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77046144}
I20260709 12:28:57.796384 21236 raft_consensus.cc:1240] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 2. Current term is 3. Ops: []
I20260709 12:28:57.801932 21896 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:57.756304 21925 consensus_queue.cc:260] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.793500 22035 raft_consensus.cc:493] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.759027 22005 consensus_queue.cc:237] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:57.822680 22035 raft_consensus.cc:515] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:57.824868 22035 leader_election.cc:290] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:57.820087 21933 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:57.821744 21504 raft_consensus.cc:359] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.827131 21504 raft_consensus.cc:385] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:57.827541 21504 raft_consensus.cc:740] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:57.830008 21504 consensus_queue.cc:260] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.847092 21386 raft_consensus.cc:1240] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:57.848951 21504 ts_tablet_manager.cc:1434] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.096s	user 0.006s	sys 0.001s
I20260709 12:28:57.831594 21925 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.849704 21386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39610 (request call id 150) took 1016 ms. Trace:
I20260709 12:28:57.858773 21505 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
I20260709 12:28:57.851644 21386 rpcz_store.cc:276] 0709 12:28:56.833055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.834632 (+  1577us) service_pool.cc:224] Handling call
0709 12:28:57.846676 (+1012044us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b5800181898. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431eee 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:57.846895 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.847628 (+   733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.849643 (+  2015us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11856256}
I20260709 12:28:57.863866 21971 consensus_queue.cc:1059] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:28:57.865289 21964 raft_consensus.cc:3055] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:57.865729 21964 raft_consensus.cc:740] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:57.866053 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:57.866678 21964 consensus_queue.cc:260] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:57.867512 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:57.867817 21964 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:57.868405 21236 raft_consensus.cc:2393] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:57.870071 21229 raft_consensus.cc:1240] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:57.871301 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:57.871796 21931 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:57.872251 21931 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:57.873258 21231 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51230 (request call id 125) took 1030 ms. Trace:
I20260709 12:28:57.874033 21931 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:57.842796 21981 consensus_queue.cc:237] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:28:57.876967 21276 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.873677 21231 rpcz_store.cc:276] 0709 12:28:56.843481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.845070 (+  1589us) service_pool.cc:224] Handling call
0709 12:28:57.872549 (+1027479us) spinlock_profiling.cc:243] Waited 7.8 ms on lock 0x7b58001a2298. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431eee 000000000036eb39 00007f4e0e4a51e0 00007f4e0e56e0c4 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:57.872713 (+   164us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.872904 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.872933 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:57.872982 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:57.873015 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.873081 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.873188 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17929088}
I20260709 12:28:57.879094 22007 consensus_queue.cc:1048] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:28:57.882310 21949 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.876112 21452 raft_consensus.cc:2393] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 1: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:57.884526 21452 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:57562 (request call id 130) took 1356 ms. Trace:
I20260709 12:28:57.884866 21452 rpcz_store.cc:276] 0709 12:28:56.528375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.528678 (+   303us) service_pool.cc:224] Handling call
0709 12:28:57.866200 (+1337522us) spinlock_profiling.cc:243] Waited 56.4 ms on lock 0x7b54000132a8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f3335f4 00007f4e0e51c227 00007f4e0e51bab8 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6
0709 12:28:57.884438 (+ 18238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129713920}
I20260709 12:28:57.886471 21458 raft_consensus.cc:1240] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:57.890027 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:57.896265 21229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:28:57.900949 21424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.901703 22010 log.cc:927] Time spent T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.895s	user 0.015s	sys 0.000s
I20260709 12:28:57.903537 21505 tablet_bootstrap.cc:654] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:57.905313 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:57.906158 21452 raft_consensus.cc:2468] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:57.912524 21351 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:57.917402 22006 raft_consensus.cc:2804] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:57.917948 22006 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:57.920442 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:57.918399 22006 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.922732 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:57.923573 21378 raft_consensus.cc:2393] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 2: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:57.924485 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:57.931818 21275 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:57.944331 21311 raft_consensus.cc:2468] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:28:57.949522 21379 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39610 (request call id 152) took 1115 ms. Trace:
I20260709 12:28:57.950029 21977 consensus_queue.cc:1048] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:28:57.974617 21278 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.977977 21979 consensus_queue.cc:1048] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:57.950299 21307 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47032 (request call id 116) took 1345 ms. Trace:
I20260709 12:28:57.980212 21307 rpcz_store.cc:276] 0709 12:28:56.605403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.606237 (+   834us) service_pool.cc:224] Handling call
0709 12:28:56.617384 (+ 11147us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:56.632603 (+ 15219us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:28:56.836633 (+204030us) log.cc:844] Serialized 14 byte log entry
0709 12:28:57.003330 (+166697us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:57.040220 (+ 36890us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:57.064531 (+ 24311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.950038 (+885507us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5400473454. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:28:57.950088 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.950248 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":290538,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":919,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":16826,"spinlock_wait_cycles":34417536,"thread_start_us":331135,"threads_started":2,"wal-append.queue_time_us":256650}
W20260709 12:28:57.959973 21201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:57.983088 21201 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:28:57.986419 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:57.987277 21977 consensus_queue.cc:1048] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:57.971076 21955 consensus_queue.cc:1059] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:28:58.002998 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:58.004958 21504 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce: Bootstrap starting.
I20260709 12:28:58.007376 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:58.007910 21230 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51230 (request call id 124) took 1167 ms. Trace:
I20260709 12:28:58.008329 21230 rpcz_store.cc:276] 0709 12:28:56.840749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.844874 (+  4125us) service_pool.cc:224] Handling call
0709 12:28:58.007203 (+1162329us) spinlock_profiling.cc:243] Waited 5.67 ms on lock 0x7b58001a2298. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:58.007402 (+   199us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:58.007580 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:58.007607 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:58.007655 (+    48us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:58.007687 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:58.007751 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:58.007855 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13048064}
I20260709 12:28:58.009347 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:58.009902 21459 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:57.991297 21236 raft_consensus.cc:1275] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:57.994067 22000 raft_consensus.cc:3060] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:57.994758 21379 rpcz_store.cc:276] 0709 12:28:56.833642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:56.834901 (+  1259us) service_pool.cc:224] Handling call
0709 12:28:57.948818 (+1113917us) spinlock_profiling.cc:243] Waited 1.54 ms on lock 0x7b5800181898. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:28:57.949018 (+   200us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:28:57.949210 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:28:57.949240 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:28:57.949280 (+    40us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:28:57.949313 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:28:57.949370 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:28:57.949466 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3533056}
I20260709 12:28:57.999893 21306 raft_consensus.cc:1275] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:58.048998 21351 leader_election.cc:304] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:58.037294 21459 raft_consensus.cc:2468] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
W20260709 12:28:58.019421 21202 leader_election.cc:343] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:58.055075 21202 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:58.059167 21228 raft_consensus.cc:1240] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:28:58.059679 21841 raft_consensus.cc:2804] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:58.059789 21933 consensus_queue.cc:1048] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:58.060492 21841 raft_consensus.cc:697] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:58.057646 21277 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:58.062522 21277 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:58.063268 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:28:58.071581 21200 leader_election.cc:343] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:58.056244 21904 raft_consensus.cc:2749] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:58.075664 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:58.082645 21916 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.066843 21971 consensus_queue.cc:1048] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:28:58.061775 21632 raft_consensus.cc:493] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:58.062525 21981 raft_consensus.cc:2804] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:28:58.074571 21200 leader_election.cc:343] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:28:58.090384 21200 leader_election.cc:304] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:58.088948 21459 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:58.070753 22025 raft_consensus.cc:2749] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.061928 21967 consensus_queue.cc:1059] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:28:58.103461 21426 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:58.093539 22039 raft_consensus.cc:2749] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:58.111510 21426 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:58.105945 21841 consensus_queue.cc:237] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:58.098542 21981 raft_consensus.cc:697] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:58.093438 21632 raft_consensus.cc:515] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:58.124527 21505 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:58.125324 21505 ts_tablet_manager.cc:1403] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.272s	user 0.019s	sys 0.004s
I20260709 12:28:58.128912 21936 raft_consensus.cc:3055] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:58.129331 21936 raft_consensus.cc:740] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:58.129439 21632 leader_election.cc:290] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:58.130260 21936 consensus_queue.cc:260] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:58.131533 21936 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.132038 21981 consensus_queue.cc:237] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.126648 21955 raft_consensus.cc:2749] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.111313 21925 raft_consensus.cc:2749] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.136497 21504 tablet_bootstrap.cc:654] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:58.092614 22006 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.136368 21237 raft_consensus.cc:1275] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:28:58.138547 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:58.136775 21977 consensus_queue.cc:1048] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:28:58.156337 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:58.154129 21977 consensus_queue.cc:1048] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:58.135000 21233 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
W20260709 12:28:58.190541 21425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:28:58.217953 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 150945us
W20260709 12:28:58.231415 21200 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 52244.3us
I20260709 12:28:58.231877 21278 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:58.233464 21278 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
W20260709 12:28:58.229547 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 167090us
I20260709 12:28:58.236624 21818 raft_consensus.cc:2749] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:28:58.263022 21379 raft_consensus.cc:2393] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 1: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:58.236388 21847 raft_consensus.cc:2749] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.268836 22006 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:28:58.269207 22047 log.cc:927] Time spent T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:28:58.275472 21278 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
W20260709 12:28:58.263015 22049 log.cc:927] Time spent T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:28:58.259765 21896 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
W20260709 12:28:58.288762 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 49171.5us
I20260709 12:28:58.290705 21277 leader_election.cc:304] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:58.292459 21947 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:58.295194 21229 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:58.295475 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:58.295940 21229 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 0.
I20260709 12:28:58.289240 21896 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.299767 21947 raft_consensus.cc:697] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:58.300849 21947 consensus_queue.cc:237] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:58.299803 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:58.303544 21456 raft_consensus.cc:2393] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 1: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:28:58.304795 21896 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:58.305217 21971 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.310214 21351 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:58.311414 21278 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:58.314491 21931 raft_consensus.cc:2749] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:58.314723 21967 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:58.315142 21967 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.315496 21967 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.317201 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:58.321312 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:58.321412 21931 consensus_queue.cc:1059] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:58.322492 21931 raft_consensus.cc:3055] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:58.322854 21931 raft_consensus.cc:740] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:58.323612 21931 consensus_queue.cc:260] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.324934 21931 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.318357 21971 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:58.327837 21971 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:58.328763 21504 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce: No bootstrap required, opened a new log
I20260709 12:28:58.329316 21504 ts_tablet_manager.cc:1403] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce: Time spent bootstrapping tablet: real 0.325s	user 0.006s	sys 0.014s
I20260709 12:28:58.330999 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:58.318351 22005 raft_consensus.cc:2749] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:28:58.331714 21234 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:28:58.333128 21352 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:58.334072 22005 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:58.334523 22005 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.334977 22005 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.306772 21904 consensus_queue.cc:1059] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:28:58.335489 21841 consensus_queue.cc:1048] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.320803 21277 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:28:58.341823 21200 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:28:58.345527 21504 raft_consensus.cc:359] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.346293 21504 raft_consensus.cc:385] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:58.346652 21504 raft_consensus.cc:740] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Initialized, Role: FOLLOWER
I20260709 12:28:58.352651 21505 raft_consensus.cc:359] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.353681 21505 raft_consensus.cc:385] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:58.354053 21505 raft_consensus.cc:740] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:58.364897 21505 consensus_queue.cc:260] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.367244 21504 consensus_queue.cc:260] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.348941 22039 raft_consensus.cc:2749] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.371882 21505 ts_tablet_manager.cc:1434] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.246s	user 0.006s	sys 0.003s
I20260709 12:28:58.372288 21305 raft_consensus.cc:1275] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 1. (term mismatch)
I20260709 12:28:58.350229 21847 raft_consensus.cc:2749] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.389958 21504 ts_tablet_manager.cc:1434] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce: Time spent starting tablet: real 0.060s	user 0.008s	sys 0.000s
I20260709 12:28:58.378883 21305 pending_rounds.cc:85] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a: Aborting all ops after (but not including) 0
I20260709 12:28:58.403045 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:58.396328 21305 pending_rounds.cc:107] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:28:58.405956 21305 raft_consensus.cc:2889] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:58.441654 22005 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:58.461349 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:58.503983 21916 raft_consensus.cc:3055] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:58.504488 21916 raft_consensus.cc:740] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:58.504875 21977 consensus_queue.cc:1048] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:58.505321 21916 consensus_queue.cc:260] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:58.507725 21425 leader_election.cc:304] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:58.537637 22005 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:58.483088 21967 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:58.541885 21937 raft_consensus.cc:2749] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.526643 21948 raft_consensus.cc:493] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:58.554232 21948 raft_consensus.cc:515] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:28:58.556916 21948 leader_election.cc:290] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:58.549971 22007 raft_consensus.cc:493] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.565793 22007 raft_consensus.cc:515] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.576083 21967 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:58.531908 21916 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.588145 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:58.589455 21309 raft_consensus.cc:2393] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 2: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:28:58.594676 21505 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666: Bootstrap starting.
W20260709 12:28:58.601343 22058 log.cc:927] Time spent T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 12:28:58.611835 22048 log.cc:927] Time spent T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce: Append to log took a long time: real 0.499s	user 0.009s	sys 0.000s
I20260709 12:28:58.622350 22007 leader_election.cc:290] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:58.624607 21899 consensus_queue.cc:1048] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.631856 22025 raft_consensus.cc:3055] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:28:58.632490 22025 raft_consensus.cc:740] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:58.636664 21896 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.637377 21896 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.639604 21896 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:58.644688 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:58.645850 21452 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:58.633689 22025 consensus_queue.cc:260] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:58.650100 22025 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.651569 21202 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:58.652966 21960 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:58.654332 21960 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.654830 21960 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.671710 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:58.672416 21457 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:28:58.675588 21182 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:28:58.677341 22030 consensus_queue.cc:1048] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:58.681739 21960 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:58.685112 21457 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:58.691634 21277 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:58.695889 22061 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:58.696060 21233 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:58.696511 22061 raft_consensus.cc:697] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:58.696795 21233 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.697479 22061 consensus_queue.cc:237] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:58.700121 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:58.700781 21234 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.693555 21131 catalog_manager.cc:5697] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } }
I20260709 12:28:58.712332 21131 catalog_manager.cc:5697] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 1 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:58.714313 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:58.715062 21131 catalog_manager.cc:5697] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } }
I20260709 12:28:58.724493 21505 tablet_bootstrap.cc:654] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666: Neither blocks nor log segments found. Creating new log.
I20260709 12:28:58.725142 21312 raft_consensus.cc:1275] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:58.704201 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:58.732489 21228 raft_consensus.cc:2468] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:58.734956 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:58.743443 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:58.744390 21456 raft_consensus.cc:2468] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 0.
I20260709 12:28:58.745705 21423 leader_election.cc:304] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:58.746861 21955 raft_consensus.cc:3060] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:58.748449 21960 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:58.751547 21352 leader_election.cc:304] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
W20260709 12:28:58.810725 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 59280.9us
I20260709 12:28:58.811736 22068 raft_consensus.cc:2804] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:28:58.812216 22068 raft_consensus.cc:493] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:58.812570 22068 raft_consensus.cc:3060] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:58.856199 21378 raft_consensus.cc:1240] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3. Ops: []
I20260709 12:28:58.879076 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:58.880667 21377 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.881697 21841 consensus_queue.cc:1048] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:58.883232 21233 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:28:58.885587 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:58.888264 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:58.888932 21451 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:28:58.885802 22076 log.cc:927] Time spent T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:28:58.903160 21234 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:28:58.904291 22062 consensus_queue.cc:1059] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:58.905313 22062 raft_consensus.cc:3055] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:28:58.905623 22062 raft_consensus.cc:740] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:28:58.906678 21955 raft_consensus.cc:2749] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:28:58.906404 22062 consensus_queue.cc:260] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:58.908813 21457 raft_consensus.cc:1275] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:28:58.909008 22062 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:58.910777 22007 consensus_queue.cc:1048] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:58.908176 21228 raft_consensus.cc:1240] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3. Ops: []
I20260709 12:28:58.922142 21505 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666: No bootstrap required, opened a new log
I20260709 12:28:58.922833 21505 ts_tablet_manager.cc:1403] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666: Time spent bootstrapping tablet: real 0.328s	user 0.010s	sys 0.012s
I20260709 12:28:58.948872 21377 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:28:58.953106 21451 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
W20260709 12:28:58.954799 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 82987.4us
I20260709 12:28:58.975018 21979 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.975741 21979 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:58.978076 21979 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:58.986399 22083 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.987317 22083 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:58.990219 21899 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:58.991030 21899 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:58.996181 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:58.996995 21456 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:59.002168 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.002264 22042 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:28:59.003671 22042 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.005754 21899 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.006585 22042 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.012653 22083 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:28:59.016903 21200 consensus_peers.cc:597] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.017472 22068 raft_consensus.cc:515] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.017930 21202 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:58.978672 22004 consensus_queue.cc:1048] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:28:59.020077 21818 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:59.020615 21818 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.021109 21896 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:59.021538 21896 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.021888 21896 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.023293 21818 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:59.023757 21505 raft_consensus.cc:359] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.024471 21505 raft_consensus.cc:385] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:28:59.024760 21505 raft_consensus.cc:740] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Initialized, Role: FOLLOWER
I20260709 12:28:59.025439 21505 consensus_queue.cc:260] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.028714 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:59.029511 21451 raft_consensus.cc:2468] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:59.033299 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.033818 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.034471 21309 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:59.043120 21505 ts_tablet_manager.cc:1434] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666: Time spent starting tablet: real 0.104s	user 0.006s	sys 0.000s
I20260709 12:28:59.047219 21899 consensus_queue.cc:1048] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:28:59.049130 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:59.049880 21460 raft_consensus.cc:3060] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:59.041164 21200 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:59.051792 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:59.055192 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.063048 21456 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:28:59.039316 22068 leader_election.cc:290] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:59.063271 21307 raft_consensus.cc:2393] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 2: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:28:59.072275 21202 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:59.076957 21277 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:59.083595 21352 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:59.088315 21202 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:59.091076 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:59.091797 21383 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:28:59.095283 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.084813 22070 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.086490 22083 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.086826 21979 raft_consensus.cc:2804] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:59.087888 21916 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:59.105755 21916 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:59.106227 21916 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.090409 22004 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.109232 22070 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.111420 22070 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.096366 22030 raft_consensus.cc:2804] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:59.112112 22030 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.112478 22030 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.121486 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.122128 22005 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.122669 22005 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.122830 21896 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.100498 21979 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.124452 21979 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.124845 21896 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.118510 22083 raft_consensus.cc:697] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 3 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:59.121480 22004 raft_consensus.cc:697] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:59.127416 22083 consensus_queue.cc:237] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.127679 22004 consensus_queue.cc:237] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.129400 21236 raft_consensus.cc:2368] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:59.121685 21377 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:59.133018 21460 raft_consensus.cc:2468] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:59.124907 22005 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.140142 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.143424 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:59.144006 21459 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.148056 21236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.153795 21459 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:28:59.154570 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:59.211735 21235 raft_consensus.cc:3060] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:28:59.212126 21131 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60588 (request call id 79) took 1134 ms. Trace:
I20260709 12:28:59.212340 21202 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:28:59.212534 21131 rpcz_store.cc:276] 0709 12:28:58.078228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.084158 (+  5930us) service_pool.cc:224] Handling call
0709 12:28:59.212068 (+1127910us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:58.809084 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:58.882848 (+ 73764us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:58.882886 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:28:58.882905 (+    19us) tablet.cc:662] Decoding operations
0709 12:28:58.883510 (+   605us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:58.883593 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:58.883619 (+    26us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:28:58.883973 (+   354us) tablet.cc:701] Row locks acquired
0709 12:28:58.883991 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:28:58.884138 (+   147us) write_op.cc:270] Start()
0709 12:28:58.884253 (+   115us) write_op.cc:276] Timestamp: P: 1783600138884092 usec, L: 0
0709 12:28:58.884274 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:28:58.884898 (+   624us) log.cc:844] Serialized 1316 byte log entry
0709 12:28:59.068520 (+183622us) op_driver.cc:464] REPLICATION: finished
0709 12:28:59.153372 (+ 84852us) write_op.cc:301] APPLY: starting
0709 12:28:59.153461 (+    89us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:59.153609 (+   148us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:59.153719 (+   110us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:59.154108 (+   389us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:59.154296 (+   188us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:28:59.154340 (+    44us) write_op.cc:312] APPLY: finished
0709 12:28:59.154526 (+   186us) log.cc:844] Serialized 34 byte log entry
0709 12:28:59.154747 (+   221us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:59.154841 (+    94us) write_op.cc:454] Released schema lock
0709 12:28:59.154881 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:59.155245 (+   364us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:59.155274 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":27531,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":84620,"apply.run_cpu_time_us":1863,"apply.run_wall_time_us":1868,"num_ops":3,"prepare.queue_time_us":9479,"prepare.run_cpu_time_us":36121,"prepare.run_wall_time_us":258406,"raft.queue_time_us":42717,"raft.run_cpu_time_us":38059,"raft.run_wall_time_us":76338,"replication_time_us":184038,"thread_start_us":163919,"threads_started":4,"wal-append.queue_time_us":104660}]]}
I20260709 12:28:59.156473 21916 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.165928 22050 consensus_queue.cc:1048] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:28:59.217257 21916 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:59.218034 21981 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.166997 21278 leader_election.cc:400] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 482723f787a14005854e038688430a80 (127.8.137.193:33017) with higher term. Message: Invalid argument: T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:59.218619 21981 raft_consensus.cc:697] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
W20260709 12:28:59.219062 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 73409.6us
I20260709 12:28:59.219499 21981 consensus_queue.cc:237] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.173841 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.176169 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.221902 21228 raft_consensus.cc:2468] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:28:59.223285 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.223907 21306 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.222115 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:59.225565 21234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:59.225971 21460 raft_consensus.cc:2368] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:59.177614 21979 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.227391 21277 leader_election.cc:400] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Vote denied by peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263) with higher term. Message: Invalid argument: T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:59.227700 21277 leader_election.cc:403] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:28:59.171854 22030 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.184988 21351 leader_election.cc:304] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
W20260709 12:28:59.228647 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 43751.4us
I20260709 12:28:59.229599 22068 raft_consensus.cc:2804] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:28:59.188787 21132 catalog_manager.cc:5697] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 2, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.230146 21236 raft_consensus.cc:2368] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 for earlier term 2. Current term is 3.
I20260709 12:28:59.229645 22030 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.209992 21135 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60570 (request call id 72) took 1131 ms. Trace:
I20260709 12:28:59.233486 21352 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:59.250497 21135 rpcz_store.cc:276] 0709 12:28:58.078959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.080052 (+  1093us) service_pool.cc:224] Handling call
0709 12:28:59.209928 (+1129876us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13116,"reported_tablets":8}
I20260709 12:28:59.242753 21979 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:59.246924 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.247846 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:59.268795 21451 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.270345 21306 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:28:59.252681 22004 raft_consensus.cc:2749] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:28:59.252806 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:28:59.272523 21377 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.233904 22068 raft_consensus.cc:697] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:59.275357 22068 consensus_queue.cc:237] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.252069 21352 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:59.279867 21972 raft_consensus.cc:2804] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:28:59.280676 21972 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.281116 21972 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:59.260913 21235 raft_consensus.cc:2468] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:28:59.286396 21451 raft_consensus.cc:2468] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:28:59.291363 21352 leader_election.cc:400] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 482723f787a14005854e038688430a80 (127.8.137.193:33017) with higher term. Message: Invalid argument: T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 for earlier term 2. Current term is 3.
I20260709 12:28:59.291808 21352 leader_election.cc:403] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:59.295836 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.296536 21307 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.310773 21899 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:59.311599 21899 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.313943 21899 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.314429 22025 consensus_queue.cc:1048] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:59.318339 21377 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:28:59.325958 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.354700 22005 raft_consensus.cc:2749] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 482723f787a14005854e038688430a80 (127.8.137.193:33017) with higher term. Message: Invalid argument: T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 for earlier term 2. Current term is 3.
I20260709 12:28:59.354774 21304 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:59.328439 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:59.337356 21307 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:28:59.356300 21456 raft_consensus.cc:2393] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:59.348603 21132 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60574 (request call id 77) took 1303 ms. Trace:
I20260709 12:28:59.349359 21134 catalog_manager.cc:5697] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.358124 21200 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:59.358371 21916 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263) with higher term. Message: Invalid argument: T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:59.358465 21134 catalog_manager.cc:5697] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } }
I20260709 12:28:59.353749 21202 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:59.360409 21979 raft_consensus.cc:2804] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.360917 21979 raft_consensus.cc:697] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:59.360074 21134 catalog_manager.cc:5697] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.361729 21979 consensus_queue.cc:237] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.364065 21134 catalog_manager.cc:5697] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 reported cstate change: term changed from 0 to 1, leader changed from <none> to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.365377 21202 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:59.357020 21972 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.392529 22030 raft_consensus.cc:2804] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.389991 21972 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.390093 22089 raft_consensus.cc:2804] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:59.402908 22089 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:59.403316 22089 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.365527 21132 rpcz_store.cc:276] 0709 12:28:58.045544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.045998 (+   454us) service_pool.cc:224] Handling call
0709 12:28:59.348537 (+1302539us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:59.316795 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:59.317297 (+   502us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:59.317327 (+    30us) write_op.cc:435] Acquired schema lock
0709 12:28:59.317348 (+    21us) tablet.cc:662] Decoding operations
0709 12:28:59.317653 (+   305us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:59.317720 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:59.317747 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:28:59.317928 (+   181us) tablet.cc:701] Row locks acquired
0709 12:28:59.317941 (+    13us) write_op.cc:260] PREPARE: finished
0709 12:28:59.318066 (+   125us) write_op.cc:270] Start()
0709 12:28:59.318157 (+    91us) write_op.cc:276] Timestamp: P: 1783600139318024 usec, L: 0
0709 12:28:59.318177 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:28:59.318774 (+   597us) log.cc:844] Serialized 540 byte log entry
0709 12:28:59.321032 (+  2258us) op_driver.cc:464] REPLICATION: finished
0709 12:28:59.321449 (+   417us) write_op.cc:301] APPLY: starting
0709 12:28:59.321539 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:59.321629 (+    90us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:59.321717 (+    88us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:59.321925 (+   208us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:59.322062 (+   137us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:28:59.322103 (+    41us) write_op.cc:312] APPLY: finished
0709 12:28:59.322284 (+   181us) log.cc:844] Serialized 22 byte log entry
0709 12:28:59.322441 (+   157us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:59.322518 (+    77us) write_op.cc:454] Released schema lock
0709 12:28:59.322555 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:59.322926 (+   371us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:59.322952 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":394116,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":253,"apply.run_cpu_time_us":1401,"apply.run_wall_time_us":1442,"num_ops":1,"prepare.queue_time_us":9267,"prepare.run_cpu_time_us":2228,"prepare.run_wall_time_us":2228,"replication_time_us":2738}]]}
I20260709 12:28:59.390373 22081 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.400980 22030 raft_consensus.cc:697] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 3 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:59.414831 22062 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:28:59.411648 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.423702 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:59.429602 21228 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:28:59.414202 22081 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.440192 22081 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.441298 21235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.441443 21916 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:28:59.442427 21916 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.446038 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:59.446457 22030 consensus_queue.cc:237] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.447785 21456 raft_consensus.cc:2368] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:59.422137 22062 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.450567 21916 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:59.454501 21277 leader_election.cc:400] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Vote denied by peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263) with higher term. Message: Invalid argument: T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:59.454934 21277 leader_election.cc:403] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:59.455590 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.459151 22062 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:28:59.461808 21306 raft_consensus.cc:3060] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:28:59.467321 22089 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.449182 21228 raft_consensus.cc:2468] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 4.
I20260709 12:28:59.470274 22089 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.472731 22062 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.460953 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.473846 21237 raft_consensus.cc:2368] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 3. Current term is 4.
I20260709 12:28:59.474843 21352 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:28:59.475656 22004 raft_consensus.cc:2804] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:28:59.476161 22004 raft_consensus.cc:697] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 4 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:28:59.477031 22004 consensus_queue.cc:237] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:28:59.482956 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:59.487641 21456 raft_consensus.cc:2393] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:59.488420 21131 catalog_manager.cc:5697] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } }
I20260709 12:28:59.491379 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.492067 21307 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:28:59.492585 21131 catalog_manager.cc:5697] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } }
I20260709 12:28:59.493934 21278 leader_election.cc:400] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Vote denied by peer 482723f787a14005854e038688430a80 (127.8.137.193:33017) with higher term. Message: Invalid argument: T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 3. Current term is 4.
I20260709 12:28:59.494293 21278 leader_election.cc:403] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:28:59.490191 21306 raft_consensus.cc:2468] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 4.
I20260709 12:28:59.497283 21818 raft_consensus.cc:2749] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 482723f787a14005854e038688430a80 (127.8.137.193:33017) with higher term. Message: Invalid argument: T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 3. Current term is 4.
I20260709 12:28:59.498644 21425 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:59.497961 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.501883 21632 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:59.502147 22062 raft_consensus.cc:2749] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263) with higher term. Message: Invalid argument: T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a for earlier term 2. Current term is 3.
I20260709 12:28:59.502394 21632 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.504285 21632 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.508371 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:28:59.512662 21306 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:59.514446 21202 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:59.515565 22089 raft_consensus.cc:2804] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:59.516057 22089 raft_consensus.cc:697] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:59.516848 22089 consensus_queue.cc:237] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.498154 21134 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60562 (request call id 73) took 1121 ms. Trace:
I20260709 12:28:59.495440 21131 catalog_manager.cc:5697] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } }
I20260709 12:28:59.520012 21134 rpcz_store.cc:276] 0709 12:28:58.393288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:28:58.393605 (+   317us) service_pool.cc:224] Handling call
0709 12:28:58.629988 (+236383us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b54006bd5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094c0090 00007f4e0955ef3a 00007f4e0f314e9c 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc 00007f4e0d8d3392 00007f4e0789148c 00007f4e078908c6
0709 12:28:59.495164 (+865176us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:28:59.425835 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:28:59.426368 (+   533us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:28:59.426408 (+    40us) write_op.cc:435] Acquired schema lock
0709 12:28:59.426426 (+    18us) tablet.cc:662] Decoding operations
0709 12:28:59.429848 (+  3422us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:28:59.429912 (+    64us) write_op.cc:641] Partition lock acquired for write op
0709 12:28:59.429938 (+    26us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:28:59.449833 (+ 19895us) tablet.cc:701] Row locks acquired
0709 12:28:59.449855 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:28:59.450005 (+   150us) write_op.cc:270] Start()
0709 12:28:59.450127 (+   122us) write_op.cc:276] Timestamp: P: 1783600139449964 usec, L: 0
0709 12:28:59.450147 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:28:59.450749 (+   602us) log.cc:844] Serialized 1705 byte log entry
0709 12:28:59.468519 (+ 17770us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b18004015c0. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e0cf8d41d 00007f4e0cf8f71a 00007f4e0cf8f6a8 00007f4e0cf8f628 00007f4e0cf8f5e3 00007f4e0cf8e7fc 00007f4e0f3c12f6 00007f4e0f3bbaa8 00007f4e09562310
0709 12:28:59.468586 (+    67us) op_driver.cc:464] REPLICATION: finished
0709 12:28:59.469092 (+   506us) write_op.cc:301] APPLY: starting
0709 12:28:59.469173 (+    81us) tablet.cc:1367] starting BulkCheckPresence
0709 12:28:59.469356 (+   183us) tablet.cc:1370] finished BulkCheckPresence
0709 12:28:59.469380 (+    24us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:28:59.469899 (+   519us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:28:59.470102 (+   203us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:28:59.470156 (+    54us) write_op.cc:312] APPLY: finished
0709 12:28:59.470396 (+   240us) log.cc:844] Serialized 40 byte log entry
0709 12:28:59.470567 (+   171us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:59.470685 (+   118us) write_op.cc:454] Released schema lock
0709 12:28:59.470725 (+    40us) write_op.cc:341] FINISH: Updating metrics
0709 12:28:59.471020 (+   295us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:28:59.471046 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":21977,"reported_tablets":16,"spinlock_wait_cycles":4673408,"child_traces":[["op",{"apply.queue_time_us":276,"apply.run_cpu_time_us":1910,"apply.run_wall_time_us":1910,"num_ops":4,"prepare.queue_time_us":123,"prepare.run_cpu_time_us":3282,"prepare.run_wall_time_us":31657,"replication_time_us":6578,"spinlock_wait_cycles":6147328}]]}
I20260709 12:28:59.536343 21632 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.545380 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:59.560153 21632 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.559538 21135 catalog_manager.cc:5697] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 1 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } }
I20260709 12:28:59.561424 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.562340 21307 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.568750 21135 catalog_manager.cc:5697] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 0 to 1, leader changed from <none> to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 1 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.580145 22030 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:59.580583 21307 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:28:59.580996 22030 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.582073 21425 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:59.582967 21955 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.583425 21955 raft_consensus.cc:697] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:59.583961 22030 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:59.584268 21955 consensus_queue.cc:237] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.588091 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.588645 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:28:59.588904 21306 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:59.589454 21456 raft_consensus.cc:2393] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:59.590415 21202 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:28:59.594852 22030 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:28:59.595346 22030 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:59.595769 22030 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:28:59.605369 22030 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.607795 22030 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:28:59.609035 21132 catalog_manager.cc:5697] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 1 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.610361 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:28:59.611196 21456 raft_consensus.cc:2393] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:28:59.612709 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.612419 21132 catalog_manager.cc:5697] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 0 to 1, leader changed from <none> to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.613698 22081 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.614183 22081 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.617426 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.617826 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.618105 21237 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:28:59.622941 21426 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:28:59.627570 22081 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:59.628471 22081 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:59.628837 22081 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.629153 22081 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.640767 21306 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:28:59.641237 21383 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:59.642012 21202 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:28:59.642802 22089 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.643605 22040 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:28:59.645138 22040 raft_consensus.cc:697] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:28:59.645149 21135 catalog_manager.cc:5697] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.646126 22040 consensus_queue.cc:237] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.665457 21132 catalog_manager.cc:5697] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.672636 21459 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:59.672793 21306 raft_consensus.cc:1275] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:59.674415 22040 consensus_queue.cc:1048] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.675203 21981 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.675987 21134 catalog_manager.cc:5697] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.678787 21134 catalog_manager.cc:5697] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.681740 21134 catalog_manager.cc:5697] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 3, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 3 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.683343 21134 catalog_manager.cc:5697] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 3, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 3 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.685122 21134 catalog_manager.cc:5697] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:28:59.695878 22004 consensus_queue.cc:1048] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:59.713419 21456 raft_consensus.cc:1275] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:59.715987 22081 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.721669 22081 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:28:59.722724 21237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:59.726536 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.727276 21307 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.730129 21632 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:28:59.730748 21632 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.732643 21981 consensus_queue.cc:1048] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.732645 21632 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.735932 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:28:59.742033 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:28:59.742813 21303 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:28:59.745447 21307 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:28:59.750118 21237 raft_consensus.cc:1275] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:59.751729 21977 consensus_queue.cc:1048] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:28:59.755993 21425 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:59.756779 22081 raft_consensus.cc:2804] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:28:59.757179 22081 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:28:59.757561 22081 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.768829 22081 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.771529 21228 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:28:59.773020 21425 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:59.775895 22004 consensus_queue.cc:1048] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:28:59.778519 21306 raft_consensus.cc:1275] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:28:59.781675 21306 pending_rounds.cc:85] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: Aborting all ops after (but not including) 1
I20260709 12:28:59.782002 21306 pending_rounds.cc:107] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:28:59.782290 21306 raft_consensus.cc:2889] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:28:59.782498 22113 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.791771 22081 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:28:59.792457 22113 raft_consensus.cc:697] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:59.800433 22113 consensus_queue.cc:237] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:28:59.815521 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:28:59.816243 21307 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:28:59.828804  8743 tablet_server.cc:179] TabletServer@127.8.137.193:0 shutting down...
I20260709 12:28:59.831365 21307 raft_consensus.cc:2468] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:28:59.828366 21131 catalog_manager.cc:5697] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:28:59.834475 21425 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:28:59.838855 22081 raft_consensus.cc:2804] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:28:59.839469 22081 raft_consensus.cc:697] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:28:59.840889 22081 consensus_queue.cc:237] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.855074 22004 consensus_queue.cc:1048] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:28:59.856593 21352 consensus_peers.cc:597] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.866091 21352 consensus_peers.cc:597] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:28:59.871505 21426 consensus_peers.cc:597] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.873282 21307 raft_consensus.cc:1275] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:28:59.876127 21426 consensus_peers.cc:597] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:28:59.884647 22004 consensus_queue.cc:1048] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:28:59.979424 21307 raft_consensus.cc:1275] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:28:59.991997 22083 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:28:59.992720 22040 consensus_queue.cc:1048] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:28:59.993240 22083 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:28:59.995842 21303 raft_consensus.cc:1275] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:29:00.007794 21352 consensus_peers.cc:597] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:00.035285 22081 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:00.036489 22081 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:00.039784 22081 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:00.062909 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:00.070046 21383 raft_consensus.cc:1275] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:00.070336 21303 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
W20260709 12:29:00.079676 21352 consensus_peers.cc:597] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:00.080710 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:00.081812 21451 raft_consensus.cc:2393] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:00.083897 21456 raft_consensus.cc:1275] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:00.088822 21304 raft_consensus.cc:1240] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:00.089617 21425 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:00.092507 22113 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:00.093021 22113 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:00.093454 22113 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:00.094573 21949 consensus_queue.cc:1048] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:00.096030 22030 consensus_queue.cc:1048] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:00.082221 22083 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:00.098403 22040 consensus_queue.cc:1048] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:29:00.117240 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:00.139605 21456 raft_consensus.cc:1240] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:00.143477 21303 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
W20260709 12:29:00.150683 21352 consensus_peers.cc:597] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:00.150808 21981 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:00.151767 21981 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:00.155095 21426 consensus_peers.cc:597] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.154964 21352 consensus_peers.cc:597] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.156945 21426 consensus_peers.cc:597] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.159318 21426 consensus_peers.cc:597] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.167137 21426 leader_election.cc:336] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:00.138301 21949 consensus_queue.cc:1059] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:29:00.169883 21981 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:00.175156 21451 raft_consensus.cc:1240] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3. Ops: []
W20260709 12:29:00.183015 22126 log.cc:927] Time spent T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:29:00.183522 21949 raft_consensus.cc:3055] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:00.187443 21949 raft_consensus.cc:740] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:00.189751 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:00.190259 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:00.191015 21949 consensus_queue.cc:260] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:00.192471 21949 raft_consensus.cc:3060] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:00.190932 21309 raft_consensus.cc:2368] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:00.202010 21304 raft_consensus.cc:1275] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:00.218773 21202 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:29:00.220512 21202 leader_election.cc:400] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173) with higher term. Message: Invalid argument: T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:00.221007 21202 leader_election.cc:403] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:00.221794 21979 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:00.222213 21979 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:00.222647 21979 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:00.223052 21981 consensus_queue.cc:1059] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:00.224431 21981 consensus_queue.cc:1243] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [LEADER]: Peer 1095bdbabc944624b91311b39a5a02ce log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:29:00.225109 21981 raft_consensus.cc:3055] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:00.225608 21981 raft_consensus.cc:740] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:00.226975 21981 consensus_queue.cc:260] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:00.222849 22083 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:29:00.203428 21352 consensus_peers.cc:597] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.254453 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 51139.7us
W20260709 12:29:00.258128 21352 consensus_peers.cc:597] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.258801 21352 consensus_peers.cc:597] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:00.261448 22116 consensus_queue.cc:1048] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:29:00.280174 21426 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.303380 21278 consensus_peers.cc:597] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:00.307112 21312 raft_consensus.cc:1240] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3. Ops: []
W20260709 12:29:00.352648 21352 consensus_peers.cc:597] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.353711 21352 consensus_peers.cc:597] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:00.362444 21981 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:00.342259 22083 raft_consensus.cc:2749] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173) with higher term. Message: Invalid argument: T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:00.395022 22113 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:00.457072 21979 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:00.465196 21979 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:00.470187 22113 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:00.483204 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 80345.1us
I20260709 12:29:00.486699 21307 raft_consensus.cc:1275] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:00.494319 21309 raft_consensus.cc:1275] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:00.506629 22113 consensus_queue.cc:1048] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:00.506991 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:00.507745 21306 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:00.512861 22113 consensus_queue.cc:1048] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:29:00.544013 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
W20260709 12:29:00.566682 21278 consensus_peers.cc:597] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.567639 21278 consensus_peers.cc:597] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.570242 21278 consensus_peers.cc:597] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.571451 21278 consensus_peers.cc:597] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:00.575973 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:00.576819 21459 raft_consensus.cc:2368] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:00.626674 21306 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:00.630254 21425 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:00.631106 22081 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:00.631651 22081 raft_consensus.cc:697] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:00.632616 22081 consensus_queue.cc:237] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:00.643512 21131 catalog_manager.cc:5697] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:00.648339 21131 catalog_manager.cc:5697] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:00.710683 21278 consensus_peers.cc:597] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.711205 21202 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 86639.3us
W20260709 12:29:00.712738 21426 consensus_peers.cc:597] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.715111 21426 consensus_peers.cc:597] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.716369 21426 consensus_peers.cc:597] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:00.717622 21426 leader_election.cc:336] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:29:00.732877 21202 leader_election.cc:400] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Vote denied by peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263) with higher term. Message: Invalid argument: T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:00.734805 21202 leader_election.cc:403] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:29:00.740916 22030 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:00.757615 22030 raft_consensus.cc:2749] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263) with higher term. Message: Invalid argument: T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:00.773917  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:00.775843  8743 tablet_replica.cc:333] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.780380  8743 raft_consensus.cc:2243] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.781266  8743 raft_consensus.cc:2272] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.780424 21131 catalog_manager.cc:5697] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 3, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:00.796866  8743 tablet_replica.cc:333] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.797963  8743 raft_consensus.cc:2243] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.801012  8743 raft_consensus.cc:2272] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.806726  8743 tablet_replica.cc:333] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.807821  8743 raft_consensus.cc:2243] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.808467  8743 raft_consensus.cc:2272] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.816071  8743 tablet_replica.cc:333] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.817116  8743 raft_consensus.cc:2243] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.817847  8743 raft_consensus.cc:2272] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.821516  8743 tablet_replica.cc:333] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.822492  8743 raft_consensus.cc:2243] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.826854  8743 raft_consensus.cc:2272] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.831934  8743 tablet_replica.cc:333] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.833160  8743 raft_consensus.cc:2243] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.834506  8743 raft_consensus.cc:2272] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.838063  8743 tablet_replica.cc:333] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.838948  8743 raft_consensus.cc:2243] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.839474  8743 pending_rounds.cc:70] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Trying to abort 1 pending ops.
I20260709 12:29:00.839727  8743 pending_rounds.cc:77] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305626171474137088 op_type: NO_OP noop_request { }
I20260709 12:29:00.840137  8743 raft_consensus.cc:2889] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:00.840488  8743 raft_consensus.cc:2272] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.844960  8743 tablet_replica.cc:333] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.845878 22030 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:00.846251  8743 raft_consensus.cc:2243] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.846082 22134 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:00.847126  8743 raft_consensus.cc:2272] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.846977 22134 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:00.846953 22030 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:00.849910 22134 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:00.850823  8743 tablet_replica.cc:333] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.851841  8743 raft_consensus.cc:2243] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.853307  8743 raft_consensus.cc:2272] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.853479 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:00.853313 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:00.858091  8743 tablet_replica.cc:333] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.859283  8743 raft_consensus.cc:2243] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.859894  8743 raft_consensus.cc:2272] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.861274 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:00.863216 22134 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:00.864053  8743 tablet_replica.cc:333] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.864008 22134 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:00.865465  8743 raft_consensus.cc:2243] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.866341  8743 raft_consensus.cc:2272] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.869831  8743 tablet_replica.cc:333] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.866397 22134 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:00.850114 22030 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:00.870894  8743 raft_consensus.cc:2243] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.872094 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:00.872200 21202 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:00.872433 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:00.874264  8743 raft_consensus.cc:2272] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.874581 22134 raft_consensus.cc:2749] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.876899 21202 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:00.877177 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:00.877485  8743 tablet_replica.cc:333] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.877727 22134 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.878934  8743 raft_consensus.cc:2243] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.879065 21202 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:00.879837 22134 raft_consensus.cc:2749] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.880268  8743 raft_consensus.cc:2272] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.883742  8743 tablet_replica.cc:333] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.884804  8743 raft_consensus.cc:2243] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.885538  8743 raft_consensus.cc:2272] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.889057  8743 tablet_replica.cc:333] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.889993  8743 raft_consensus.cc:2243] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.891074  8743 raft_consensus.cc:2272] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.894668  8743 tablet_replica.cc:333] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.895591  8743 raft_consensus.cc:2243] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.896260  8743 raft_consensus.cc:2272] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.903230  8743 tablet_replica.cc:333] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.905769  8743 raft_consensus.cc:2243] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.907335 21981 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:00.907965 21981 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:00.910363 21981 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:00.912009  8743 pending_rounds.cc:70] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Trying to abort 1 pending ops.
I20260709 12:29:00.911729 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:00.912386  8743 pending_rounds.cc:77] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305626172002402304 op_type: NO_OP noop_request { }
I20260709 12:29:00.912840  8743 raft_consensus.cc:2889] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:00.913246  8743 raft_consensus.cc:2272] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.916807  8743 tablet_replica.cc:333] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.917861  8743 raft_consensus.cc:2243] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.918560  8743 raft_consensus.cc:2272] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.920344 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:00.922016  8743 tablet_replica.cc:333] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.923439  8743 raft_consensus.cc:2243] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.924417 21202 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:00.924886  8743 raft_consensus.cc:2272] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:00.925179 21202 raft_consensus.cc:2686] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:29:00.927970  8743 tablet_replica.cc:333] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.928995  8743 raft_consensus.cc:2243] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.929708  8743 raft_consensus.cc:2272] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.933111  8743 tablet_replica.cc:333] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.934146  8743 raft_consensus.cc:2243] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.935381  8743 raft_consensus.cc:2272] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.938794  8743 tablet_replica.cc:333] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.939786  8743 raft_consensus.cc:2243] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.940446  8743 raft_consensus.cc:2272] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.944365  8743 tablet_replica.cc:333] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.945307  8743 raft_consensus.cc:2243] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.945984  8743 raft_consensus.cc:2272] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.949182  8743 tablet_replica.cc:333] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.950013  8743 raft_consensus.cc:2243] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.950639  8743 raft_consensus.cc:2272] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.953747  8743 tablet_replica.cc:333] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.954730  8743 raft_consensus.cc:2243] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.955879  8743 raft_consensus.cc:2272] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.959267  8743 tablet_replica.cc:333] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.960155  8743 raft_consensus.cc:2243] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.961270  8743 raft_consensus.cc:2272] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.964291  8743 tablet_replica.cc:333] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.965173  8743 raft_consensus.cc:2243] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.965854  8743 raft_consensus.cc:2272] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.969123  8743 tablet_replica.cc:333] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.970001  8743 raft_consensus.cc:2243] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.970955  8743 raft_consensus.cc:2272] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.974249  8743 tablet_replica.cc:333] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.975137  8743 raft_consensus.cc:2243] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.976389  8743 raft_consensus.cc:2272] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.979962  8743 tablet_replica.cc:333] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.980902  8743 raft_consensus.cc:2243] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.982193  8743 raft_consensus.cc:2272] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.982550 22134 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:00.983170 22134 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:00.985391 22134 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:00.985584  8743 tablet_replica.cc:333] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.986204 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:00.986619  8743 raft_consensus.cc:2243] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.987371  8743 raft_consensus.cc:2272] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.988641 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:00.989951 21202 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:00.990635 22134 raft_consensus.cc:2749] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:00.990978  8743 tablet_replica.cc:333] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.991901  8743 raft_consensus.cc:2243] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:00.993081  8743 raft_consensus.cc:2272] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:00.996407  8743 tablet_replica.cc:333] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:00.997352  8743 raft_consensus.cc:2243] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:00.998011  8743 raft_consensus.cc:2272] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.001096 22134 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:01.002028  8743 tablet_replica.cc:333] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.003252  8743 raft_consensus.cc:2243] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.004058  8743 raft_consensus.cc:2272] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.006038 22134 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:01.008026  8743 tablet_replica.cc:333] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.008900 22134 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:01.009672  8743 raft_consensus.cc:2243] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.010937  8743 raft_consensus.cc:2272] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.011250 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:01.014405 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:01.024940 21202 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:01.026019 22134 raft_consensus.cc:2749] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:01.033023  8743 tablet_replica.cc:333] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.034082  8743 raft_consensus.cc:2243] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.034886  8743 raft_consensus.cc:2272] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.038281  8743 tablet_replica.cc:333] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.039978  8743 raft_consensus.cc:2243] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.040568  8743 raft_consensus.cc:2272] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.043540  8743 tablet_replica.cc:333] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: stopping tablet replica
W20260709 12:29:01.044044 21426 consensus_peers.cc:597] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:01.044257  8743 raft_consensus.cc:2243] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.045461  8743 raft_consensus.cc:2272] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.048920  8743 tablet_replica.cc:333] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.049722  8743 raft_consensus.cc:2243] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.050827  8743 raft_consensus.cc:2272] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.054044  8743 tablet_replica.cc:333] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.054847  8743 raft_consensus.cc:2243] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.055408  8743 raft_consensus.cc:2272] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.058321  8743 tablet_replica.cc:333] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.058960  8743 raft_consensus.cc:2243] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.059528  8743 raft_consensus.cc:2272] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.062615  8743 tablet_replica.cc:333] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.063263  8743 raft_consensus.cc:2243] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.064319  8743 raft_consensus.cc:2272] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.067368  8743 tablet_replica.cc:333] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.068104  8743 raft_consensus.cc:2243] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.068658  8743 raft_consensus.cc:2272] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.071383  8743 tablet_replica.cc:333] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.071951  8743 raft_consensus.cc:2243] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:01.073001  8743 raft_consensus.cc:2272] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.075946  8743 tablet_replica.cc:333] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: stopping tablet replica
I20260709 12:29:01.076556  8743 raft_consensus.cc:2243] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:01.077046  8743 raft_consensus.cc:2272] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:01.173173  8743 tablet_server.cc:196] TabletServer@127.8.137.193:0 shutdown complete.
I20260709 12:29:01.199549 21307 raft_consensus.cc:1275] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:01.201241 22113 consensus_queue.cc:1048] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:01.323577  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:01.340911 22153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:01.356564  8743 server_base.cc:1061] running on GCE node
W20260709 12:29:01.357503 22154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:01.357662 22156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:01.365548  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:01.365919  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:01.366174  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600141366157 us; error 0 us; skew 500 ppm
I20260709 12:29:01.367058  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:01.370888  8743 webserver.cc:533] Webserver started at http://127.8.137.193:41199/ using document root <none> and password file <none>
I20260709 12:29:01.371807  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:01.372083  8743 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:29:01.378051  8743 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.007s	sys 0.000s
I20260709 12:29:01.389019 22161 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:01.390082  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:29:01.390527  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "482723f787a14005854e038688430a80"
format_stamp: "Formatted at 2026-07-09 12:28:43 on dist-test-slave-kpwv"
I20260709 12:29:01.391079  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:01.474351  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:01.475797  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:01.477886  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:01.484826 22169 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:29:01.669157 22113 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:01.669817 22113 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:01.672348 22113 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:01.674425 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
W20260709 12:29:01.678865 21426 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:01.679307 21426 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:01.680007 22113 raft_consensus.cc:2749] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:01.823971  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:29:01.824378  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.341s	user 0.008s	sys 0.001s
I20260709 12:29:01.824832  8743 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:29:01.831609 22169 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Bootstrap starting.
W20260709 12:29:01.856978 21426 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111) [suppressed 109 similar messages]
I20260709 12:29:01.864380 22169 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:01.866303 22169 tablet_bootstrap.cc:492] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:01.867141 22169 ts_tablet_manager.cc:1403] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.036s	user 0.025s	sys 0.004s
I20260709 12:29:01.875042 22169 raft_consensus.cc:359] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:01.877174 22169 raft_consensus.cc:740] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.878247 22169 consensus_queue.cc:260] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:01.896471 22169 ts_tablet_manager.cc:1434] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.014s
I20260709 12:29:01.897516 22169 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:01.916510 22169 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:01.917537 22169 tablet_bootstrap.cc:492] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:01.918218 22169 ts_tablet_manager.cc:1403] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.000s
I20260709 12:29:01.921098 22169 raft_consensus.cc:359] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:01.921873 22169 raft_consensus.cc:740] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.922664 22169 consensus_queue.cc:260] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:01.925004 22169 ts_tablet_manager.cc:1434] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.004s
I20260709 12:29:01.925823 22169 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:01.949187 22169 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:29:01.950232 22169 tablet_bootstrap.cc:492] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:01.950941 22169 ts_tablet_manager.cc:1403] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.025s	user 0.016s	sys 0.008s
I20260709 12:29:01.956151 22169 raft_consensus.cc:359] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:01.957080 22169 raft_consensus.cc:740] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:01.957962 22169 consensus_queue.cc:260] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:01.960042 22169 ts_tablet_manager.cc:1434] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:29:01.961145 22169 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:02.006086 22169 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:02.007160 22169 tablet_bootstrap.cc:492] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:02.008170 22169 ts_tablet_manager.cc:1403] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.047s	user 0.019s	sys 0.007s
I20260709 12:29:02.012672 22169 raft_consensus.cc:359] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.013458 22169 raft_consensus.cc:740] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.014214 22169 consensus_queue.cc:260] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.016654 22169 ts_tablet_manager.cc:1434] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:02.017517 22169 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:02.036675 22169 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:02.037722 22169 tablet_bootstrap.cc:492] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:02.038432 22169 ts_tablet_manager.cc:1403] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.001s
I20260709 12:29:02.041369 22169 raft_consensus.cc:359] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.042127 22169 raft_consensus.cc:740] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.042946 22169 consensus_queue.cc:260] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.045125 22169 ts_tablet_manager.cc:1434] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:02.045998 22169 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:02.046092 22025 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.046799 22025 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.052450 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:29:02.057785 21278 leader_election.cc:336] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.058250 21278 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:02.059056 22025 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.059689 22163 raft_consensus.cc:2749] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:02.066843 22068 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.072981 22068 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.067587 21977 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.075397 21977 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.077842 21977 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.079907 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:02.081871 22113 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.082408 22113 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.087469 22081 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.088081 22081 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.090140 22081 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.092135 22113 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.093248  8743 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:29:02.093636  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.269s	user 0.265s	sys 0.000s
I20260709 12:29:02.094842 21977 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
W20260709 12:29:02.095929 21426 leader_election.cc:336] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.095602 21977 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.097888 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:02.098568 21303 raft_consensus.cc:2468] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:02.099303 22068 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.099676 21977 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.101739 21425 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:02.102429 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:02.107059 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:29:02.107820 21426 leader_election.cc:336] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.096490 22163 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.110236 22163 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.111467 22081 raft_consensus.cc:2804] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:02.113111 22081 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.113487 22081 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:02.113897 22163 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:02.122210 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.123314 21377 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:02.133841 21275 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:02.115258 22068 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.134809 22025 raft_consensus.cc:2804] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:02.135933 22025 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.136327 22025 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:29:02.138880 21278 leader_election.cc:336] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.142786 22068 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.145080 22068 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.155586 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.160761 22025 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.167447 22025 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:02.164079 21423 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
W20260709 12:29:02.165599 21352 leader_election.cc:336] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.169997 21352 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:02.169435 22113 raft_consensus.cc:2749] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.154430 22081 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.173460 22081 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.180116 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:02.180896 21306 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:29:02.183781 21426 leader_election.cc:336] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.192770 21977 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:02.179145 21278 leader_election.cc:336] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.198452 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 72675us
I20260709 12:29:02.201820 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.202440 21383 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:29:02.215874 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 99175.2us
I20260709 12:29:02.217425 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:02.219667 21306 raft_consensus.cc:2468] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 4.
I20260709 12:29:02.220283 22169 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:02.220712 21383 raft_consensus.cc:2468] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 4.
W20260709 12:29:02.227126 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 61555.9us
I20260709 12:29:02.229954 22169 tablet_bootstrap.cc:492] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:02.229919 21425 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
W20260709 12:29:02.230139 21352 leader_election.cc:336] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.231009 22169 ts_tablet_manager.cc:1403] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.185s	user 0.029s	sys 0.035s
I20260709 12:29:02.230939 21352 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:02.231295 22113 raft_consensus.cc:2804] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:02.231822 22113 raft_consensus.cc:697] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 4 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:02.232692 22113 consensus_queue.cc:237] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.234051 22169 raft_consensus.cc:359] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.235917 22169 raft_consensus.cc:740] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.236619 22169 consensus_queue.cc:260] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.239392 22169 ts_tablet_manager.cc:1434] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:02.240195 22169 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Bootstrap starting.
W20260709 12:29:02.245132 21352 leader_election.cc:336] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.246151 21352 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:02.248293 22081 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.249130 22081 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.251348 22183 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.251897 22183 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.252995 22163 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.251876 22081 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.255172 22163 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.255342 22081 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.277930 22068 raft_consensus.cc:2749] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:02.309852 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 80983.6us
I20260709 12:29:02.300419 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
W20260709 12:29:02.313512 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 68639.8us
I20260709 12:29:02.319862 22185 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.320874 22185 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.306233 22180 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.311039 22181 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.312041 22183 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.312516 21977 raft_consensus.cc:2749] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.334666 22187 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.337275 22193 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.301573 22081 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:02.357731 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 64740.6us
I20260709 12:29:02.358433 22187 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.327957 21306 raft_consensus.cc:2468] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:02.360018 22193 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.365032 22193 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.361723 22113 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
W20260709 12:29:02.367626 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 54921.5us
I20260709 12:29:02.361210 22181 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.368592 21275 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.357086 22182 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.381852 22182 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.382640 22179 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.383221 22179 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.384217 22182 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.361444 22177 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.387036 22191 raft_consensus.cc:2804] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:02.341522 22180 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.393736 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:02.373641 22113 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.395387 22180 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.400805 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:02.401525 21451 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:02.403730 22081 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.401659 22181 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.406059 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:29:02.406380 21278 leader_election.cc:336] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.408187 22185 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:02.396651 22163 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.387809 22177 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:29:02.410416 21278 leader_election.cc:336] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.411033 21278 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.411516 21278 consensus_peers.cc:597] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.412150 21278 leader_election.cc:336] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.396311 22179 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.416493 22163 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.417961 21132 catalog_manager.cc:5697] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 3 to 4, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 4 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.418624 22163 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.419855 22187 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.411995 22177 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.420763 22163 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:02.416782 22113 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.423688 22186 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.415838 21451 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:02.424276 22186 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.415413 22191 raft_consensus.cc:697] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 4 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
W20260709 12:29:02.427098 21352 leader_election.cc:336] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.427716 21352 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
W20260709 12:29:02.431852 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 49834.8us
I20260709 12:29:02.435142 22200 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.435807 22200 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.436331 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:02.437619 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.438313 22200 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:02.439857 22169 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:02.439628 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.440392 21378 raft_consensus.cc:2468] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:02.440877 22169 tablet_bootstrap.cc:492] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:02.438422 21383 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:02.441609 22169 ts_tablet_manager.cc:1403] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.202s	user 0.014s	sys 0.026s
I20260709 12:29:02.426148 22187 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.442684 22187 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.443029 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.444918 22187 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.445451 22195 raft_consensus.cc:2749] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.446372 21377 raft_consensus.cc:2468] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:02.448979 22186 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:02.453105 21352 leader_election.cc:336] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.453976 21352 leader_election.cc:336] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.454907 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:02.455415 22197 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.455672 21451 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:02.455960 22197 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.456779 21425 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:02.460099 21423 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.460163 21277 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:02.461109 21423 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.461896 21423 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.462172 22193 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.462777 22113 raft_consensus.cc:2804] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.463290 22113 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.462983 22193 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:02.464151 21278 leader_election.cc:336] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.464726 21278 leader_election.cc:336] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.464813 22190 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.465304 22190 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:02.465840 21352 leader_election.cc:336] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.466004 22193 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.466780 22188 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:02.467188 22188 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.467535 22188 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:02.470255 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.461910 21277 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.471065 21377 raft_consensus.cc:2468] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:02.458271 22197 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:02.462242 21426 leader_election.cc:336] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.471889 22180 raft_consensus.cc:2749] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.446014 22169 raft_consensus.cc:359] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.472905 22180 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.473492 22180 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.473583 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.474050 22180 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.474350 21377 raft_consensus.cc:2468] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
W20260709 12:29:02.475991 21352 leader_election.cc:336] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.476850 21423 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.471087 22194 raft_consensus.cc:2804] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.477664 22197 raft_consensus.cc:2804] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.477674 22194 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.478108 22197 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.478609 22197 raft_consensus.cc:3060] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.481432 22191 consensus_queue.cc:237] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.478226 22194 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.485700 22185 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.486510 22185 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.488097 22197 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.489225 22185 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.473088 22169 raft_consensus.cc:740] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:02.491071 22169 consensus_queue.cc:260] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.491811 22081 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.492411 21423 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.492327 22081 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.463716 22113 raft_consensus.cc:3060] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.495218 22169 ts_tablet_manager.cc:1434] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.053s	user 0.009s	sys 0.000s
I20260709 12:29:02.499094 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:02.508791 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.509513 21377 raft_consensus.cc:3060] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.509910 21309 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:02.458544 22187 raft_consensus.cc:2804] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.510682 22190 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.503396 22193 raft_consensus.cc:2804] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.536707 21351 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:02.540889 21351 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.533492 22187 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.532716 22197 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.544675 21277 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
W20260709 12:29:02.546029 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 53708.9us
I20260709 12:29:02.545531 22199 raft_consensus.cc:2749] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.538992 22193 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.543463 22187 raft_consensus.cc:3060] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.548242 22193 raft_consensus.cc:3060] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.546530 22179 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.551609 22169 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:02.551347 22179 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.552755 22200 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:02.553149 22200 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.553498 22200 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:02.553586 22179 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.555722 22113 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.557894 22113 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.560279 22081 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.577104 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:02.577566 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:02.577910 21306 raft_consensus.cc:2468] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:29:02.577188 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:02.579583 21309 raft_consensus.cc:2468] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:02.584421 22179 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.585078 22179 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.578384 21308 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
W20260709 12:29:02.589661 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 48939.7us
I20260709 12:29:02.570685 22191 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.592355 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.592216 22191 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.593048 21380 raft_consensus.cc:2468] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:02.594333 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.595023 21386 raft_consensus.cc:2468] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:02.594539 22193 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.596767 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.597352 21379 raft_consensus.cc:2468] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:02.597481 22193 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:02.591305 21352 leader_election.cc:336] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.599807 21352 consensus_peers.cc:597] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.600626 21352 consensus_peers.cc:597] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.601219 21352 consensus_peers.cc:597] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.599890 21275 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.566691 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:02.606917 21275 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.613905 21275 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
W20260709 12:29:02.604866 21426 consensus_peers.cc:597] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.562260 22187 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.599320 22202 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:02.616549 21303 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:29:02.621861 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:29:02.615962 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 68764.5us
I20260709 12:29:02.607977 22194 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.622772 21451 raft_consensus.cc:2468] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:29:02.623369 22202 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.623780 22202 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:02.624805 22194 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.621551 22179 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:02.616595 21426 leader_election.cc:336] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.613487 22185 raft_consensus.cc:2804] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.635612 22202 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.636344 22185 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.636771 22185 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.638183 22202 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.643134 21377 raft_consensus.cc:2468] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:02.649497 22188 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.626073 22200 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.651554 22188 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.653056 22200 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.654703 22187 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.619995 21351 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.658450 22185 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.659267 22203 raft_consensus.cc:2804] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.659636 22203 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.660001 22203 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.662271 22185 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:29:02.663398 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 43480.1us
W20260709 12:29:02.628974 21426 leader_election.cc:336] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.627322 21131 catalog_manager.cc:5697] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 3 to 4, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 4 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.617188 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:02.630044 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.666316 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.626356 22163 raft_consensus.cc:2804] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:02.666777 21385 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:02.667207 22163 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.667721 22163 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:29:02.632180 21278 consensus_peers.cc:597] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.668289 21278 leader_election.cc:336] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.668996 21278 consensus_peers.cc:597] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.669512 21278 consensus_peers.cc:597] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.616004 22183 raft_consensus.cc:2804] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.670833 22183 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.671394 22183 raft_consensus.cc:3060] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.635875 22191 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:02.665249 21351 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.668303 21275 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:02.676138 22191 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.676795 22191 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
W20260709 12:29:02.676213 21426 leader_election.cc:336] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.677278 22191 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.675202 22199 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:29:02.663275 21352 leader_election.cc:336] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.673941 22203 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.679963 22199 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.685716 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:02.686681 21303 raft_consensus.cc:3060] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.686187 22199 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.690397 21425 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.691883 22188 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:02.692348 22188 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.692811 22188 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:02.695232 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:02.696484 21451 raft_consensus.cc:2393] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 4: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:02.696846 22183 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.701095 22188 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:02.701431 21278 leader_election.cc:336] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.704859 21425 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:29:02.680907 21352 consensus_peers.cc:597] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.705731 21425 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:29:02.705878 21352 consensus_peers.cc:597] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.683681 21377 raft_consensus.cc:2393] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 2: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:29:02.707489 22081 raft_consensus.cc:2804] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.710443 22081 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.681351 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:02.710978 22081 raft_consensus.cc:3060] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.711566 21306 raft_consensus.cc:2393] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 2: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:02.712980 22209 raft_consensus.cc:2804] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.713515 21351 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.713604 22209 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.714334 22209 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.716920 21423 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.718477 22190 raft_consensus.cc:2804] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.719096 22190 raft_consensus.cc:697] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:02.720099 22190 consensus_queue.cc:237] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.685437 22203 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.729310 21977 raft_consensus.cc:2804] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.729792 21977 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.730193 21977 raft_consensus.cc:3060] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:02.732182 21278 consensus_peers.cc:597] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.680451 22180 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.730506 22188 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.742352 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:02.742614 21307 raft_consensus.cc:1275] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:02.724370 22191 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.743441 21306 raft_consensus.cc:2393] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 4: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:02.744154 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.744831 21377 raft_consensus.cc:3060] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.745177 21977 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.746584 22191 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:29:02.741458 21278 leader_election.cc:336] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.749396 21278 leader_election.cc:336] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.747771 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.749959 22188 consensus_queue.cc:1048] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:02.750224 21278 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:02.751637 22200 raft_consensus.cc:2749] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.749164 22163 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.737867 22209 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.755635 21309 raft_consensus.cc:2393] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 3: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:02.754703 22183 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:29:02.727917 21352 consensus_peers.cc:597] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.758388 21352 consensus_peers.cc:597] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.758811 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:02.759467 21377 raft_consensus.cc:2468] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:02.766950 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:02.747295 21977 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.767711 21451 raft_consensus.cc:3060] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.770373 21275 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:02.771075 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
W20260709 12:29:02.764297 21352 leader_election.cc:336] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.770216 22081 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:02.773424 21352 consensus_peers.cc:597] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.775807 21278 leader_election.cc:336] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.776016 22081 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.776840 21351 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:02.777873 21977 raft_consensus.cc:2749] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.778367 21351 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:02.779357 21977 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:02.787240 21307 raft_consensus.cc:2393] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 2: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:02.792518 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:02.771118 22180 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.799505 22163 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:02.800279 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.806185 21459 raft_consensus.cc:2393] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:02.766906 21309 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.829118 21131 catalog_manager.cc:5697] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:02.811595 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:02.831918 21312 raft_consensus.cc:3060] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:02.832358 21278 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.832929 21278 leader_election.cc:336] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.833395 21278 consensus_peers.cc:597] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.833766 21278 leader_election.cc:336] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.834975 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:29:02.835428 21426 leader_election.cc:336] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.835639 21377 raft_consensus.cc:3060] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:02.836184 21426 leader_election.cc:336] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.838176 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:02.838858 21460 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.814985 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.840188 21378 raft_consensus.cc:3060] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.815812 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:02.757000 22209 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.842036 21379 raft_consensus.cc:2468] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
W20260709 12:29:02.842273 21352 leader_election.cc:336] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.842844 21352 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:02.846237 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.846714 22202 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:02.819551 22199 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.848016 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:02.848814 21455 raft_consensus.cc:2393] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 4: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
W20260709 12:29:02.848925 21352 consensus_peers.cc:597] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.849423 21352 consensus_peers.cc:597] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.849498 22199 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:02.847462 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:29:02.849817 21352 leader_election.cc:336] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.850193 21380 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.850265 21352 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:02.849677 21385 raft_consensus.cc:2393] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 2: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:29:02.855826 22199 raft_consensus.cc:2749] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.857049 21381 raft_consensus.cc:2393] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 4: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:29:02.857273 21275 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:02.824600 21451 raft_consensus.cc:2468] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:29:02.869112 21377 raft_consensus.cc:2468] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:02.869958 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:02.873138 21423 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.874692 22190 raft_consensus.cc:2804] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:02.875108 22190 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:02.875521 22190 raft_consensus.cc:3060] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.876103 21378 raft_consensus.cc:2468] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:02.878381 21977 raft_consensus.cc:2749] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:29:02.879179 21352 leader_election.cc:336] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:02.880944 21278 consensus_peers.cc:597] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:02.883077 21423 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
W20260709 12:29:02.863050 21426 consensus_peers.cc:597] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.791558 22183 raft_consensus.cc:2804] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.887408 21423 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.888279 22188 raft_consensus.cc:2804] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.890921 22188 raft_consensus.cc:697] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:02.891849 22188 consensus_queue.cc:237] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.894273 21309 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
W20260709 12:29:02.895387 21352 leader_election.cc:336] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.897063 22183 raft_consensus.cc:697] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:02.898144 22183 consensus_queue.cc:237] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.900168 21312 raft_consensus.cc:2468] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:02.900072 22190 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:02.901870 21426 leader_election.cc:336] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.867810 22180 raft_consensus.cc:2749] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.902622 21351 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.908757 21380 raft_consensus.cc:2468] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:02.903568 21386 raft_consensus.cc:2468] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:02.903103 22209 raft_consensus.cc:2804] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.913803 22209 raft_consensus.cc:697] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:02.906654 21425 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:02.914736 22209 consensus_queue.cc:237] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:02.915208 21275 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:02.915592 21425 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:02.916363 22081 raft_consensus.cc:2804] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.916669 22200 raft_consensus.cc:2804] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:02.916827 22081 raft_consensus.cc:697] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:02.917438 22200 raft_consensus.cc:697] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:02.907296 21303 raft_consensus.cc:2468] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:02.911273 21460 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:02.912581 21977 raft_consensus.cc:2804] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.921551 21977 raft_consensus.cc:697] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:02.921581 21307 raft_consensus.cc:2393] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 2: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:02.917865 22081 consensus_queue.cc:237] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.918905 22187 raft_consensus.cc:2804] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.920256 22200 consensus_queue.cc:237] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:02.930866 21351 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:02.925550 21423 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.926908 21977 consensus_queue.cc:237] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:02.905078 21426 consensus_peers.cc:597] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:02.933386 21426 consensus_peers.cc:597] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:02.925089 22190 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:02.947243 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:02.947975 21386 raft_consensus.cc:3060] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:02.934075 22203 raft_consensus.cc:2749] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:02.936757 22187 raft_consensus.cc:697] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:02.949728 21425 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:29:02.954744 21426 leader_election.cc:336] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:02.962844 21386 raft_consensus.cc:2468] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:02.965773 22187 consensus_queue.cc:237] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:02.966771 22169 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:02.987886 21423 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:02.955399 22190 raft_consensus.cc:2804] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.978399 22212 raft_consensus.cc:2804] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:02.992502 22169 tablet_bootstrap.cc:492] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:02.993341 22169 ts_tablet_manager.cc:1403] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.442s	user 0.025s	sys 0.018s
I20260709 12:29:02.993949 22190 raft_consensus.cc:697] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:03.005756 22190 consensus_queue.cc:237] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:03.012652 21277 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:02.993608 22194 raft_consensus.cc:2804] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.013489 22212 raft_consensus.cc:697] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:03.019006 22180 raft_consensus.cc:2749] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.056281 21386 raft_consensus.cc:1275] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:03.058416 22185 consensus_queue.cc:1048] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:29:03.063413 21426 leader_election.cc:336] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.064853 21426 consensus_peers.cc:597] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:03.065420 21426 leader_election.cc:336] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.066260 21426 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:03.066807 21426 leader_election.cc:336] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:03.067274 21426 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
W20260709 12:29:03.068745 22224 log.cc:927] Time spent T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:29:03.074254 22212 consensus_queue.cc:237] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:03.102666 21426 leader_election.cc:336] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.102188 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 84664.7us
I20260709 12:29:03.107916 21277 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:03.105764 21132 catalog_manager.cc:5697] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.115329 21277 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
W20260709 12:29:03.115775 21426 leader_election.cc:336] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:03.116343 21426 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:03.073432 22209 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:03.119746 22232 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:03.117229 22025 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:03.121059 22025 raft_consensus.cc:697] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:03.121213 22233 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:03.122615 22025 consensus_queue.cc:237] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:03.118074 22169 raft_consensus.cc:359] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:03.131171 22169 raft_consensus.cc:740] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.134996 22169 consensus_queue.cc:260] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:29:03.184252 21426 leader_election.cc:336] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.185139 21426 leader_election.cc:336] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.186010 21426 consensus_peers.cc:597] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:03.193181 21352 consensus_peers.cc:597] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:03.132052 22194 raft_consensus.cc:697] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
W20260709 12:29:03.186470 21426 consensus_peers.cc:597] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:03.221113 21426 leader_election.cc:336] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.228211 21426 leader_election.cc:336] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.230966 21426 leader_election.cc:336] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.235072 21426 leader_election.cc:336] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:03.241843 21426 leader_election.cc:336] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:03.262367 22169 ts_tablet_manager.cc:1434] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.269s	user 0.006s	sys 0.005s
I20260709 12:29:03.263669 22169 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:03.241020 21132 catalog_manager.cc:5697] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.220924 22194 consensus_queue.cc:237] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:03.301087 21426 consensus_peers.cc:597] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:03.309720 22169 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:03.312403 22169 tablet_bootstrap.cc:492] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:03.313095 22169 ts_tablet_manager.cc:1403] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.050s	user 0.020s	sys 0.000s
I20260709 12:29:03.315759 21132 catalog_manager.cc:5697] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.316146 21131 catalog_manager.cc:5697] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 2 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.317837 21386 raft_consensus.cc:1275] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.317749 21132 catalog_manager.cc:5697] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.320370 22169 raft_consensus.cc:359] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:03.321667 22169 raft_consensus.cc:740] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.322213 21132 catalog_manager.cc:5697] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.323828 22169 consensus_queue.cc:260] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:03.325184 21132 catalog_manager.cc:5697] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.326135 22169 ts_tablet_manager.cc:1434] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:29:03.327010 22169 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:03.326658 21132 catalog_manager.cc:5697] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.328109 21132 catalog_manager.cc:5697] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.331373 22194 consensus_queue.cc:1048] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.334349 21386 raft_consensus.cc:1275] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.345705 22194 consensus_queue.cc:1048] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.360695 22239 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.361236 22239 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:03.377741 22239 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:03.380872 21460 raft_consensus.cc:1275] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.384585 21135 catalog_manager.cc:5697] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:03.387293 21135 catalog_manager.cc:5697] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 1 to 2, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 2 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:03.406890 22239 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.416635 22239 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:03.419828 22239 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:29:03.420605 21426 consensus_peers.cc:597] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:03.421646 21352 consensus_peers.cc:597] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:03.422222 21426 consensus_peers.cc:597] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:03.430580 22239 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.431300 22239 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:03.433907 22239 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:03.453176 22199 consensus_queue.cc:1048] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
W20260709 12:29:03.492681 21278 consensus_peers.cc:597] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:03.503707 21304 raft_consensus.cc:1275] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.527247 21380 raft_consensus.cc:1275] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.528749 21303 raft_consensus.cc:1275] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.532688 22200 consensus_queue.cc:1048] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.541574 22081 consensus_queue.cc:1048] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:03.539839 22197 consensus_queue.cc:1048] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:29:03.544724 21278 consensus_peers.cc:597] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:03.574842 21426 consensus_peers.cc:597] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:03.583536 21306 raft_consensus.cc:1275] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.584059 21381 raft_consensus.cc:1275] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.585671 22212 consensus_queue.cc:1048] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.595330 21380 raft_consensus.cc:1275] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:03.609964 21426 consensus_peers.cc:597] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:03.610558 21426 consensus_peers.cc:597] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:03.614511 21426 consensus_peers.cc:597] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:03.614913 21380 raft_consensus.cc:1275] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:03.615561 22169 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:03.616731 22169 tablet_bootstrap.cc:492] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:03.617584 22169 ts_tablet_manager.cc:1403] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.291s	user 0.039s	sys 0.046s
I20260709 12:29:03.617362 22233 consensus_queue.cc:1048] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.618752 22183 consensus_queue.cc:1048] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.621114 21455 raft_consensus.cc:1275] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.627177 22239 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.627779 22239 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:03.629868 22239 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:03.630774 22163 consensus_queue.cc:1048] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.635210 22212 consensus_queue.cc:1048] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:03.641556 22169 raft_consensus.cc:359] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:03.642342 22169 raft_consensus.cc:740] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.670928 22169 consensus_queue.cc:260] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:03.673105 21278 consensus_peers.cc:597] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:03.673923 21426 consensus_peers.cc:597] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:03.676476 22169 ts_tablet_manager.cc:1434] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.058s	user 0.010s	sys 0.000s
I20260709 12:29:03.677839 22169 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:03.678881 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:03.679957 21451 raft_consensus.cc:2393] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 4: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:03.690653 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:03.707016 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:03.707623 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:03.708696 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:03.709555 21307 raft_consensus.cc:2393] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 4: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:03.713557 22165 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:03.719539 22239 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:03.719571 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:03.738893 22162 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:03.753708 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:03.754632 21386 raft_consensus.cc:2393] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:03.749744 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:03.804584 21426 consensus_peers.cc:597] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:03.808396 22274 raft_consensus.cc:2749] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.809228 22274 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.809669 22274 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:29:03.808223 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 69430.2us
W20260709 12:29:03.854707 21426 consensus_peers.cc:597] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:03.855943 22274 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:03.859419 21380 raft_consensus.cc:1275] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:03.860749 22239 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:03.863375 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:03.892828 22193 consensus_queue.cc:1048] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:03.905241 21426 consensus_peers.cc:597] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:03.906092 22239 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.906430 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:03.906731 22239 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:03.909036 22239 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:03.910805 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:03.912811 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:03.914436 22165 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:03.915643 22239 raft_consensus.cc:2749] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:03.920014 22169 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:03.921092 22169 tablet_bootstrap.cc:492] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:03.921779 22169 ts_tablet_manager.cc:1403] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.244s	user 0.015s	sys 0.006s
I20260709 12:29:03.923586 22162 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:03.926550 22239 raft_consensus.cc:3060] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:03.927569 22162 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:03.928886 22162 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:03.930706 22275 raft_consensus.cc:2749] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:03.931430 22275 raft_consensus.cc:2749] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:03.933310 22169 raft_consensus.cc:359] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:03.933977 22169 raft_consensus.cc:740] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:03.938302 22239 raft_consensus.cc:2749] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:03.946290 22239 raft_consensus.cc:493] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:03.951326 22169 consensus_queue.cc:260] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:03.960040 22169 ts_tablet_manager.cc:1434] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.035s	user 0.004s	sys 0.001s
I20260709 12:29:03.962298 22169 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:03.958036 22239 raft_consensus.cc:515] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:03.977628 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:03.987843 22239 leader_election.cc:290] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:03.989092 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:03.990751 22162 leader_election.cc:304] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:03.992497 22239 raft_consensus.cc:2749] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.022833 22169 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.023962 22169 tablet_bootstrap.cc:492] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.024693 22169 ts_tablet_manager.cc:1403] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.063s	user 0.017s	sys 0.007s
I20260709 12:29:04.027779 22169 raft_consensus.cc:359] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.028544 22169 raft_consensus.cc:740] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.029287 22169 consensus_queue.cc:260] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.031299 22169 ts_tablet_manager.cc:1434] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:04.032112 22169 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.078326 22169 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:29:04.079445 22169 tablet_bootstrap.cc:492] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.090260 22169 ts_tablet_manager.cc:1403] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.058s	user 0.027s	sys 0.015s
I20260709 12:29:04.093080 22169 raft_consensus.cc:359] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.094102 22169 raft_consensus.cc:740] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.095085 22169 consensus_queue.cc:260] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.097741 22169 ts_tablet_manager.cc:1434] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:29:04.098688 22169 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.147272 22169 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.148301 22169 tablet_bootstrap.cc:492] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.154666 22169 ts_tablet_manager.cc:1403] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.008s
I20260709 12:29:04.157913 22169 raft_consensus.cc:359] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.158808 22169 raft_consensus.cc:740] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.159754 22169 consensus_queue.cc:260] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.171502 22169 ts_tablet_manager.cc:1434] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 12:29:04.172526 22169 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.227447 22169 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.281627 22169 tablet_bootstrap.cc:492] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.282575 22169 ts_tablet_manager.cc:1403] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.110s	user 0.010s	sys 0.013s
I20260709 12:29:04.289053 22169 raft_consensus.cc:359] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.290405 22169 raft_consensus.cc:740] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.293722 22169 consensus_queue.cc:260] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.363137 22169 ts_tablet_manager.cc:1434] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.080s	user 0.007s	sys 0.000s
I20260709 12:29:04.367576 22169 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.391166 22239 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.391789 22239 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.394001 22239 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:04.396240 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:04.396489 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:04.400449 22162 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:04.403211 22239 raft_consensus.cc:2749] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:04.438130 22025 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.438849 22025 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.441035 22025 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:04.448293 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:04.449090 21381 raft_consensus.cc:2468] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:04.450645 21275 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:04.452353 22232 raft_consensus.cc:2804] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:04.452771 22232 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:04.453135 22232 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:29:04.456627 21278 leader_election.cc:336] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:04.485993 22232 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.490535 22232 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:04.491683 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:04.492424 21381 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:04.502238 21381 raft_consensus.cc:2468] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:04.504025 22239 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.504202 21275 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:29:04.505164 21278 leader_election.cc:336] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:04.515707 22169 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.507772 22232 raft_consensus.cc:2804] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:04.520233 22169 tablet_bootstrap.cc:492] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.521530 22232 raft_consensus.cc:697] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:04.522995 22169 ts_tablet_manager.cc:1403] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.157s	user 0.023s	sys 0.000s
I20260709 12:29:04.505726 22239 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.528839 22232 consensus_queue.cc:237] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.529986 22239 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:04.532239 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:04.531930 22169 raft_consensus.cc:359] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.544589 22169 raft_consensus.cc:740] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.545644 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:04.547860 22169 consensus_queue.cc:260] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.549961 22162 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:04.551074 22239 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:04.553270 22169 ts_tablet_manager.cc:1434] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.027s	user 0.007s	sys 0.000s
I20260709 12:29:04.559456 22169 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.582861 21135 catalog_manager.cc:5697] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 1 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:04.591856 22169 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.592993 22169 tablet_bootstrap.cc:492] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.591491 22316 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.593693 22169 ts_tablet_manager.cc:1403] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.008s
I20260709 12:29:04.593768 22316 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.598362 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:04.599285 21304 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:29:04.599911 22169 raft_consensus.cc:359] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.600778 22169 raft_consensus.cc:740] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.601317 21351 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:04.601629 22169 consensus_queue.cc:260] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.602547 22177 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:04.603096 22177 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:04.603480 22177 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:04.604091 22169 ts_tablet_manager.cc:1434] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.002s
I20260709 12:29:04.605083 22169 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.611097 22316 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:04.633162 22169 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.634446 22169 tablet_bootstrap.cc:492] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.635525 22169 ts_tablet_manager.cc:1403] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.031s	user 0.022s	sys 0.000s
W20260709 12:29:04.639636 21352 leader_election.cc:336] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:04.638512 22169 raft_consensus.cc:359] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.643651 22169 raft_consensus.cc:740] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.645180 22169 consensus_queue.cc:260] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.650671 22169 ts_tablet_manager.cc:1434] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:29:04.655432 22169 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.667697 22177 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.671396 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:04.672036 21304 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:04.681874 21304 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:04.686517 21351 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:04.687487 22203 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:04.687994 22203 raft_consensus.cc:697] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:04.688867 22203 consensus_queue.cc:237] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.696482 22232 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.697263 22232 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.700456 22232 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:04.706082 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:04.707319 21455 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 4.
I20260709 12:29:04.708998 21277 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:04.710394 22232 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:04.710951 22232 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:04.711417 22232 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:29:04.711691 21278 leader_election.cc:336] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:04.736801 21131 catalog_manager.cc:5697] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 1 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:04.749012 22177 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:04.754014 21352 consensus_peers.cc:597] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:04.755332 22169 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.756325 22169 tablet_bootstrap.cc:492] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.758219 22169 ts_tablet_manager.cc:1403] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.106s	user 0.039s	sys 0.022s
I20260709 12:29:04.763000 22169 raft_consensus.cc:359] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.764613 22169 raft_consensus.cc:740] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
W20260709 12:29:04.763026 21352 leader_election.cc:336] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:04.765977 22169 consensus_queue.cc:260] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.767982 22169 ts_tablet_manager.cc:1434] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:29:04.768750 22169 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.773463 22232 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.788699 22169 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.789911 22169 tablet_bootstrap.cc:492] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.791767 22169 ts_tablet_manager.cc:1403] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.023s	user 0.019s	sys 0.000s
I20260709 12:29:04.794992 22232 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:04.795631 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:04.796545 21451 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:04.796146 22169 raft_consensus.cc:359] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.797178 22169 raft_consensus.cc:740] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.797940 22169 consensus_queue.cc:260] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.800799 22169 ts_tablet_manager.cc:1434] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:29:04.801446 22169 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.806190 21451 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 5.
I20260709 12:29:04.808275 21277 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:04.809079 22232 raft_consensus.cc:2804] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:04.809638 22232 raft_consensus.cc:697] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:04.810541 22232 consensus_queue.cc:237] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.822109  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:33017
I20260709 12:29:04.822184 22336 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:33017 every 8 connection(s)
I20260709 12:29:04.852465 22169 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.854318 22169 tablet_bootstrap.cc:492] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.855113 22169 ts_tablet_manager.cc:1403] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.054s	user 0.016s	sys 0.000s
I20260709 12:29:04.857824 22169 raft_consensus.cc:359] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.858525 22169 raft_consensus.cc:740] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.859228 22169 consensus_queue.cc:260] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.863282 22292 raft_consensus.cc:3060] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.863590 22290 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.863962 22169 ts_tablet_manager.cc:1434] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:29:04.864871 22169 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.867782 22310 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.868317 22310 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.872869 22177 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:04.873560 22310 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:04.873456 22177 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.874153 21131 catalog_manager.cc:5697] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 3 to 5, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 5 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:04.876570 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:04.877252 21380 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 4.
I20260709 12:29:04.879577 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:04.880237 21455 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 4.
I20260709 12:29:04.880995 22289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:04.881613 21351 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:04.882534 22203 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:04.882934 22203 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:04.883298 22203 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:04.883587 22177 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:04.885924 21426 leader_election.cc:343] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:04.889467 22289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:04.890213 22289 raft_consensus.cc:3060] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:04.906055 21423 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:04.909705 22203 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:04.915373 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:04.916071 21455 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:04.919639 22288 raft_consensus.cc:3060] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.929261 21455 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 5.
I20260709 12:29:04.930356 21351 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:04.931434 22316 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:04.931919 22316 raft_consensus.cc:697] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 5 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:04.932770 22316 consensus_queue.cc:237] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:04.933321 21426 consensus_peers.cc:597] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:04.934337 22169 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.935396 22169 tablet_bootstrap.cc:492] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.930557 22310 raft_consensus.cc:2764] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:29:04.936151 22169 ts_tablet_manager.cc:1403] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.071s	user 0.027s	sys 0.010s
I20260709 12:29:04.939491 22169 raft_consensus.cc:359] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.940286 22169 raft_consensus.cc:740] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.941087 22169 consensus_queue.cc:260] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:04.943152 22290 pending_rounds.cc:85] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Aborting all ops after (but not including) 1
I20260709 12:29:04.943446 22169 ts_tablet_manager.cc:1434] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.001s
I20260709 12:29:04.943579 22290 pending_rounds.cc:107] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:29:04.944239 22290 raft_consensus.cc:2889] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:04.944897 22169 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:04.929926 22203 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:04.961930 22290 raft_consensus.cc:3060] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:04.969614 21380 raft_consensus.cc:1275] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:04.973528 22025 consensus_queue.cc:1048] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:04.976882 22169 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:04.978387 22169 tablet_bootstrap.cc:492] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:04.979360 22169 ts_tablet_manager.cc:1403] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.003s
I20260709 12:29:04.975989 22289 raft_consensus.cc:2468] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 5.
I20260709 12:29:04.982892 22169 raft_consensus.cc:359] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.983768 22169 raft_consensus.cc:740] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:04.984606 22169 consensus_queue.cc:260] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:04.987607 22169 ts_tablet_manager.cc:1434] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:04.988595 22169 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.035617 22289 raft_consensus.cc:3060] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:05.052699 22169 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.044245 21131 catalog_manager.cc:5697] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 3 to 5, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 5 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:05.053964 22169 tablet_bootstrap.cc:492] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.054668 22169 ts_tablet_manager.cc:1403] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.066s	user 0.027s	sys 0.013s
I20260709 12:29:05.057888 22169 raft_consensus.cc:359] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.058583 22169 raft_consensus.cc:740] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.059329 22169 consensus_queue.cc:260] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.062022 22287 raft_consensus.cc:3060] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:05.064577 22320 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:05.065095 22320 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.067262 22320 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:05.070299 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:05.071285 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:29:05.074276 21426 consensus_peers.cc:597] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:05.075804 22165 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:29:05.077256 21278 consensus_peers.cc:597] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:05.077417 22292 raft_consensus.cc:3060] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.089670 22320 raft_consensus.cc:2749] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:05.113410 22290 pending_rounds.cc:85] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Aborting all ops after (but not including) 1
I20260709 12:29:05.113767 22290 pending_rounds.cc:107] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:29:05.114050 22290 raft_consensus.cc:2889] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:05.121794 22288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:05.126144 22169 ts_tablet_manager.cc:1434] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.071s	user 0.015s	sys 0.011s
I20260709 12:29:05.127202 22169 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.130069 21303 raft_consensus.cc:1275] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:05.132632 22288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:05.133705 22177 consensus_queue.cc:1048] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:05.172004 21352 leader_election.cc:343] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 pre-election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:05.180048 22169 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.181419 22169 tablet_bootstrap.cc:492] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.182855 22169 ts_tablet_manager.cc:1403] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.056s	user 0.012s	sys 0.011s
I20260709 12:29:05.185722 22169 raft_consensus.cc:359] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.187052 22169 raft_consensus.cc:740] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.187798 22169 consensus_queue.cc:260] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.189769 22169 ts_tablet_manager.cc:1434] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.002s
I20260709 12:29:05.190758 22169 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.222234 22289 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:05.230118 22169 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.231387 22169 tablet_bootstrap.cc:492] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.232513 22169 ts_tablet_manager.cc:1403] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.042s	user 0.022s	sys 0.015s
W20260709 12:29:05.234944 21426 consensus_peers.cc:597] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:05.235781 22169 raft_consensus.cc:359] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.236552 22169 raft_consensus.cc:740] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.237264 22169 consensus_queue.cc:260] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.238914 22169 ts_tablet_manager.cc:1434] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:29:05.239719 22169 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.275678 22169 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.276702 22169 tablet_bootstrap.cc:492] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.277416 22169 ts_tablet_manager.cc:1403] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.038s	user 0.021s	sys 0.011s
I20260709 12:29:05.282512 22169 raft_consensus.cc:359] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.283501 22169 raft_consensus.cc:740] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.284215 22169 consensus_queue.cc:260] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.285907 22169 ts_tablet_manager.cc:1434] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:05.286942 22169 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Bootstrap starting.
W20260709 12:29:05.298918 21352 consensus_peers.cc:597] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:05.300225 21352 leader_election.cc:343] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 election: Tablet error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:05.301229 21352 consensus_peers.cc:597] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:05.303367 21352 consensus_peers.cc:597] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:05.303838 22288 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:05.304077 22169 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.306190 22169 tablet_bootstrap.cc:492] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.306941 22169 ts_tablet_manager.cc:1403] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.020s	user 0.018s	sys 0.000s
W20260709 12:29:05.304021 21352 consensus_peers.cc:597] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:05.315667 21426 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:05.313030 22169 raft_consensus.cc:359] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.316681 22169 raft_consensus.cc:740] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.317611 22169 consensus_queue.cc:260] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.319382 22169 ts_tablet_manager.cc:1434] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.001s
I20260709 12:29:05.320241 22169 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.322019 22288 raft_consensus.cc:1275] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:29:05.323190 21352 consensus_peers.cc:597] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:05.338243 21426 consensus_peers.cc:597] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:05.348371 22287 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:05.361017 22169 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.362636 22169 tablet_bootstrap.cc:492] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.363718 22169 ts_tablet_manager.cc:1403] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.044s	user 0.027s	sys 0.004s
I20260709 12:29:05.367165 22169 raft_consensus.cc:359] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.368029 22169 raft_consensus.cc:740] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.368839 22169 consensus_queue.cc:260] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.371390 22169 ts_tablet_manager.cc:1434] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:05.371565 22232 consensus_queue.cc:1048] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:05.372644 22169 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.380924 21455 raft_consensus.cc:1275] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:05.383149 22232 consensus_queue.cc:1048] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:05.391271 22287 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:29:05.408699 21352 consensus_peers.cc:597] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:05.420109 22288 raft_consensus.cc:1275] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:05.422319 22338 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36393
I20260709 12:29:05.422927 22338 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:05.423923 22338 heartbeater.cc:507] Master 127.8.137.254:36393 requested a full tablet report, sending...
I20260709 12:29:05.423570 22232 consensus_queue.cc:1048] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:05.445358 22287 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:05.454259 22372 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:05.456493 22169 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.458336 22169 tablet_bootstrap.cc:492] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.459467 22169 ts_tablet_manager.cc:1403] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.087s	user 0.027s	sys 0.032s
W20260709 12:29:05.460412 21426 consensus_peers.cc:597] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:05.462409 22169 raft_consensus.cc:359] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.463176 22169 raft_consensus.cc:740] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.463897 22169 consensus_queue.cc:260] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.466157 22169 ts_tablet_manager.cc:1434] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:29:05.467043 22169 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.507274 21451 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:05.509989 22372 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:05.531121 22169 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.532112 22169 tablet_bootstrap.cc:492] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.532764 22169 ts_tablet_manager.cc:1403] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.066s	user 0.037s	sys 0.013s
I20260709 12:29:05.535413 22169 raft_consensus.cc:359] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.536140 22169 raft_consensus.cc:740] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.536877 22169 consensus_queue.cc:260] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.539304 22169 ts_tablet_manager.cc:1434] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:05.544202 22169 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.556567 22289 raft_consensus.cc:3060] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:05.557914 21135 ts_manager.cc:194] Re-registered known tserver with Master: 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:05.660404 22169 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.661530 22169 tablet_bootstrap.cc:492] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.667059 22169 ts_tablet_manager.cc:1403] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.123s	user 0.027s	sys 0.033s
I20260709 12:29:05.671700 22169 raft_consensus.cc:359] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.672503 22169 raft_consensus.cc:740] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.673405 22169 consensus_queue.cc:260] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.676225 22169 ts_tablet_manager.cc:1434] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:29:05.677508 22169 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.694553 21135 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49894
I20260709 12:29:05.708599 22338 heartbeater.cc:499] Master 127.8.137.254:36393 was elected leader, sending a full tablet report...
I20260709 12:29:05.709298 22169 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.710423 22169 tablet_bootstrap.cc:492] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.711162 22169 ts_tablet_manager.cc:1403] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.009s
I20260709 12:29:05.713995 22169 raft_consensus.cc:359] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.714759 22169 raft_consensus.cc:740] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.715687 22169 consensus_queue.cc:260] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.721841 22169 ts_tablet_manager.cc:1434] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 12:29:05.722779 22169 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.758268 22169 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.759421 22169 tablet_bootstrap.cc:492] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.760327 22169 ts_tablet_manager.cc:1403] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.001s
I20260709 12:29:05.763042 22169 raft_consensus.cc:359] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.763758 22169 raft_consensus.cc:740] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.764571 22169 consensus_queue.cc:260] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:05.772138 22169 ts_tablet_manager.cc:1434] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.000s
I20260709 12:29:05.773068 22169 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.800093 22169 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.801533 22169 tablet_bootstrap.cc:492] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.802376 22169 ts_tablet_manager.cc:1403] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.004s
I20260709 12:29:05.805404 22169 raft_consensus.cc:359] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.806147 22169 raft_consensus.cc:740] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.806913 22169 consensus_queue.cc:260] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:05.809181 22169 ts_tablet_manager.cc:1434] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:05.809996 22169 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.855993 22169 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.857102 22169 tablet_bootstrap.cc:492] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.857779 22169 ts_tablet_manager.cc:1403] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.007s
I20260709 12:29:05.860118 22169 raft_consensus.cc:359] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.860623 22169 raft_consensus.cc:740] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.861223 22169 consensus_queue.cc:260] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.862401 22169 ts_tablet_manager.cc:1434] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.001s
I20260709 12:29:05.863183 22169 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.887080 22169 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.888140 22169 tablet_bootstrap.cc:492] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.888820 22169 ts_tablet_manager.cc:1403] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.004s
I20260709 12:29:05.892247 22169 raft_consensus.cc:359] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.893322 22169 raft_consensus.cc:740] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.893925 22169 consensus_queue.cc:260] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.895496 22169 ts_tablet_manager.cc:1434] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:29:05.896287 22169 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:05.981521 22288 raft_consensus.cc:3060] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:05.988580 22169 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:05.989787 22169 tablet_bootstrap.cc:492] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:05.990520 22169 ts_tablet_manager.cc:1403] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.094s	user 0.035s	sys 0.045s
I20260709 12:29:05.998440 22169 raft_consensus.cc:359] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:05.999243 22169 raft_consensus.cc:740] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:05.999980 22169 consensus_queue.cc:260] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:06.002708 22169 ts_tablet_manager.cc:1434] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:06.003599 22169 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:06.022979 22288 raft_consensus.cc:3060] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:06.039108 22169 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.040452 22169 tablet_bootstrap.cc:492] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:06.041265 22169 ts_tablet_manager.cc:1403] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.038s	user 0.020s	sys 0.007s
I20260709 12:29:06.044211 22169 raft_consensus.cc:359] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:06.044919 22169 raft_consensus.cc:740] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.045761 22169 consensus_queue.cc:260] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:06.048128 22169 ts_tablet_manager.cc:1434] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:06.049117 22169 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:06.072279 22169 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.073277 22169 tablet_bootstrap.cc:492] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:06.074050 22169 ts_tablet_manager.cc:1403] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.005s
I20260709 12:29:06.076720 22169 raft_consensus.cc:359] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:06.077446 22169 raft_consensus.cc:740] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.078123 22169 consensus_queue.cc:260] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:06.080504 22169 ts_tablet_manager.cc:1434] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.002s
I20260709 12:29:06.081344 22169 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:06.106204 22169 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.107287 22169 tablet_bootstrap.cc:492] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:06.107987 22169 ts_tablet_manager.cc:1403] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.003s
I20260709 12:29:06.110689 22169 raft_consensus.cc:359] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:06.111393 22169 raft_consensus.cc:740] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.112128 22169 consensus_queue.cc:260] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:06.113906 22169 ts_tablet_manager.cc:1434] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:06.114733 22169 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:06.137455 22169 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.138496 22169 tablet_bootstrap.cc:492] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:06.139238 22169 ts_tablet_manager.cc:1403] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.004s
I20260709 12:29:06.141948 22169 raft_consensus.cc:359] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:06.142649 22169 raft_consensus.cc:740] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.143343 22169 consensus_queue.cc:260] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:06.146781 22169 ts_tablet_manager.cc:1434] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:06.147588 22169 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:06.166198 22169 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.167011 22169 tablet_bootstrap.cc:492] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:06.167726 22169 ts_tablet_manager.cc:1403] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.020s	user 0.006s	sys 0.009s
I20260709 12:29:06.171550 22169 raft_consensus.cc:359] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:06.172250 22169 raft_consensus.cc:740] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.172904 22169 consensus_queue.cc:260] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:06.175297 22169 ts_tablet_manager.cc:1434] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 12:29:06.183578 22169 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:06.200968 22169 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.201978 22169 tablet_bootstrap.cc:492] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:06.202713 22169 ts_tablet_manager.cc:1403] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 12:29:06.205514 22169 raft_consensus.cc:359] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:06.206239 22169 raft_consensus.cc:740] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.207021 22169 consensus_queue.cc:260] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:06.209420 22169 ts_tablet_manager.cc:1434] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:06.210489 22169 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Bootstrap starting.
I20260709 12:29:06.246553 22169 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:06.250916 22169 tablet_bootstrap.cc:492] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Bootstrap complete.
I20260709 12:29:06.251762 22169 ts_tablet_manager.cc:1403] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Time spent bootstrapping tablet: real 0.042s	user 0.018s	sys 0.006s
I20260709 12:29:06.254616 22169 raft_consensus.cc:359] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:06.255350 22169 raft_consensus.cc:740] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Initialized, Role: FOLLOWER
I20260709 12:29:06.256322 22169 consensus_queue.cc:260] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:06.258610 22169 ts_tablet_manager.cc:1434] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:06.619431 22288 raft_consensus.cc:3060] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:06.676467 22288 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:29:10.184154 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 54000.4us
I20260709 12:29:13.307586 22433 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:13.308595 22433 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:13.311348 22433 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:13.318192 22434 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:13.318864 22434 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:13.321118 22434 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:13.336108 22431 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:13.336752 22431 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:13.337919 22435 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:13.338680 22435 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:13.340512 22430 raft_consensus.cc:493] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:13.341181 22430 raft_consensus.cc:515] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:13.340930 22434 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:13.342777 22434 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:13.343220 22431 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:13.343897 22430 leader_election.cc:290] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:13.345135 22435 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:13.343918 22431 raft_consensus.cc:493] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:13.351663 22431 raft_consensus.cc:515] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:13.352458 22434 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:13.353883 22434 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:13.361697 22431 leader_election.cc:290] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:13.363119 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:13.354467 22434 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:13.366863 22434 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:13.371191 22442 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:13.372119 22442 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:13.373065 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:13.381731 22444 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:13.383137 22444 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:13.391705 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:13.394567 22446 raft_consensus.cc:493] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:13.399082 22443 raft_consensus.cc:493] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:13.399972 22443 raft_consensus.cc:515] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:13.397804 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:13.488942 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:13.489413 22437 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:13.428030 22434 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:13.490003 22437 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:13.520013 22436 raft_consensus.cc:493] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:13.511829 22434 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:13.494813 22446 raft_consensus.cc:515] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:13.530040 22436 raft_consensus.cc:515] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:14.129997 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:14.151108 21381 raft_consensus.cc:2468] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:14.152621 22447 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:14.153383 22447 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:13.939682 22450 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
W20260709 12:29:14.298658 22164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.778s	user 0.004s	sys 0.016s
W20260709 12:29:14.299220 22164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.004s	sys 0.016s
I20260709 12:29:14.326637 22452 raft_consensus.cc:493] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:14.327612 22452 raft_consensus.cc:515] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:14.378515 22429 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.378921 22449 raft_consensus.cc:493] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:14.379344 22429 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:14.379482 22449 raft_consensus.cc:515] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:14.381616 22454 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:14.381877 22429 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:14.382256 22454 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:14.386984 21333 debug-util.cc:398] Leaking SignalData structure 0x7b08002c1f40 after lost signal to thread 8746
I20260709 12:29:14.399382 22437 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:14.402235 21426 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.004s	sys 0.017s
W20260709 12:29:14.402776 21426 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.881s	user 0.004s	sys 0.018s
I20260709 12:29:14.404910 22436 leader_election.cc:290] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:14.336287 22450 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:14.412883 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:14.419047 22442 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:14.421015 22284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35724 (request call id 187) took 1031 ms. Trace:
I20260709 12:29:14.423195 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:14.424207 22283 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:14.440244 22441 raft_consensus.cc:493] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:14.421715 22284 rpcz_store.cc:276] 0709 12:29:13.389116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:13.398165 (+  9049us) service_pool.cc:224] Handling call
0709 12:29:14.420104 (+1021939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:14.420350 (+   246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:14.420397 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:14.420463 (+    66us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:29:14.420513 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:14.420594 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:14.420737 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:14.440508 22442 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.440528 22283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35724 (request call id 188) took 1048 ms. Trace:
W20260709 12:29:14.455021 21351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.008s	sys 0.012s
W20260709 12:29:14.457280 22162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.008s	sys 0.039s
I20260709 12:29:14.457752 22432 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.457826 22453 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
W20260709 12:29:14.457865 22162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.009s	sys 0.039s
I20260709 12:29:14.458515 22432 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:14.458698 22453 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:14.457340 22442 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:14.450986 22283 rpcz_store.cc:276] 0709 12:29:13.389445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:13.395274 (+  5829us) service_pool.cc:224] Handling call
0709 12:29:14.437392 (+1042118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:14.453914 22458 raft_consensus.cc:493] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:14.451759 22452 leader_election.cc:290] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:14.494622 22432 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:14.494983 22451 raft_consensus.cc:493] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:14.495822 22451 raft_consensus.cc:515] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:14.486763 22447 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:14.454602 22441 raft_consensus.cc:515] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:29:14.500384 21424 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.022s	sys 0.031s
W20260709 12:29:14.511721 21424 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.022s	sys 0.031s
I20260709 12:29:14.504973 21308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47030 (request call id 506) took 1101 ms. Trace:
W20260709 12:29:14.496078 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 50458.2us
I20260709 12:29:14.500103 22447 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:14.539736 22455 raft_consensus.cc:493] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:14.539893 22445 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.540485 22455 raft_consensus.cc:515] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:14.540637 22445 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:14.542814 22455 leader_election.cc:290] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:14.543169 22434 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:14.543298 22455 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:14.543603 21386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39594 (request call id 497) took 1218 ms. Trace:
I20260709 12:29:14.543933 22455 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:14.544010 21386 rpcz_store.cc:276] 0709 12:29:13.324818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:14.542745 (+1217927us) service_pool.cc:224] Handling call
0709 12:29:14.543046 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:14.543205 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:14.543228 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:14.543266 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:14.543294 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:14.543350 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:14.543447 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:14.546167 22446 leader_election.cc:290] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:14.546456 22445 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:14.501603 22442 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:14.541071 22449 leader_election.cc:290] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:14.500351 22453 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:14.499402 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:14.548105 21378 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39594 (request call id 509) took 1186 ms (client timeout 1511 ms). Trace:
W20260709 12:29:14.548547 21378 rpcz_store.cc:269] 0709 12:29:13.361937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:14.463779 (+1101842us) service_pool.cc:224] Handling call
0709 12:29:14.548026 (+ 84247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:14.514485 21308 rpcz_store.cc:276] 0709 12:29:13.403949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:14.502167 (+1098218us) service_pool.cc:224] Handling call
0709 12:29:14.504085 (+  1918us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:14.504365 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:14.504419 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:14.504484 (+    65us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:14.504531 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:14.504606 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:14.504751 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:14.549486 22432 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.546442 22455 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:14.550154 22432 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:14.550921 22455 raft_consensus.cc:493] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:14.551496 22455 raft_consensus.cc:515] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:14.552708 22432 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:14.553303 22432 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:14.553570 22455 leader_election.cc:290] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:14.554469 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:14.557001 22432 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:14.503993 22451 leader_election.cc:290] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:14.502238 22441 leader_election.cc:290] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:14.514307 22459 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.576398 22449 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.628458 22449 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:14.573415 22445 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.564612 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:14.562239 22442 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:14.631651 22459 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:14.631991 22442 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:14.603070 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:14.630954 22445 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:14.636137 22442 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:14.629395 22450 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:14.567610 22453 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:14.638381 22459 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.313985 22453 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.314790 22445 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.315374 22445 raft_consensus.cc:493] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
W20260709 12:29:15.315876 22166 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.862s	user 0.001s	sys 0.006s
I20260709 12:29:14.561896 22464 raft_consensus.cc:493] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
W20260709 12:29:15.316254 22166 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.862s	user 0.001s	sys 0.006s
I20260709 12:29:15.316699 22464 raft_consensus.cc:515] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.315889 22445 raft_consensus.cc:515] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.318961 22464 leader_election.cc:290] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:14.561816 22447 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:14.635743 22449 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:14.500296 22458 raft_consensus.cc:515] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.321231 22447 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.318965 22445 leader_election.cc:290] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.322144 22458 leader_election.cc:290] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.316188 22453 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:14.868521 22443 leader_election.cc:290] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.286187 22444 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.295399 22471 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.297322 22472 raft_consensus.cc:493] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.328118 22449 raft_consensus.cc:493] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.299335 22474 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.328647 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.328987 22474 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.329716 21309 raft_consensus.cc:2468] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:15.299481 22473 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:14.641355 22432 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.325937 22430 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.331853 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.332731 22471 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.333527 22473 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.333446 22430 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.328715 22472 raft_consensus.cc:515] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.328785 22449 raft_consensus.cc:515] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.335912 22474 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.338402 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.339831 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.347992 22449 leader_election.cc:290] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.348605 22473 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.349532 22449 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.350550 22472 leader_election.cc:290] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.350317 22449 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.344527 22471 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.341717 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.357048 22449 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.357458 22430 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.360338 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.363453 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.364382 21378 raft_consensus.cc:2468] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
W20260709 12:29:15.365859 21278 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.850s	user 0.073s	sys 0.161s
W20260709 12:29:15.366554 21278 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.851s	user 0.073s	sys 0.161s
I20260709 12:29:15.372047 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.376776 21377 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:15.369416 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:15.379881 21426 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.931s	user 0.012s	sys 0.025s
W20260709 12:29:15.380368 21426 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.012s	sys 0.025s
I20260709 12:29:15.381913 22478 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.382453 22478 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.382966 22471 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.383558 22471 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.385002 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.385388 22478 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.385550 22471 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.386005 22478 raft_consensus.cc:493] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.386572 22478 raft_consensus.cc:515] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.403414 22439 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.403986 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.404109 22439 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.404525 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.404798 21452 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:15.403440 22478 leader_election.cc:290] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.406394 22439 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.388028 22448 raft_consensus.cc:493] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.389725 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:15.386282 21379 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39594 (request call id 510) took 2024 ms (client timeout 2241 ms). Trace:
I20260709 12:29:15.408226 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.415607 22448 raft_consensus.cc:515] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:29:15.387528 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 1.87189e+06us
I20260709 12:29:15.417431 21278 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.424158 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.424610 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.425424 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.425899 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:29:15.418118 21379 rpcz_store.cc:269] 0709 12:29:13.362261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.384509 (+2022248us) service_pool.cc:224] Handling call
0709 12:29:15.386189 (+  1680us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:15.427263 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.420354 21132 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60588 (request call id 126) took 2157 ms. Trace:
I20260709 12:29:15.420782 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.406913 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.423628 22448 leader_election.cc:290] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.429471 22473 raft_consensus.cc:493] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.430115 22473 raft_consensus.cc:515] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.429247 21132 rpcz_store.cc:276] 0709 12:29:13.262546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.417470 (+2154924us) service_pool.cc:224] Handling call
0709 12:29:15.420164 (+  2694us) inbound_call.cc:177] Queueing success response
Metrics: {"reported_tablets":0}
I20260709 12:29:15.432049 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.432652 22473 leader_election.cc:290] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.425606 22430 raft_consensus.cc:2749] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:15.433252 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.433398 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.421236 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.420392 21275 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:29:15.436128 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 93859.2us
I20260709 12:29:15.436902 21275 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:15.437674 22430 raft_consensus.cc:2804] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.437956 21425 leader_election.cc:304] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:15.438138 22430 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.438622 22480 raft_consensus.cc:2804] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.442550 22430 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.442833 22480 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.443310 22480 raft_consensus.cc:3060] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.441684 22454 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:29:15.448025 21333 debug-util.cc:398] Leaking SignalData structure 0x7b08001584c0 after lost signal to thread 21169
W20260709 12:29:15.449713 21333 debug-util.cc:398] Leaking SignalData structure 0x7b08003264c0 after lost signal to thread 21336
W20260709 12:29:15.450870 21333 debug-util.cc:398] Leaking SignalData structure 0x7b080029a8c0 after lost signal to thread 21410
I20260709 12:29:15.450886 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:15.451443 22162 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.991s	user 0.010s	sys 0.003s
W20260709 12:29:15.451787 22162 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.010s	sys 0.003s
W20260709 12:29:15.452039 21333 debug-util.cc:398] Leaking SignalData structure 0x7b0800320280 after lost signal to thread 21484
W20260709 12:29:15.452456 21333 debug-util.cc:398] Leaking SignalData structure 0x7b0800392900 after lost signal to thread 21485
I20260709 12:29:15.452976 22479 raft_consensus.cc:2804] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.453419 22479 raft_consensus.cc:493] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
W20260709 12:29:15.453576 21333 debug-util.cc:398] Leaking SignalData structure 0x7b0800385420 after lost signal to thread 22336
I20260709 12:29:15.453794 22479 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:15.453955 21333 debug-util.cc:398] Leaking SignalData structure 0x7b08001ef320 after lost signal to thread 22338
I20260709 12:29:15.457803 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.465472 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.466408 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.466908 21304 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:15.425424 21278 leader_election.cc:304] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
W20260709 12:29:15.468554 21350 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.894s	user 0.010s	sys 0.009s
W20260709 12:29:15.472620 21350 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.011s	sys 0.009s
I20260709 12:29:15.469817 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.470572 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.471889 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.475075 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.479288 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.479826 21385 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39594 (request call id 507) took 2118 ms. Trace:
I20260709 12:29:15.480232 21385 rpcz_store.cc:276] 0709 12:29:13.361146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:15.478936 (+2117790us) service_pool.cc:224] Handling call
0709 12:29:15.479238 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:15.479421 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:15.479445 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:15.479482 (+    37us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:15.479508 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:15.479554 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:15.479662 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:15.481107 21277 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.482461 22448 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.482646 22288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35724 (request call id 181) took 2095 ms. Trace:
I20260709 12:29:15.483086 22288 rpcz_store.cc:276] 0709 12:29:13.387189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:14.624205 (+1237016us) service_pool.cc:224] Handling call
0709 12:29:14.624628 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:15.430313 (+805685us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:15.430345 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:15.430386 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:15.430414 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:15.482294 (+ 51880us) spinlock_profiling.cc:243] Waited 805 ms on lock 0x7b540034d0d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09556e01 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2 00007f4e0789148c 00007f4e078908c6
0709 12:29:15.482347 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:15.482462 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1852493568}
I20260709 12:29:15.480952 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.479866 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:15.490121 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 64784.8us
I20260709 12:29:15.491453 22481 raft_consensus.cc:2749] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.492453 21278 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.494355 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.495079 22469 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.534427 22469 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.536399 21423 leader_election.cc:304] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.536953 22469 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.537572 22469 raft_consensus.cc:493] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.537802 22477 raft_consensus.cc:2749] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.495790 22470 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.499346 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.498101 21277 leader_election.cc:304] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.501155 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.538889 22470 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.539726 21277 leader_election.cc:304] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.540548 22431 raft_consensus.cc:2749] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.540848 21423 leader_election.cc:304] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.541054 22470 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.541280 22431 raft_consensus.cc:2749] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.538120 22469 raft_consensus.cc:515] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.541659 21277 leader_election.cc:304] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.541923 22477 raft_consensus.cc:2749] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.506381 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.507218 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.503203 21425 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:15.507941 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.543412 22469 leader_election.cc:290] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.543802 21380 raft_consensus.cc:2468] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:15.512845 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.544487 22470 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.515025 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.545068 22470 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.546128 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.483057 22448 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.516848 22442 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.546854 22448 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.546988 22442 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.547405 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.549189 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.519368 22453 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.515729 22444 raft_consensus.cc:493] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.525866 22458 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.550469 22470 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.550810 22472 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.551409 22472 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.551829 22472 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.527094 22450 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.554680 22469 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.527554 22443 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.555567 21423 leader_election.cc:304] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.555451 22443 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.531106 22465 raft_consensus.cc:493] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.532553 22487 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.556584 22475 raft_consensus.cc:2749] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.556387 22465 raft_consensus.cc:515] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.533164 22485 raft_consensus.cc:493] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.563416 22432 raft_consensus.cc:2749] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:15.563815 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.563474 22485 raft_consensus.cc:515] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.556511 22487 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:15.566546 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 74183.2us
I20260709 12:29:15.536988 22486 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.549377 22442 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.517411 22447 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.549976 22453 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.525275 22459 raft_consensus.cc:493] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.550518 22444 raft_consensus.cc:515] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.526365 22454 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.555361 22469 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.528369 22464 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.527647 22479 raft_consensus.cc:515] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.557705 22443 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.554965 22450 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.561714 22465 leader_election.cc:290] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.550855 22458 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.562336 22431 raft_consensus.cc:2749] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.551112 22470 raft_consensus.cc:493] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.566761 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.567808 22447 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.582671 21277 leader_election.cc:304] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.583488 22431 raft_consensus.cc:2749] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.567512 22442 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.584939 22442 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.585666 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.585954 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.572079 22454 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.567521 22486 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.587499 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.588104 21308 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:15.572739 22464 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.589344 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.574349 22479 leader_election.cc:290] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.575117 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.575506 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.576233 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.576632 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.577087 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.577607 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.578506 22445 raft_consensus.cc:493] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.569864 22459 raft_consensus.cc:515] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.599180 22445 raft_consensus.cc:515] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.601225 22445 leader_election.cc:290] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.567857 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.602437 21456 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 1.
I20260709 12:29:15.581453 22470 raft_consensus.cc:515] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.604847 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.581521 22284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.605829 22284 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:15.578094 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.608486 22448 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.612200 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.613340 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.614135 21308 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
W20260709 12:29:15.616421 21351 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.028s	user 0.024s	sys 0.041s
W20260709 12:29:15.617004 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 1.02851e+06us
I20260709 12:29:15.618109 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.618585 22465 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.619215 22465 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.620692 22454 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.621371 22465 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.622555 22431 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.623260 22459 leader_election.cc:290] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.623189 22431 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.624677 22450 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.625417 22431 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.626233 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.627080 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.627733 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.623469 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.626163 22447 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.629671 22453 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.636399 22470 leader_election.cc:290] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.638633 22487 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.639366 22486 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.639551 22472 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.640863 22485 leader_election.cc:290] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.629469 22430 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.630561 22464 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.632041 22469 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.633965 22458 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.632877 22444 leader_election.cc:290] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.642321 22448 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.643749 22430 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.643895 22442 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.645357 22472 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.647270 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.648272 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.648449 22288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.649787 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.650133 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.651196 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.650633 22283 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:15.655824 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.657042 22442 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.657656 22442 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.659983 22442 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.660871 21278 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.586648 22480 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.662465 22432 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:15.663975 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.668151 22288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.663221 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.708707 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.664809 22432 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.695403 22480 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.668131 21423 leader_election.cc:304] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:15.696977 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.697314 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.712872 21423 leader_election.cc:304] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.713688 22472 raft_consensus.cc:2804] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.714120 22472 raft_consensus.cc:493] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.714556 22472 raft_consensus.cc:3060] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.714776 22478 raft_consensus.cc:2749] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.716609 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.715852 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.718143 22285 raft_consensus.cc:2468] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:15.720003 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.713493 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.722746 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.723764 22292 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:15.697502 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.698529 22284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.725770 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8c8433d1e274f879bb2b84ad59f5c3f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.701043 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.726778 22287 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:15.704893 22442 raft_consensus.cc:493] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.727279 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:15.727614 22442 raft_consensus.cc:515] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.729092 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.729849 22442 leader_election.cc:290] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.709770 22432 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:15.731040 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.725998 22284 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:15.732998 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.726172 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:15.734899 21308 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.728109 21304 raft_consensus.cc:2393] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 2: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:15.703738 21426 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.739385 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.740748 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.741743 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:15.741966 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:15.742738 21456 raft_consensus.cc:2393] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:15.743451 22439 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.743856 22439 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.744218 22439 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.745366 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.746048 22287 raft_consensus.cc:2468] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:15.748073 22284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.755017 22288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.756294 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.757437 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.757725 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.757835 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.758639 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.760037 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.761665 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.763572 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.764845 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.747335 21426 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.766469 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.767115 22478 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:29:15.768275 21426 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.769017 22478 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.769387 22478 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.769763 22478 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.770452 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.773756 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:15.774580 22282 raft_consensus.cc:2468] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 1.
I20260709 12:29:15.775760 22162 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:15.776482 22439 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.776971 22442 raft_consensus.cc:2804] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:15.777444 22442 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.777897 22442 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.778854 22439 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.725775 22472 raft_consensus.cc:515] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.781075 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:15.781407 21278 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.793079 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.795240 22480 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.794520 21278 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.797788 22480 raft_consensus.cc:2749] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.762221 22432 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.797974 21278 leader_election.cc:304] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.799247 21278 leader_election.cc:304] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.799381 22480 raft_consensus.cc:2749] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.799997 22431 raft_consensus.cc:2804] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.800396 22431 raft_consensus.cc:493] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.800781 22431 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.802037 22472 leader_election.cc:290] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.803164 21278 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.803710 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.812322 21278 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.812417 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.813120 22448 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:15.813490 21278 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.813508 22448 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.814109 22448 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:15.814752 22430 raft_consensus.cc:2749] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.815012 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.815426 22430 raft_consensus.cc:2749] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.816010 21278 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.816694 22430 raft_consensus.cc:2749] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:15.816998 21426 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.818113 21426 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.818446 22476 raft_consensus.cc:2749] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.818698 21278 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.819185 21426 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.774168 22165 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:15.819180 22476 raft_consensus.cc:2804] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.819998 22476 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.820200 22444 raft_consensus.cc:2804] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:15.820420 22476 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.810139 22431 raft_consensus.cc:515] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.820681 22444 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.821105 22444 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:15.819733 22480 raft_consensus.cc:2749] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:15.822423 22431 leader_election.cc:290] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.824044 21352 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.833453 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.834462 21312 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.834851 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.835681 22489 raft_consensus.cc:2804] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:15.836078 22489 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.836438 22489 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:15.831586 22448 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.824678 21308 raft_consensus.cc:2468] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:15.839100 21352 leader_election.cc:304] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:15.825883 22432 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.827303 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.840754 22484 raft_consensus.cc:2804] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.841142 22484 raft_consensus.cc:493] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:15.841544 22484 raft_consensus.cc:3060] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:15.841984 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 175708us
I20260709 12:29:15.842070 22478 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.844347 22478 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.844883 22489 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.845394 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.827404 22442 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.847051 22489 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:15.829527 22444 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.853102 22444 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:15.830201 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 56207.5us
I20260709 12:29:15.806070 21425 leader_election.cc:304] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.830973 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.832034 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.832249 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.832708 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.833181 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.855350 22165 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:15.856568 22472 raft_consensus.cc:2749] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.828370 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.848508 21351 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.859697 22483 raft_consensus.cc:2749] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.859851 21351 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.860747 22490 raft_consensus.cc:2749] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.861356 22444 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:15.862741 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.847448 22442 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.863536 22444 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:15.850610 22448 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.848577 22476 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.866423 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.861848 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.867637 21304 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:29:15.868589 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
W20260709 12:29:15.862623 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 43495.4us
I20260709 12:29:15.863631 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:15.870739 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:15.864017 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:15.871966 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:15.873124 22165 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.874461 22165 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:29:15.854171 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 48191.7us
I20260709 12:29:15.875131 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:15.875715 22165 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.876384 22469 raft_consensus.cc:2749] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.876438 22458 raft_consensus.cc:2749] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.876824 22442 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:29:15.865095 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:15.880219 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:15.882846 22283 raft_consensus.cc:1240] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:15.875142 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.861874 22472 raft_consensus.cc:2804] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.885396 22472 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.866163 22444 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:15.885860 22472 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.865702 21351 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.871373 22285 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:15.888656 22490 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.890101 21351 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.895545 22490 raft_consensus.cc:2749] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.849815 22484 raft_consensus.cc:515] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.900496 22484 leader_election.cc:290] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.884544 22165 leader_election.cc:304] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.924531 22165 leader_election.cc:304] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.924568 22442 raft_consensus.cc:2749] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.925891 22442 raft_consensus.cc:2749] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.926182 22165 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.928349 22442 raft_consensus.cc:2749] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.928289 22165 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.929862 21455 raft_consensus.cc:1240] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:15.930167 22442 raft_consensus.cc:2804] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:15.930567 22442 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.930944 22442 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:15.934301 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:15.903929 21425 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.893841 21277 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:15.942916 22431 raft_consensus.cc:2749] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:15.907701 22285 raft_consensus.cc:2468] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 4.
I20260709 12:29:15.907449 21278 leader_election.cc:304] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.944490 22431 raft_consensus.cc:2749] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.908756 22476 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.908499 22444 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:15.909586 22472 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.912127 22448 raft_consensus.cc:493] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.913781 22480 raft_consensus.cc:493] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:15.915867 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:15.917104 21459 raft_consensus.cc:1240] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:15.920353 21453 raft_consensus.cc:1240] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:15.962522 22448 raft_consensus.cc:515] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.962915 22480 raft_consensus.cc:515] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.964746 22478 raft_consensus.cc:2749] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.967449 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:15.969476 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 47735.9us
I20260709 12:29:15.970781 21351 leader_election.cc:304] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.983160 21426 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.983997 22445 raft_consensus.cc:2749] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.977072 22444 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:15.977842 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 84102.7us
I20260709 12:29:15.985594 21277 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.986470 22431 raft_consensus.cc:2749] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.987910 21277 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.974601 22472 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:15.960523 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:15.995178 22432 raft_consensus.cc:2749] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:29:15.980151 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 72918.6us
I20260709 12:29:15.979833 21352 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:15.980365 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:15.981309 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.982079 22480 leader_election.cc:290] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:15.981734 22442 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:15.982964 22448 leader_election.cc:290] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:29:15.988359 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 84519.4us
I20260709 12:29:15.997936 22484 raft_consensus.cc:2804] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:15.999310 21425 leader_election.cc:304] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:16.000057 22484 raft_consensus.cc:697] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 4 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:16.001314 22479 raft_consensus.cc:2804] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:16.001937 22479 raft_consensus.cc:697] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:16.003961 22448 consensus_queue.cc:1059] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:16.005203 22480 raft_consensus.cc:3055] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:16.005635 22480 raft_consensus.cc:740] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:16.005976 21278 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:15.988946 22477 consensus_queue.cc:1059] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:16.009614 22477 raft_consensus.cc:3055] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:16.010074 22477 raft_consensus.cc:740] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:15.990046 22478 raft_consensus.cc:2749] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:15.990669 22431 raft_consensus.cc:493] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:16.010991 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:16.011679 22431 raft_consensus.cc:515] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:15.978876 22165 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:15.992321 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.992688 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.993211 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.014045 22431 leader_election.cc:290] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:15.991636 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:15.993367 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.006454 22479 consensus_queue.cc:237] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:15.994017 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.016180 22442 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:16.011018 22477 consensus_queue.cc:260] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:16.016213 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.016860 22442 raft_consensus.cc:2749] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:16.006563 22480 consensus_queue.cc:260] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:16.001330 21425 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:16.018370 22480 raft_consensus.cc:3060] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:16.018771 22475 raft_consensus.cc:2749] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:16.019847 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:16.020828 22165 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:16.021894 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:16.022123 22458 raft_consensus.cc:2749] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.022445 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:16.023101 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:16.001250 22484 consensus_queue.cc:237] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:16.024199 22431 raft_consensus.cc:2749] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.023980 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:16.025765 21278 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:16.022168 22165 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:16.010094 21277 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:16.030534 22444 raft_consensus.cc:2749] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:16.017225 22477 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:16.031864 22165 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:29:16.032416 22431 raft_consensus.cc:2749] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:16.032754 22444 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:16.033133 22444 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:16.033515 22431 raft_consensus.cc:2749] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:16.033516 22444 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:16.037043 22431 consensus_queue.cc:1059] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:16.038079 22448 raft_consensus.cc:3055] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:16.038452 22448 raft_consensus.cc:740] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:16.039361 22448 consensus_queue.cc:260] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:16.040872 22448 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:16.041510 22165 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:16.042905 22458 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:16.050973 22478 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:16.051681 22478 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:16.053781 22478 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:16.055292 22439 raft_consensus.cc:493] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:16.055804 22439 raft_consensus.cc:515] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:16.057941 22439 leader_election.cc:290] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:16.059187 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.060513 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.060915 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:16.062515 22444 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:16.070528 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3c29a71139f45fc849c307af6259abb" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.075922 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.079463 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:16.081252 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.081358 22284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:16.082866 22444 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:16.088466 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5de0c984317b4a16bd865892a98794b7" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.091656 22475 raft_consensus.cc:493] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:16.093080 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:16.095880 21426 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:16.098117 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:16.098187 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:16.098701 22478 raft_consensus.cc:493] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:16.099223 22478 raft_consensus.cc:515] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:16.101423 22478 leader_election.cc:290] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:16.103268 22431 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:16.103874 22431 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:16.107586 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:16.097970 22475 raft_consensus.cc:515] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:16.111084 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.111228 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.115429 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.116211 22475 leader_election.cc:290] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:16.117511 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.118409 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.116583 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.119578 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.119659 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.121361 21305 raft_consensus.cc:1240] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:16.123342 21385 raft_consensus.cc:1240] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:16.128084 21303 raft_consensus.cc:1240] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:16.146731 21426 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:16.156319 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:16.116876 22283 raft_consensus.cc:1240] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:16.142326 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:16.122586 22475 raft_consensus.cc:2749] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:16.165251 22476 raft_consensus.cc:2749] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:16.487470 22325 debug-util.cc:398] Leaking SignalData structure 0x7b08001f0c20 after lost signal to thread 8746
I20260709 12:29:16.489318 22431 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:29:16.498785 22325 debug-util.cc:398] Leaking SignalData structure 0x7b08002541c0 after lost signal to thread 21169
W20260709 12:29:16.500385 22325 debug-util.cc:398] Leaking SignalData structure 0x7b0800369e80 after lost signal to thread 21336
W20260709 12:29:16.503984 22325 debug-util.cc:398] Leaking SignalData structure 0x7b08003f4a40 after lost signal to thread 21410
I20260709 12:29:16.504973 22292 raft_consensus.cc:1240] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 3. Current term is 4. Ops: []
W20260709 12:29:16.506878 22325 debug-util.cc:398] Leaking SignalData structure 0x7b08003e4be0 after lost signal to thread 21411
W20260709 12:29:16.508888 22325 debug-util.cc:398] Leaking SignalData structure 0x7b0800177800 after lost signal to thread 21484
W20260709 12:29:16.509824 22325 debug-util.cc:398] Leaking SignalData structure 0x7b08001c1160 after lost signal to thread 21485
W20260709 12:29:16.511770 22325 debug-util.cc:398] Leaking SignalData structure 0x7b08000598c0 after lost signal to thread 22336
W20260709 12:29:16.512302 22325 debug-util.cc:398] Leaking SignalData structure 0x7b080017e120 after lost signal to thread 22338
I20260709 12:29:17.029940 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.056309 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.061748 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.128921 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.149892 21305 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47030 (request call id 551) took 1027 ms. Trace:
I20260709 12:29:17.151966 21305 rpcz_store.cc:276] 0709 12:29:16.122508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.142059 (+ 19551us) service_pool.cc:224] Handling call
0709 12:29:17.148997 (+1006938us) spinlock_profiling.cc:243] Waited 1.5 ms on lock 0x7b5800174098. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:29:17.149245 (+   248us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:17.149472 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:17.149509 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:17.149562 (+    53us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:29:17.149606 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.149675 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:17.149814 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3451776}
W20260709 12:29:17.157471 21425 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.905s	user 0.000s	sys 0.009s
W20260709 12:29:17.157967 21425 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.001s	sys 0.009s
I20260709 12:29:17.158327 22497 consensus_queue.cc:1059] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:17.159597 22495 raft_consensus.cc:3055] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:17.159987 22495 raft_consensus.cc:740] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:17.163908 22288 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35724 (request call id 217) took 1143 ms. Trace:
I20260709 12:29:17.164314 22288 rpcz_store.cc:276] 0709 12:29:16.020571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.020684 (+   113us) service_pool.cc:224] Handling call
0709 12:29:17.163224 (+1142540us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b5800115e98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:29:17.163418 (+   194us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:17.163595 (+   177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:17.163621 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:17.163668 (+    47us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:29:17.163702 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.163755 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:17.163840 (+    85us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15658240}
I20260709 12:29:17.160878 22495 consensus_queue.cc:260] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.167105 22495 raft_consensus.cc:3060] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:17.192785 21304 consensus_meta.cc:287] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a: Time spent flushing consensus metadata: real 1.093s	user 0.011s	sys 0.006s
I20260709 12:29:17.193483 21304 raft_consensus.cc:2468] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:16.163655 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.196717 21303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47030 (request call id 554) took 1041 ms. Trace:
I20260709 12:29:17.196980 21303 rpcz_store.cc:276] 0709 12:29:16.155502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.157343 (+  1841us) service_pool.cc:224] Handling call
0709 12:29:17.196663 (+1039320us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:17.203424 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.206233 21452 raft_consensus.cc:1240] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:17.210876 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.214126 22498 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.214754 22498 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.217361 22498 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.224536 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.225374 21308 raft_consensus.cc:2468] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:17.225931 21308 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47030 (request call id 553) took 1101 ms. Trace:
I20260709 12:29:17.226255 21308 rpcz_store.cc:276] 0709 12:29:16.142691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.149011 (+  6320us) service_pool.cc:224] Handling call
0709 12:29:17.225840 (+1076829us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:17.228966 21453 raft_consensus.cc:1240] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:17.236155 22290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35732 (request call id 241) took 1189 ms. Trace:
I20260709 12:29:17.237980 21383 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35346 (request call id 22) took 1122 ms. Trace:
I20260709 12:29:17.238408 21383 rpcz_store.cc:276] 0709 12:29:16.115455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.117275 (+  1820us) service_pool.cc:224] Handling call
0709 12:29:17.237884 (+1120609us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:17.244035 22285 raft_consensus.cc:1240] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 3. Current term is 4. Ops: []
I20260709 12:29:17.244871 22285 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35724 (request call id 216) took 1224 ms. Trace:
I20260709 12:29:17.245240 22285 rpcz_store.cc:276] 0709 12:29:16.020266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.020410 (+   144us) service_pool.cc:224] Handling call
0709 12:29:17.243570 (+1223160us) spinlock_profiling.cc:243] Waited 6.8 ms on lock 0x7b5800115e98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:29:17.243812 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:17.244570 (+   758us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.244810 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15646592}
I20260709 12:29:17.253024 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "54443d308bbc43359e598ace1951bd70" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.254061 22287 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35724 (request call id 218) took 1212 ms. Trace:
I20260709 12:29:17.254405 22287 rpcz_store.cc:276] 0709 12:29:16.041992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.042122 (+   130us) service_pool.cc:224] Handling call
0709 12:29:17.252838 (+1210716us) spinlock_profiling.cc:243] Waited 7.74 ms on lock 0x7b5800115e98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:29:17.253966 (+  1128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17794304}
I20260709 12:29:17.263715 22497 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.264446 22497 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.264802 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.266435 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:17.267289 21386 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35346 (request call id 20) took 1167 ms (client timeout 1539 ms). Trace:
I20260709 12:29:17.267619 22497 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:29:17.267649 21386 rpcz_store.cc:269] 0709 12:29:16.099914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.149771 (+ 49857us) service_pool.cc:224] Handling call
0709 12:29:17.267229 (+1117458us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:17.236689 22290 rpcz_store.cc:276] 0709 12:29:16.046848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.047014 (+   166us) service_pool.cc:224] Handling call
0709 12:29:17.235241 (+1188227us) spinlock_profiling.cc:243] Waited 5.62 ms on lock 0x7b5800115e98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:29:17.235505 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:17.235735 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:17.235774 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:17.235823 (+    49us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:17.235865 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.235931 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:17.236088 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12923648}
I20260709 12:29:17.276046 21460 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:29:17.285363 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.286440 22283 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35716 (request call id 334) took 1126 ms. Trace:
I20260709 12:29:17.288061 22283 rpcz_store.cc:276] 0709 12:29:16.159695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.308242 (+148547us) service_pool.cc:224] Handling call
0709 12:29:17.285171 (+976929us) spinlock_profiling.cc:243] Waited 6.49 ms on lock 0x7b5800115e98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:29:17.286367 (+  1196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14931200}
I20260709 12:29:17.293844 22497 raft_consensus.cc:493] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.294497 22497 raft_consensus.cc:515] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.296478 21304 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:47030 (request call id 538) took 1203 ms. Trace:
I20260709 12:29:17.296867 22497 leader_election.cc:290] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:17.296928 21304 rpcz_store.cc:276] 0709 12:29:16.092731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.092847 (+   116us) service_pool.cc:224] Handling call
0709 12:29:17.296269 (+1203422us) spinlock_profiling.cc:243] Waited 5.6 ms on lock 0x7b54003e7954. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:29:17.296413 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12877696}
I20260709 12:29:17.303401 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:17.303844 21303 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:17.305420 21425 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.307572 22499 raft_consensus.cc:2749] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.307761 21308 raft_consensus.cc:1240] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 3. Current term is 4. Ops: []
I20260709 12:29:17.316192 21278 leader_election.cc:304] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.319207 22495 consensus_queue.cc:1059] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:17.320215 22495 raft_consensus.cc:2749] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.320888 22495 raft_consensus.cc:3055] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:17.320724 22500 consensus_queue.cc:1059] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:17.321260 22495 raft_consensus.cc:740] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:17.321897 22473 raft_consensus.cc:3055] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:17.322266 22473 raft_consensus.cc:740] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:17.322491 22495 consensus_queue.cc:260] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.323153 22473 consensus_queue.cc:260] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.323772 22495 raft_consensus.cc:3060] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:17.324817 22499 consensus_queue.cc:1059] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:17.325968 22479 raft_consensus.cc:3055] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:17.326316 22479 raft_consensus.cc:740] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:17.327237 22479 consensus_queue.cc:260] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.329892 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:17.335969 22496 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.336637 22496 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.337054 21311 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47030 (request call id 552) took 1212 ms. Trace:
I20260709 12:29:17.337559 21311 rpcz_store.cc:276] 0709 12:29:16.124327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.162380 (+ 38053us) service_pool.cc:224] Handling call
0709 12:29:17.336084 (+1173704us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:17.336626 (+   542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:17.336663 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:17.336716 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:29:17.336762 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.336834 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:17.336966 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:17.338071 21425 leader_election.cc:304] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.334451 21277 leader_election.cc:304] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.343840 22162 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.344205 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.343878 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.346352 22496 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.347710 21425 leader_election.cc:304] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.348953 21425 leader_election.cc:304] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.351518 21277 leader_election.cc:304] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.352587 22502 raft_consensus.cc:2749] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.344887 22283 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:17.363070 22501 raft_consensus.cc:2749] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.357221 22496 raft_consensus.cc:2749] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.366122 22503 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.364795 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:17.367178 21311 raft_consensus.cc:1275] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:17.367656 21305 raft_consensus.cc:2468] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:29:17.370307 22479 raft_consensus.cc:3060] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.378746 22505 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.379343 22505 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:17.380170 21460 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58790 (request call id 34) took 1358 ms (client timeout 1614 ms). Trace:
W20260709 12:29:17.380756 21460 rpcz_store.cc:269] 0709 12:29:16.021798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.022868 (+  1070us) service_pool.cc:224] Handling call
0709 12:29:17.379918 (+1357050us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54004fe2d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:29:17.380070 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2797231488}
I20260709 12:29:17.381552 22505 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:17.382126 22505 raft_consensus.cc:2749] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.383260 21310 raft_consensus.cc:1240] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:17.383579 22492 raft_consensus.cc:2804] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:17.373492 22502 raft_consensus.cc:2804] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:17.384310 22492 raft_consensus.cc:697] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:17.373324 22165 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.386757 22165 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.384441 22502 raft_consensus.cc:493] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
W20260709 12:29:17.406297 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 62544.7us
I20260709 12:29:17.406978 22162 leader_election.cc:304] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.385746 22492 consensus_queue.cc:237] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.419597 22162 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.374094 22503 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.404135 22473 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.431732 22507 raft_consensus.cc:2749] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:17.432065 22500 raft_consensus.cc:493] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.432484 22507 raft_consensus.cc:2749] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.432572 22500 raft_consensus.cc:515] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.433161 22507 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:29:17.434049 22506 raft_consensus.cc:493] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.434579 22506 raft_consensus.cc:515] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.435695 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.436640 22507 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.436971 22506 leader_election.cc:290] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.427359 21309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47030 (request call id 549) took 1301 ms. Trace:
I20260709 12:29:17.404336 22502 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.437218 22507 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:17.445246 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 58572.1us
I20260709 12:29:17.446395 22507 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:17.404623 22505 raft_consensus.cc:2749] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:29:17.429567 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 51286.3us
I20260709 12:29:17.447803 22510 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.448169 22287 raft_consensus.cc:1240] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:29:17.430162 22508 raft_consensus.cc:493] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.447422 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.453372 22508 raft_consensus.cc:515] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.464134 22508 leader_election.cc:290] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.464784 22508 raft_consensus.cc:493] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.465245 22508 raft_consensus.cc:515] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.467449 22499 raft_consensus.cc:493] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:17.443387 21309 rpcz_store.cc:276] 0709 12:29:16.117545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:16.122258 (+  4713us) service_pool.cc:224] Handling call
0709 12:29:16.162523 (+ 40265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:29:17.391390 (+1228867us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:17.391423 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:29:17.403805 (+ 12382us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:17.403889 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:17.405647 (+  1758us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:17.418812 (+ 13165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:29:17.468029 22499 raft_consensus.cc:515] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.470083 22499 leader_election.cc:290] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.456837 22500 leader_election.cc:290] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.454038 22284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.472256 22284 raft_consensus.cc:2468] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:17.456065 22503 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.487267 22503 raft_consensus.cc:493] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.488003 22503 raft_consensus.cc:515] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.486639 21132 catalog_manager.cc:5697] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:17.492434 22165 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.448428 22510 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.495764 22509 raft_consensus.cc:2804] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:17.496349 22509 raft_consensus.cc:697] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 3 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:17.497344 22509 consensus_queue.cc:237] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.470551 22499 raft_consensus.cc:493] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:17.503110 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.490440 22503 leader_election.cc:290] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.507053 21305 raft_consensus.cc:2393] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 4: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:17.521180 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.546921 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.559384 22512 raft_consensus.cc:493] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:17.560567 22512 raft_consensus.cc:515] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.565306 22512 leader_election.cc:290] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.565851 22500 raft_consensus.cc:493] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.566656 22512 consensus_queue.cc:1059] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:17.566679 22500 raft_consensus.cc:515] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.566830 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.569343 22512 raft_consensus.cc:3055] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:17.569456 21381 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:17.569783 22512 raft_consensus.cc:740] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:17.568187 22520 consensus_queue.cc:1059] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:17.567785 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.571899 22512 consensus_queue.cc:260] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.573536 22500 leader_election.cc:290] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.574460 22500 raft_consensus.cc:3055] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:17.575047 22500 raft_consensus.cc:740] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:17.576330 22500 consensus_queue.cc:260] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.582729 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.585126 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.585300 22510 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.585939 21304 raft_consensus.cc:2393] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:17.586639 22498 raft_consensus.cc:493] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.587100 22498 raft_consensus.cc:515] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.573549 22512 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:17.589250 22498 leader_election.cc:290] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.590499 21426 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.517722 22495 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.597065 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.590950 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.518764 22473 raft_consensus.cc:493] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:17.507599 22499 raft_consensus.cc:515] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.609184 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
W20260709 12:29:17.609906 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 190379us
I20260709 12:29:17.610844 22162 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.611716 22503 raft_consensus.cc:2749] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.613056 22162 leader_election.cc:304] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.613919 22503 raft_consensus.cc:2749] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.615454 22162 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.616281 22503 raft_consensus.cc:2749] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.617842 22162 leader_election.cc:304] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.618693 22503 raft_consensus.cc:2749] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.620482 22162 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.621672 22503 raft_consensus.cc:2749] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:17.622990 22162 leader_election.cc:304] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.623903 22503 raft_consensus.cc:2749] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.625723 22162 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.582484 21310 raft_consensus.cc:2468] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:17.628295 22508 leader_election.cc:290] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.521829 22479 consensus_queue.cc:1048] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 12:29:17.638774 22496 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.639448 22496 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.641356 22502 raft_consensus.cc:515] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.641939 22496 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.604394 22495 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.627482 22473 raft_consensus.cc:515] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.647908 22473 leader_election.cc:290] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.631350 22162 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.606273 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.626536 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.598721 22521 raft_consensus.cc:2749] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.634735 22497 raft_consensus.cc:493] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.666572 22497 raft_consensus.cc:515] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.669006 22497 leader_election.cc:290] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.636317 22501 raft_consensus.cc:493] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.663662 22162 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.669693 22501 raft_consensus.cc:515] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.670161 22505 raft_consensus.cc:2749] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.671085 22495 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.642030 22499 leader_election.cc:290] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:17.672080 22501 leader_election.cc:290] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.672290 22523 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.672590 22524 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.672866 22523 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.673305 22524 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.676057 22524 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:17.665360 22503 raft_consensus.cc:2749] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.677748 22162 leader_election.cc:304] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.678635 22503 raft_consensus.cc:2749] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.678731 22489 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.679250 22489 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.679319 21135 catalog_manager.cc:5697] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 3 to 4, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 4 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:17.681646 22489 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.677872 21386 raft_consensus.cc:1240] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 4 LEADER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 3. Current term is 4. Ops: []
I20260709 12:29:17.683444 22523 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.687543 22521 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:17.688037 22521 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.686816 22504 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.688910 22504 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.676254 22483 raft_consensus.cc:493] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.664618 22509 raft_consensus.cc:2749] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:17.685873 22445 raft_consensus.cc:493] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.686468 22484 raft_consensus.cc:493] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.686993 22490 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.686729 22519 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.685037 22162 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.682135 22489 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.690289 22521 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:17.695060 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:17.694245 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 49078us
I20260709 12:29:17.706518 22484 raft_consensus.cc:515] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.695327 22445 raft_consensus.cc:515] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.707026 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.683535 22429 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.707746 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:17.693863 21424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:17.705709 21425 leader_election.cc:304] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:17.708307 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.710489 22483 raft_consensus.cc:515] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.713863 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:17.714893 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 89075.5us
I20260709 12:29:17.719684 21275 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:17.717394 21426 leader_election.cc:304] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
W20260709 12:29:17.720189 21425 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:17.718070 22162 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.723097 22473 raft_consensus.cc:2804] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:17.723522 22473 raft_consensus.cc:493] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.710901 22489 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.723918 22473 raft_consensus.cc:3060] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.717053 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac0d0feb45a42ccaa8859902754853c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.714499 22519 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.726871 22519 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:17.727030 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.714422 22504 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:17.721808 22496 raft_consensus.cc:2804] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:17.728734 22496 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.729110 22496 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.715757 22503 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.713135 22490 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.733619 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c956710bf64e4bb5b40784ba8060147c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.733906 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "81d20a4b91864a4787df9fdd31977f44" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.734992 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.722102 22429 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.686524 22165 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:29:17.756469 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 70022.9us
I20260709 12:29:17.757202 22165 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.757396 22429 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.758055 22511 consensus_queue.cc:1059] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:17.758368 22514 raft_consensus.cc:2749] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:17.743505 22500 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:17.745250 22503 raft_consensus.cc:2749] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:17.745814 22484 leader_election.cc:290] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.746197 22162 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:29:17.798267 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 52157.7us
I20260709 12:29:17.799031 22162 leader_election.cc:304] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.747740 22502 leader_election.cc:290] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.748384 22445 leader_election.cc:290] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.753871 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.802423 21278 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.802762 22507 raft_consensus.cc:2749] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.803715 22507 raft_consensus.cc:493] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:17.804150 22507 raft_consensus.cc:515] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.804697 22495 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:17.805150 22495 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.805518 22495 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:17.806246 22507 leader_election.cc:290] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.809502 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.810364 21309 raft_consensus.cc:2393] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:29:17.813338 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.775133 21383 raft_consensus.cc:3055] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:17.814448 21383 raft_consensus.cc:740] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:17.815486 21383 consensus_queue.cc:260] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.816512 21383 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.767580 22514 raft_consensus.cc:2749] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:17.779670 21460 raft_consensus.cc:1240] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 1. Current term is 2. Ops: []
W20260709 12:29:17.776069 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 80092.4us
I20260709 12:29:17.780447 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.817255 22495 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.781443 22483 leader_election.cc:290] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.782815 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "79a82803d22242a991a35ece42323ea7" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.781153 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.819548 22283 raft_consensus.cc:2468] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:17.784240 22490 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.785915 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.787267 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.788332 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.821496 21456 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:17.789577 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.790649 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "315529067cd1423da537bde53365bbbe" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.791250 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.823015 22495 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.791311 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.792424 22289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.792517 21423 leader_election.cc:304] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.824800 22500 raft_consensus.cc:2749] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.825065 21423 leader_election.cc:304] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.825834 22500 raft_consensus.cc:2749] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.766773 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.775130 22489 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.772055 22510 raft_consensus.cc:2804] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:17.826795 21423 leader_election.cc:304] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.827057 22510 raft_consensus.cc:493] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.827484 22510 raft_consensus.cc:3060] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.828462 21423 leader_election.cc:304] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.829227 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.794090 22531 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.829883 21452 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:17.830317 22531 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.832574 22531 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.832741 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.797457 22431 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.833904 22521 raft_consensus.cc:2749] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.834021 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.762176 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.802839 22505 raft_consensus.cc:2749] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:17.834942 22287 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 5.
I20260709 12:29:17.794235 22506 raft_consensus.cc:3055] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:17.836041 22506 raft_consensus.cc:740] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:17.836565 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.837247 21456 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 4.
I20260709 12:29:17.836970 22506 consensus_queue.cc:260] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.838675 22506 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:17.834116 22431 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.840865 22431 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.827620 22500 raft_consensus.cc:2749] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.813614 22496 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.844352 22496 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:17.847105 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.850847 22496 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.851423 22496 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.816097 22473 raft_consensus.cc:515] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:29:17.802546 22162 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.8.137.195:44499 timed out after 1.539s (SENT) [suppressed 254 similar messages]
I20260709 12:29:17.855111 22496 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:17.857000 22496 consensus_queue.cc:1059] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:17.858302 22431 raft_consensus.cc:3055] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:17.858738 22431 raft_consensus.cc:740] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:17.859666 22431 consensus_queue.cc:260] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.860992 22431 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.861337 21277 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
W20260709 12:29:17.867488 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 49851.4us
W20260709 12:29:17.868221 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 54553.8us
I20260709 12:29:17.869784 22483 consensus_queue.cc:1059] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:17.870918 22445 raft_consensus.cc:3055] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:17.871331 22445 raft_consensus.cc:740] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:17.872213 22445 consensus_queue.cc:260] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.873160 22522 consensus_queue.cc:1059] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:17.873392 22445 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.874465 22484 raft_consensus.cc:3055] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:17.874873 22484 raft_consensus.cc:740] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:17.875836 22484 consensus_queue.cc:260] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.879999 21383 raft_consensus.cc:1275] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:17.881384 21456 raft_consensus.cc:1240] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:17.883128 22484 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:17.885196 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 82675.2us
I20260709 12:29:17.886883 22505 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:17.887565 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.887414 22505 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:17.888720 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 149518us
I20260709 12:29:17.889374 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.890079 21378 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:17.892985 21426 leader_election.cc:304] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.893177 22431 raft_consensus.cc:2749] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:17.894717 21309 raft_consensus.cc:1240] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 3. Current term is 4. Ops: []
I20260709 12:29:17.896417 21309 raft_consensus.cc:1240] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:17.897377 22502 raft_consensus.cc:2749] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:29:17.898036 21309 raft_consensus.cc:1240] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 3. Current term is 4. Ops: []
I20260709 12:29:17.899654 21425 leader_election.cc:304] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.900769 22479 raft_consensus.cc:2749] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.903775 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.903878 21426 leader_election.cc:304] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.904475 21459 raft_consensus.cc:2468] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:17.905052 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.899683 21277 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:17.906008 22479 raft_consensus.cc:2804] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:17.906071 21456 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:29:17.906486 22479 raft_consensus.cc:493] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.906957 22479 raft_consensus.cc:3060] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.907343 22431 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:17.907765 22431 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.908114 22431 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:17.896843 22505 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:17.911847 21277 leader_election.cc:304] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.912721 22495 raft_consensus.cc:2749] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:17.913884 21426 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:17.914808 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:17.915014 21426 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:17.915705 22500 raft_consensus.cc:2749] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:17.915462 21309 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.917637 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.918059 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.918951 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.919450 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.920141 21458 raft_consensus.cc:2468] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:29:17.920480 21275 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:17.921062 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.921240 22495 raft_consensus.cc:2804] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:17.921625 22495 raft_consensus.cc:493] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:17.921937 22495 raft_consensus.cc:3060] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:17.917656 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.923487 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.923938 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.924706 22289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.925417 22289 raft_consensus.cc:2468] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:17.925527 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d0bfa2f2d224261803bbc1ee6978296" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.924889 22283 raft_consensus.cc:2393] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 4: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:29:17.927263 21456 raft_consensus.cc:1275] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:17.928601 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.930234 22479 raft_consensus.cc:515] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:17.930859 22495 raft_consensus.cc:515] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:17.931785 22431 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.937403 22511 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:29:17.937858 22511 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:17.938232 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.938230 22511 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:17.939579 21309 raft_consensus.cc:2468] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:17.883198 22510 raft_consensus.cc:515] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.943054 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96593726ac324bc78be4e7c2bdb7d850" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:17.943338 22510 leader_election.cc:290] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.943746 21310 raft_consensus.cc:3060] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:17.948626 22500 consensus_queue.cc:1048] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:17.949162 22505 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:17.949612 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.949880 22505 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:17.948679 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.950285 21378 raft_consensus.cc:3060] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:17.951905 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.951930 22495 leader_election.cc:290] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:17.953580 22500 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:17.954962 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f78e89ba4d49471aaf6eb57ea8f738bf" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.956529 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.957378 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:17.957569 22284 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.958153 22292 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:17.958186 22505 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:17.959405 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.957636 22282 raft_consensus.cc:2393] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 4: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:29:17.960986 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.961753 21459 raft_consensus.cc:2393] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:17.958346 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:17.962944 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:17.963653 22290 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:17.954174 22500 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:17.966760 22500 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.967213 22522 consensus_queue.cc:1059] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:17.967911 22505 consensus_queue.cc:1048] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:17.968479 22484 raft_consensus.cc:3055] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:17.969141 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.982991 21310 raft_consensus.cc:2468] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:17.931226 22473 leader_election.cc:290] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.983561 22484 raft_consensus.cc:740] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:17.969251 21351 leader_election.cc:304] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:17.971472 21307 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:17.970170 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.972069 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:17.973585 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:17.974398 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:17.987829 21308 raft_consensus.cc:2368] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:17.991144 21459 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:17.991992 21459 raft_consensus.cc:2393] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
W20260709 12:29:17.992781 21481 debug-util.cc:398] Leaking SignalData structure 0x7b0800153740 after lost signal to thread 8746
I20260709 12:29:17.990161 22522 raft_consensus.cc:2804] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:17.993060 21378 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:17.993633 22522 raft_consensus.cc:697] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
W20260709 12:29:17.993898 21481 debug-util.cc:398] Leaking SignalData structure 0x7b08000e0d20 after lost signal to thread 21169
W20260709 12:29:17.995409 21481 debug-util.cc:398] Leaking SignalData structure 0x7b0800149ee0 after lost signal to thread 21336
W20260709 12:29:17.996660 21481 debug-util.cc:398] Leaking SignalData structure 0x7b08001bcce0 after lost signal to thread 21410
W20260709 12:29:17.998049 21481 debug-util.cc:398] Leaking SignalData structure 0x7b080028af60 after lost signal to thread 21484
W20260709 12:29:17.999290 21481 debug-util.cc:398] Leaking SignalData structure 0x7b0800256a80 after lost signal to thread 22336
I20260709 12:29:17.995141 22522 consensus_queue.cc:237] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.003829 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:18.004745 21307 raft_consensus.cc:1275] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:29:17.975651 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 59595.8us
I20260709 12:29:17.977983  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:18.007195 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:18.008041 21310 raft_consensus.cc:2368] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 for earlier term 2. Current term is 3.
I20260709 12:29:17.971796 22292 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:17.932509 22479 leader_election.cc:290] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:17.942724 22431 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:17.984576 22509 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:17.984649 22484 consensus_queue.cc:260] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.010088 22509 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:18.010690 22484 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:18.011289 22445 consensus_queue.cc:1059] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:18.012291 22509 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:18.012562 22489 raft_consensus.cc:3055] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:18.013005 22489 raft_consensus.cc:740] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.013970 22489 consensus_queue.cc:260] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.017077 22489 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:18.017659 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:18.018307 22290 raft_consensus.cc:3060] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.004686 21305 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:18.005021 21275 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:17.978417 21131 catalog_manager.cc:5697] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 2 to 3, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 3 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.023619 22496 raft_consensus.cc:2804] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:18.024133 22496 raft_consensus.cc:697] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:18.025035 22496 consensus_queue.cc:237] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.025902 22290 raft_consensus.cc:2468] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:17.974828 22511 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.029047 22511 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:18.030367 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:18.030963 21310 raft_consensus.cc:1240] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.031597 22484 consensus_queue.cc:1059] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:18.033525 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:18.034134 22285 raft_consensus.cc:3060] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.034171 22281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.034970 22281 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
I20260709 12:29:18.036863 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:18.037974 22445 consensus_queue.cc:1059] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:18.038357 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.038533 22281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:18.039234 22281 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:18.039525 22533 raft_consensus.cc:3055] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:18.039943 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:18.040156 22533 raft_consensus.cc:740] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.040644 21460 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:18.041116 22533 consensus_queue.cc:260] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.042330 22285 raft_consensus.cc:2468] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:18.042975 22490 raft_consensus.cc:3055] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:18.043457 22490 raft_consensus.cc:740] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.044751 21309 raft_consensus.cc:1240] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.044466 22490 consensus_queue.cc:260] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.063414 22490 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.065358 22489 raft_consensus.cc:493] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:18.065955 22489 raft_consensus.cc:515] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.068153 22489 leader_election.cc:290] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:18.045617 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:18.048029 21305 raft_consensus.cc:1240] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.048192 22165 leader_election.cc:400] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173) with higher term. Message: Invalid argument: T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:18.081949 22165 leader_election.cc:403] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:18.049153 21460 raft_consensus.cc:1240] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 5. Current term is 6. Ops: []
I20260709 12:29:18.049259 22533 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:18.082764 22509 raft_consensus.cc:2749] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173) with higher term. Message: Invalid argument: T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3.
I20260709 12:29:18.050288 21452 raft_consensus.cc:1240] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.050922 21425 leader_election.cc:304] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.051185 22285 raft_consensus.cc:1240] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.051229 21456 raft_consensus.cc:1240] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.052340 22282 raft_consensus.cc:1240] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.052834 22281 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 6.
I20260709 12:29:18.049212 22288 raft_consensus.cc:1240] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.085726 21425 leader_election.cc:304] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.086541 22510 raft_consensus.cc:2804] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:18.087052 22510 raft_consensus.cc:697] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:18.087888 22510 consensus_queue.cc:237] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.054876 22484 consensus_queue.cc:1059] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:29:18.091382 21352 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:18.055830 21458 raft_consensus.cc:1240] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.056193 21351 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:18.056628 22496 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.092433 21351 leader_election.cc:304] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:18.092712 22496 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.093752 22290 raft_consensus.cc:1275] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:18.094038 21310 raft_consensus.cc:2393] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 2: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:18.094800 22522 raft_consensus.cc:2804] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:18.062132 21426 leader_election.cc:304] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.095098 22496 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:18.095264 22522 raft_consensus.cc:493] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:18.095789 22522 raft_consensus.cc:3060] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.096118 21426 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.097100 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:18.091307 22484 raft_consensus.cc:3055] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:18.098194 22484 raft_consensus.cc:740] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.098461 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.099164 21378 raft_consensus.cc:2468] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 1.
I20260709 12:29:18.099259 22484 consensus_queue.cc:260] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.100703 22484 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.101438 22473 raft_consensus.cc:2804] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:18.101989 22473 raft_consensus.cc:697] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:18.055735 21455 raft_consensus.cc:1240] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.102260 21275 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:18.102331 22509 consensus_queue.cc:1048] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:18.102890 22473 consensus_queue.cc:237] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.103292 22479 raft_consensus.cc:2804] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:18.094129 21351 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:18.104063 22479 raft_consensus.cc:697] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:18.105086 22479 consensus_queue.cc:237] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.103775 22511 raft_consensus.cc:2804] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:18.105113 22429 raft_consensus.cc:2804] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:18.107048 22165 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:29:18.103374 22489 raft_consensus.cc:2764] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:29:18.108348 22509 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:18.107198 22429 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:18.108752 22509 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:18.108914 22429 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.109229 22509 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.106734 22511 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.110457 22511 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.108371 22496 raft_consensus.cc:2804] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:18.112486 22496 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.112818 22496 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.115459 22483 consensus_queue.cc:1059] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:18.115825 22445 consensus_queue.cc:1059] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:18.115906 22517 consensus_queue.cc:1059] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:18.116626 22483 consensus_queue.cc:1059] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:18.117233 22518 raft_consensus.cc:3055] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:29:18.117722 22518 raft_consensus.cc:740] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.117830 22516 consensus_queue.cc:1059] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:18.118893 22516 raft_consensus.cc:3055] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:18.119324 22516 raft_consensus.cc:740] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.120146 22516 consensus_queue.cc:260] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.121356 22516 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.117900 22489 raft_consensus.cc:3055] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:18.121677 22517 raft_consensus.cc:3055] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:18.122174 22489 raft_consensus.cc:740] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.122351 22517 raft_consensus.cc:740] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.118780 22518 consensus_queue.cc:260] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.123241 22517 consensus_queue.cc:260] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.124233 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.124573 22517 raft_consensus.cc:3060] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.125126 22285 raft_consensus.cc:2393] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 3: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:29:18.124334 22489 consensus_queue.cc:260] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.126173 22489 raft_consensus.cc:3060] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.130630 22504 raft_consensus.cc:3055] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:18.131160 22504 raft_consensus.cc:740] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.131726 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:18.132143 22504 consensus_queue.cc:260] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.132746 22509 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:18.134022 21351 leader_election.cc:400] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Vote denied by peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173) with higher term. Message: Invalid argument: T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 for earlier term 2. Current term is 3.
I20260709 12:29:18.133263 22504 raft_consensus.cc:3060] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.136101 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.136567 21277 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.136894 21455 raft_consensus.cc:2393] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:18.140462 21378 raft_consensus.cc:3060] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.141476 22288 raft_consensus.cc:1240] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Rejecting Update request from peer a3220692a27d44e49348c63d58fcf666 for earlier term 5. Current term is 6. Ops: []
I20260709 12:29:18.123673 22518 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:18.144701 22429 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.156442 22429 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:18.148972 22288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5192ed91b2d4c14905f4a0a7410da9f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.150110 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.150981 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e36b9296b6e408da863f2bd3bb7b36b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.151082 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.146821 22496 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.158160 22281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cc850ba2d5144b66bf78197d972b70d8" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.186916 22496 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:29:18.148461 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 52430.4us
I20260709 12:29:18.159387 22289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.187757 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:18.159168 22522 raft_consensus.cc:515] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.160303 22509 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:18.188467 21384 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.162551 22495 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:18.189143 22495 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.189519 22495 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.162969 21278 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.190635 22522 leader_election.cc:290] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:18.191632 21278 leader_election.cc:304] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.188441 21426 leader_election.cc:304] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.192389 22431 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:18.192842 21278 leader_election.cc:304] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.193149 22431 raft_consensus.cc:2804] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:18.193068 21426 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.193670 22431 raft_consensus.cc:493] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.193971 21278 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.166998 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31b54df203a347c993ed917ad39bbf1c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.167311 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f68cddadeb0443b28b03d76d5195fc45" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.194916 21383 raft_consensus.cc:2468] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:18.168170 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:18.169019 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fb7aa63607864dce845a979faebae247" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.169750 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaea00780b02433fa3a95bf6e01cb78d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.200908 22531 raft_consensus.cc:2749] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:18.158792 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.202420 21278 leader_election.cc:304] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.203832 22524 raft_consensus.cc:2749] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:18.205250 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.205924 21456 raft_consensus.cc:3060] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.194079 22431 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.200944 22525 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:18.178543 22516 consensus_queue.cc:1048] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:18.164829 21277 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:18.209797 22502 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:29:18.239445 22549 log.cc:927] Time spent T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce: Append to log took a long time: real 0.099s	user 0.005s	sys 0.012s
I20260709 12:29:18.210171 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.240625 21458 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:29:18.240998 22536 log.cc:927] Time spent T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
W20260709 12:29:18.242244 22543 log.cc:927] Time spent T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce: Append to log took a long time: real 0.184s	user 0.000s	sys 0.002s
I20260709 12:29:18.193989 22506 raft_consensus.cc:2804] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:18.222416 22495 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.225469 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.246387 22495 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:18.209527 21455 raft_consensus.cc:2468] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:29:18.248386 21351 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:18.232241 21384 raft_consensus.cc:2468] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:18.248631 22502 raft_consensus.cc:697] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 6 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
W20260709 12:29:18.222822 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 58048.6us
I20260709 12:29:18.249747 22502 consensus_queue.cc:237] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.252326 22429 raft_consensus.cc:2764] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:29:18.252480 21307 raft_consensus.cc:1240] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:18.253212 22506 raft_consensus.cc:697] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:18.253340 21310 raft_consensus.cc:1275] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:18.254253 22506 consensus_queue.cc:237] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.220499 21456 raft_consensus.cc:2468] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:18.258095 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:18.260641 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:18.261005 21452 raft_consensus.cc:2468] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:18.262501 21275 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:29:18.264103 21351 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:18.265106 21351 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:18.266776 22522 raft_consensus.cc:2804] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:18.267318 22522 raft_consensus.cc:697] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:18.268246 22522 consensus_queue.cc:237] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.224231 22531 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:18.248394 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:18.270043 22531 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:18.270203 21377 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.270502 22531 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:18.230731 21278 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.273635 21278 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.274495 22498 raft_consensus.cc:2804] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:18.275068 22498 raft_consensus.cc:697] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:18.275676 22496 raft_consensus.cc:2749] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:18.276360 22496 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:18.276011 22498 consensus_queue.cc:237] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.276785 22496 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.277185 22496 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:18.234918 22431 raft_consensus.cc:515] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.230858 22162 leader_election.cc:304] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:18.280997 21308 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:18.280632 21278 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:18.282116 22431 leader_election.cc:290] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:18.282791 22431 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:18.229964 22525 raft_consensus.cc:2749] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:18.283272 22431 raft_consensus.cc:697] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:18.284278 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:29:18.284272 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 53497.8us
I20260709 12:29:18.285079 21384 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.285395 22162 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:18.291898 22162 leader_election.cc:304] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:29:18.295390 22162 leader_election.cc:304] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.295815 21384 raft_consensus.cc:2468] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:18.301815 21275 leader_election.cc:304] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:18.303539 22162 leader_election.cc:304] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:18.307447 22162 leader_election.cc:304] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.307566 22525 raft_consensus.cc:2804] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:18.308670 22525 raft_consensus.cc:697] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:18.309940 22525 consensus_queue.cc:237] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.306396 22507 raft_consensus.cc:2749] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:18.339129 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.351192 21308 raft_consensus.cc:1275] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:18.369339 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:18.305434 22505 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:18.305385 22514 raft_consensus.cc:2764] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:29:18.302220 22509 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.343645 21460 raft_consensus.cc:3060] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.369350 21277 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.331617 22503 raft_consensus.cc:2749] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:29:18.343546 22544 log.cc:927] Time spent T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.289s	user 0.003s	sys 0.003s
I20260709 12:29:18.305343 22523 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.385263 21277 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.350772 21311 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9e2b7f6dfa92472f8a79868af4c7d106" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:18.366164 22477 consensus_queue.cc:1059] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:29:18.379873 22507 raft_consensus.cc:2749] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:18.397305 21311 raft_consensus.cc:2393] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 2: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:18.368340 22500 consensus_queue.cc:1048] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:29:18.349376 22511 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.366448 22165 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:18.377573 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:18.440989 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.441764 21453 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.442570 22281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:18.456328 22531 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:18.457293 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:18.457907 21384 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.458724 22531 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:18.461504 22479 raft_consensus.cc:3055] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:18.447469 21135 catalog_manager.cc:5697] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:18.406312 21460 raft_consensus.cc:2468] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:18.463498 22509 raft_consensus.cc:2749] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:18.466753 21377 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
W20260709 12:29:18.421973 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 90951.6us
I20260709 12:29:18.390297 22523 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.423004 22507 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:18.423851 21378 raft_consensus.cc:1275] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:18.425611 21458 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:18.391250 22431 consensus_queue.cc:237] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:18.415759 22559 raft_consensus.cc:2749] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:18.427040 22555 raft_consensus.cc:2749] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:29:18.432899 22511 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:18.462173 22479 raft_consensus.cc:740] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
W20260709 12:29:18.420259 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 112899us
W20260709 12:29:18.414624 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 112940us
I20260709 12:29:18.473971 21132 catalog_manager.cc:5697] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 1 to 3, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:18.543862 22162 leader_election.cc:304] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:18.479982 21384 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:18.450176 22496 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.493275 22523 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:18.546336 21275 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:18.553068 22289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f7e6811f72d0447781abab4178b3bdd4" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:18.556491 21352 leader_election.cc:304] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.560740 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f52db4db2acf4abfa01aa7604963edbc" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:18.561635 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.563319 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.578580 21384 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 4.
I20260709 12:29:18.578637 21383 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:18.580653 21275 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:18.583666 22162 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:18.586560 22287 raft_consensus.cc:2393] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 3: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:29:18.588271 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.588825 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4d4f5802ffb348c78eaca62ca6fac2fc" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:18.609719 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 221306us
I20260709 12:29:18.590842 21352 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.589583 21385 raft_consensus.cc:2468] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:18.614274 22162 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:18.576108 22507 raft_consensus.cc:3060] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.629781 21352 leader_election.cc:304] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.641403 21423 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:18.648393 21352 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.653698 22162 leader_election.cc:304] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:29:18.654989 21352 leader_election.cc:304] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.656400 21352 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:18.657716 22518 raft_consensus.cc:2749] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:18.574791 22477 consensus_queue.cc:1048] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
I20260709 12:29:18.589563 22525 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:18.608664 22429 raft_consensus.cc:2749] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:18.628595 22509 raft_consensus.cc:2749] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:18.658480 22518 raft_consensus.cc:2749] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:18.647142 22517 raft_consensus.cc:2749] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:18.650507 22504 raft_consensus.cc:2749] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:18.691922 22556 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:18.692451 22556 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:18.692994 22556 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:18.699502 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:18.700523 22283 raft_consensus.cc:2393] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 4: Already voted for candidate 482723f787a14005854e038688430a80 in this term.
I20260709 12:29:18.687887 22557 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:18.665109 22490 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:18.682806 22525 raft_consensus.cc:697] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:18.660063 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:18.689531 22479 consensus_queue.cc:260] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.673516 22503 raft_consensus.cc:2764] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:29:18.674750 22506 raft_consensus.cc:2804] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:29:18.732421 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 120307us
I20260709 12:29:18.715132 22502 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.715904 22557 raft_consensus.cc:493] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:18.761288 21310 raft_consensus.cc:1275] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:18.762034 21351 leader_election.cc:304] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:18.730237 22506 raft_consensus.cc:697] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:18.765938 22506 consensus_queue.cc:237] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:18.777602 22289 raft_consensus.cc:1275] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:18.760267 22557 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:18.767860 22518 raft_consensus.cc:2749] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:18.795300 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 87153.2us
I20260709 12:29:18.795862 22570 consensus_queue.cc:1048] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:18.781636 22505 raft_consensus.cc:2749] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:18.782572 22527 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:18.768041 22522 consensus_queue.cc:1048] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.031s
I20260709 12:29:18.757359 22502 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:18.805197 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 44841.4us
I20260709 12:29:18.792255 22479 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:18.767414 22165 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:29:18.816430 21459 raft_consensus.cc:1275] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:18.827174 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:18.828750 22287 raft_consensus.cc:3060] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:18.830919 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:18.819774 22496 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:18.821493 21382 raft_consensus.cc:1275] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:18.843976 21460 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 4.
I20260709 12:29:18.812435 22527 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:18.830911 21453 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
W20260709 12:29:18.850284 22564 log.cc:927] Time spent T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:29:18.840397 22477 consensus_queue.cc:1048] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 12:29:18.797397 22525 consensus_queue.cc:237] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:18.916106 21380 raft_consensus.cc:1275] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:18.922369 22287 raft_consensus.cc:1275] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:18.925007 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:18.833984 22572 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:29:18.841311 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 81398.2us
W20260709 12:29:18.949607 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 123262us
I20260709 12:29:18.958293 21277 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:18.959167 22498 consensus_queue.cc:1048] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:18.960103 22575 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:18.960704 22575 raft_consensus.cc:697] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 4 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:18.961616 22575 consensus_queue.cc:237] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:18.979039 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 194977us
I20260709 12:29:18.984741 21381 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:18.954628 22572 raft_consensus.cc:697] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:18.991125 21305 raft_consensus.cc:1275] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:19.009280 22162 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:29:18.965224 22431 consensus_queue.cc:1048] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 12:29:19.016428 22514 raft_consensus.cc:2804] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:19.018967 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:19.019325 22514 raft_consensus.cc:697] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:19.019490 22574 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:19.020061 22574 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:19.020364 22514 consensus_queue.cc:237] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:19.022969 22574 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:29:19.038131 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 94727.8us
I20260709 12:29:19.037940 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:18.991309 22572 consensus_queue.cc:237] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:19.064872 22534 consensus_queue.cc:1048] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:29:19.070127 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:19.071296 21383 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:19.055751 22527 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:19.091733 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
W20260709 12:29:19.097640 22580 log.cc:927] Time spent T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:29:19.110641 22583 consensus_queue.cc:1048] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 12:29:19.116291 21383 raft_consensus.cc:2468] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 5.
I20260709 12:29:19.119154 22556 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:19.123306 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
W20260709 12:29:19.129868 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 161326us
I20260709 12:29:19.136569 22283 raft_consensus.cc:1275] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:19.151857 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:19.159865 22579 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.160497 22579 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:19.161144 21378 raft_consensus.cc:1275] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:19.150054 21452 raft_consensus.cc:1275] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:19.173954 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:19.183259 21385 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:19.174122 21382 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:29:19.175514 21377 raft_consensus.cc:1275] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:19.108080 22557 raft_consensus.cc:515] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:19.189459 22282 raft_consensus.cc:1275] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:19.193558 22512 consensus_queue.cc:1048] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:19.182628 22579 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:19.195544 22484 consensus_queue.cc:1048] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:19.170899 22556 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:19.218093 22489 consensus_queue.cc:1048] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:19.212049 22534 consensus_queue.cc:1048] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 12:29:19.223973 22590 log.cc:927] Time spent T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.148s	user 0.004s	sys 0.000s
I20260709 12:29:19.225648 22557 leader_election.cc:290] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:19.215396 21378 raft_consensus.cc:1275] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:19.236887 22589 log.cc:927] Time spent T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:29:19.237483 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:19.233788 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
W20260709 12:29:19.232165 22507 consensus_meta.cc:287] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80: Time spent flushing consensus metadata: real 0.573s	user 0.009s	sys 0.004s
I20260709 12:29:19.239140 21385 raft_consensus.cc:2368] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 4. Current term is 5.
I20260709 12:29:19.234995 22165 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:19.239156 22507 raft_consensus.cc:2749] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:19.253646 22288 raft_consensus.cc:1275] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:19.254508 22514 raft_consensus.cc:2749] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:19.257571 22284 raft_consensus.cc:1275] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:19.261591 22162 leader_election.cc:400] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Vote denied by peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499) with higher term. Message: Invalid argument: T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 4. Current term is 5.
I20260709 12:29:19.262068 22162 leader_election.cc:403] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:29:19.277195 22514 raft_consensus.cc:3060] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:19.280071 22562 consensus_queue.cc:1048] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:29:19.300621 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f08ca19d90734deca4d113875d7def85" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:19.301487 21307 raft_consensus.cc:2368] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 4. Current term is 5.
I20260709 12:29:19.302942 21453 raft_consensus.cc:1275] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:19.320365 21132 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60570 (request call id 116) took 1191 ms. Trace:
I20260709 12:29:19.330009 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:19.331022 21307 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:19.337143 21458 raft_consensus.cc:1275] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:19.344485 21306 raft_consensus.cc:1275] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:19.346110 22431 consensus_queue.cc:1048] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:29:19.335023 22609 log.cc:927] Time spent T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:29:19.355150 21425 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:19.325598 22556 consensus_queue.cc:1048] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:29:19.353724 22165 leader_election.cc:400] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Vote denied by peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173) with higher term. Message: Invalid argument: T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 4. Current term is 5.
I20260709 12:29:19.368826 22514 raft_consensus.cc:2749] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499) with higher term. Message: Invalid argument: T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 4. Current term is 5.
I20260709 12:29:19.373421 22571 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:19.373947 22571 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.374282 22571 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:19.321007 21132 rpcz_store.cc:276] 0709 12:29:18.114410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.114727 (+   317us) service_pool.cc:224] Handling call
0709 12:29:18.323349 (+208622us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b54006bd5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094c0090 00007f4e0955ef3a 00007f4e0f314e9c 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc 00007f4e0d8d3392 00007f4e0789148c 00007f4e078908c6
0709 12:29:19.305818 (+982469us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:18.588833 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:18.848733 (+259900us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b54006bd428. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f9129e1 00007f4e0f91164c 00007f4e0cfa0d5d 00007f4e0cf8c0a6 00007f4e0cf8bd58 00007f4e0cf909f1 00007f4e0cf909a9 00007f4e0cf90939 00007f4e0cf90901 00007f4e0cf8fb2d 0000000000388634
0709 12:29:18.848803 (+    70us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:18.848837 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:29:18.848857 (+    20us) tablet.cc:662] Decoding operations
0709 12:29:18.922988 (+ 74131us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:18.931144 (+  8156us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:18.942460 (+ 11316us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:18.979449 (+ 36989us) tablet.cc:701] Row locks acquired
0709 12:29:18.979470 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:29:19.003604 (+ 24134us) write_op.cc:270] Start()
0709 12:29:19.017803 (+ 14199us) write_op.cc:276] Timestamp: P: 1783600158995580 usec, L: 0
0709 12:29:19.024620 (+  6817us) op_driver.cc:348] REPLICATION: starting
0709 12:29:19.071498 (+ 46878us) log.cc:844] Serialized 540 byte log entry
0709 12:29:19.173506 (+102008us) op_driver.cc:464] REPLICATION: finished
0709 12:29:19.179574 (+  6068us) write_op.cc:301] APPLY: starting
0709 12:29:19.179684 (+   110us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:19.179808 (+   124us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:19.179842 (+    34us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:19.180180 (+   338us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:19.180411 (+   231us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:29:19.180482 (+    71us) write_op.cc:312] APPLY: finished
0709 12:29:19.180703 (+   221us) log.cc:844] Serialized 22 byte log entry
0709 12:29:19.180914 (+   211us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:19.181029 (+   115us) write_op.cc:454] Released schema lock
0709 12:29:19.181080 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:19.181532 (+   452us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:19.181578 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":2,"spinlock_wait_cycles":476315392,"child_traces":[["op",{"apply.queue_time_us":5805,"apply.run_cpu_time_us":1946,"apply.run_wall_time_us":1946,"num_ops":1,"prepare.queue_time_us":1079,"prepare.run_cpu_time_us":9013,"prepare.run_wall_time_us":510314,"replication_time_us":148706,"spinlock_wait_cycles":15859456,"thread_start_us":5337,"threads_started":1}]]}
I20260709 12:29:19.379659 22502 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:19.387567 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:19.388283 21382 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:19.389197 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:19.404572 21381 raft_consensus.cc:1275] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:19.406828 21378 raft_consensus.cc:1275] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:19.416989 21423 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:19.425765 22507 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.427672 22507 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:19.429500 22290 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:19.413612 22285 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:29:19.429903 22507 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:19.431365 21378 raft_consensus.cc:1275] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:19.399627 22506 consensus_queue.cc:1048] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:29:19.437062 22290 raft_consensus.cc:1275] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:19.450673 21275 leader_election.cc:304] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:19.451838 22559 raft_consensus.cc:2804] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:19.452507 22559 raft_consensus.cc:697] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:19.452739 21275 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:19.453840 22559 consensus_queue.cc:237] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:19.454720 22495 raft_consensus.cc:2749] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:19.455494 22495 consensus_queue.cc:1048] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:19.466370 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 46506.7us
I20260709 12:29:19.460196 22523 consensus_queue.cc:1048] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:29:19.463562 22477 raft_consensus.cc:2804] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:19.552438 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:19.557132 21455 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
W20260709 12:29:19.581207 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 60939.6us
W20260709 12:29:19.581169 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 89416.7us
I20260709 12:29:19.586361 21134 catalog_manager.cc:5697] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 1 to 2, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.587095 22528 consensus_queue.cc:1048] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:29:19.472142 22534 consensus_queue.cc:1048] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 12:29:19.567550 22574 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:19.606935 22510 consensus_queue.cc:1048] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:19.607609 22574 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:19.609989 22574 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:19.610622 22484 consensus_queue.cc:1048] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:19.611287 22559 consensus_queue.cc:1048] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:19.612010 22484 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.612524 22484 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:19.566844 22477 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.616285 21378 raft_consensus.cc:1275] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:19.501255 22502 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 12:29:19.632593 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
W20260709 12:29:19.639534 22616 log.cc:927] Time spent T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:29:19.645598 21304 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 4.
I20260709 12:29:19.627957 22484 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:19.629096 22477 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:19.649262 22507 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.644766 22574 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:19.717242 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 60557.6us
I20260709 12:29:19.650784 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:19.648898 22499 consensus_queue.cc:1048] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 12:29:19.700098 22579 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:29:19.755538 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:19.752456 21306 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:19.770051 22285 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
W20260709 12:29:19.771143 22618 log.cc:927] Time spent T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:29:19.772315 22614 log.cc:927] Time spent T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:29:19.738261 22574 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:19.733651 22507 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:19.805976 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 170897us
I20260709 12:29:19.811474 21351 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:19.832796 21135 catalog_manager.cc:5697] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.844713 22521 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.846941 21135 catalog_manager.cc:5697] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 5 to 6, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 6 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.818934 22571 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:19.859556 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:19.861117 22571 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:19.865232 22507 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:19.867327 22517 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:19.867914 22517 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:19.850395 22521 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:19.871676 22574 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:19.872720 22521 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:19.801143 22165 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:29:19.845659 22587 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:29:19.873713 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 85378.3us
I20260709 12:29:19.874110 22587 raft_consensus.cc:493] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.874480 22587 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:19.874892 22572 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:19.875308 22572 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.875631 22572 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:19.881899 21456 raft_consensus.cc:1240] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:19.848148 22514 consensus_queue.cc:1048] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:29:19.887513 22289 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:19.888212 22289 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 6.
I20260709 12:29:19.897365 21352 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:19.898304 21377 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382"
dest_uuid: "a3220692a27d44e49348c63d58fcf666"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:47062
I20260709 12:29:19.898831 22504 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:29:19.899235 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 63514.2us
I20260709 12:29:19.899710 22504 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:19.900254 22504 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:29:19.900821 21134 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60574 (request call id 126) took 1680 ms. Trace:
I20260709 12:29:19.899261 21377 raft_consensus.cc:606] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:19.919778 21135 catalog_manager.cc:5697] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:19.922425 21135 catalog_manager.cc:5697] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:19.924860 21135 catalog_manager.cc:5697] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 2, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 2 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:19.927376 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 176794us
I20260709 12:29:19.870337 22517 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:19.937151 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 86041.1us
W20260709 12:29:19.936041 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 219928us
I20260709 12:29:19.959405 21455 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:19.901250 21134 rpcz_store.cc:276] 0709 12:29:18.220856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.221023 (+   167us) service_pool.cc:224] Handling call
0709 12:29:18.325602 (+104579us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54006bd5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094c0090 00007f4e0955ef3a 00007f4e0f314e9c 00007f4e0f41304e 00007f4e0d8d42a4 00007f4e0d8d4232 00007f4e0d8d4161 00007f4e0d8d40dc 00007f4e0d8d3392 00007f4e0789148c 00007f4e078908c6
0709 12:29:19.900750 (+1575148us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:19.596912 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:19.604481 (+  7569us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:19.604529 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:29:19.604555 (+    26us) tablet.cc:662] Decoding operations
0709 12:29:19.604979 (+   424us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:19.605040 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:19.605085 (+    45us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:19.605401 (+   316us) tablet.cc:701] Row locks acquired
0709 12:29:19.605425 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:29:19.605624 (+   199us) write_op.cc:270] Start()
0709 12:29:19.605761 (+   137us) write_op.cc:276] Timestamp: P: 1783600159605541 usec, L: 0
0709 12:29:19.605790 (+    29us) op_driver.cc:348] REPLICATION: starting
0709 12:29:19.628458 (+ 22668us) log.cc:844] Serialized 540 byte log entry
0709 12:29:19.641264 (+ 12806us) op_driver.cc:464] REPLICATION: finished
0709 12:29:19.656049 (+ 14785us) write_op.cc:301] APPLY: starting
0709 12:29:19.656183 (+   134us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:19.656334 (+   151us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:19.704818 (+ 48484us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:19.755717 (+ 50899us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:19.780500 (+ 24783us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:29:19.784352 (+  3852us) write_op.cc:312] APPLY: finished
0709 12:29:19.806475 (+ 22123us) log.cc:844] Serialized 22 byte log entry
0709 12:29:19.815963 (+  9488us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:19.833999 (+ 18036us) write_op.cc:454] Released schema lock
0709 12:29:19.838539 (+  4540us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":73799,"reported_tablets":2,"spinlock_wait_cycles":237514496,"child_traces":[["op",{"apply.queue_time_us":5168,"num_ops":1,"prepare.queue_time_us":1654,"prepare.run_cpu_time_us":3299,"prepare.run_wall_time_us":49258,"replication_time_us":35354,"spinlock_wait_cycles":48274688}]]}
I20260709 12:29:19.962258 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:19.963186 21304 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:19.944322 21458 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:29:20.010634 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 48806.5us
I20260709 12:29:20.031474 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:20.018610 22292 raft_consensus.cc:1275] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:20.021427 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:20.034111 21384 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 1.
W20260709 12:29:20.022439 22622 log.cc:927] Time spent T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:29:20.023386 22587 raft_consensus.cc:515] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:20.024636 21308 raft_consensus.cc:1275] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:19.946316 22574 consensus_queue.cc:1059] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.097s
I20260709 12:29:19.955500 21456 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:20.043598 22587 leader_election.cc:290] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:20.044348 21380 raft_consensus.cc:2368] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 4. Current term is 5.
I20260709 12:29:20.026940 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:20.045948 22507 raft_consensus.cc:3055] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:20.046399 22507 raft_consensus.cc:740] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:20.047341 22507 consensus_queue.cc:260] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:20.048641 21458 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:20.030304 21425 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:20.049867 21457 raft_consensus.cc:1275] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:20.050531 22479 raft_consensus.cc:2804] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:20.051237 22479 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:20.051723 22479 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:20.055348 21451 raft_consensus.cc:1275] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:29:20.037149 21382 consensus_meta.cc:287] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666: Time spent flushing consensus metadata: real 0.648s	user 0.008s	sys 0.004s
I20260709 12:29:20.018568 21381 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:20.067723 21304 raft_consensus.cc:2393] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 4: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:20.161950 21384 raft_consensus.cc:1275] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:20.132781 21382 raft_consensus.cc:2468] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 5.
I20260709 12:29:20.252930 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
W20260709 12:29:20.255740 22162 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:20.260172 21307 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:20.263530 21384 raft_consensus.cc:1275] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:29:20.254056 22477 consensus_meta.cc:287] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce: Time spent flushing consensus metadata: real 0.523s	user 0.001s	sys 0.009s
W20260709 12:29:20.231389 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 308768us
I20260709 12:29:20.269042 22556 consensus_queue.cc:1048] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:29:20.273499 22477 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:20.288160 21451 raft_consensus.cc:1240] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Rejecting Update request from peer 482723f787a14005854e038688430a80 for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:20.288327 22289 raft_consensus.cc:1275] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:20.283576 22504 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:20.303563 22479 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:20.306022 22477 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:20.306123 22479 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:20.302855 22572 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:20.312454 22504 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 7 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:20.313061 22572 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:20.365155 21275 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:20.373967 21378 raft_consensus.cc:1240] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:29:20.379799 21385 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:20.383340 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:20.385829 21383 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:29:20.425547 22165 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:20.439124 22642 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:20.439740 22642 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:20.440204 22642 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:20.440524 22523 raft_consensus.cc:493] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:20.432502 22504 raft_consensus.cc:493] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:20.433442 21307 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 5.
I20260709 12:29:20.443115 22572 consensus_queue.cc:1059] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:20.444422 22431 consensus_queue.cc:1048] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:20.466380 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:20.467525 21454 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:20.469772 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3cd8e0c4a39c4a85bc4a2663a8a39c5f" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:20.449174 22585 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.476488 22162 leader_election.cc:400] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499) with higher term. Message: Invalid argument: T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 4. Current term is 5.
I20260709 12:29:20.477011 22162 leader_election.cc:403] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:29:20.449672 22525 consensus_queue.cc:1048] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:20.451983 22556 raft_consensus.cc:2804] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:20.457104 21383 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:20.430675 22516 raft_consensus.cc:493] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:20.481371 22556 raft_consensus.cc:697] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:20.481420 22516 raft_consensus.cc:515] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:20.422230 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:20.473783 22585 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:20.485651 21306 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:20.484617 22572 raft_consensus.cc:3055] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:20.498698 22292 raft_consensus.cc:3060] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:29:20.484951 22516 leader_election.cc:290] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:20.482537 22507 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:20.485929 22431 consensus_queue.cc:1048] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:29:20.510524 22572 raft_consensus.cc:740] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:20.539953 22572 consensus_queue.cc:260] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:20.539966 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:20.541615 21308 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:20.541862 22572 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:20.529979 22523 raft_consensus.cc:515] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:20.530974 22504 raft_consensus.cc:515] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:20.483275 22556 consensus_queue.cc:237] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:20.560552 22504 leader_election.cc:290] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:20.561058 22523 leader_election.cc:290] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
W20260709 12:29:20.570986 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 48620.5us
I20260709 12:29:20.578274 22292 raft_consensus.cc:2468] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 5.
I20260709 12:29:20.593888 21454 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 5.
I20260709 12:29:20.596349 22165 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:29:20.601548 22574 raft_consensus.cc:493] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:20.602787 22514 raft_consensus.cc:2749] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499) with higher term. Message: Invalid argument: T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for earlier term 4. Current term is 5.
W20260709 12:29:20.604727 22616 log.cc:927] Time spent T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:29:20.608137 21306 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:20.602272 22574 raft_consensus.cc:515] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:20.616156 22504 raft_consensus.cc:493] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:20.625414 21423 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:20.615017 22642 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:20.641861 22479 consensus_queue.cc:1059] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 12:29:20.642784 22511 raft_consensus.cc:2804] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:20.644114 22504 raft_consensus.cc:515] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:20.648432 22574 leader_election.cc:290] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:20.663127 22479 raft_consensus.cc:3055] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:29:20.663734 22479 raft_consensus.cc:740] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:20.665164 22479 consensus_queue.cc:260] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:20.670868 22511 raft_consensus.cc:697] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:20.672676 22511 consensus_queue.cc:237] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
W20260709 12:29:20.673529 22636 log.cc:927] Time spent T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.390s	user 0.004s	sys 0.000s
I20260709 12:29:20.676005 21386 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39594 (request call id 609) took 1016 ms. Trace:
I20260709 12:29:20.676131 21351 leader_election.cc:304] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:20.678767 22484 raft_consensus.cc:2804] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:20.676680 21386 rpcz_store.cc:276] 0709 12:29:19.659859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.678567 (+ 18708us) service_pool.cc:224] Handling call
0709 12:29:19.707630 (+ 29063us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:19.731282 (+ 23652us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:19.731311 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:19.830292 (+ 98981us) spinlock_profiling.cc:243] Waited 948 us on lock 0x7b2c00168bb8. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:29:19.830723 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:29:19.935649 (+104926us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:19.935709 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:19.935816 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:20.672562 (+736746us) spinlock_profiling.cc:243] Waited 1 ms on lock 0x7b4400022b4c. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e0638fc85 00007f4e063906a2 00007f4e091021db 00007f4e090fdaaf 00007f4e090f186e 00007f4e090f0bb6 00007f4e090f07d8 00007f4e090fa221 00007f4e090fa1d9 00007f4e090fa169
0709 12:29:20.675518 (+  2956us) raft_consensus.cc:1685] finished
0709 12:29:20.675545 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:20.675782 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23159,"raft.queue_time_us":25843,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":13151,"spinlock_wait_cycles":4488448,"thread_start_us":104495,"threads_started":1,"wal-append.queue_time_us":336320}
I20260709 12:29:20.679593 22484 raft_consensus.cc:697] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 5 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
W20260709 12:29:20.658895 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 62837.8us
I20260709 12:29:20.681344 22484 consensus_queue.cc:237] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:20.685158 22574 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:20.686147 22574 raft_consensus.cc:697] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 5 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:20.687700 22574 consensus_queue.cc:237] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:20.696444 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:20.699412 22575 raft_consensus.cc:493] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:20.700296 22585 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:20.700569 22575 raft_consensus.cc:515] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:20.705575 22575 leader_election.cc:290] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:20.705777 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
W20260709 12:29:20.705592 21424 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:20.710320 21453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:20.713964 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:20.723568 21425 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:20.730732 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:20.731067 22283 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:20.731678 21457 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
W20260709 12:29:20.732815 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 256780us
I20260709 12:29:20.733822 22162 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:20.735980 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:20.737748 21309 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7d3bfcb7836d457caed630ee21058a19" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:20.739846 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:20.741847 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:20.745448 21310 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:20.747225 21308 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:20.751214 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:20.751268 22583 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:20.752718 22583 raft_consensus.cc:697] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:20.754103 22583 consensus_queue.cc:237] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:20.756717 22504 leader_election.cc:290] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:20.759538 21378 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:20.779882 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 46142.5us
I20260709 12:29:20.790959 22653 raft_consensus.cc:2749] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:20.792558 22653 raft_consensus.cc:493] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:20.793174 22653 raft_consensus.cc:515] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:20.795099 22284 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35732 (request call id 282) took 1060 ms. Trace:
I20260709 12:29:20.796766 22281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dbd17ab2b7d04cc19cf64cdd44c442a7" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:20.795549 22284 rpcz_store.cc:276] 0709 12:29:19.734324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.736420 (+  2096us) service_pool.cc:224] Handling call
0709 12:29:19.906916 (+170496us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b5800115e98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e52ffc0 00007f4e0e51a929 00007f4e0e51a3d2 00007f4e07a91bd4
0709 12:29:19.907167 (+   251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:19.907596 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:19.907636 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:19.908210 (+   574us) log.cc:844] Serialized 26 byte log entry
0709 12:29:20.271668 (+363458us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:29:20.290244 (+ 18576us) log.cc:844] Serialized 14 byte log entry
0709 12:29:20.317744 (+ 27500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:20.317856 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:20.791942 (+474086us) spinlock_profiling.cc:243] Waited 4.23 ms on lock 0x7b1800170140. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e0f331ab1 00007f4e0f33e281 00007f4e064693f3 00007f4e0646b7c1 00007f4e0646b779 00007f4e0646b709 00007f4e0646b6d1 00007f4e0646a9cd 0000000000388634 0000000000388479 00007f4e0644c83a
0709 12:29:20.794667 (+  2725us) raft_consensus.cc:1685] finished
0709 12:29:20.794692 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:20.794898 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5780,"prepare.queue_time_us":6896,"prepare.run_cpu_time_us":149,"prepare.run_wall_time_us":5841,"raft.queue_time_us":22198,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":22211,"spinlock_wait_cycles":18829568,"thread_start_us":350478,"threads_started":1,"wal-append.queue_time_us":389513}
I20260709 12:29:20.801054 22292 raft_consensus.cc:1240] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:29:20.802351 22653 leader_election.cc:290] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:20.805002 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:20.805835 22654 raft_consensus.cc:493] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:20.808243 22654 raft_consensus.cc:515] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:20.809531 22285 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:20.811648 22654 leader_election.cc:290] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:20.811936 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:20.812177 21131 catalog_manager.cc:5697] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:20.815538 22479 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:29:20.825621 22654 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:20.828167 22654 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:20.830739 22654 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:20.836694 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:20.841648 21135 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60588 (request call id 129) took 2158 ms. Trace:
W20260709 12:29:20.846343 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 59554.5us
I20260709 12:29:20.847164 21351 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:20.857178 22281 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:20.860073 22292 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13c370d75f794be68da9f93a6a876a55" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:20.864617 22283 raft_consensus.cc:2410] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:29:20.874543 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4da9543a3474f228769c7d28609a702" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:20.877594 21352 leader_election.cc:304] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:20.814199 21131 catalog_manager.cc:5697] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:20.880791 21310 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 5.
I20260709 12:29:20.882009 21351 leader_election.cc:304] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:20.875890 22556 consensus_queue.cc:1059] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.022s
I20260709 12:29:20.907033 22518 raft_consensus.cc:2749] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:20.907979 22570 raft_consensus.cc:2749] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:20.914311 22556 raft_consensus.cc:3055] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:29:20.914907 22556 raft_consensus.cc:740] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 4 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:20.916028 22556 consensus_queue.cc:260] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:20.918450 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:20.919440 21452 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:20.919826 22584 raft_consensus.cc:493] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:20.920545 22584 raft_consensus.cc:515] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:20.917793 22556 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:20.842276 21135 rpcz_store.cc:276] 0709 12:29:18.689485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:18.689778 (+   293us) service_pool.cc:224] Handling call
0709 12:29:20.841181 (+2151403us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:20.028525 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:20.053129 (+ 24604us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:20.053163 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:29:20.053175 (+    12us) tablet.cc:662] Decoding operations
0709 12:29:20.053956 (+   781us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:20.054027 (+    71us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:20.054056 (+    29us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:29:20.054679 (+   623us) tablet.cc:701] Row locks acquired
0709 12:29:20.054714 (+    35us) write_op.cc:260] PREPARE: finished
0709 12:29:20.054891 (+   177us) write_op.cc:270] Start()
0709 12:29:20.055019 (+   128us) write_op.cc:276] Timestamp: P: 1783600160054842 usec, L: 0
0709 12:29:20.055046 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:29:20.281336 (+226290us) log.cc:844] Serialized 2085 byte log entry
0709 12:29:20.305941 (+ 24605us) op_driver.cc:464] REPLICATION: finished
0709 12:29:20.306292 (+   351us) write_op.cc:301] APPLY: starting
0709 12:29:20.306414 (+   122us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:20.306689 (+   275us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:20.306843 (+   154us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:20.307738 (+   895us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:20.308024 (+   286us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:29:20.308102 (+    78us) write_op.cc:312] APPLY: finished
0709 12:29:20.308540 (+   438us) log.cc:844] Serialized 46 byte log entry
0709 12:29:20.308794 (+   254us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:20.308958 (+   164us) write_op.cc:454] Released schema lock
0709 12:29:20.309025 (+    67us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:20.562416 (+253391us) spinlock_profiling.cc:243] Waited 98.7 ms on lock 0x7b54006bd5d4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e09551e28 00007f4e0cf8c581 00007f4e0cf8bd58 00007f4e0cf909f1 00007f4e0cf909a9 00007f4e0cf90939 00007f4e0cf90901 00007f4e0cf8fb2d 0000000000388634 0000000000388479
0709 12:29:20.581809 (+ 19393us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:20.588096 (+  6287us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":60994,"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":103,"apply.run_cpu_time_us":3113,"apply.run_wall_time_us":3130,"num_ops":5,"prepare.queue_time_us":18785,"prepare.run_cpu_time_us":7197,"prepare.run_wall_time_us":534089,"replication_time_us":250742,"spinlock_wait_cycles":227073408}]]}
I20260709 12:29:20.932767 21384 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85044ca33b9947dface5200a544f79a2" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:20.933739 21384 raft_consensus.cc:2393] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 5: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:20.924546 21131 catalog_manager.cc:5697] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 reported cstate change: term changed from 2 to 3, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } }
I20260709 12:29:20.941932 22516 raft_consensus.cc:993] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:20.942835 21453 raft_consensus.cc:2410] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:29:20.937762 22585 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:20.955469 21382 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "de84b253ee104e63be65f2ef9e9173dd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:20.943676 22642 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:20.957940 21351 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:20.959134 22484 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:29:20.936671 21131 catalog_manager.cc:5697] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 3, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } }
I20260709 12:29:20.942572 22516 raft_consensus.cc:1081] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:20.978746 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f44f7e65b47e48649e1e7ab852e0fc84" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:20.969432 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:20.989022 22162 leader_election.cc:304] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:20.983261 21131 catalog_manager.cc:5697] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } }
I20260709 12:29:20.984812 22584 leader_election.cc:290] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:21.002447 22507 raft_consensus.cc:3060] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:21.013427 22290 raft_consensus.cc:2410] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:29:21.003130 22162 leader_election.cc:304] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:21.022837 22287 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f6dbe944bf742148d2688474712308b" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:21.026726 22162 leader_election.cc:304] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:21.030471 22585 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:21.031134 22585 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:21.031486 22290 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35716 (request call id 419) took 1078 ms. Trace:
I20260709 12:29:21.031946 22290 rpcz_store.cc:276] 0709 12:29:19.952851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:19.954997 (+  2146us) service_pool.cc:224] Handling call
0709 12:29:20.694388 (+739391us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b5800115e98. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e062f1db1 00007f4e06431b71 00007f4e06431a22 00007f4e0f34a8d9 00007f4e0e4a4ee0 00007f4e0e56e0ab 00007f4e0e56e131 00007f4e0e51c0a0 00007f4e0e51b984 00007f4e07a966d4 00007f4e07a96662
0709 12:29:21.031216 (+336828us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b5400007dd4. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e094bf571 00007f4e0955a2fa 00007f4e0e51bb17 00007f4e07a966d4 00007f4e07a96662 00007f4e07a96591 00007f4e07a9650c 00007f4e07a957c2 00007f4e0789148c 00007f4e078908c6 00007f4e0789026f
0709 12:29:21.031388 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685098496}
I20260709 12:29:21.041023 22287 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35732
I20260709 12:29:21.041860 22287 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:21.042263 22287 raft_consensus.cc:3060] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:21.036612 22572 raft_consensus.cc:2749] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:21.051187 22577 consensus_queue.cc:1048] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:21.059906 21352 leader_election.cc:304] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:21.060894 21278 leader_election.cc:304] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:21.061337 22517 raft_consensus.cc:2749] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:29:21.061899 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 73562us
I20260709 12:29:21.034533 22642 raft_consensus.cc:2749] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:21.068588 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:21.067095 22577 raft_consensus.cc:2749] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:21.065313 21452 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 5.
W20260709 12:29:21.088322 22661 log.cc:927] Time spent T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:29:21.091982 22165 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:21.099846 21383 raft_consensus.cc:1275] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:21.113652 21426 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:21.116452 22631 raft_consensus.cc:2749] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:29:21.116858 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0cc4e1a190343f19f100f757eab0c65" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:21.107173 22572 raft_consensus.cc:2749] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:21.097234 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:21.135185 22165 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:29:21.138091 22574 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:21.138847 22574 raft_consensus.cc:697] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 5 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:21.140069 22574 consensus_queue.cc:237] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:21.141032 22577 consensus_queue.cc:1048] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:21.143610 21306 raft_consensus.cc:2410] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:29:21.155902 22662 log.cc:927] Time spent T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:29:21.160754 21453 raft_consensus.cc:1275] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:21.162741 22572 consensus_queue.cc:1048] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:21.164739 22507 raft_consensus.cc:2749] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:21.170502 22287 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:21.197475 21277 leader_election.cc:304] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:21.199572 21460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:21.200371 21460 raft_consensus.cc:606] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Received request to transfer leadership to TS b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:21.201754 22585 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:21.202386 22577 raft_consensus.cc:2749] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:29:21.235430 22658 log.cc:927] Time spent T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:29:21.235939 21383 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:21.236629 21383 raft_consensus.cc:3055] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:21.236925 21383 raft_consensus.cc:740] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:21.237685 21383 consensus_queue.cc:260] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:21.244445 21380 raft_consensus.cc:1240] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Rejecting Update request from peer b565aef1b93143c78c5c3dc4a861a68a for earlier term 3. Current term is 4. Ops: []
I20260709 12:29:21.243923 22585 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:21.262288 22287 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:21.264724 21383 raft_consensus.cc:3060] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:21.265991 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:21.268801 21306 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
W20260709 12:29:21.278959 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 196194us
I20260709 12:29:21.289283 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:21.289541 21423 leader_election.cc:304] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:21.289979 21460 raft_consensus.cc:3060] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:21.298159 21457 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1f486fb8a1874281ab5022d0aea60379"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:21.297578 21306 raft_consensus.cc:1275] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:21.298985 21457 raft_consensus.cc:606] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Received request to transfer leadership to TS b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:21.299419 21457 raft_consensus.cc:614] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:21.300021 21308 raft_consensus.cc:1275] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:21.300894 21351 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:21.305653 21460 raft_consensus.cc:2468] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:21.264524 22577 consensus_queue.cc:1059] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:29:21.316749 22499 consensus_queue.cc:1048] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:29:21.317075 22517 raft_consensus.cc:2749] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:21.317873 22284 raft_consensus.cc:1275] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:21.318466 22507 consensus_queue.cc:1048] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:21.320055 22165 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:29:21.316012 22523 raft_consensus.cc:3055] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:21.321223 22523 raft_consensus.cc:740] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:21.322197 22523 consensus_queue.cc:260] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:21.323040 22572 raft_consensus.cc:2804] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:21.323493 22523 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:21.323599 22572 raft_consensus.cc:697] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:21.324752 22572 consensus_queue.cc:237] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:21.333815 21386 raft_consensus.cc:1275] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:21.336108 21304 raft_consensus.cc:1275] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:21.346093 22290 raft_consensus.cc:1275] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:29:21.346915 21308 raft_consensus.cc:1275] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:21.346925 22290 pending_rounds.cc:85] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Aborting all ops after (but not including) 1
I20260709 12:29:21.352223 22290 pending_rounds.cc:107] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:29:21.352627 22290 raft_consensus.cc:2889] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:21.355899 22516 consensus_queue.cc:1048] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:21.355937 22581 consensus_queue.cc:1048] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:29:21.362782 22582 consensus_queue.cc:1048] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:21.364631 22631 consensus_queue.cc:1048] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:21.363023 22284 raft_consensus.cc:1275] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:21.366981 22499 consensus_queue.cc:1048] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:21.371555 21383 raft_consensus.cc:2468] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:21.382975 21460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:21.383868 21460 raft_consensus.cc:606] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:21.384285 21460 raft_consensus.cc:614] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:21.393486 22518 consensus_queue.cc:1048] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260709 12:29:21.397818 21460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:21.400365 21460 raft_consensus.cc:606] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Received request to transfer leadership to TS b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:21.439754 21460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:21.440369 21460 raft_consensus.cc:606] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:21.451433 22584 raft_consensus.cc:2749] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:21.443921  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:29:21.462388 22583 raft_consensus.cc:993] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower b565aef1b93143c78c5c3dc4a861a68a to start an election
I20260709 12:29:21.463796 22583 raft_consensus.cc:1081] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Signalling peer b565aef1b93143c78c5c3dc4a861a68a to start an election
I20260709 12:29:21.473459  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:21.474915  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:21.478091 22569 raft_consensus.cc:993] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower b565aef1b93143c78c5c3dc4a861a68a to start an election
I20260709 12:29:21.478875 22569 raft_consensus.cc:1081] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Signalling peer b565aef1b93143c78c5c3dc4a861a68a to start an election
I20260709 12:29:21.482303 21311 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494"
dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
 from {username='slave'} at 127.0.0.1:47030
I20260709 12:29:21.482957 21311 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:21.483326 21311 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:21.509215 21309 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f"
dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
 from {username='slave'} at 127.0.0.1:47030
I20260709 12:29:21.510177 21309 raft_consensus.cc:493] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:21.510550 21309 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:21.529193 21309 raft_consensus.cc:515] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:21.542878 21309 leader_election.cc:290] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:21.545985 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:21.546774 21377 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:21.560537 22572 raft_consensus.cc:493] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:21.561175 22572 raft_consensus.cc:515] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:21.557080 21131 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49894 (request call id 35) took 1198 ms. Trace:
I20260709 12:29:21.563333 22572 leader_election.cc:290] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:21.564522 21311 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:21.570219 21311 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:21.586303 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:21.591914 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:21.599318 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a8f34093df9d4e6db1b428ae3b532fec" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:21.604580 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:21.605275 21454 raft_consensus.cc:3055] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:21.605602 21454 raft_consensus.cc:740] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:21.606525 21454 consensus_queue.cc:260] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:21.607693 22165 leader_election.cc:304] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:21.608978 22572 raft_consensus.cc:2749] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:21.607692 21454 raft_consensus.cc:3060] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:29:21.613936 22684 log.cc:927] Time spent T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:29:21.615965 21132 catalog_manager.cc:5697] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 1 to 3, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:21.625793 21377 raft_consensus.cc:2468] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:21.631340 21386 raft_consensus.cc:1240] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:29:21.682844 21132 catalog_manager.cc:5697] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 3 to 5, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 5 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:21.562666 21131 rpcz_store.cc:276] 0709 12:29:20.349640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.350192 (+   552us) service_pool.cc:224] Handling call
0709 12:29:21.547489 (+1197297us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:21.125049 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:21.125873 (+   824us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:21.125947 (+    74us) write_op.cc:435] Acquired schema lock
0709 12:29:21.125968 (+    21us) tablet.cc:662] Decoding operations
0709 12:29:21.126914 (+   946us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:21.127030 (+   116us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:21.127066 (+    36us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:29:21.127993 (+   927us) tablet.cc:701] Row locks acquired
0709 12:29:21.128087 (+    94us) write_op.cc:260] PREPARE: finished
0709 12:29:21.128332 (+   245us) write_op.cc:270] Start()
0709 12:29:21.128524 (+   192us) write_op.cc:276] Timestamp: P: 1783600161128245 usec, L: 0
0709 12:29:21.128662 (+   138us) op_driver.cc:348] REPLICATION: starting
0709 12:29:21.129681 (+  1019us) log.cc:844] Serialized 2093 byte log entry
0709 12:29:21.339178 (+209497us) op_driver.cc:464] REPLICATION: finished
0709 12:29:21.444843 (+105665us) write_op.cc:301] APPLY: starting
0709 12:29:21.480066 (+ 35223us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:21.487332 (+  7266us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:21.490421 (+  3089us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:21.515396 (+ 24975us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:21.515744 (+   348us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:29:21.515813 (+    69us) write_op.cc:312] APPLY: finished
0709 12:29:21.516139 (+   326us) log.cc:844] Serialized 46 byte log entry
0709 12:29:21.516321 (+   182us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:21.516447 (+   126us) write_op.cc:454] Released schema lock
0709 12:29:21.516511 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:21.517031 (+   520us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:21.517066 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":414378,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":97778,"apply.run_cpu_time_us":3444,"apply.run_wall_time_us":79717,"num_ops":5,"prepare.queue_time_us":248,"prepare.run_cpu_time_us":5088,"prepare.run_wall_time_us":25009,"replication_time_us":210310,"thread_start_us":73330,"threads_started":1}]]}
W20260709 12:29:21.726897 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 98337.4us
W20260709 12:29:21.778803 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 46903.2us
I20260709 12:29:21.780664 21132 catalog_manager.cc:5697] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 3 to 4, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 4 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:21.786897 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:21.787739 21452 raft_consensus.cc:3055] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:21.788115 21452 raft_consensus.cc:740] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:21.789001 21452 consensus_queue.cc:260] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:21.782665 21132 catalog_manager.cc:5697] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 1 to 3, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:21.793118 21377 raft_consensus.cc:1275] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:21.796398 22572 consensus_queue.cc:1048] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:21.792172 22516 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:21.802529 22516 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:21.805176 21275 leader_election.cc:304] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:21.805312 22516 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:21.807034 22634 raft_consensus.cc:2804] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:21.807514 22634 raft_consensus.cc:697] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:21.808339 22634 consensus_queue.cc:237] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:21.826653 21308 raft_consensus.cc:1275] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:29:21.829540 21308 pending_rounds.cc:85] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a: Aborting all ops after (but not including) 2
I20260709 12:29:21.829913 21308 pending_rounds.cc:107] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:29:21.830235 21308 raft_consensus.cc:2889] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:21.836386 22282 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:21.837216 22282 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:21.845383 21352 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:21.854902 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:21.855757 21460 raft_consensus.cc:2393] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 3: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
W20260709 12:29:21.858528 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 43697.2us
I20260709 12:29:21.850797 22576 raft_consensus.cc:2804] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:21.863068 22576 raft_consensus.cc:493] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:21.863473 22576 raft_consensus.cc:3060] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:21.864714 21457 raft_consensus.cc:1275] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:29:21.865463 22686 log.cc:927] Time spent T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:29:21.866838 21453 raft_consensus.cc:1275] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:21.868559 22642 consensus_queue.cc:1048] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:21.849830 21452 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:29:21.884451 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 78349.4us
W20260709 12:29:21.931057 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 101802us
I20260709 12:29:21.984627 22642 consensus_queue.cc:1048] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:29:22.044518 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 114332us
W20260709 12:29:22.051080 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 109297us
I20260709 12:29:22.094234 21452 raft_consensus.cc:2468] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 4.
I20260709 12:29:22.068938 21454 raft_consensus.cc:2468] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
W20260709 12:29:22.167547 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 143365us
W20260709 12:29:22.175974 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 135465us
W20260709 12:29:22.181718 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 197525us
I20260709 12:29:22.190915 22572 consensus_queue.cc:1048] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:22.194149 22676 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:22.194818 22676 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:22.193691 22576 raft_consensus.cc:515] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:22.197067 22676 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:22.210665 22520 raft_consensus.cc:993] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:22.211200 22520 raft_consensus.cc:1081] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:22.214160 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:22.215188 21377 raft_consensus.cc:2393] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 7: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:29:22.219097 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:22.220059 21452 raft_consensus.cc:2393] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 3: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:22.238271 22288 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:29:22.238957 22288 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:22.239336 22288 raft_consensus.cc:3060] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:22.245601 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:22.246486 21452 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 7.
I20260709 12:29:22.249212 21277 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:22.252866 22288 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:22.253016 22165 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:22.254566 22676 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:29:22.254969 22676 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:22.255398 22676 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:29:22.260804 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:22.261619 21452 raft_consensus.cc:3055] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:22.261973 21452 raft_consensus.cc:740] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:22.263087 21452 consensus_queue.cc:260] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:22.264353 21452 raft_consensus.cc:3060] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:22.265856 22676 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:22.267010 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:22.263110 22576 leader_election.cc:290] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:22.273416 22288 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:29:22.516018 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 244878us
W20260709 12:29:22.518889 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 269753us
I20260709 12:29:22.523905 21377 raft_consensus.cc:3060] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:22.529883 22695 raft_consensus.cc:2804] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:22.589522 22290 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdd6e626d31d4da8a2e4a407e1710ba2" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80"
I20260709 12:29:22.585644 22676 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 7 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:22.524101 22525 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.588404 21377 raft_consensus.cc:2468] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:22.596510 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:22.602141 22525 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:22.605304 22525 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:22.608142 21423 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:29:22.601188 21452 raft_consensus.cc:2468] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:22.617911 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:22.618860 21377 raft_consensus.cc:2393] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 482723f787a14005854e038688430a80 in current term 7: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:29:22.616654 22162 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
W20260709 12:29:22.626415 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 95942us
I20260709 12:29:22.626374 22705 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:22.651391 21379 raft_consensus.cc:1240] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:29:22.662994 21460 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 7.
I20260709 12:29:22.630455 22695 raft_consensus.cc:697] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 4 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:22.638399 22676 raft_consensus.cc:2804] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:22.673825 22288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
I20260709 12:29:22.674858 22676 raft_consensus.cc:697] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:22.667022 22705 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:22.690739 22165 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666
W20260709 12:29:22.724484 22162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 47033.8us
I20260709 12:29:22.734683 22705 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:22.800262 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 80530.1us
W20260709 12:29:22.793300 22714 log.cc:927] Time spent T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 12:29:22.833101 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 77417.1us
I20260709 12:29:22.713161 22695 consensus_queue.cc:237] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:22.848665 22288 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "482723f787a14005854e038688430a80" is_pre_election: true
W20260709 12:29:22.848762 22165 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 173154us
I20260709 12:29:22.828816 22290 raft_consensus.cc:2468] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:22.838826 22724 raft_consensus.cc:493] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:22.890933 22724 raft_consensus.cc:515] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:29:22.901889 22718 log.cc:927] Time spent T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce: Append to log took a long time: real 0.305s	user 0.001s	sys 0.002s
I20260709 12:29:22.811225 22676 consensus_queue.cc:237] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:22.886533 22584 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:22.911284 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:22.851540 22723 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:29:22.888546 21379 raft_consensus.cc:1275] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:29:22.918576 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 105133us
I20260709 12:29:22.922433 21277 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:22.927380 22694 raft_consensus.cc:2749] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:29:22.937728 21452 raft_consensus.cc:1275] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:22.928448 22723 raft_consensus.cc:697] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 7 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:22.939715 22723 consensus_queue.cc:237] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:22.944288 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 122453us
I20260709 12:29:22.945626 22527 consensus_queue.cc:1048] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:22.917588 22584 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:23.039523 22584 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:23.046759 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:23.047597 21377 raft_consensus.cc:2393] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 4: Already voted for candidate a3220692a27d44e49348c63d58fcf666 in this term.
I20260709 12:29:22.959224 22634 consensus_queue.cc:1048] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 12:29:23.062220 22572 raft_consensus.cc:493] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a3220692a27d44e49348c63d58fcf666)
I20260709 12:29:23.062948 22572 raft_consensus.cc:515] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:23.065359 22572 leader_election.cc:290] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:23.067884 21352 leader_election.cc:304] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce
I20260709 12:29:23.070027 22732 raft_consensus.cc:2804] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:23.070569 22732 raft_consensus.cc:697] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:23.071583 22732 consensus_queue.cc:237] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:23.078403 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:23.079314 21457 raft_consensus.cc:2410] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:29:23.083702 22649 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:23.084314 22649 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:23.087308 22649 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:23.090780 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:23.091660 21454 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:23.093010 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:23.093732 21380 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 4.
I20260709 12:29:23.099987 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:23.103359 21305 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 4.
I20260709 12:29:23.101411 21277 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:23.110100 22577 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:23.110561 22577 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:23.110953 22577 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:23.116238 22667 raft_consensus.cc:493] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1095bdbabc944624b91311b39a5a02ce)
I20260709 12:29:23.116950 22667 raft_consensus.cc:515] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:23.119982 22667 leader_election.cc:290] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:23.125522 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:23.100319 21453 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58790 (request call id 70) took 1158 ms. Trace:
I20260709 12:29:23.133810 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:23.128334 21453 rpcz_store.cc:276] 0709 12:29:21.941696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.942689 (+   993us) service_pool.cc:224] Handling call
0709 12:29:22.021103 (+ 78414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:29:22.525681 (+504578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:29:22.531704 (+  6023us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:29:23.038832 (+507128us) spinlock_profiling.cc:243] Waited 2.51 ms on lock 0x7b2c00168d18. stack: 00007f4e06437343 00007f4e06437240 00007f4e05dc61d4 00007f4e0fb04304 00007f4e0fb04299 00007f4e07e4fa11 00007f4e07e5f372 00007f4e09591a01 00007f4e0955643a 00007f4e0955583e 00007f4e0e51a419 00007f4e07a91bd4 00007f4e07a91b62 00007f4e07a91a91 00007f4e07a91a0c 00007f4e07a90cc2
0709 12:29:23.039472 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 12:29:23.097578 (+ 58106us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:29:23.097649 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:29:23.097753 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:29:23.099966 (+  2213us) raft_consensus.cc:1685] finished
0709 12:29:23.099985 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:29:23.100225 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2720,"raft.queue_time_us":64026,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":4293,"spinlock_wait_cycles":5765888,"thread_start_us":393796,"threads_started":2,"wal-append.queue_time_us":56056}
I20260709 12:29:23.138010 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b17439cec0b24d57853f995ce7c8e494" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:23.143627 21352 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:23.162433 21454 raft_consensus.cc:2393] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 4: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
W20260709 12:29:23.156889 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 47478.5us
I20260709 12:29:23.156932 22705 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:29:23.173071 21425 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: a3220692a27d44e49348c63d58fcf666
I20260709 12:29:23.176410 22581 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:23.177007 22581 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:23.177467 22581 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:23.202107 21351 leader_election.cc:304] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:23.128135  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:23.196949 22577 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:23.225741 22577 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:23.228770 22576 raft_consensus.cc:3060] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:23.239239 22576 raft_consensus.cc:2749] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:23.253429 22581 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:23.253960 22724 leader_election.cc:290] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:23.261819 21132 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60570 (request call id 117) took 2892 ms. Trace:
I20260709 12:29:23.263949 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:23.266283 21380 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:29:23.271957 21278 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 125088us
I20260709 12:29:23.276537 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ed20ce36b0e46b6a043a3d1b558fb8f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:23.277920 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:23.221253 21134 catalog_manager.cc:5697] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 1 to 3, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } }
I20260709 12:29:23.320284 22165 leader_election.cc:304] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:29:23.331462 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 45043.3us
I20260709 12:29:23.333160 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93df7ba57f144211b8429c8c773bd74e" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:23.327471 22511 raft_consensus.cc:493] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b565aef1b93143c78c5c3dc4a861a68a)
I20260709 12:29:23.360224 22162 leader_election.cc:304] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 482723f787a14005854e038688430a80; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:23.361091 22702 raft_consensus.cc:2749] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:23.358328 22511 raft_consensus.cc:515] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:23.334450 22724 raft_consensus.cc:2749] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:23.367651 21380 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 5.
W20260709 12:29:23.366789 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 164622us
I20260709 12:29:23.378578 21275 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:23.264978 21132 rpcz_store.cc:276] 0709 12:29:20.369795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.370065 (+   270us) service_pool.cc:224] Handling call
0709 12:29:23.261755 (+2891690us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:22.120484 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:22.130456 (+  9972us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:22.130490 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:29:22.130508 (+    18us) tablet.cc:662] Decoding operations
0709 12:29:22.131160 (+   652us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:22.131248 (+    88us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:22.131274 (+    26us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:29:22.141673 (+ 10399us) tablet.cc:701] Row locks acquired
0709 12:29:22.143411 (+  1738us) write_op.cc:260] PREPARE: finished
0709 12:29:22.148678 (+  5267us) write_op.cc:270] Start()
0709 12:29:22.153010 (+  4332us) write_op.cc:276] Timestamp: P: 1783600162147562 usec, L: 0
0709 12:29:22.155285 (+  2275us) op_driver.cc:348] REPLICATION: starting
0709 12:29:22.163268 (+  7983us) log.cc:844] Serialized 1705 byte log entry
0709 12:29:22.595107 (+431839us) op_driver.cc:464] REPLICATION: finished
0709 12:29:22.793714 (+198607us) write_op.cc:301] APPLY: starting
0709 12:29:22.815156 (+ 21442us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:22.837207 (+ 22051us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:22.862372 (+ 25165us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:22.895985 (+ 33613us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:22.896247 (+   262us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:29:22.896319 (+    72us) write_op.cc:312] APPLY: finished
0709 12:29:22.896661 (+   342us) log.cc:844] Serialized 40 byte log entry
0709 12:29:22.896890 (+   229us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:22.897019 (+   129us) write_op.cc:454] Released schema lock
0709 12:29:22.897062 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:22.897484 (+   422us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:22.897526 (+    42us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":286576,"reported_tablets":11,"child_traces":[["op",{"apply.queue_time_us":141438,"apply.run_cpu_time_us":3009,"apply.run_wall_time_us":150415,"num_ops":4,"prepare.queue_time_us":264464,"prepare.run_cpu_time_us":4139,"prepare.run_wall_time_us":94150,"replication_time_us":439652,"thread_start_us":220285,"threads_started":2}]]}
I20260709 12:29:23.381409 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:23.382074 21451 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:23.388769 22581 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:23.391422 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:23.392215 21379 raft_consensus.cc:2368] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce for earlier term 4. Current term is 5.
I20260709 12:29:23.393250 21423 leader_election.cc:400] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Vote denied by peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499) with higher term. Message: Invalid argument: T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce for earlier term 4. Current term is 5.
I20260709 12:29:23.393530 21423 leader_election.cc:403] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 12:29:23.396675 21451 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 5.
I20260709 12:29:23.397184 22631 raft_consensus.cc:2749] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499) with higher term. Message: Invalid argument: T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce for earlier term 4. Current term is 5.
I20260709 12:29:23.390579 22699 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:23.405980 22699 raft_consensus.cc:697] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:23.406953 22699 consensus_queue.cc:237] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:23.436827 21454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1cb329d7c3274501a5c1c9f03eebd18f"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "a3220692a27d44e49348c63d58fcf666"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:23.437595 21454 raft_consensus.cc:606] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Received request to transfer leadership to TS a3220692a27d44e49348c63d58fcf666
I20260709 12:29:23.441257 21454 raft_consensus.cc:614] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:23.448496 22511 leader_election.cc:290] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:23.449394 21379 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:29:23.451565 22724 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:23.452973 21380 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:23.454945 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:23.458990 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3dbb15b421ee44b48cdf0afeed6430e9" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:23.466446 21425 leader_election.cc:304] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:23.467031 21379 raft_consensus.cc:1275] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:23.475515 22708 raft_consensus.cc:2749] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:29:23.480252 21454 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "a3220692a27d44e49348c63d58fcf666"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:23.480922 21454 raft_consensus.cc:606] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Received request to transfer leadership to TS a3220692a27d44e49348c63d58fcf666
I20260709 12:29:23.481973 21134 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60574 (request call id 127) took 2526 ms. Trace:
I20260709 12:29:23.482795 21134 rpcz_store.cc:276] 0709 12:29:20.955529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:20.955737 (+   208us) service_pool.cc:224] Handling call
0709 12:29:23.481910 (+2526173us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:23.375990 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:23.379085 (+  3095us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:23.386404 (+  7319us) write_op.cc:435] Acquired schema lock
0709 12:29:23.387691 (+  1287us) tablet.cc:662] Decoding operations
0709 12:29:23.402011 (+ 14320us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:23.402090 (+    79us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:23.402122 (+    32us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:23.402317 (+   195us) tablet.cc:701] Row locks acquired
0709 12:29:23.402334 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:29:23.402469 (+   135us) write_op.cc:270] Start()
0709 12:29:23.402585 (+   116us) write_op.cc:276] Timestamp: P: 1783600163402427 usec, L: 0
0709 12:29:23.402635 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 12:29:23.403277 (+   642us) log.cc:844] Serialized 540 byte log entry
0709 12:29:23.430626 (+ 27349us) op_driver.cc:464] REPLICATION: finished
0709 12:29:23.451934 (+ 21308us) write_op.cc:301] APPLY: starting
0709 12:29:23.452019 (+    85us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:23.452114 (+    95us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:23.452230 (+   116us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:23.452443 (+   213us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:23.452604 (+   161us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:29:23.452652 (+    48us) write_op.cc:312] APPLY: finished
0709 12:29:23.459537 (+  6885us) log.cc:844] Serialized 22 byte log entry
0709 12:29:23.460809 (+  1272us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:23.460894 (+    85us) write_op.cc:454] Released schema lock
0709 12:29:23.460936 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:23.461318 (+   382us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:23.461348 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":132570,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":21021,"apply.run_cpu_time_us":1649,"apply.run_wall_time_us":9376,"num_ops":1,"prepare.queue_time_us":94227,"prepare.run_cpu_time_us":2871,"prepare.run_wall_time_us":36461,"replication_time_us":27805,"thread_start_us":102351,"threads_started":2}]]}
I20260709 12:29:23.485954 22745 consensus_queue.cc:1048] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:23.488472 21131 catalog_manager.cc:5697] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 reported cstate change: term changed from 4 to 5, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 5 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.491739 21131 catalog_manager.cc:5697] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:23.494380 21131 catalog_manager.cc:5697] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } }
I20260709 12:29:23.496537 21131 catalog_manager.cc:5697] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 3 to 5, leader changed from a3220692a27d44e49348c63d58fcf666 (127.8.137.195) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 5 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } }
I20260709 12:29:23.499240 21131 catalog_manager.cc:5697] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 reported cstate change: term changed from 3 to 5, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 5 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.519842 21454 raft_consensus.cc:1275] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:23.508327 21131 catalog_manager.cc:5697] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } }
I20260709 12:29:23.527220 22704 consensus_queue.cc:1048] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 12:29:23.553810 21380 raft_consensus.cc:1275] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:23.555845 22724 consensus_queue.cc:1048] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:23.572510 21454 raft_consensus.cc:1275] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:23.574273 22642 consensus_queue.cc:1048] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:23.673339 21453 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:29:23.689095 21452 raft_consensus.cc:1275] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:23.708166 21131 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:49894 (request call id 36) took 1890 ms. Trace:
I20260709 12:29:23.719053 22516 consensus_queue.cc:1048] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:23.724852 22724 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:23.708691 21131 rpcz_store.cc:276] 0709 12:29:21.817475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.817606 (+   131us) service_pool.cc:224] Handling call
0709 12:29:23.708101 (+1890495us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:23.537916 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:23.538492 (+   576us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:23.538530 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:29:23.538549 (+    19us) tablet.cc:662] Decoding operations
0709 12:29:23.539362 (+   813us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:23.539432 (+    70us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:23.539466 (+    34us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:29:23.540102 (+   636us) tablet.cc:701] Row locks acquired
0709 12:29:23.540126 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:29:23.540304 (+   178us) write_op.cc:270] Start()
0709 12:29:23.540410 (+   106us) write_op.cc:276] Timestamp: P: 1783600163540237 usec, L: 0
0709 12:29:23.540432 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:29:23.541080 (+   648us) log.cc:844] Serialized 2481 byte log entry
0709 12:29:23.548206 (+  7126us) op_driver.cc:464] REPLICATION: finished
0709 12:29:23.556331 (+  8125us) write_op.cc:301] APPLY: starting
0709 12:29:23.556409 (+    78us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:23.556568 (+   159us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:23.556662 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:23.557284 (+   622us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:23.557453 (+   169us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:29:23.557497 (+    44us) write_op.cc:312] APPLY: finished
0709 12:29:23.557726 (+   229us) log.cc:844] Serialized 52 byte log entry
0709 12:29:23.621146 (+ 63420us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:23.628644 (+  7498us) write_op.cc:454] Released schema lock
0709 12:29:23.638200 (+  9556us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:23.652848 (+ 14648us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:23.652881 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":85,"reported_tablets":14,"child_traces":[["op",{"apply.queue_time_us":5806,"apply.run_cpu_time_us":2499,"apply.run_wall_time_us":96479,"num_ops":6,"prepare.queue_time_us":110,"prepare.run_cpu_time_us":3475,"prepare.run_wall_time_us":5791,"replication_time_us":7630}]]}
I20260709 12:29:23.728430 21135 catalog_manager.cc:5697] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:23.750960 21304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5"
dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
mode: GRACEFUL
new_leader_uuid: "a3220692a27d44e49348c63d58fcf666"
 from {username='slave'} at 127.0.0.1:51008
I20260709 12:29:23.751811 21304 raft_consensus.cc:606] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Received request to transfer leadership to TS a3220692a27d44e49348c63d58fcf666
I20260709 12:29:23.755992 22283 raft_consensus.cc:1275] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:23.790561 22705 consensus_queue.cc:1048] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 12:29:23.813627 21460 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:23.814328 21460 raft_consensus.cc:606] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
W20260709 12:29:23.851738 22757 log.cc:927] Time spent T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666: Append to log took a long time: real 0.058s	user 0.001s	sys 0.001s
I20260709 12:29:23.894984 22584 raft_consensus.cc:993] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:23.895527 22584 raft_consensus.cc:1081] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:23.904333 22285 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:29:23.905649 22285 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:23.906291 22285 raft_consensus.cc:3060] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:23.911381 21134 catalog_manager.cc:5697] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 2 to 3, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:23.917241 21135 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:60588 (request call id 130) took 2333 ms. Trace:
I20260709 12:29:23.922900 21135 rpcz_store.cc:276] 0709 12:29:21.583628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:29:21.583785 (+   157us) service_pool.cc:224] Handling call
0709 12:29:23.917175 (+2333390us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:29:23.795058 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:29:23.835468 (+ 40410us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:29:23.838137 (+  2669us) write_op.cc:435] Acquired schema lock
0709 12:29:23.838151 (+    14us) tablet.cc:662] Decoding operations
0709 12:29:23.848389 (+ 10238us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:29:23.849004 (+   615us) write_op.cc:641] Partition lock acquired for write op
0709 12:29:23.849038 (+    34us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:29:23.851925 (+  2887us) tablet.cc:701] Row locks acquired
0709 12:29:23.851941 (+    16us) write_op.cc:260] PREPARE: finished
0709 12:29:23.860179 (+  8238us) write_op.cc:270] Start()
0709 12:29:23.860317 (+   138us) write_op.cc:276] Timestamp: P: 1783600163860115 usec, L: 0
0709 12:29:23.860337 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:29:23.875501 (+ 15164us) log.cc:844] Serialized 540 byte log entry
0709 12:29:23.877213 (+  1712us) op_driver.cc:464] REPLICATION: finished
0709 12:29:23.877672 (+   459us) write_op.cc:301] APPLY: starting
0709 12:29:23.877774 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 12:29:23.877879 (+   105us) tablet.cc:1370] finished BulkCheckPresence
0709 12:29:23.877978 (+    99us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:29:23.878207 (+   229us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:29:23.878374 (+   167us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:29:23.878424 (+    50us) write_op.cc:312] APPLY: finished
0709 12:29:23.878692 (+   268us) log.cc:844] Serialized 22 byte log entry
0709 12:29:23.878864 (+   172us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:23.878957 (+    93us) write_op.cc:454] Released schema lock
0709 12:29:23.878999 (+    42us) write_op.cc:341] FINISH: Updating metrics
0709 12:29:23.879364 (+   365us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:29:23.879395 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7587,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":239,"apply.run_cpu_time_us":1643,"apply.run_wall_time_us":1642,"num_ops":1,"prepare.queue_time_us":4012,"prepare.run_cpu_time_us":3331,"prepare.run_wall_time_us":82501,"replication_time_us":16753}]]}
I20260709 12:29:23.914095 21134 catalog_manager.cc:5697] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.955415 21134 catalog_manager.cc:5697] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 3 to 4, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 4 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.966478 21452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:23.967496 21452 raft_consensus.cc:606] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:23.969367 22631 raft_consensus.cc:993] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower a3220692a27d44e49348c63d58fcf666 to start an election
I20260709 12:29:23.969940 22631 raft_consensus.cc:1081] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Signalling peer a3220692a27d44e49348c63d58fcf666 to start an election
I20260709 12:29:23.972146 21377 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf"
dest_uuid: "a3220692a27d44e49348c63d58fcf666"
 from {username='slave'} at 127.0.0.1:39588
I20260709 12:29:23.972731 21377 raft_consensus.cc:493] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:23.973093 21377 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:23.978710 21131 catalog_manager.cc:5697] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:23.980911 21131 catalog_manager.cc:5697] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 6 to 7, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 7 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } }
I20260709 12:29:23.983954 21131 catalog_manager.cc:5697] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 1 to 2, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } }
I20260709 12:29:23.956669 21451 raft_consensus.cc:1275] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:23.990474 21377 raft_consensus.cc:515] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:24.036998 21379 raft_consensus.cc:1275] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:29:24.053956 22708 raft_consensus.cc:993] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:24.054507 22511 raft_consensus.cc:1081] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:24.059401 21377 leader_election.cc:290] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:24.083163 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:24.063355 22695 consensus_queue.cc:1048] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:29:24.058513 22285 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:24.095585 21310 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:24.065544 22283 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:29:24.097160 22283 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.097569 22283 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:24.067509 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ac1f07547154488aa8d69f3b35d29bf" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:24.100484 21457 raft_consensus.cc:3055] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:24.100823 21457 raft_consensus.cc:740] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:24.101553 21457 consensus_queue.cc:260] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:24.105583 21310 raft_consensus.cc:2468] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:24.106648 21457 raft_consensus.cc:3060] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:24.109829 22285 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:24.110532 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:24.111306 21454 raft_consensus.cc:3055] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:24.111630 21454 raft_consensus.cc:740] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:24.112537 21454 consensus_queue.cc:260] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:24.113818 21454 raft_consensus.cc:3060] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.131229 22694 consensus_queue.cc:1048] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:29:24.069024 21452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:24.143808 21452 raft_consensus.cc:606] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:24.144285 21452 raft_consensus.cc:614] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:29:24.145838  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:29:24.152552  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:24.153532  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:24.172912 21457 raft_consensus.cc:2468] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:24.193547 21351 leader_election.cc:304] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:24.194712 22705 raft_consensus.cc:2804] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:24.195235 22705 raft_consensus.cc:697] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:24.196183 22705 consensus_queue.cc:237] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:24.158459 22283 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:24.167935 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 74722.7us
I20260709 12:29:24.235720 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:24.236395 21307 raft_consensus.cc:3060] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:24.237751 21454 raft_consensus.cc:2468] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:24.242447 22165 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:29:24.245167 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:24.245824 21308 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:24.249564 21307 raft_consensus.cc:2468] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:24.252350 22642 raft_consensus.cc:2804] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:24.252924 22642 raft_consensus.cc:697] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:24.253870 22642 consensus_queue.cc:237] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:24.258317 21308 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:24.289386 22283 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:24.290324 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:24.291630 21454 raft_consensus.cc:3055] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:24.292088 21454 raft_consensus.cc:740] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:24.299899 22165 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:24.305068 22724 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:24.305627 22724 raft_consensus.cc:697] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 4 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:24.306471 22724 consensus_queue.cc:237] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:24.301012 21454 consensus_queue.cc:260] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:24.319804 21454 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:24.342525 21304 raft_consensus.cc:1240] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Rejecting Update request from peer 1095bdbabc944624b91311b39a5a02ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:29:24.391160 21454 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:24.402426 22525 raft_consensus.cc:993] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a: : Instructing follower a3220692a27d44e49348c63d58fcf666 to start an election
I20260709 12:29:24.402994 22695 raft_consensus.cc:1081] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Signalling peer a3220692a27d44e49348c63d58fcf666 to start an election
I20260709 12:29:24.404790 21386 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5"
dest_uuid: "a3220692a27d44e49348c63d58fcf666"
 from {username='slave'} at 127.0.0.1:39594
I20260709 12:29:24.405360 21386 raft_consensus.cc:493] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:24.405736 21386 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:24.407620 21131 catalog_manager.cc:5697] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.433897 21386 raft_consensus.cc:515] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:24.457670 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:24.458346 21308 raft_consensus.cc:3055] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:29:24.458688 21308 raft_consensus.cc:740] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:24.457995 21131 catalog_manager.cc:5697] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 3 to 4, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 4 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:24.459457 21308 consensus_queue.cc:260] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:24.460520 21308 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:24.468322 21132 catalog_manager.cc:5697] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 2 to 3, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.473958 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc7d870ec3c54968ba293f0a1e2afab5" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:24.474757 21454 raft_consensus.cc:3060] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:24.480126 21386 leader_election.cc:290] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 6 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
I20260709 12:29:24.477882 21308 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 6.
I20260709 12:29:24.485816 21351 leader_election.cc:304] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:24.486763 22705 raft_consensus.cc:2804] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:29:24.488215 22705 raft_consensus.cc:697] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 6 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:24.489195 22705 consensus_queue.cc:237] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } }
I20260709 12:29:24.492579 21454 raft_consensus.cc:2468] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 6.
I20260709 12:29:24.554032 21132 catalog_manager.cc:5697] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 5 to 6, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 6 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:24.638927 21454 raft_consensus.cc:1275] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:24.645640 22705 consensus_queue.cc:1048] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:24.671996 21307 raft_consensus.cc:1275] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:24.676304 22724 consensus_queue.cc:1048] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:24.704595 21308 raft_consensus.cc:1275] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:24.711889 22576 consensus_queue.cc:1048] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:24.739383 21451 raft_consensus.cc:1275] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:24.740978 22724 consensus_queue.cc:1048] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:24.767314 21451 raft_consensus.cc:1275] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:24.768707 22724 consensus_queue.cc:1048] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:24.789021 21307 raft_consensus.cc:1275] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:24.840447 22702 consensus_queue.cc:1048] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:24.999974 21457 raft_consensus.cc:1275] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:29:25.002554 22780 consensus_queue.cc:1048] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:25.159261 21307 raft_consensus.cc:1275] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:29:25.199683 22779 consensus_queue.cc:1048] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:25.527510  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:25.581384 21308 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279"
dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:51008
I20260709 12:29:25.582058 21308 raft_consensus.cc:606] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:25.585232 21451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:25.585827 21451 raft_consensus.cc:606] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:25.589520 21451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:25.590094 21451 raft_consensus.cc:606] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:25.593091 21451 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08"
dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35106
I20260709 12:29:25.593645 21451 raft_consensus.cc:606] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:25.639523 21308 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a"
dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:51008
I20260709 12:29:25.640203 21308 raft_consensus.cc:606] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:25.644534  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:29:25.656695  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:25.657646  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:25.804883 22784 raft_consensus.cc:993] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:25.805470 22784 raft_consensus.cc:1081] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:25.807879 22283 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35724
I20260709 12:29:25.808478 22283 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:25.808828 22283 raft_consensus.cc:3060] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.864670 22283 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:25.867753 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:25.868435 21308 raft_consensus.cc:3055] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:25.868811 21308 raft_consensus.cc:740] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:25.869654 21308 consensus_queue.cc:260] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:25.871145 21308 raft_consensus.cc:3060] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.890039 21308 raft_consensus.cc:2468] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:25.892254 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:25.936422 21377 raft_consensus.cc:3060] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.919557 22708 raft_consensus.cc:993] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:25.942042 22708 raft_consensus.cc:1081] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:25.928930 22283 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:25.957975 22281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:29:25.958745 22281 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:25.958822 22165 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:25.959184 22281 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:25.959857 22724 raft_consensus.cc:2804] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:25.960382 22724 raft_consensus.cc:697] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:25.961566 22584 raft_consensus.cc:993] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:25.961287 22724 consensus_queue.cc:237] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:25.962157 22584 raft_consensus.cc:1081] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:25.968356 22285 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:29:25.968935 22285 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:25.969329 22285 raft_consensus.cc:3060] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:25.980443 22285 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:25.987556 22285 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:25.991065 22281 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:25.994323 22281 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:25.998951 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:25.999555 21451 raft_consensus.cc:3055] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:25.999856 21451 raft_consensus.cc:740] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:26.000592 21451 consensus_queue.cc:260] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:26.001767 21451 raft_consensus.cc:3060] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.002964 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:26.003561 21457 raft_consensus.cc:3055] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:26.003854 21457 raft_consensus.cc:740] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:26.004549 21457 consensus_queue.cc:260] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:26.005741 21457 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:26.014334 21304 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:26.015211 21304 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:26.012411 21377 raft_consensus.cc:2468] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:26.026733 22511 raft_consensus.cc:993] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:26.027411 22511 raft_consensus.cc:1081] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:26.029311 22281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35716
I20260709 12:29:26.029695 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:26.029884 22281 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:26.030290 21308 raft_consensus.cc:3060] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:26.046564 21132 catalog_manager.cc:5697] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.072535 22695 raft_consensus.cc:993] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:26.074501 22784 raft_consensus.cc:1081] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:26.089177 22283 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35724
I20260709 12:29:26.089798 22283 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:26.090154 22283 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.112797 21308 raft_consensus.cc:2468] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:26.115828 22165 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:26.117059 21451 raft_consensus.cc:2468] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:26.117228 22724 raft_consensus.cc:2804] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:26.121711 21304 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:26.125268 22165 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:26.126044 22283 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:26.126806 22702 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:26.127550 22702 raft_consensus.cc:697] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 4 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:26.128858 22702 consensus_queue.cc:237] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:26.123607 21457 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:26.030299 22281 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:26.149084 22283 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:26.169422 21457 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:26.170131 21457 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.172228 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:26.173530 21308 raft_consensus.cc:3055] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:26.173929 21308 raft_consensus.cc:740] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:26.174943 21308 consensus_queue.cc:260] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:26.176597 21308 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:26.188263 21457 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:29:26.189837 22165 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:29:26.178081 22724 raft_consensus.cc:697] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:26.194060 22788 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:26.194583 22788 raft_consensus.cc:697] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 3 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:26.195573 22788 consensus_queue.cc:237] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:26.200918 21308 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 3.
I20260709 12:29:26.211041 22281 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:26.223510 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:26.224094 21377 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:26.227789 22724 consensus_queue.cc:237] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:26.236210 21377 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:26.250444 22162 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:26.255713 22787 raft_consensus.cc:2804] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:26.256299 22787 raft_consensus.cc:697] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 4 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:26.257277 22787 consensus_queue.cc:237] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:26.259104 22281 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263)
W20260709 12:29:26.261603 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 79117.1us
I20260709 12:29:26.261973 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:26.262701 21454 raft_consensus.cc:3055] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:26.263098 21454 raft_consensus.cc:740] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:26.266963 21454 consensus_queue.cc:260] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:26.268354 21454 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:26.279765 21454 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:26.314819 21132 catalog_manager.cc:5697] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.316771 21132 catalog_manager.cc:5697] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 reported cstate change: term changed from 2 to 3, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 3 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.320783 21132 catalog_manager.cc:5697] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 3 to 4, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 4 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:26.322427 21132 catalog_manager.cc:5697] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 reported cstate change: term changed from 3 to 4, leader changed from 1095bdbabc944624b91311b39a5a02ce (127.8.137.196) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 4 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:26.509876 21454 raft_consensus.cc:1275] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:26.511803 22724 consensus_queue.cc:1048] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:26.577015 21304 raft_consensus.cc:1275] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:26.578843 21304 raft_consensus.cc:1275] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:26.588570 22724 consensus_queue.cc:1048] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:26.597349 22724 consensus_queue.cc:1048] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:26.623189 21377 raft_consensus.cc:1275] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:26.627952 22788 consensus_queue.cc:1048] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:26.628435 21311 raft_consensus.cc:1275] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:26.630292 22724 consensus_queue.cc:1048] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:26.716135 21452 raft_consensus.cc:1275] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:26.718924 22787 consensus_queue.cc:1048] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:26.726795 21454 raft_consensus.cc:1275] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:26.734858 21451 raft_consensus.cc:1275] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:26.745311 22724 consensus_queue.cc:1048] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:26.752878 22788 consensus_queue.cc:1048] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:26.767939 21308 raft_consensus.cc:1275] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:26.780128 22788 consensus_queue.cc:1048] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:26.796393  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:26.853626 21377 raft_consensus.cc:1275] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:26.859220 22788 consensus_queue.cc:1048] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:26.937726 21304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c"
dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:51008
I20260709 12:29:26.938324 21304 raft_consensus.cc:606] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:26.945012 21304 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd"
dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
mode: GRACEFUL
new_leader_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:51008
I20260709 12:29:26.945663 21304 raft_consensus.cc:606] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Received request to transfer leadership to TS 482723f787a14005854e038688430a80
I20260709 12:29:26.948022  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:26.949981  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:26.950809  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:27.196529 22776 raft_consensus.cc:993] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:27.197151 22776 raft_consensus.cc:1081] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:27.213533 22281 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35724
I20260709 12:29:27.324790 22281 raft_consensus.cc:493] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:27.326942 22281 raft_consensus.cc:3060] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:27.427695 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 86941.2us
I20260709 12:29:27.478040 22695 raft_consensus.cc:993] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a: : Instructing follower 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:27.478729 22813 raft_consensus.cc:1081] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Signalling peer 482723f787a14005854e038688430a80 to start an election
I20260709 12:29:27.500926 22281 raft_consensus.cc:515] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:29:27.507961 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 50942.3us
I20260709 12:29:27.511206 22287 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c"
dest_uuid: "482723f787a14005854e038688430a80"
 from {username='slave'} at 127.0.0.1:35724
I20260709 12:29:27.511931 22287 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:27.512326 22287 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:27.521462 22281 leader_election.cc:290] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:27.546702 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:27.546972 21377 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:27.547333 21307 raft_consensus.cc:3055] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:27.547528 21377 raft_consensus.cc:3060] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.547662 21307 raft_consensus.cc:740] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:27.548452 21307 consensus_queue.cc:260] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:27.549448 21307 raft_consensus.cc:3060] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:27.555931 21377 raft_consensus.cc:2468] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:27.556810 21307 raft_consensus.cc:2468] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 2.
I20260709 12:29:27.557317 22162 leader_election.cc:304] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:27.561277 22702 raft_consensus.cc:2804] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:27.563024 22702 raft_consensus.cc:697] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 2 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:27.564126 22702 consensus_queue.cc:237] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:27.582293 22287 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:27.584594 22287 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:27.586041 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:27.591701 21452 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:27.586859 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "482723f787a14005854e038688430a80" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:27.595991 21307 raft_consensus.cc:3055] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:27.596412 21307 raft_consensus.cc:740] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:27.597393 21307 consensus_queue.cc:260] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:27.601248 21307 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:27.605585 21452 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:27.598991 21132 catalog_manager.cc:5697] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 reported cstate change: term changed from 1 to 2, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.609086 22165 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80; no voters: 
I20260709 12:29:27.610715 22702 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:27.611359 22702 raft_consensus.cc:697] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 4 LEADER]: Becoming Leader. State: Replica: 482723f787a14005854e038688430a80, State: Running, Role: LEADER
I20260709 12:29:27.612491 22702 consensus_queue.cc:237] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:27.618523 21307 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 482723f787a14005854e038688430a80 in term 4.
I20260709 12:29:27.731350 21132 catalog_manager.cc:5697] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 reported cstate change: term changed from 3 to 4, leader changed from b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194) to 482723f787a14005854e038688430a80 (127.8.137.193). New cstate: current_term: 4 leader_uuid: "482723f787a14005854e038688430a80" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:27.950317 21308 raft_consensus.cc:1275] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:27.956343 22787 consensus_queue.cc:1048] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:28.041181  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:28.051211 21377 raft_consensus.cc:1275] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:28.052877 22787 consensus_queue.cc:1048] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:28.135720  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:29:28.137743  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:28.138710  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:28.174749 21308 raft_consensus.cc:1275] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:28.176190 22787 consensus_queue.cc:1048] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:28.200695 21454 raft_consensus.cc:1275] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Refusing update from remote peer 482723f787a14005854e038688430a80: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:28.208467 22724 consensus_queue.cc:1048] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:28.264149  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:28.358687  8743 tablet_server.cc:179] TabletServer@127.8.137.193:33017 shutting down...
W20260709 12:29:28.374089 21278 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 12:29:28.411891 21278 consensus_peers.cc:597] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.426016 21426 consensus_peers.cc:597] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.431174 21278 consensus_peers.cc:597] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.434238 21278 consensus_peers.cc:597] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.441630 21426 consensus_peers.cc:597] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.454762  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:28.456838  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.459143  8743 raft_consensus.cc:2243] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:28.463042 21426 consensus_peers.cc:597] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.474879  8743 raft_consensus.cc:2272] T 9e36b9296b6e408da863f2bd3bb7b36b P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.478657  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.479655  8743 raft_consensus.cc:2243] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.480257  8743 raft_consensus.cc:2272] T f68cddadeb0443b28b03d76d5195fc45 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.486976 21278 consensus_peers.cc:597] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.498514  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.499712  8743 raft_consensus.cc:2243] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.500346  8743 raft_consensus.cc:2272] T a8f34093df9d4e6db1b428ae3b532fec P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.503577  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.504588  8743 raft_consensus.cc:2243] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.505235  8743 raft_consensus.cc:2272] T 0d0bfa2f2d224261803bbc1ee6978296 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.510720 21278 consensus_peers.cc:597] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.513289  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.514376  8743 raft_consensus.cc:2243] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.515483  8743 raft_consensus.cc:2272] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.517084 21278 consensus_peers.cc:597] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.518501 21352 consensus_peers.cc:597] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.530134 21352 consensus_peers.cc:597] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.540655  8743 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:28.542286 21426 consensus_peers.cc:597] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.548424 21426 consensus_peers.cc:597] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.550046  8743 raft_consensus.cc:2243] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.553242  8743 raft_consensus.cc:2272] T 54443d308bbc43359e598ace1951bd70 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.550798 21426 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.562470  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.566175  8743 raft_consensus.cc:2243] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.566850  8743 raft_consensus.cc:2272] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.572695 21278 consensus_peers.cc:597] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.573949  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.577301  8743 raft_consensus.cc:2243] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.584550  8743 raft_consensus.cc:2272] T a2384bff30cd4d3f9801e52ba3c28c08 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.593554  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.597167  8743 raft_consensus.cc:2243] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.597711  8743 raft_consensus.cc:2272] T f44f7e65b47e48649e1e7ab852e0fc84 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.605931  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.606801  8743 raft_consensus.cc:2243] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 7 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.619433  8743 raft_consensus.cc:2272] T 63656d4d714a4ce89de05b1e5ba3fe43 P 482723f787a14005854e038688430a80 [term 7 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.620522 21426 consensus_peers.cc:597] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.608024 21352 consensus_peers.cc:597] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.622454  8743 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:28.622911 21352 consensus_peers.cc:597] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.623425  8743 raft_consensus.cc:2243] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.624006  8743 raft_consensus.cc:2272] T d4da9543a3474f228769c7d28609a702 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.627142  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.627952  8743 raft_consensus.cc:2243] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.628931  8743 raft_consensus.cc:2272] T f6048d81937c4eaab03874e7a6577982 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.633292  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.643321  8743 raft_consensus.cc:2243] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.643975  8743 raft_consensus.cc:2272] T 85044ca33b9947dface5200a544f79a2 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.646977  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.647940  8743 raft_consensus.cc:2243] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.648514  8743 raft_consensus.cc:2272] T dbd17ab2b7d04cc19cf64cdd44c442a7 P 482723f787a14005854e038688430a80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.651506  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.663311  8743 raft_consensus.cc:2243] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.664024  8743 raft_consensus.cc:2272] T de84b253ee104e63be65f2ef9e9173dd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.667021  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.667897  8743 raft_consensus.cc:2243] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.668491  8743 raft_consensus.cc:2272] T 31b54df203a347c993ed917ad39bbf1c P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.671427  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.672338  8743 raft_consensus.cc:2243] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.673686  8743 raft_consensus.cc:2272] T f52db4db2acf4abfa01aa7604963edbc P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.679368 21352 consensus_peers.cc:597] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.681768  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.683321  8743 raft_consensus.cc:2243] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.684513  8743 raft_consensus.cc:2272] T d22f7d658c5f48e7badd08e41f3e6e0d P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.692772 21426 consensus_peers.cc:597] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.707824  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.708930  8743 raft_consensus.cc:2243] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.710309  8743 raft_consensus.cc:2272] T c4c805fca65a4771bf0e2270955c14c0 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.713950  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.715049  8743 raft_consensus.cc:2243] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.715698  8743 raft_consensus.cc:2272] T e5192ed91b2d4c14905f4a0a7410da9f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.718426 21426 consensus_peers.cc:597] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.718796  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.719857  8743 raft_consensus.cc:2243] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.720515  8743 raft_consensus.cc:2272] T 79a82803d22242a991a35ece42323ea7 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.723869  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.725142  8743 raft_consensus.cc:2243] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.725839  8743 raft_consensus.cc:2272] T 5de0c984317b4a16bd865892a98794b7 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.729272  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.730216  8743 raft_consensus.cc:2243] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.731427  8743 raft_consensus.cc:2272] T 14c3f29aa4ff473db2aec020d298d279 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.734422  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.735335  8743 raft_consensus.cc:2243] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.737449  8743 raft_consensus.cc:2272] T 5dc226a03f0e4040aabb242f51269a5b P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.740026 21352 consensus_peers.cc:597] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.740782  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.741715  8743 raft_consensus.cc:2243] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.742379  8743 raft_consensus.cc:2272] T fb7aa63607864dce845a979faebae247 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.745648  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.746488  8743 raft_consensus.cc:2243] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.747583  8743 raft_consensus.cc:2272] T 27e40fdd4a714c729e2bcc297a44ec4f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.750792  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.751705  8743 raft_consensus.cc:2243] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.752590  8743 raft_consensus.cc:2272] T d49430f4da5e4c108a5cf967c8844659 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.755656  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.756608  8743 raft_consensus.cc:2243] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.758368  8743 raft_consensus.cc:2272] T f08ca19d90734deca4d113875d7def85 P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.762154  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.764658  8743 raft_consensus.cc:2243] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.765354  8743 raft_consensus.cc:2272] T cc850ba2d5144b66bf78197d972b70d8 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.768572  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.769495  8743 raft_consensus.cc:2243] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.770164  8743 raft_consensus.cc:2272] T 4d4f5802ffb348c78eaca62ca6fac2fc P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.773516  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.774456  8743 raft_consensus.cc:2243] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.775126  8743 raft_consensus.cc:2272] T d3c29a71139f45fc849c307af6259abb P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.776237 21426 consensus_peers.cc:597] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.778869  8743 tablet_replica.cc:333] stopping tablet replica
W20260709 12:29:28.779525 21352 consensus_peers.cc:597] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.779959  8743 raft_consensus.cc:2243] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.781190  8743 raft_consensus.cc:2272] T 482afeed6d904bb9a5133bff3cb22b8b P 482723f787a14005854e038688430a80 [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.781311 21352 consensus_peers.cc:597] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.784544  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.785497  8743 raft_consensus.cc:2243] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.786624  8743 raft_consensus.cc:2272] T e8a5b040cb6442da9a9cfca14c12225a P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.789819  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.790799  8743 raft_consensus.cc:2243] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.791467  8743 raft_consensus.cc:2272] T a8c8433d1e274f879bb2b84ad59f5c3f P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.810276 21426 consensus_peers.cc:597] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.812460  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.813947  8743 raft_consensus.cc:2243] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.814574  8743 raft_consensus.cc:2272] T cdd6e626d31d4da8a2e4a407e1710ba2 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.817565  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.818405  8743 raft_consensus.cc:2243] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.819556  8743 raft_consensus.cc:2272] T 1b2bb57292a3448bba84e7e4ade90382 P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.822510  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.823285  8743 raft_consensus.cc:2243] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.823854  8743 raft_consensus.cc:2272] T 2f6dbe944bf742148d2688474712308b P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.831821  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.832619  8743 raft_consensus.cc:2243] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.833655  8743 raft_consensus.cc:2272] T b1620754a7ca4c3ca00eff22b9493e9c P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.836747  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.837571  8743 raft_consensus.cc:2243] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.838286  8743 raft_consensus.cc:2272] T eaea00780b02433fa3a95bf6e01cb78d P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.840987  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.841711  8743 raft_consensus.cc:2243] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:28.842738  8743 raft_consensus.cc:2272] T d0ceb29ef8444e9d96f0276fe5fe93fd P 482723f787a14005854e038688430a80 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.846446  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.847345  8743 raft_consensus.cc:2243] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.848019  8743 raft_consensus.cc:2272] T f7e6811f72d0447781abab4178b3bdd4 P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.851497  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.852447  8743 raft_consensus.cc:2243] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.853148  8743 raft_consensus.cc:2272] T 93df7ba57f144211b8429c8c773bd74e P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.857190 21426 consensus_peers.cc:597] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.873010  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.874055  8743 raft_consensus.cc:2243] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.874818  8743 raft_consensus.cc:2272] T 5ed20ce36b0e46b6a043a3d1b558fb8f P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:28.879429  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.881668  8743 raft_consensus.cc:2243] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.882421  8743 raft_consensus.cc:2272] T 13c370d75f794be68da9f93a6a876a55 P 482723f787a14005854e038688430a80 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.882663 21278 consensus_peers.cc:597] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:28.885766  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:28.886715  8743 raft_consensus.cc:2243] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:28.887421  8743 raft_consensus.cc:2272] T f78e89ba4d49471aaf6eb57ea8f738bf P 482723f787a14005854e038688430a80 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:28.899516 21352 consensus_peers.cc:597] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:28.905133 21426 consensus_peers.cc:597] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:29.574811 22839 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.575412 22839 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:29.577677 22839 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:29.581876 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
W20260709 12:29:29.623060 21426 leader_election.cc:336] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.624253 21426 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:29.625159 22708 raft_consensus.cc:2749] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:29:29.625684 22814 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.626639 22814 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:29.628993 22814 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:29.634109 21278 leader_election.cc:336] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.636737 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:29.642175 21277 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:29.643484 22814 raft_consensus.cc:2749] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:29.651396 22817 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.652037 22817 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:29.655531 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:29.658666 22839 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.659281 22839 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:29.661556 22839 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:29.664942 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:29.666872 22817 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:29.665625 21308 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 4.
W20260709 12:29:29.672947 21352 leader_election.cc:336] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.673427 21425 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:29.673496 21352 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:29.674276 22839 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:29.674718 22817 raft_consensus.cc:2749] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:29.674798 22839 raft_consensus.cc:493] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.675414 22839 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:29.685180 22839 raft_consensus.cc:515] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:29.694973 22839 leader_election.cc:290] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:29.696138 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2384bff30cd4d3f9801e52ba3c28c08" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:29.696861 21308 raft_consensus.cc:3060] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:29.709411 21308 raft_consensus.cc:2468] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 5.
I20260709 12:29:29.714524 22839 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.718327 22839 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:29.720760 22839 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:29.722954 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:29.723877 21425 leader_election.cc:304] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:29.725587 22839 raft_consensus.cc:2804] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:29.726616 22839 raft_consensus.cc:697] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 5 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
W20260709 12:29:29.714802 21426 leader_election.cc:336] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:29.727802 21426 leader_election.cc:336] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.727589 22839 consensus_queue.cc:237] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:29.744982 21426 leader_election.cc:336] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.745520 21426 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:29.753762 22708 raft_consensus.cc:2749] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:29.758755 21132 catalog_manager.cc:5697] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 4 to 5, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 5 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:29.761623 22839 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.763238 22839 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:29.769037 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:29.769711 21386 raft_consensus.cc:2468] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:29.771380 21423 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:29.772538 22708 raft_consensus.cc:2804] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:29.773012 22708 raft_consensus.cc:493] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.773413 22708 raft_consensus.cc:3060] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:29:29.777848 21426 leader_election.cc:336] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.779235 22839 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:29.799346 22708 raft_consensus.cc:515] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:29.805316 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6048d81937c4eaab03874e7a6577982" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:29.805999 21386 raft_consensus.cc:3060] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:29:29.810284 21426 leader_election.cc:336] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.817134 22708 leader_election.cc:290] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:29.827118 22695 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.831892 22695 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:29.839051 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:29.845410 22695 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:29.852627 21278 leader_election.cc:336] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.854058 21278 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:29.866020 22695 raft_consensus.cc:2749] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:29:29.867189 21386 raft_consensus.cc:2468] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:29.868909 21423 leader_election.cc:304] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:29.870157 22708 raft_consensus.cc:2804] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:29.870669 22708 raft_consensus.cc:697] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:29.871480 22708 consensus_queue.cc:237] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:29.882658 22695 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.883428 22695 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:29.886655 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:29.886744 22695 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:29.907624 22839 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.908275 22839 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:29.911911 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:29.910341 22839 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:29.917162 21426 leader_election.cc:336] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.917635 21426 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:29.910051 22695 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.921788 22839 raft_consensus.cc:2749] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:29:29.921254 22695 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:29.940258 22817 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.940971 22817 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:29.945132 22817 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:29.945979 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:29.946816 21454 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 4.
I20260709 12:29:29.948370 21351 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 
I20260709 12:29:29.949357 22817 raft_consensus.cc:2804] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:29.949832 22817 raft_consensus.cc:493] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.950307 22817 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:29:29.955348 21352 leader_election.cc:336] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.962944 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:29.983501 22708 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.984390 22708 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:29.987731 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:29.988361 21308 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:29.989626 21425 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:29:29.989917 21278 leader_election.cc:336] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:29.990522 22839 raft_consensus.cc:2804] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:29.991086 22839 raft_consensus.cc:493] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:29.991482 22839 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:29.996472 22695 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:29.990592 21278 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:29.998942 22708 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:29.999779 22812 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.021361 22812 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.023815 22812 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.001629 22839 raft_consensus.cc:515] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:30.002027 21426 leader_election.cc:336] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.026561 22814 raft_consensus.cc:2749] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:29:30.012226 21132 catalog_manager.cc:5697] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:30.028189 22839 leader_election.cc:290] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:30.017881 21275 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 54204.8us
W20260709 12:29:30.039041 21426 leader_election.cc:336] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.040091 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:30.040882 21454 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 5.
I20260709 12:29:30.044073 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c4c805fca65a4771bf0e2270955c14c0" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:30.044309 21277 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:30.044477 22814 raft_consensus.cc:493] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.044775 21308 raft_consensus.cc:3060] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:30.045153 22814 raft_consensus.cc:515] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:30.047549 22814 leader_election.cc:290] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:30.048193 22814 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:29:30.048920 22814 raft_consensus.cc:493] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.049389 22814 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:30.058872 22839 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.059552 22839 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.061765 22839 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.063665 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:30.066973 21386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:30.072937 22837 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.073553 22837 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.078240 22817 raft_consensus.cc:515] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.085603 21308 raft_consensus.cc:2468] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 4.
W20260709 12:29:30.083705 21426 leader_election.cc:336] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.086782 21426 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:30.086910 21425 leader_election.cc:304] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.088934 22852 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.089807 22852 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.091745 22708 raft_consensus.cc:2749] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:30.093137 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:30.079301 22837 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:30.098747 21352 leader_election.cc:336] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.100589 22839 raft_consensus.cc:2804] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:30.101366 22839 raft_consensus.cc:697] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 4 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:30.095330 21451 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 7.
I20260709 12:29:30.124172 22708 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.115525 22812 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
W20260709 12:29:30.123574 21278 leader_election.cc:336] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.124996 22708 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:30.127121 21278 leader_election.cc:336] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.127990  8743 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22325)
I20260709 12:29:30.128206 21278 leader_election.cc:304] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80, a3220692a27d44e49348c63d58fcf666
I20260709 12:29:30.102531 22839 consensus_queue.cc:237] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.129314 22812 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.133152 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:30.133826 21451 raft_consensus.cc:2468] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:30.134795 21277 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:30.134965 22695 raft_consensus.cc:2749] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:30.128235 21351 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.137458 22695 raft_consensus.cc:2804] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:30.137830 22695 raft_consensus.cc:493] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.138199 22695 raft_consensus.cc:3060] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.139395 22728 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.139940 22728 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:30.140064 22814 raft_consensus.cc:515] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.141844 22728 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.142230 22814 leader_election.cc:290] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.144274 22856 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:29:30.146134 22856 raft_consensus.cc:493] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
W20260709 12:29:30.145498 21278 leader_election.cc:336] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.147277 21278 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:30.147341 22695 raft_consensus.cc:515] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:30.146859 21352 leader_election.cc:336] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.153682 22792 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.154271 22792 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.154513 22812 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.146807 22856 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:29:30.156445 22792 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.159364 22817 leader_election.cc:290] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:30.160568 21426 leader_election.cc:336] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.163450 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27cdc0acc5734dd6ac5fb17c5805ce11" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:30.164171 21451 raft_consensus.cc:3060] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:30.167484 22708 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:30.167754 21426 leader_election.cc:336] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.168807 21425 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 81991.2us
W20260709 12:29:30.176148 21352 leader_election.cc:336] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.179388 21451 raft_consensus.cc:2468] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 5.
I20260709 12:29:30.183909 21351 leader_election.cc:304] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.186457 22695 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:30.187331 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 52536.3us
I20260709 12:29:30.187281 22817 raft_consensus.cc:2804] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:30.178893 22814 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.195926 22814 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:30.198025 22814 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:30.189993 22852 leader_election.cc:290] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.178719 22812 raft_consensus.cc:2749] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:29:30.195395 22817 raft_consensus.cc:697] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 5 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:30.199405 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666" is_pre_election: true
I20260709 12:29:30.199728 22817 consensus_queue.cc:237] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.200126 21379 raft_consensus.cc:2468] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 2.
I20260709 12:29:30.184573 22837 raft_consensus.cc:493] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.202477 22837 raft_consensus.cc:515] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:30.203753 22856 raft_consensus.cc:515] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.204623 22837 leader_election.cc:290] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.207001 21275 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:30.210357 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dc226a03f0e4040aabb242f51269a5b" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:30.211956 21451 raft_consensus.cc:3060] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:30.211866 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:30.211428 22695 raft_consensus.cc:2804] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:30.212781 21452 raft_consensus.cc:3060] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.217885 22695 raft_consensus.cc:493] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.218621 22695 raft_consensus.cc:3060] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.220821 22792 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.253501 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:30.257304 21308 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 5.
W20260709 12:29:30.253460 21278 leader_election.cc:336] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.264979 22695 raft_consensus.cc:515] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:30.265218 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:30.268962 22851 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.269104 21308 raft_consensus.cc:2468] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 2.
I20260709 12:29:30.269557 22851 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.271234 22695 leader_election.cc:290] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499)
I20260709 12:29:30.271580 22851 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.271880 22695 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
W20260709 12:29:30.272063 21426 consensus_peers.cc:597] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:30.272894 21426 leader_election.cc:336] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.272396 22695 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.274449 21308 raft_consensus.cc:1275] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:29:30.253840 22857 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.275971 22857 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.276414 21307 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d22f7d658c5f48e7badd08e41f3e6e0d" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:30.277181 21307 raft_consensus.cc:2393] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 3: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:30.276599 21132 catalog_manager.cc:5697] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 3 to 4, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 4 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:30.278213 22857 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:30.278754 21278 leader_election.cc:336] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.279517 21278 leader_election.cc:336] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.279899 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:30.280680 21310 raft_consensus.cc:2393] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a3220692a27d44e49348c63d58fcf666 in current term 3: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:30.280916 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:30.281543 21308 raft_consensus.cc:2468] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:29:30.260794 22792 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.283968 21425 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.285276 22792 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.285141 22839 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:29:30.285825 21352 leader_election.cc:336] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.286026 22839 raft_consensus.cc:493] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.286435 22839 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:30.288483 22695 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.288372 21425 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.289620 22859 raft_consensus.cc:2804] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:30.289992 22859 raft_consensus.cc:493] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.290354 22859 raft_consensus.cc:3060] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.292842 22854 consensus_queue.cc:1048] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:30.295032 21425 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:30.295172 22695 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.296258 22695 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.296864 21351 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
I20260709 12:29:30.305380 22728 raft_consensus.cc:3060] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.309139 21451 raft_consensus.cc:2468] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 6.
I20260709 12:29:30.310704 21452 raft_consensus.cc:2468] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:30.316653 21351 leader_election.cc:304] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.317368 22695 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.317952 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
W20260709 12:29:30.315879 21278 leader_election.cc:336] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.318751 21452 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:30.319646 21451 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:30.321137 21277 leader_election.cc:304] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.321497 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:30.322011 22695 raft_consensus.cc:2804] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:29:30.322350 21277 leader_election.cc:304] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.322505 22695 raft_consensus.cc:697] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 6 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:30.321457 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:30.319701 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:30.323527 21310 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:30.323668 21308 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 4.
I20260709 12:29:30.314100 22857 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.346051 22857 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.347415 21454 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 4.
I20260709 12:29:30.348784 22857 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.327733 22695 consensus_queue.cc:237] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.314229 22854 raft_consensus.cc:2749] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:30.351105 21277 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:30.359885 22851 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:30.360303 22851 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.360700 22851 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:30.331940 22856 leader_election.cc:290] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 8 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.318130 22837 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.343261 22817 raft_consensus.cc:2804] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:30.330703 22812 raft_consensus.cc:2804] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:30.366223 22812 raft_consensus.cc:697] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:30.367174 22812 consensus_queue.cc:237] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:30.352566 21426 leader_election.cc:336] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.363237 21425 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:30.370085 21425 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.363987 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
W20260709 12:29:30.371301 21352 leader_election.cc:336] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.371474 22862 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:29:30.371910 22862 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.372269 22862 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:30.372421 22854 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:30.371328 21308 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 4.
I20260709 12:29:30.365159 22817 raft_consensus.cc:493] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.373674 22817 raft_consensus.cc:3060] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.367198 22837 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.352449 22728 raft_consensus.cc:2749] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:30.372853 22854 raft_consensus.cc:493] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
W20260709 12:29:30.364388 21278 leader_election.cc:336] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.376684 21278 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.378089 21278 leader_election.cc:336] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.363452 21132 catalog_manager.cc:5697] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 4 to 5, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 5 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:30.377681 22854 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:30.388011 22837 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:30.395337 21426 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.397629 22854 raft_consensus.cc:515] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:30.399935 21352 leader_election.cc:336] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.382246 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 65649.2us
I20260709 12:29:30.382091 21277 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:30.380896 21425 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
W20260709 12:29:30.403898 21278 leader_election.cc:336] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.404801 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "63656d4d714a4ce89de05b1e5ba3fe43" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:30.405435 21454 raft_consensus.cc:3060] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:29:30.406682 21426 leader_election.cc:336] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.424888 22862 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.405054 22852 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:29:30.430042 21426 consensus_peers.cc:597] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.406720 22817 raft_consensus.cc:515] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:30.381767 22859 raft_consensus.cc:515] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.435141 21451 raft_consensus.cc:2393] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 3: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:30.436087 22817 leader_election.cc:290] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.436805 22859 leader_election.cc:290] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.432673 21454 raft_consensus.cc:2468] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 8.
I20260709 12:29:30.438211 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27e40fdd4a714c729e2bcc297a44ec4f" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:30.438972 21308 raft_consensus.cc:3060] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.405146 22814 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:30.440840 22814 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.441226 22814 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:30.441375 21379 raft_consensus.cc:1275] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:30.440476 22852 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.443019 22852 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:30.410272 21452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce" is_pre_election: true
I20260709 12:29:30.448004 21452 raft_consensus.cc:2468] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 2.
I20260709 12:29:30.448099 21308 raft_consensus.cc:2468] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 3.
I20260709 12:29:30.449954 21351 leader_election.cc:304] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.450949 21425 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 
I20260709 12:29:30.453241 21277 leader_election.cc:304] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:30.454547 21351 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.454368 22792 raft_consensus.cc:2804] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:30.453523 22839 raft_consensus.cc:515] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.460160 22792 raft_consensus.cc:697] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:30.461216 22792 consensus_queue.cc:237] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.463531 22854 leader_election.cc:290] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: Requested vote from peers b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.475293 22728 raft_consensus.cc:2804] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:29:30.475769 22728 raft_consensus.cc:493] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 482723f787a14005854e038688430a80)
I20260709 12:29:30.476155 22728 raft_consensus.cc:3060] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.456247 22837 raft_consensus.cc:2804] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:29:30.467646 22812 raft_consensus.cc:2749] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:29:30.482654 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d49430f4da5e4c108a5cf967c8844659" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:30.483237 21310 raft_consensus.cc:3060] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:30.484443 22814 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.488737 22814 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.482304 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0ceb29ef8444e9d96f0276fe5fe93fd" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:30.491811 21308 raft_consensus.cc:3060] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.490114 21460 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
W20260709 12:29:30.495854 21352 leader_election.cc:336] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.496062 22837 raft_consensus.cc:697] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 8 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:30.497224 22837 consensus_queue.cc:237] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.499823 21308 raft_consensus.cc:2468] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:30.500300 21310 raft_consensus.cc:2468] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 5.
W20260709 12:29:30.500574 21426 leader_election.cc:336] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.502072 21426 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.502825 21426 leader_election.cc:336] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.505913 22728 raft_consensus.cc:515] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.453536 22859 consensus_queue.cc:1048] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:29:30.509473 22728 leader_election.cc:290] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
I20260709 12:29:30.515537 21379 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14c3f29aa4ff473db2aec020d298d279" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a3220692a27d44e49348c63d58fcf666"
I20260709 12:29:30.517148 21425 leader_election.cc:304] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.521283 22708 raft_consensus.cc:2804] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:30.521219 21310 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:30.521917 22708 raft_consensus.cc:697] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 5 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:30.506915 22862 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.522840 22708 consensus_queue.cc:237] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
I20260709 12:29:30.552345 22839 leader_election.cc:290] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.588461 22852 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.591215 22852 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:30.576687 22851 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.594318 22851 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Requested vote from peers 1095bdbabc944624b91311b39a5a02ce (127.8.137.196:36263), 482723f787a14005854e038688430a80 (127.8.137.193:33017)
W20260709 12:29:30.635809 21278 leader_election.cc:336] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.638190 21310 raft_consensus.cc:2393] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 4: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
I20260709 12:29:30.643231 21308 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "482afeed6d904bb9a5133bff3cb22b8b" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:30.645151 21308 raft_consensus.cc:3060] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:30.651410 21131 catalog_manager.cc:5697] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 5 to 6, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 6 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:30.655704 21131 catalog_manager.cc:5697] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:30.662385 21460 raft_consensus.cc:2393] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 5: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:30.682886 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "b565aef1b93143c78c5c3dc4a861a68a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:30.684250 21454 raft_consensus.cc:2393] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate b565aef1b93143c78c5c3dc4a861a68a in current term 4: Already voted for candidate 1095bdbabc944624b91311b39a5a02ce in this term.
I20260709 12:29:30.692111 21425 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:29:30.711488 21352 leader_election.cc:336] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.692701 21277 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:30.734642 22708 raft_consensus.cc:2749] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:30.735728 21132 catalog_manager.cc:5697] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 3 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:30.741688 21426 leader_election.cc:336] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.745823 21426 leader_election.cc:336] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:30.745512 21133 catalog_manager.cc:5697] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 7 to 8, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 8 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:30.760195 21278 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
W20260709 12:29:30.763319 21352 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 44158.2us
W20260709 12:29:30.766415 21278 consensus_peers.cc:597] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:30.768767 21277 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.196:36263, user_credentials={real_user=slave}} blocked reactor thread for 76497.1us
I20260709 12:29:30.768755 22814 raft_consensus.cc:2749] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:29:30.770430 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:30.771486 21305 raft_consensus.cc:2393] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1095bdbabc944624b91311b39a5a02ce in current term 5: Already voted for candidate b565aef1b93143c78c5c3dc4a861a68a in this term.
W20260709 12:29:30.799716 21278 consensus_peers.cc:597] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:30.806562 21351 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41173, user_credentials={real_user=slave}} blocked reactor thread for 328371us
I20260709 12:29:30.815665 21307 raft_consensus.cc:1275] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:29:30.822322 21278 consensus_peers.cc:597] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.824175 21425 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce; no voters: 482723f787a14005854e038688430a80, b565aef1b93143c78c5c3dc4a861a68a
W20260709 12:29:30.828059 21426 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:33017, user_credentials={real_user=slave}} blocked reactor thread for 81536.4us
I20260709 12:29:30.828177 22708 raft_consensus.cc:2749] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:29:30.835062 21379 raft_consensus.cc:2468] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b565aef1b93143c78c5c3dc4a861a68a in term 3.
I20260709 12:29:30.835351 22879 consensus_queue.cc:1048] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:30.842129 21308 raft_consensus.cc:2468] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 6.
I20260709 12:29:30.880285 21452 raft_consensus.cc:1275] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:29:30.848215 21132 catalog_manager.cc:5697] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 4 to 5, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 5 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:30.880313 21275 leader_election.cc:304] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
W20260709 12:29:30.859004 21278 consensus_peers.cc:597] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.883443 22871 raft_consensus.cc:2804] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:30.884039 22871 raft_consensus.cc:697] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Becoming Leader. State: Replica: b565aef1b93143c78c5c3dc4a861a68a, State: Running, Role: LEADER
I20260709 12:29:30.885380 22871 consensus_queue.cc:237] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
W20260709 12:29:30.886134 21426 consensus_peers.cc:597] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:30.869396 21277 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b565aef1b93143c78c5c3dc4a861a68a; no voters: 1095bdbabc944624b91311b39a5a02ce, 482723f787a14005854e038688430a80
I20260709 12:29:30.862622 21454 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1b2bb57292a3448bba84e7e4ade90382" candidate_uuid: "a3220692a27d44e49348c63d58fcf666" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "1095bdbabc944624b91311b39a5a02ce"
I20260709 12:29:30.892906 21454 raft_consensus.cc:3060] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:30.900183 21351 leader_election.cc:304] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a3220692a27d44e49348c63d58fcf666, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.902483 21425 leader_election.cc:304] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
W20260709 12:29:30.904287 21278 consensus_peers.cc:597] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:30.911360 22812 raft_consensus.cc:2749] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:29:30.918375 21454 raft_consensus.cc:2468] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3220692a27d44e49348c63d58fcf666 in term 3.
I20260709 12:29:30.923256 22728 raft_consensus.cc:2804] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:30.923969 22728 raft_consensus.cc:697] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:30.925071 22728 consensus_queue.cc:237] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } }
I20260709 12:29:30.926412 22837 consensus_queue.cc:1048] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:29:30.926793 22879 raft_consensus.cc:2804] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:29:30.927534 22879 raft_consensus.cc:697] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 6 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:30.928761 22879 consensus_queue.cc:237] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:30.949165 21351 leader_election.cc:304] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, a3220692a27d44e49348c63d58fcf666; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:30.950414 22817 raft_consensus.cc:2804] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:30.952898 22817 raft_consensus.cc:697] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Becoming Leader. State: Replica: a3220692a27d44e49348c63d58fcf666, State: Running, Role: LEADER
I20260709 12:29:30.954197 22817 consensus_queue.cc:237] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } }
W20260709 12:29:30.963156 21278 consensus_peers.cc:597] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:30.968650 21352 consensus_peers.cc:597] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:31.002467 21460 raft_consensus.cc:1275] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:31.018765 21308 raft_consensus.cc:1275] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:29:31.021438 21132 catalog_manager.cc:5697] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194). New cstate: current_term: 3 leader_uuid: "b565aef1b93143c78c5c3dc4a861a68a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:31.030997 21133 catalog_manager.cc:5697] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:31.046833 22875 consensus_queue.cc:1048] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:29:31.060819 21278 consensus_peers.cc:597] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:31.066044 21310 raft_consensus.cc:1275] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:31.072577 21454 raft_consensus.cc:1275] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 8 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:29:31.074321 22856 consensus_queue.cc:1048] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:29:31.077538 21278 consensus_peers.cc:597] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:31.077667 22875 consensus_queue.cc:1048] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:29:31.087886 21452 raft_consensus.cc:1275] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:29:31.104440 21426 consensus_peers.cc:597] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.105082 21426 consensus_peers.cc:597] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.105535 21426 consensus_peers.cc:597] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.112663 21352 consensus_peers.cc:597] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:31.127950 22868 consensus_queue.cc:1048] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
W20260709 12:29:31.128644  8743 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22325)
W20260709 12:29:31.134953 21352 consensus_peers.cc:597] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.135715 21352 consensus_peers.cc:597] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:31.169816 22814 consensus_queue.cc:1048] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:29:31.174444 21278 consensus_peers.cc:597] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.162774 21426 consensus_peers.cc:597] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:31.178431 21426 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.190038 21426 consensus_peers.cc:597] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:31.201030 21133 catalog_manager.cc:5697] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 reported cstate change: term changed from 2 to 3, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to a3220692a27d44e49348c63d58fcf666 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "a3220692a27d44e49348c63d58fcf666" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:31.215839 21352 consensus_peers.cc:597] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:31.218856 21352 consensus_peers.cc:597] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.233131 21426 consensus_peers.cc:597] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.234709 21423 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:44499, user_credentials={real_user=slave}} blocked reactor thread for 85007.7us
W20260709 12:29:31.236974 21426 consensus_peers.cc:597] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:31.254693 21131 catalog_manager.cc:5697] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 5 to 6, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 6 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:31.307319 21426 consensus_peers.cc:597] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:31.322718 21305 raft_consensus.cc:1275] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:29:31.324391 21426 consensus_peers.cc:597] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:31.324364 22839 consensus_queue.cc:1048] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:29:31.338943 21352 consensus_peers.cc:597] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.359850 21426 consensus_peers.cc:597] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.362963 21352 consensus_peers.cc:597] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.363874 21426 consensus_peers.cc:597] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:31.380784 21454 raft_consensus.cc:1275] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:31.382396 22817 consensus_queue.cc:1048] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:31.394630 21305 raft_consensus.cc:1275] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Refusing update from remote peer a3220692a27d44e49348c63d58fcf666: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:31.394717 21377 raft_consensus.cc:1275] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Refusing update from remote peer b565aef1b93143c78c5c3dc4a861a68a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:31.396736 22817 consensus_queue.cc:1048] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:31.400051 22814 consensus_queue.cc:1048] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3220692a27d44e49348c63d58fcf666" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 44499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:29:31.403519 21278 consensus_peers.cc:597] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.423584 21352 consensus_peers.cc:597] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.448683 21278 consensus_peers.cc:597] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:31.451819 21352 consensus_peers.cc:597] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:31.464121 21426 consensus_peers.cc:597] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:31.493722 21352 consensus_peers.cc:597] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:31.499454 21426 consensus_peers.cc:597] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.500000 21426 consensus_peers.cc:597] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.507107 21352 consensus_peers.cc:597] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.507781 21352 consensus_peers.cc:597] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:31.631412 21426 consensus_peers.cc:597] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:32.129707  8743 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22325)
I20260709 12:29:32.523404 22792 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.524019 22792 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:32.526422 22792 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:32.531610 21426 leader_election.cc:336] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:32.534580 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:32.536639 21305 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 5.
I20260709 12:29:32.550966 21425 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
W20260709 12:29:32.552127 21426 consensus_peers.cc:597] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:29:32.553370 22792 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:29:32.554870 22792 raft_consensus.cc:493] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.558899 22792 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:32.626402 22903 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:32.627156 22903 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:32.629474 22903 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
W20260709 12:29:32.634739 21426 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:32.659794 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a" is_pre_election: true
I20260709 12:29:32.660631 21305 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 4.
I20260709 12:29:32.665858 21425 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:32.668709 22903 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:29:32.669214 22903 raft_consensus.cc:493] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:32.669603 22903 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:32.685961 22792 raft_consensus.cc:515] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:32.699460 21426 leader_election.cc:336] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:32.707383 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b1620754a7ca4c3ca00eff22b9493e9c" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:32.708040 21305 raft_consensus.cc:3060] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:29:32.741588 22792 leader_election.cc:290] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:32.742676 21305 raft_consensus.cc:2468] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 6.
I20260709 12:29:32.754032 21425 leader_election.cc:304] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:32.755187 22708 raft_consensus.cc:2804] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:29:32.756319 22708 raft_consensus.cc:697] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 6 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:32.757201 22708 consensus_queue.cc:237] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:32.745404 22903 raft_consensus.cc:515] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
W20260709 12:29:32.774215 21426 leader_election.cc:336] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111)
I20260709 12:29:32.783279 22903 leader_election.cc:290] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: Requested vote from peers 482723f787a14005854e038688430a80 (127.8.137.193:33017), b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173)
I20260709 12:29:32.784746 21305 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e8a5b040cb6442da9a9cfca14c12225a" candidate_uuid: "1095bdbabc944624b91311b39a5a02ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "b565aef1b93143c78c5c3dc4a861a68a"
I20260709 12:29:32.794775 21305 raft_consensus.cc:3060] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:29:32.789705 21133 catalog_manager.cc:5697] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 4 to 6, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 6 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:32.817067 21305 raft_consensus.cc:2468] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1095bdbabc944624b91311b39a5a02ce in term 5.
I20260709 12:29:32.818781 21425 leader_election.cc:304] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1095bdbabc944624b91311b39a5a02ce, b565aef1b93143c78c5c3dc4a861a68a; no voters: 482723f787a14005854e038688430a80
I20260709 12:29:32.820358 22903 raft_consensus.cc:2804] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:29:32.821039 22903 raft_consensus.cc:697] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 5 LEADER]: Becoming Leader. State: Replica: 1095bdbabc944624b91311b39a5a02ce, State: Running, Role: LEADER
I20260709 12:29:32.824380 22903 consensus_queue.cc:237] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } }
I20260709 12:29:32.842877 21133 catalog_manager.cc:5697] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce reported cstate change: term changed from 3 to 5, leader changed from 482723f787a14005854e038688430a80 (127.8.137.193) to 1095bdbabc944624b91311b39a5a02ce (127.8.137.196). New cstate: current_term: 5 leader_uuid: "1095bdbabc944624b91311b39a5a02ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1095bdbabc944624b91311b39a5a02ce" member_type: VOTER last_known_addr { host: "127.8.137.196" port: 36263 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "482723f787a14005854e038688430a80" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:33.072445 21278 consensus_peers.cc:597] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.121786 21278 consensus_peers.cc:597] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.130501  8743 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 22325)
W20260709 12:29:33.141083 21426 consensus_peers.cc:597] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.195993 21426 consensus_peers.cc:597] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.245800 21426 consensus_peers.cc:597] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.253010 21278 consensus_peers.cc:597] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:33.299834 21305 raft_consensus.cc:1275] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:29:33.301303 22792 consensus_queue.cc:1048] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:29:33.331763 21278 consensus_peers.cc:597] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.333243 21426 consensus_peers.cc:597] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:33.340374 21278 consensus_peers.cc:597] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.341046 21278 consensus_peers.cc:597] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.341137 21352 consensus_peers.cc:597] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.370935 21278 consensus_peers.cc:597] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:33.376458 21305 raft_consensus.cc:1275] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Refusing update from remote peer 1095bdbabc944624b91311b39a5a02ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:29:33.377882 22792 consensus_queue.cc:1048] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "b565aef1b93143c78c5c3dc4a861a68a" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:29:33.381353 21352 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111) [suppressed 418 similar messages]
W20260709 12:29:33.384963 21352 consensus_peers.cc:597] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.420341 21426 consensus_peers.cc:597] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.436218 21426 consensus_peers.cc:597] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.460071 21426 consensus_peers.cc:597] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.504071 21426 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.526892 21352 consensus_peers.cc:597] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.537199 21426 consensus_peers.cc:597] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.552685 21426 consensus_peers.cc:597] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.605443 21278 consensus_peers.cc:597] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.618105 21352 consensus_peers.cc:597] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.619899 21426 consensus_peers.cc:597] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.620428 21426 consensus_peers.cc:597] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.638511 21278 consensus_peers.cc:597] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.672770 21352 consensus_peers.cc:597] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.684170 21352 consensus_peers.cc:597] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.738487 21352 consensus_peers.cc:597] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.761049 21426 consensus_peers.cc:597] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.792964 21352 consensus_peers.cc:597] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.819265 21426 consensus_peers.cc:597] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.824471 21426 consensus_peers.cc:597] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.841898 21426 consensus_peers.cc:597] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.890338 21352 consensus_peers.cc:597] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.907852 21352 consensus_peers.cc:597] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.913900 21352 consensus_peers.cc:597] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:33.933224 21278 consensus_peers.cc:597] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:33.934798 21426 consensus_peers.cc:597] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:34.020713 21426 consensus_peers.cc:597] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:34.079021 21426 consensus_peers.cc:597] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:34.126997 21352 consensus_peers.cc:597] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:34.131068  8743 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 22325)
W20260709 12:29:34.211035 21278 consensus_peers.cc:597] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:34.301057 21352 consensus_peers.cc:597] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:34.384298 21426 consensus_peers.cc:597] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:35.083009 21426 consensus_peers.cc:597] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:35.131587  8743 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 22325)
W20260709 12:29:35.474581 21278 consensus_peers.cc:597] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:35.482915 21278 consensus_peers.cc:597] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:35.586778 21426 consensus_peers.cc:597] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:35.588311 21278 consensus_peers.cc:597] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:35.630878 21278 consensus_peers.cc:597] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:35.770026 21426 consensus_peers.cc:597] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:35.781420 21278 consensus_peers.cc:597] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:35.844977 21278 consensus_peers.cc:597] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:35.861063 21426 consensus_peers.cc:597] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:35.913686 21426 consensus_peers.cc:597] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:35.960896 21278 consensus_peers.cc:597] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:35.990473 21426 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.003891 21278 consensus_peers.cc:597] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:36.015240 21352 consensus_peers.cc:597] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:36.049451 21426 consensus_peers.cc:597] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:36.059494 21352 consensus_peers.cc:597] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.086946 21352 consensus_peers.cc:597] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.098435 21426 consensus_peers.cc:597] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.099362 21352 consensus_peers.cc:597] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.116978 21426 consensus_peers.cc:597] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.121551 21426 consensus_peers.cc:597] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:36.132182  8743 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 22325)
W20260709 12:29:36.137604 21426 consensus_peers.cc:597] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.153851 21278 consensus_peers.cc:597] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.153357 21352 consensus_peers.cc:597] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.215754 21278 consensus_peers.cc:597] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:36.223166 21426 consensus_peers.cc:597] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.225267 21352 consensus_peers.cc:597] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.225387 21426 consensus_peers.cc:597] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:36.277632  8743 tablet_server.cc:196] TabletServer@127.8.137.193:33017 shutdown complete.
W20260709 12:29:36.311151 21426 consensus_peers.cc:597] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.315183 21352 consensus_peers.cc:597] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.316421 21352 consensus_peers.cc:597] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.317576 21426 consensus_peers.cc:597] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:36.338666 21426 consensus_peers.cc:597] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.352999 21352 consensus_peers.cc:597] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.371091 21426 consensus_peers.cc:597] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.406715 21352 consensus_peers.cc:597] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:36.422245  8743 tablet_server.cc:179] TabletServer@127.8.137.194:0 shutting down...
W20260709 12:29:36.480517 21425 consensus_peers.cc:597] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.482892  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:36.486053  8743 tablet_replica.cc:333] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
W20260709 12:29:36.486498 21351 consensus_peers.cc:597] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.487222  8743 raft_consensus.cc:2243] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.487937  8743 raft_consensus.cc:2272] T a2384bff30cd4d3f9801e52ba3c28c08 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.491132  8743 tablet_replica.cc:333] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.492136  8743 raft_consensus.cc:2243] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:36.492628 21425 consensus_peers.cc:597] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.492838 21426 consensus_peers.cc:597] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:36.492833  8743 raft_consensus.cc:2272] T bc7d870ec3c54968ba293f0a1e2afab5 P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.496364  8743 tablet_replica.cc:333] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.497438  8743 raft_consensus.cc:2243] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.498996  8743 raft_consensus.cc:2272] T 1cb329d7c3274501a5c1c9f03eebd18f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.501536  8743 tablet_replica.cc:333] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.502239  8743 raft_consensus.cc:2243] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.503196  8743 raft_consensus.cc:2272] T f08ca19d90734deca4d113875d7def85 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.507295  8743 tablet_replica.cc:333] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.508186  8743 raft_consensus.cc:2243] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.508877  8743 raft_consensus.cc:2272] T e8a5b040cb6442da9a9cfca14c12225a P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.511845  8743 tablet_replica.cc:333] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.512524  8743 raft_consensus.cc:2243] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:29:36.512619 21351 consensus_peers.cc:597] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.513155  8743 raft_consensus.cc:2272] T d3c29a71139f45fc849c307af6259abb P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:36.513244 21425 consensus_peers.cc:597] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.515772  8743 tablet_replica.cc:333] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.516525  8743 raft_consensus.cc:2243] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.517139  8743 raft_consensus.cc:2272] T 482afeed6d904bb9a5133bff3cb22b8b P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.519490  8743 tablet_replica.cc:333] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.520084  8743 raft_consensus.cc:2243] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.521003  8743 raft_consensus.cc:2272] T c0cc4e1a190343f19f100f757eab0c65 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.523250  8743 tablet_replica.cc:333] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.523819  8743 raft_consensus.cc:2243] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.524266  8743 raft_consensus.cc:2272] T f78e89ba4d49471aaf6eb57ea8f738bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.526221  8743 tablet_replica.cc:333] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.526891  8743 raft_consensus.cc:2243] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.527314  8743 raft_consensus.cc:2272] T 9e2b7f6dfa92472f8a79868af4c7d106 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.529379  8743 tablet_replica.cc:333] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.529959  8743 raft_consensus.cc:2243] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.530414  8743 raft_consensus.cc:2272] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.532810  8743 tablet_replica.cc:333] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.533510  8743 raft_consensus.cc:2243] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.534073  8743 raft_consensus.cc:2272] T b1620754a7ca4c3ca00eff22b9493e9c P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:36.535558 21425 consensus_peers.cc:597] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.536563  8743 tablet_replica.cc:333] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.537237  8743 raft_consensus.cc:2243] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.538120  8743 raft_consensus.cc:2272] T f7e6811f72d0447781abab4178b3bdd4 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.541370  8743 tablet_replica.cc:333] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.542243  8743 raft_consensus.cc:2243] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.542918  8743 raft_consensus.cc:2272] T d0ceb29ef8444e9d96f0276fe5fe93fd P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.545737  8743 tablet_replica.cc:333] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.546506  8743 raft_consensus.cc:2243] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.547194  8743 raft_consensus.cc:2272] T 96593726ac324bc78be4e7c2bdb7d850 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.549585  8743 tablet_replica.cc:333] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.550226  8743 raft_consensus.cc:2243] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.551108  8743 raft_consensus.cc:2272] T 14c3f29aa4ff473db2aec020d298d279 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.553141  8743 tablet_replica.cc:333] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.553869  8743 raft_consensus.cc:2243] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.555264  8743 raft_consensus.cc:2272] T 5de0c984317b4a16bd865892a98794b7 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.557538  8743 tablet_replica.cc:333] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.558185  8743 raft_consensus.cc:2243] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.559105  8743 raft_consensus.cc:2272] T 79a82803d22242a991a35ece42323ea7 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.561370  8743 tablet_replica.cc:333] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.562023  8743 raft_consensus.cc:2243] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.562739  8743 raft_consensus.cc:2272] T 0d0bfa2f2d224261803bbc1ee6978296 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.566931  8743 tablet_replica.cc:333] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.567834  8743 raft_consensus.cc:2243] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 6 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.568815  8743 raft_consensus.cc:2272] T 5dc226a03f0e4040aabb242f51269a5b P b565aef1b93143c78c5c3dc4a861a68a [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.571158  8743 tablet_replica.cc:333] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.571808  8743 raft_consensus.cc:2243] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.572769  8743 raft_consensus.cc:2272] T fb7aa63607864dce845a979faebae247 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.575384  8743 tablet_replica.cc:333] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.575953  8743 raft_consensus.cc:2243] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.576423  8743 raft_consensus.cc:2272] T d49430f4da5e4c108a5cf967c8844659 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.578320  8743 tablet_replica.cc:333] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.578936  8743 raft_consensus.cc:2243] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.579382  8743 raft_consensus.cc:2272] T 27e40fdd4a714c729e2bcc297a44ec4f P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.581372  8743 tablet_replica.cc:333] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.581929  8743 raft_consensus.cc:2243] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.582383  8743 raft_consensus.cc:2272] T 5ed20ce36b0e46b6a043a3d1b558fb8f P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.584453  8743 tablet_replica.cc:333] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.584998  8743 raft_consensus.cc:2243] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.585464  8743 raft_consensus.cc:2272] T 13c370d75f794be68da9f93a6a876a55 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.587548  8743 tablet_replica.cc:333] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.588119  8743 raft_consensus.cc:2243] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.588966  8743 raft_consensus.cc:2272] T 93df7ba57f144211b8429c8c773bd74e P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.591163  8743 tablet_replica.cc:333] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.591837  8743 raft_consensus.cc:2243] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.592281  8743 raft_consensus.cc:2272] T 315529067cd1423da537bde53365bbbe P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.594852  8743 tablet_replica.cc:333] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.595567  8743 raft_consensus.cc:2243] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.596129  8743 raft_consensus.cc:2272] T d4da9543a3474f228769c7d28609a702 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:36.597391 21352 consensus_peers.cc:597] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:36.598721  8743 tablet_replica.cc:333] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.599313  8743 raft_consensus.cc:2243] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.600082  8743 raft_consensus.cc:2272] T 7d3bfcb7836d457caed630ee21058a19 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.602233  8743 tablet_replica.cc:333] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.602854  8743 raft_consensus.cc:2243] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.603726  8743 raft_consensus.cc:2272] T c956710bf64e4bb5b40784ba8060147c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.605913  8743 tablet_replica.cc:333] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.606694  8743 raft_consensus.cc:2243] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.607200  8743 raft_consensus.cc:2272] T a8c8433d1e274f879bb2b84ad59f5c3f P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.609975  8743 tablet_replica.cc:333] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
W20260709 12:29:36.610328 21426 consensus_peers.cc:597] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:29:36.610750  8743 raft_consensus.cc:2243] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.611414  8743 raft_consensus.cc:2272] T 31b54df203a347c993ed917ad39bbf1c P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:36.612145 21351 consensus_peers.cc:597] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.613929  8743 tablet_replica.cc:333] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.614642  8743 raft_consensus.cc:2243] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.615581  8743 raft_consensus.cc:2272] T 85044ca33b9947dface5200a544f79a2 P b565aef1b93143c78c5c3dc4a861a68a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.617851  8743 tablet_replica.cc:333] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.618444  8743 raft_consensus.cc:2243] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.619510  8743 raft_consensus.cc:2272] T 1f486fb8a1874281ab5022d0aea60379 P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.622630  8743 tablet_replica.cc:333] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.623423  8743 raft_consensus.cc:2243] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.624733  8743 raft_consensus.cc:2272] T dbd17ab2b7d04cc19cf64cdd44c442a7 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:36.625201 21425 consensus_peers.cc:597] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.627840  8743 tablet_replica.cc:333] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.628762  8743 raft_consensus.cc:2243] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.629359  8743 raft_consensus.cc:2272] T 81d20a4b91864a4787df9fdd31977f44 P b565aef1b93143c78c5c3dc4a861a68a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.632112  8743 tablet_replica.cc:333] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
W20260709 12:29:36.632273 21352 consensus_peers.cc:597] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:36.632876  8743 raft_consensus.cc:2243] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.633512  8743 raft_consensus.cc:2272] T a8f34093df9d4e6db1b428ae3b532fec P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.635896  8743 tablet_replica.cc:333] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.636427  8743 raft_consensus.cc:2243] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.637180  8743 raft_consensus.cc:2272] T b17439cec0b24d57853f995ce7c8e494 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.639406  8743 tablet_replica.cc:333] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.639914  8743 raft_consensus.cc:2243] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.640280  8743 raft_consensus.cc:2272] T 54443d308bbc43359e598ace1951bd70 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.642311  8743 tablet_replica.cc:333] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.642869  8743 raft_consensus.cc:2243] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.643430  8743 raft_consensus.cc:2272] T c4c805fca65a4771bf0e2270955c14c0 P b565aef1b93143c78c5c3dc4a861a68a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.646268  8743 tablet_replica.cc:333] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.646997  8743 raft_consensus.cc:2243] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.648068  8743 raft_consensus.cc:2272] T d22f7d658c5f48e7badd08e41f3e6e0d P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:36.648123 21351 consensus_peers.cc:597] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.650544  8743 tablet_replica.cc:333] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.651345  8743 raft_consensus.cc:2243] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.652370  8743 raft_consensus.cc:2272] T 3dbb15b421ee44b48cdf0afeed6430e9 P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.655279  8743 tablet_replica.cc:333] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.655855  8743 raft_consensus.cc:2243] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.656329  8743 raft_consensus.cc:2272] T 7ac1f07547154488aa8d69f3b35d29bf P b565aef1b93143c78c5c3dc4a861a68a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.658414  8743 tablet_replica.cc:333] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a: stopping tablet replica
I20260709 12:29:36.658928  8743 raft_consensus.cc:2243] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.659338  8743 raft_consensus.cc:2272] T 0ac0d0feb45a42ccaa8859902754853c P b565aef1b93143c78c5c3dc4a861a68a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:36.665620 21426 consensus_peers.cc:597] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.680361 21425 consensus_peers.cc:597] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.683445 21351 consensus_peers.cc:597] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.701540 21351 consensus_peers.cc:597] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.721732 21277 connection.cc:466] server connection from 127.0.0.1:47030 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:47030 (request call id 1006) could send its response
W20260709 12:29:36.723076 21425 connection.cc:573] client connection to 127.8.137.194:41173 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:29:36.723619 21425 consensus_peers.cc:597] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.734299 21351 consensus_peers.cc:597] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.734746 21351 consensus_peers.cc:597] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.739074 21425 consensus_peers.cc:597] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.744910 21425 consensus_peers.cc:597] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.763394  8743 tablet_server.cc:196] TabletServer@127.8.137.194:0 shutdown complete.
W20260709 12:29:36.813247 21351 consensus_peers.cc:597] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.832749 21351 consensus_peers.cc:597] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.851317  8743 tablet_server.cc:179] TabletServer@127.8.137.195:0 shutting down...
W20260709 12:29:36.859951 21425 consensus_peers.cc:597] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.881139 21423 consensus_peers.cc:597] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.882473 21426 consensus_peers.cc:597] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce -> Peer 482723f787a14005854e038688430a80 (127.8.137.193:33017): Couldn't send request to peer 482723f787a14005854e038688430a80. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:33017: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:29:36.895339 21425 consensus_peers.cc:597] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.898859 21425 consensus_peers.cc:597] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:36.899380 21425 consensus_peers.cc:597] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.909816  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:36.910769  8743 tablet_replica.cc:333] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.911716  8743 raft_consensus.cc:2243] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.912348  8743 raft_consensus.cc:2272] T f68cddadeb0443b28b03d76d5195fc45 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.914909  8743 tablet_replica.cc:333] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.915453  8743 raft_consensus.cc:2243] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.916235  8743 raft_consensus.cc:2272] T 315529067cd1423da537bde53365bbbe P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.918310  8743 tablet_replica.cc:333] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.918958  8743 raft_consensus.cc:2243] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.919363  8743 raft_consensus.cc:2272] T f6048d81937c4eaab03874e7a6577982 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.921273  8743 tablet_replica.cc:333] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.921793  8743 raft_consensus.cc:2243] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 6 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.922537  8743 raft_consensus.cc:2272] T bc7d870ec3c54968ba293f0a1e2afab5 P a3220692a27d44e49348c63d58fcf666 [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.925441  8743 tablet_replica.cc:333] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.926275  8743 raft_consensus.cc:2243] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.926887  8743 raft_consensus.cc:2272] T 1cb329d7c3274501a5c1c9f03eebd18f P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.929713  8743 tablet_replica.cc:333] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.930553  8743 raft_consensus.cc:2243] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.931668  8743 raft_consensus.cc:2272] T d3c29a71139f45fc849c307af6259abb P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.934407  8743 tablet_replica.cc:333] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.935212  8743 raft_consensus.cc:2243] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.935714  8743 raft_consensus.cc:2272] T c0cc4e1a190343f19f100f757eab0c65 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.938539  8743 tablet_replica.cc:333] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.939409  8743 raft_consensus.cc:2243] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.939947  8743 raft_consensus.cc:2272] T 9e2b7f6dfa92472f8a79868af4c7d106 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.942309  8743 tablet_replica.cc:333] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.942943  8743 raft_consensus.cc:2243] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.943745  8743 raft_consensus.cc:2272] T 1b2bb57292a3448bba84e7e4ade90382 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.945811  8743 tablet_replica.cc:333] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.946377  8743 raft_consensus.cc:2243] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.947238  8743 raft_consensus.cc:2272] T cdd6e626d31d4da8a2e4a407e1710ba2 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.949378  8743 tablet_replica.cc:333] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.949993  8743 raft_consensus.cc:2243] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.950472  8743 raft_consensus.cc:2272] T e5192ed91b2d4c14905f4a0a7410da9f P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.952771  8743 tablet_replica.cc:333] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.953348  8743 raft_consensus.cc:2243] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.954141  8743 raft_consensus.cc:2272] T 3cd8e0c4a39c4a85bc4a2663a8a39c5f P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.956467  8743 tablet_replica.cc:333] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.957051  8743 raft_consensus.cc:2243] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.957469  8743 raft_consensus.cc:2272] T 2f6dbe944bf742148d2688474712308b P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.959662  8743 tablet_replica.cc:333] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.960280  8743 raft_consensus.cc:2243] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.960739  8743 raft_consensus.cc:2272] T f08ca19d90734deca4d113875d7def85 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.962854  8743 tablet_replica.cc:333] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.963415  8743 raft_consensus.cc:2243] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.963874  8743 raft_consensus.cc:2272] T cc850ba2d5144b66bf78197d972b70d8 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.965924  8743 tablet_replica.cc:333] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.966645  8743 raft_consensus.cc:2243] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.967114  8743 raft_consensus.cc:2272] T 96593726ac324bc78be4e7c2bdb7d850 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.969436  8743 tablet_replica.cc:333] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.970304  8743 raft_consensus.cc:2243] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.970871  8743 raft_consensus.cc:2272] T 14c3f29aa4ff473db2aec020d298d279 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.973631  8743 tablet_replica.cc:333] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
W20260709 12:29:36.974107 21351 consensus_peers.cc:597] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.974423  8743 raft_consensus.cc:2243] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.975127  8743 raft_consensus.cc:2272] T 5de0c984317b4a16bd865892a98794b7 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:36.976943 21425 consensus_peers.cc:597] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce -> Peer b565aef1b93143c78c5c3dc4a861a68a (127.8.137.194:41173): Couldn't send request to peer b565aef1b93143c78c5c3dc4a861a68a. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41173: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:36.977821  8743 tablet_replica.cc:333] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.978447  8743 raft_consensus.cc:2243] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.978953  8743 raft_consensus.cc:2272] T 3dbb15b421ee44b48cdf0afeed6430e9 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.981314  8743 tablet_replica.cc:333] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.981925  8743 raft_consensus.cc:2243] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.983045  8743 raft_consensus.cc:2272] T 5ed20ce36b0e46b6a043a3d1b558fb8f P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.985680  8743 tablet_replica.cc:333] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.986291  8743 raft_consensus.cc:2243] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.987151  8743 raft_consensus.cc:2272] T 13c370d75f794be68da9f93a6a876a55 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.989198  8743 tablet_replica.cc:333] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.989831  8743 raft_consensus.cc:2243] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.990351  8743 raft_consensus.cc:2272] T 93df7ba57f144211b8429c8c773bd74e P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.992429  8743 tablet_replica.cc:333] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.992978  8743 raft_consensus.cc:2243] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:36.993439  8743 raft_consensus.cc:2272] T f7e6811f72d0447781abab4178b3bdd4 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.995520  8743 tablet_replica.cc:333] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.996137  8743 raft_consensus.cc:2243] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.996892  8743 raft_consensus.cc:2272] T d0ceb29ef8444e9d96f0276fe5fe93fd P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:36.998648  8743 tablet_replica.cc:333] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:36.999187  8743 raft_consensus.cc:2243] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:36.999897  8743 raft_consensus.cc:2272] T eaea00780b02433fa3a95bf6e01cb78d P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.001720  8743 tablet_replica.cc:333] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.002329  8743 raft_consensus.cc:2243] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.003341  8743 raft_consensus.cc:2272] T 7ac1f07547154488aa8d69f3b35d29bf P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.005934  8743 tablet_replica.cc:333] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.006657  8743 raft_consensus.cc:2243] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.007745  8743 raft_consensus.cc:2272] T 0ac0d0feb45a42ccaa8859902754853c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.010332  8743 tablet_replica.cc:333] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.010990  8743 raft_consensus.cc:2243] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.011449  8743 raft_consensus.cc:2272] T 9e36b9296b6e408da863f2bd3bb7b36b P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.013989  8743 tablet_replica.cc:333] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.014786  8743 raft_consensus.cc:2243] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.015331  8743 raft_consensus.cc:2272] T 85044ca33b9947dface5200a544f79a2 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.017336  8743 tablet_replica.cc:333] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.017848  8743 raft_consensus.cc:2243] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.018230  8743 raft_consensus.cc:2272] T 1f486fb8a1874281ab5022d0aea60379 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.020421  8743 tablet_replica.cc:333] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.021133  8743 raft_consensus.cc:2243] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.021677  8743 raft_consensus.cc:2272] T dbd17ab2b7d04cc19cf64cdd44c442a7 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.024278  8743 tablet_replica.cc:333] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.024979  8743 raft_consensus.cc:2243] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 8 LEADER]: Raft consensus shutting down.
W20260709 12:29:37.025022 21423 consensus_peers.cc:597] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:37.025964  8743 raft_consensus.cc:2272] T 63656d4d714a4ce89de05b1e5ba3fe43 P a3220692a27d44e49348c63d58fcf666 [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.028064  8743 tablet_replica.cc:333] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.028601  8743 raft_consensus.cc:2243] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.028956  8743 raft_consensus.cc:2272] T f44f7e65b47e48649e1e7ab852e0fc84 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.030875  8743 tablet_replica.cc:333] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.031427  8743 raft_consensus.cc:2243] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.032223  8743 raft_consensus.cc:2272] T f52db4db2acf4abfa01aa7604963edbc P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.034178  8743 tablet_replica.cc:333] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.034778  8743 raft_consensus.cc:2243] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.035640  8743 raft_consensus.cc:2272] T 31b54df203a347c993ed917ad39bbf1c P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.038300  8743 tablet_replica.cc:333] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
W20260709 12:29:37.038800 21423 consensus_peers.cc:597] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:37.039083  8743 raft_consensus.cc:2243] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.039606  8743 raft_consensus.cc:2272] T de84b253ee104e63be65f2ef9e9173dd P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.041848  8743 tablet_replica.cc:333] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.042630  8743 raft_consensus.cc:2243] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.043619  8743 raft_consensus.cc:2272] T 81d20a4b91864a4787df9fdd31977f44 P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.046422  8743 tablet_replica.cc:333] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.046981  8743 raft_consensus.cc:2243] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.047374  8743 raft_consensus.cc:2272] T 4d4f5802ffb348c78eaca62ca6fac2fc P a3220692a27d44e49348c63d58fcf666 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.049202  8743 tablet_replica.cc:333] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.049681  8743 raft_consensus.cc:2243] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.050055  8743 raft_consensus.cc:2272] T b17439cec0b24d57853f995ce7c8e494 P a3220692a27d44e49348c63d58fcf666 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.051915  8743 tablet_replica.cc:333] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.052434  8743 raft_consensus.cc:2243] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.053215  8743 raft_consensus.cc:2272] T 54443d308bbc43359e598ace1951bd70 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.055289  8743 tablet_replica.cc:333] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.055822  8743 raft_consensus.cc:2243] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.056273  8743 raft_consensus.cc:2272] T fb7aa63607864dce845a979faebae247 P a3220692a27d44e49348c63d58fcf666 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.058234  8743 tablet_replica.cc:333] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.058801  8743 raft_consensus.cc:2243] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.059625  8743 raft_consensus.cc:2272] T 27cdc0acc5734dd6ac5fb17c5805ce11 P a3220692a27d44e49348c63d58fcf666 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.061605  8743 tablet_replica.cc:333] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.062099  8743 raft_consensus.cc:2243] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.062692  8743 raft_consensus.cc:2272] T 7d3bfcb7836d457caed630ee21058a19 P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.064970  8743 tablet_replica.cc:333] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666: stopping tablet replica
I20260709 12:29:37.065534  8743 raft_consensus.cc:2243] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.066015  8743 raft_consensus.cc:2272] T c956710bf64e4bb5b40784ba8060147c P a3220692a27d44e49348c63d58fcf666 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:37.127799 21352 connection.cc:466] server connection from 127.0.0.1:39588 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39588 (request call id 871) could send its response
W20260709 12:29:37.129499 21423 consensus_peers.cc:597] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Network error: recv got EOF from 127.8.137.195:44499 (error 108). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:37.140527 21423 consensus_peers.cc:597] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.195:44499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:37.143303  8743 tablet_server.cc:196] TabletServer@127.8.137.195:0 shutdown complete.
W20260709 12:29:37.163390 21423 consensus_peers.cc:597] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.195:44499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:37.173302 21423 consensus_peers.cc:597] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.195:44499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:37.231642  8743 tablet_server.cc:179] TabletServer@127.8.137.196:0 shutting down...
I20260709 12:29:37.283109  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:37.284086  8743 tablet_replica.cc:333] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.285044  8743 raft_consensus.cc:2243] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.286082  8743 raft_consensus.cc:2272] T f68cddadeb0443b28b03d76d5195fc45 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:37.286716 21423 consensus_peers.cc:597] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.195:44499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:37.288390  8743 tablet_replica.cc:333] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.289090  8743 raft_consensus.cc:2243] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.289731  8743 raft_consensus.cc:2272] T bc7d870ec3c54968ba293f0a1e2afab5 P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.292791  8743 tablet_replica.cc:333] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.293648  8743 raft_consensus.cc:2243] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.294237  8743 raft_consensus.cc:2272] T 1cb329d7c3274501a5c1c9f03eebd18f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.297144  8743 tablet_replica.cc:333] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.297999  8743 raft_consensus.cc:2243] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.299031  8743 raft_consensus.cc:2272] T f6048d81937c4eaab03874e7a6577982 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.301237  8743 tablet_replica.cc:333] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.301862  8743 raft_consensus.cc:2243] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.302748  8743 raft_consensus.cc:2272] T e8a5b040cb6442da9a9cfca14c12225a P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.304629  8743 tablet_replica.cc:333] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.305243  8743 raft_consensus.cc:2243] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 6 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.306007  8743 raft_consensus.cc:2272] T 482afeed6d904bb9a5133bff3cb22b8b P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.308245  8743 tablet_replica.cc:333] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.308802  8743 raft_consensus.cc:2243] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.309213  8743 raft_consensus.cc:2272] T c0cc4e1a190343f19f100f757eab0c65 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.311362  8743 tablet_replica.cc:333] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.312244  8743 raft_consensus.cc:2243] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.313225  8743 raft_consensus.cc:2272] T f78e89ba4d49471aaf6eb57ea8f738bf P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:37.315594 21423 consensus_peers.cc:597] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.195:44499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:37.315905  8743 tablet_replica.cc:333] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.317001  8743 raft_consensus.cc:2243] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.317965  8743 raft_consensus.cc:2272] T 9e2b7f6dfa92472f8a79868af4c7d106 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.320501  8743 tablet_replica.cc:333] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.321280  8743 raft_consensus.cc:2243] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.321661  8743 raft_consensus.cc:2272] T 5dc226a03f0e4040aabb242f51269a5b P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.323575  8743 tablet_replica.cc:333] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.324095  8743 raft_consensus.cc:2243] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.324833  8743 raft_consensus.cc:2272] T e5192ed91b2d4c14905f4a0a7410da9f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.326786  8743 tablet_replica.cc:333] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.327316  8743 raft_consensus.cc:2243] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 6 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.328025  8743 raft_consensus.cc:2272] T b1620754a7ca4c3ca00eff22b9493e9c P 1095bdbabc944624b91311b39a5a02ce [term 6 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.329748  8743 tablet_replica.cc:333] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.330233  8743 raft_consensus.cc:2243] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.330651  8743 raft_consensus.cc:2272] T 1f486fb8a1874281ab5022d0aea60379 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.333123  8743 tablet_replica.cc:333] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.333900  8743 raft_consensus.cc:2243] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.334913  8743 raft_consensus.cc:2272] T 2f6dbe944bf742148d2688474712308b P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.337692  8743 tablet_replica.cc:333] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.338279  8743 raft_consensus.cc:2243] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.339120  8743 raft_consensus.cc:2272] T 96593726ac324bc78be4e7c2bdb7d850 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.341956  8743 tablet_replica.cc:333] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.342891  8743 raft_consensus.cc:2243] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.343519  8743 raft_consensus.cc:2272] T 79a82803d22242a991a35ece42323ea7 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:37.345906 21423 consensus_peers.cc:597] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce -> Peer a3220692a27d44e49348c63d58fcf666 (127.8.137.195:44499): Couldn't send request to peer a3220692a27d44e49348c63d58fcf666. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.195:44499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:37.346351  8743 tablet_replica.cc:333] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.347067  8743 raft_consensus.cc:2243] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.347890  8743 raft_consensus.cc:2272] T cc850ba2d5144b66bf78197d972b70d8 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.350018  8743 tablet_replica.cc:333] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.350883  8743 raft_consensus.cc:2243] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.351759  8743 raft_consensus.cc:2272] T d49430f4da5e4c108a5cf967c8844659 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.354473  8743 tablet_replica.cc:333] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.355347  8743 raft_consensus.cc:2243] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.356208  8743 raft_consensus.cc:2272] T 27e40fdd4a714c729e2bcc297a44ec4f P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.358220  8743 tablet_replica.cc:333] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.358857  8743 raft_consensus.cc:2243] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.359647  8743 raft_consensus.cc:2272] T a2384bff30cd4d3f9801e52ba3c28c08 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.361685  8743 tablet_replica.cc:333] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.362280  8743 raft_consensus.cc:2243] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.363229  8743 raft_consensus.cc:2272] T f44f7e65b47e48649e1e7ab852e0fc84 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.365520  8743 tablet_replica.cc:333] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.366220  8743 raft_consensus.cc:2243] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 8 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.366739  8743 raft_consensus.cc:2272] T 63656d4d714a4ce89de05b1e5ba3fe43 P 1095bdbabc944624b91311b39a5a02ce [term 8 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.369186  8743 tablet_replica.cc:333] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.369779  8743 raft_consensus.cc:2243] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.370201  8743 raft_consensus.cc:2272] T 315529067cd1423da537bde53365bbbe P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.372135  8743 tablet_replica.cc:333] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.372712  8743 raft_consensus.cc:2243] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.373561  8743 raft_consensus.cc:2272] T d4da9543a3474f228769c7d28609a702 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.375679  8743 tablet_replica.cc:333] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.376202  8743 raft_consensus.cc:2243] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.376892  8743 raft_consensus.cc:2272] T 0d0bfa2f2d224261803bbc1ee6978296 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.378957  8743 tablet_replica.cc:333] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.379472  8743 raft_consensus.cc:2243] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.379846  8743 raft_consensus.cc:2272] T eaea00780b02433fa3a95bf6e01cb78d P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.381785  8743 tablet_replica.cc:333] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.382305  8743 raft_consensus.cc:2243] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.382791  8743 raft_consensus.cc:2272] T 27cdc0acc5734dd6ac5fb17c5805ce11 P 1095bdbabc944624b91311b39a5a02ce [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.384776  8743 tablet_replica.cc:333] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.385298  8743 raft_consensus.cc:2243] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.386021  8743 raft_consensus.cc:2272] T c4c805fca65a4771bf0e2270955c14c0 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.387967  8743 tablet_replica.cc:333] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.388521  8743 raft_consensus.cc:2243] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.388880  8743 raft_consensus.cc:2272] T d22f7d658c5f48e7badd08e41f3e6e0d P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.390801  8743 tablet_replica.cc:333] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.391292  8743 raft_consensus.cc:2243] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.391665  8743 raft_consensus.cc:2272] T f52db4db2acf4abfa01aa7604963edbc P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.393519  8743 tablet_replica.cc:333] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.394068  8743 raft_consensus.cc:2243] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.394958  8743 raft_consensus.cc:2272] T de84b253ee104e63be65f2ef9e9173dd P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.396927  8743 tablet_replica.cc:333] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.397512  8743 raft_consensus.cc:2243] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.397975  8743 raft_consensus.cc:2272] T 81d20a4b91864a4787df9fdd31977f44 P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.399842  8743 tablet_replica.cc:333] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.400401  8743 raft_consensus.cc:2243] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.401176  8743 raft_consensus.cc:2272] T 4d4f5802ffb348c78eaca62ca6fac2fc P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.403146  8743 tablet_replica.cc:333] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.403680  8743 raft_consensus.cc:2243] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.404409  8743 raft_consensus.cc:2272] T a8f34093df9d4e6db1b428ae3b532fec P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.406363  8743 tablet_replica.cc:333] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.406966  8743 raft_consensus.cc:2243] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.407464  8743 raft_consensus.cc:2272] T 3dbb15b421ee44b48cdf0afeed6430e9 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.409639  8743 tablet_replica.cc:333] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.410275  8743 raft_consensus.cc:2243] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.411118  8743 raft_consensus.cc:2272] T 9e36b9296b6e408da863f2bd3bb7b36b P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.413295  8743 tablet_replica.cc:333] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.413870  8743 raft_consensus.cc:2243] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.414417  8743 raft_consensus.cc:2272] T 0ac0d0feb45a42ccaa8859902754853c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.416524  8743 tablet_replica.cc:333] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.417047  8743 raft_consensus.cc:2243] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.417483  8743 raft_consensus.cc:2272] T 7ac1f07547154488aa8d69f3b35d29bf P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.419420  8743 tablet_replica.cc:333] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.419909  8743 raft_consensus.cc:2243] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.420295  8743 raft_consensus.cc:2272] T 7d3bfcb7836d457caed630ee21058a19 P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.422017  8743 tablet_replica.cc:333] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.422520  8743 raft_consensus.cc:2243] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.422928  8743 raft_consensus.cc:2272] T c956710bf64e4bb5b40784ba8060147c P 1095bdbabc944624b91311b39a5a02ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.424736  8743 tablet_replica.cc:333] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.425228  8743 raft_consensus.cc:2243] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.425653  8743 raft_consensus.cc:2272] T 1b2bb57292a3448bba84e7e4ade90382 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.427321  8743 tablet_replica.cc:333] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.427798  8743 raft_consensus.cc:2243] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.428242  8743 raft_consensus.cc:2272] T cdd6e626d31d4da8a2e4a407e1710ba2 P 1095bdbabc944624b91311b39a5a02ce [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.430090  8743 tablet_replica.cc:333] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.430558  8743 raft_consensus.cc:2243] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.431318  8743 raft_consensus.cc:2272] T a8c8433d1e274f879bb2b84ad59f5c3f P 1095bdbabc944624b91311b39a5a02ce [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.433079  8743 tablet_replica.cc:333] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce: stopping tablet replica
I20260709 12:29:37.433553  8743 raft_consensus.cc:2243] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:37.433971  8743 raft_consensus.cc:2272] T b17439cec0b24d57853f995ce7c8e494 P 1095bdbabc944624b91311b39a5a02ce [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.543551  8743 tablet_server.cc:196] TabletServer@127.8.137.196:0 shutdown complete.
I20260709 12:29:37.607754  8743 master.cc:562] Master@127.8.137.254:36393 shutting down...
I20260709 12:29:37.635118  8743 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:37.635740  8743 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:37.636106  8743 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9f2356d6f7cb41419b59884869890214: stopping tablet replica
I20260709 12:29:37.658555  8743 master.cc:584] Master@127.8.137.254:36393 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (54043 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:29:37.752417  8743 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.137.254:33643
I20260709 12:29:37.754562  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:37.762516 22926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:37.764385 22927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:37.768368  8743 server_base.cc:1061] running on GCE node
W20260709 12:29:37.769943 22929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:37.771229  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:37.771540  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:37.771721  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600177771705 us; error 0 us; skew 500 ppm
I20260709 12:29:37.772380  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:37.775381  8743 webserver.cc:533] Webserver started at http://127.8.137.254:39825/ using document root <none> and password file <none>
I20260709 12:29:37.775902  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:37.776108  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:37.776403  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:37.777531  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/master-0-root/instance:
uuid: "b38031e4847f4e9da457e2a8be0cdd0b"
format_stamp: "Formatted at 2026-07-09 12:29:37 on dist-test-slave-kpwv"
I20260709 12:29:37.783390  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.003s
I20260709 12:29:37.788407 22934 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:37.791822  8743 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.004s	sys 0.001s
I20260709 12:29:37.792248  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/master-0-root
uuid: "b38031e4847f4e9da457e2a8be0cdd0b"
format_stamp: "Formatted at 2026-07-09 12:29:37 on dist-test-slave-kpwv"
I20260709 12:29:37.792670  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:37.824918  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:37.826164  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:37.864814  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.254:33643
I20260709 12:29:37.864930 22985 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.254:33643 every 8 connection(s)
I20260709 12:29:37.869002 22986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:37.880002 22986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b: Bootstrap starting.
I20260709 12:29:37.885299 22986 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:37.889810 22986 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b: No bootstrap required, opened a new log
I20260709 12:29:37.892143 22986 raft_consensus.cc:359] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38031e4847f4e9da457e2a8be0cdd0b" member_type: VOTER }
I20260709 12:29:37.892560 22986 raft_consensus.cc:385] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:37.892791 22986 raft_consensus.cc:740] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b38031e4847f4e9da457e2a8be0cdd0b, State: Initialized, Role: FOLLOWER
I20260709 12:29:37.893476 22986 consensus_queue.cc:260] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38031e4847f4e9da457e2a8be0cdd0b" member_type: VOTER }
I20260709 12:29:37.893950 22986 raft_consensus.cc:399] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:37.894168 22986 raft_consensus.cc:493] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:37.894410 22986 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:37.899694 22986 raft_consensus.cc:515] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38031e4847f4e9da457e2a8be0cdd0b" member_type: VOTER }
I20260709 12:29:37.900336 22986 leader_election.cc:304] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b38031e4847f4e9da457e2a8be0cdd0b; no voters: 
I20260709 12:29:37.901582 22986 leader_election.cc:290] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:37.902091 22989 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:37.903733 22989 raft_consensus.cc:697] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 1 LEADER]: Becoming Leader. State: Replica: b38031e4847f4e9da457e2a8be0cdd0b, State: Running, Role: LEADER
I20260709 12:29:37.904522 22989 consensus_queue.cc:237] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38031e4847f4e9da457e2a8be0cdd0b" member_type: VOTER }
I20260709 12:29:37.904968 22986 sys_catalog.cc:565] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:37.909327 22990 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b38031e4847f4e9da457e2a8be0cdd0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38031e4847f4e9da457e2a8be0cdd0b" member_type: VOTER } }
I20260709 12:29:37.910214 22990 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:37.912075 22997 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:37.912134 22991 sys_catalog.cc:455] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b38031e4847f4e9da457e2a8be0cdd0b. Latest consensus state: current_term: 1 leader_uuid: "b38031e4847f4e9da457e2a8be0cdd0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b38031e4847f4e9da457e2a8be0cdd0b" member_type: VOTER } }
I20260709 12:29:37.912981 22991 sys_catalog.cc:458] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:37.917326 22997 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:37.923862  8743 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:37.926748 22997 catalog_manager.cc:1383] Generated new cluster ID: 86c94eeaab04469a875c14391600589e
I20260709 12:29:37.927064 22997 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:37.944964 22997 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:37.946209 22997 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:37.963088 22997 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b: Generated new TSK 0
I20260709 12:29:37.963687 22997 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:37.993278  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:37.999440 23007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:38.000638 23008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:38.002341 23010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:38.002919  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:38.004021  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:38.004225  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:38.004400  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600178004380 us; error 0 us; skew 500 ppm
I20260709 12:29:38.004979  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:38.007427  8743 webserver.cc:533] Webserver started at http://127.8.137.193:42667/ using document root <none> and password file <none>
I20260709 12:29:38.007922  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:38.008113  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:38.008394  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:38.009543  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root/instance:
uuid: "fb2b3ec3aae740189bc991bee9e52925"
format_stamp: "Formatted at 2026-07-09 12:29:38 on dist-test-slave-kpwv"
I20260709 12:29:38.015398  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.003s
I20260709 12:29:38.018936 23015 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.019680  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:29:38.019997  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "fb2b3ec3aae740189bc991bee9e52925"
format_stamp: "Formatted at 2026-07-09 12:29:38 on dist-test-slave-kpwv"
I20260709 12:29:38.020287  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:38.028673  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:38.029913  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:38.031566  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:38.033833  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:38.034065  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.034328  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:38.034518  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.076589  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:37909
I20260709 12:29:38.076749 23077 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:37909 every 8 connection(s)
I20260709 12:29:38.083144  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:38.090438 23082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:38.091507 23083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:38.094332 23085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:38.095249  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:38.096733  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:38.096992  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:38.097177  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600178097137 us; error 0 us; skew 500 ppm
I20260709 12:29:38.097823 23078 heartbeater.cc:344] Connected to a master server at 127.8.137.254:33643
I20260709 12:29:38.097858  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:38.098366 23078 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:38.099210 23078 heartbeater.cc:507] Master 127.8.137.254:33643 requested a full tablet report, sending...
I20260709 12:29:38.100948  8743 webserver.cc:533] Webserver started at http://127.8.137.194:40721/ using document root <none> and password file <none>
I20260709 12:29:38.101509  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:38.101511 22951 ts_manager.cc:194] Registered new tserver with Master: fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909)
I20260709 12:29:38.101959  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:38.102327  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:38.103470  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-1-root/instance:
uuid: "bfa64f51bcd64967ad3ada1069c46b56"
format_stamp: "Formatted at 2026-07-09 12:29:38 on dist-test-slave-kpwv"
I20260709 12:29:38.103655 22951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51104
I20260709 12:29:38.108405  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:29:38.112061 23090 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.112840  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:38.113153  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-1-root
uuid: "bfa64f51bcd64967ad3ada1069c46b56"
format_stamp: "Formatted at 2026-07-09 12:29:38 on dist-test-slave-kpwv"
I20260709 12:29:38.113412  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:38.138118  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:38.139441  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:38.140950  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:38.143265  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:38.143500  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.143760  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:38.143934  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.188900  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.194:43125
I20260709 12:29:38.189051 23152 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.194:43125 every 8 connection(s)
I20260709 12:29:38.195878  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:38.204221 23156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:38.209334  8743 server_base.cc:1061] running on GCE node
W20260709 12:29:38.209260 23159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:38.210806 23153 heartbeater.cc:344] Connected to a master server at 127.8.137.254:33643
W20260709 12:29:38.210971 23157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:38.211321 23153 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:38.212046  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:38.212347  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:38.212188 23153 heartbeater.cc:507] Master 127.8.137.254:33643 requested a full tablet report, sending...
I20260709 12:29:38.212646  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600178212633 us; error 0 us; skew 500 ppm
I20260709 12:29:38.213392  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:38.214937 22951 ts_manager.cc:194] Registered new tserver with Master: bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:38.216475 22951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51114
I20260709 12:29:38.216496  8743 webserver.cc:533] Webserver started at http://127.8.137.195:45309/ using document root <none> and password file <none>
I20260709 12:29:38.217259  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:38.217489  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:38.217751  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:38.219182  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-2-root/instance:
uuid: "09d06605a7724754b17ada9579d17c7b"
format_stamp: "Formatted at 2026-07-09 12:29:38 on dist-test-slave-kpwv"
I20260709 12:29:38.223798  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 12:29:38.227546 23164 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.228436  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:29:38.228840  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-2-root
uuid: "09d06605a7724754b17ada9579d17c7b"
format_stamp: "Formatted at 2026-07-09 12:29:38 on dist-test-slave-kpwv"
I20260709 12:29:38.229162  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:38.252235  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:38.253463  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:38.255004  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:38.257258  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:38.257445  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.257750  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:38.257898  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:38.300311  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.195:34907
I20260709 12:29:38.300427 23226 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.195:34907 every 8 connection(s)
I20260709 12:29:38.314028 23227 heartbeater.cc:344] Connected to a master server at 127.8.137.254:33643
I20260709 12:29:38.314533 23227 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:38.315423 23227 heartbeater.cc:507] Master 127.8.137.254:33643 requested a full tablet report, sending...
I20260709 12:29:38.317380 22951 ts_manager.cc:194] Registered new tserver with Master: 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:38.318156  8743 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014626809s
I20260709 12:29:38.319006 22951 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51128
I20260709 12:29:38.344585 22951 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51138:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:38.347582 22951 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:38.419955 23118 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4414e54d6c465084d581f259215562 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:38.421496 23118 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4414e54d6c465084d581f259215562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.421386 23115 tablet_service.cc:1511] Processing CreateTablet for tablet bfd40c9cf4d94976b3b4d908e5e3e258 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:38.422940 23115 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd40c9cf4d94976b3b4d908e5e3e258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.423821 23117 tablet_service.cc:1511] Processing CreateTablet for tablet 25a9b271f2894c91b543e458a845cc89 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:38.430826 23192 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4414e54d6c465084d581f259215562 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:38.426702 23116 tablet_service.cc:1511] Processing CreateTablet for tablet 17f39d2dc2f34a3abd7b7a76f767c99f (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:38.432283 23192 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4414e54d6c465084d581f259215562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.433077 23116 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f39d2dc2f34a3abd7b7a76f767c99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.435173 23117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a9b271f2894c91b543e458a845cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.437501 23114 tablet_service.cc:1511] Processing CreateTablet for tablet 695fef5616834fc595570c13e142e255 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:38.439585 23114 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695fef5616834fc595570c13e142e255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.445637 23113 tablet_service.cc:1511] Processing CreateTablet for tablet 0c17bf5ce614455191209f3a51483c53 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:38.447170 23113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c17bf5ce614455191209f3a51483c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.452764 23112 tablet_service.cc:1511] Processing CreateTablet for tablet eb14294be772495c913f2227f7ab9f50 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:38.454276 23112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb14294be772495c913f2227f7ab9f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.460723 23111 tablet_service.cc:1511] Processing CreateTablet for tablet bc2fdb33ad7b46c39013e6df43d80d28 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:38.462234 23111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2fdb33ad7b46c39013e6df43d80d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.464128 23110 tablet_service.cc:1511] Processing CreateTablet for tablet 48857b99fb4f4b489ad2a4fca48e4013 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:38.466085 23110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48857b99fb4f4b489ad2a4fca48e4013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.468541 23109 tablet_service.cc:1511] Processing CreateTablet for tablet e744cd5af851467ab5041b62f4f5abb0 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:38.474020 23191 tablet_service.cc:1511] Processing CreateTablet for tablet 25a9b271f2894c91b543e458a845cc89 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:38.478314 23191 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a9b271f2894c91b543e458a845cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.479562 23042 tablet_service.cc:1511] Processing CreateTablet for tablet 25a9b271f2894c91b543e458a845cc89 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:29:38.483412 23042 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25a9b271f2894c91b543e458a845cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.470553 23109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e744cd5af851467ab5041b62f4f5abb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.480106 23043 tablet_service.cc:1511] Processing CreateTablet for tablet 0b4414e54d6c465084d581f259215562 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:29:38.486025 23043 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b4414e54d6c465084d581f259215562. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.532387 23041 tablet_service.cc:1511] Processing CreateTablet for tablet 17f39d2dc2f34a3abd7b7a76f767c99f (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:38.540882 23041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f39d2dc2f34a3abd7b7a76f767c99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.533612 23190 tablet_service.cc:1511] Processing CreateTablet for tablet 17f39d2dc2f34a3abd7b7a76f767c99f (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:29:38.535645 23189 tablet_service.cc:1511] Processing CreateTablet for tablet bfd40c9cf4d94976b3b4d908e5e3e258 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:38.543409 23190 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 17f39d2dc2f34a3abd7b7a76f767c99f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.537228 23040 tablet_service.cc:1511] Processing CreateTablet for tablet bfd40c9cf4d94976b3b4d908e5e3e258 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:29:38.545550 23040 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd40c9cf4d94976b3b4d908e5e3e258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.538867 23188 tablet_service.cc:1511] Processing CreateTablet for tablet 695fef5616834fc595570c13e142e255 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:38.548501 23188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695fef5616834fc595570c13e142e255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.555022 23189 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd40c9cf4d94976b3b4d908e5e3e258. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.522517 23242 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:38.559952 23039 tablet_service.cc:1511] Processing CreateTablet for tablet 695fef5616834fc595570c13e142e255 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:29:38.560604 23038 tablet_service.cc:1511] Processing CreateTablet for tablet 0c17bf5ce614455191209f3a51483c53 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:38.561627 23039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 695fef5616834fc595570c13e142e255. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.562366 23038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c17bf5ce614455191209f3a51483c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.563838 23187 tablet_service.cc:1511] Processing CreateTablet for tablet 0c17bf5ce614455191209f3a51483c53 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:29:38.563959 23037 tablet_service.cc:1511] Processing CreateTablet for tablet eb14294be772495c913f2227f7ab9f50 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:38.565419 23037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb14294be772495c913f2227f7ab9f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.565459 23187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c17bf5ce614455191209f3a51483c53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.567837 23036 tablet_service.cc:1511] Processing CreateTablet for tablet bc2fdb33ad7b46c39013e6df43d80d28 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:38.569239 23036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2fdb33ad7b46c39013e6df43d80d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.571802 23035 tablet_service.cc:1511] Processing CreateTablet for tablet 48857b99fb4f4b489ad2a4fca48e4013 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:38.573688 23035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48857b99fb4f4b489ad2a4fca48e4013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.575838 23034 tablet_service.cc:1511] Processing CreateTablet for tablet e744cd5af851467ab5041b62f4f5abb0 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:38.577244 23034 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e744cd5af851467ab5041b62f4f5abb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.579876 23186 tablet_service.cc:1511] Processing CreateTablet for tablet eb14294be772495c913f2227f7ab9f50 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:29:38.581333 23186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eb14294be772495c913f2227f7ab9f50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.583848 23185 tablet_service.cc:1511] Processing CreateTablet for tablet bc2fdb33ad7b46c39013e6df43d80d28 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:29:38.585314 23185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2fdb33ad7b46c39013e6df43d80d28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.587711 23184 tablet_service.cc:1511] Processing CreateTablet for tablet 48857b99fb4f4b489ad2a4fca48e4013 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:29:38.589087 23184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 48857b99fb4f4b489ad2a4fca48e4013. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.591723 23183 tablet_service.cc:1511] Processing CreateTablet for tablet e744cd5af851467ab5041b62f4f5abb0 (DEFAULT_TABLE table=test-workload [id=6c9fac57a1f4471ebfd289c58e92183b]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:29:38.593086 23183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e744cd5af851467ab5041b62f4f5abb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:38.638345 23243 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:38.657153 23244 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:38.667476 23242 tablet_bootstrap.cc:654] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.670012 23243 tablet_bootstrap.cc:654] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.687355 23244 tablet_bootstrap.cc:654] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.703378 23243 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:38.703862 23244 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:38.704002 23243 ts_tablet_manager.cc:1403] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.066s	user 0.017s	sys 0.003s
I20260709 12:29:38.704115 23242 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:38.704562 23244 ts_tablet_manager.cc:1403] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.048s	user 0.010s	sys 0.006s
I20260709 12:29:38.704790 23242 ts_tablet_manager.cc:1403] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.183s	user 0.014s	sys 0.005s
I20260709 12:29:38.707650 23244 raft_consensus.cc:359] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.708571 23244 raft_consensus.cc:385] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.708940 23244 raft_consensus.cc:740] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.708699 23242 raft_consensus.cc:359] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.709501 23242 raft_consensus.cc:385] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.709802 23242 raft_consensus.cc:740] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.709766 23244 consensus_queue.cc:260] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.710951 23242 consensus_queue.cc:260] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.713092 23243 raft_consensus.cc:359] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.714244 23243 raft_consensus.cc:385] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.714754 23243 raft_consensus.cc:740] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.715869 23243 consensus_queue.cc:260] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.719425 23227 heartbeater.cc:499] Master 127.8.137.254:33643 was elected leader, sending a full tablet report...
I20260709 12:29:38.719575 23244 ts_tablet_manager.cc:1434] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.006s
I20260709 12:29:38.719805 23243 ts_tablet_manager.cc:1434] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.001s
I20260709 12:29:38.720513 23078 heartbeater.cc:499] Master 127.8.137.254:33643 was elected leader, sending a full tablet report...
I20260709 12:29:38.721164 23243 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:38.733470 23248 raft_consensus.cc:493] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.734292 23248 raft_consensus.cc:515] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.736169 23244 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:38.745530 23250 raft_consensus.cc:493] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.746212 23153 heartbeater.cc:499] Master 127.8.137.254:33643 was elected leader, sending a full tablet report...
I20260709 12:29:38.746152 23250 raft_consensus.cc:515] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.747329 23242 ts_tablet_manager.cc:1434] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.042s	user 0.017s	sys 0.023s
I20260709 12:29:38.748446 23242 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:38.748989 23250 leader_election.cc:290] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909)
I20260709 12:29:38.750496 23244 tablet_bootstrap.cc:654] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.757148 23242 tablet_bootstrap.cc:654] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.777745 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
W20260709 12:29:38.780325 23166 leader_election.cc:343] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:38.784922 23243 tablet_bootstrap.cc:654] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.786856 23248 leader_election.cc:290] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:38.800076 23242 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:38.800724 23242 ts_tablet_manager.cc:1403] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.053s	user 0.028s	sys 0.006s
W20260709 12:29:38.804376 23228 tablet.cc:2406] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:29:38.804085 23242 raft_consensus.cc:359] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.808801 23242 raft_consensus.cc:385] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.809123 23242 raft_consensus.cc:740] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.809968 23242 consensus_queue.cc:260] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.812321 23242 ts_tablet_manager.cc:1434] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:29:38.813544 23242 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:38.818044 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a9b271f2894c91b543e458a845cc89" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:38.820677 23242 tablet_bootstrap.cc:654] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:38.820845 23017 leader_election.cc:343] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:38.838088 23242 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:38.838831 23242 ts_tablet_manager.cc:1403] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.026s	user 0.017s	sys 0.007s
I20260709 12:29:38.840849 23243 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:38.841423 23243 ts_tablet_manager.cc:1403] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.001s
I20260709 12:29:38.842041 23242 raft_consensus.cc:359] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.842860 23242 raft_consensus.cc:385] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.843149 23242 raft_consensus.cc:740] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.843964 23242 consensus_queue.cc:260] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.844579 23243 raft_consensus.cc:359] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.845360 23243 raft_consensus.cc:385] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.845672 23243 raft_consensus.cc:740] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.846280 23242 ts_tablet_manager.cc:1434] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:29:38.847383 23242 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:38.848439 23244 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:38.848987 23244 ts_tablet_manager.cc:1403] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.113s	user 0.045s	sys 0.052s
I20260709 12:29:38.851192 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a9b271f2894c91b543e458a845cc89" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:38.851759 23127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:38.852002 23128 raft_consensus.cc:2468] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb2b3ec3aae740189bc991bee9e52925 in term 0.
I20260709 12:29:38.852468 23127 raft_consensus.cc:2468] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 0.
I20260709 12:29:38.852382 23244 raft_consensus.cc:359] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.853296 23244 raft_consensus.cc:385] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.853603 23244 raft_consensus.cc:740] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.854033 23242 tablet_bootstrap.cc:654] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.853859 23165 leader_election.cc:304] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:38.854434 23244 consensus_queue.cc:260] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.855324 23251 raft_consensus.cc:2804] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:38.855785 23251 raft_consensus.cc:493] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:38.856134 23251 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.856810 23244 ts_tablet_manager.cc:1434] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:29:38.857476 23243 consensus_queue.cc:260] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.860368 23016 leader_election.cc:304] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:38.863404 23243 ts_tablet_manager.cc:1434] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:29:38.864589 23243 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:38.872143 23243 tablet_bootstrap.cc:654] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.873270 23251 raft_consensus.cc:515] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.875016 23248 raft_consensus.cc:2804] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:38.875491 23248 raft_consensus.cc:493] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:38.875854 23248 raft_consensus.cc:3060] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.877329 23244 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:38.878835 23242 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:38.879398 23242 ts_tablet_manager.cc:1403] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.010s
I20260709 12:29:38.882681 23242 raft_consensus.cc:359] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.883473 23242 raft_consensus.cc:385] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.883522 23248 raft_consensus.cc:515] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.883831 23242 raft_consensus.cc:740] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.884958 23242 consensus_queue.cc:260] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.887019 23127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:38.887805 23127 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.888199 23242 ts_tablet_manager.cc:1434] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:29:38.886950 23244 tablet_bootstrap.cc:654] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.889842 23242 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:38.894816 23251 leader_election.cc:290] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 election: Requested vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909)
I20260709 12:29:38.897432 23243 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:38.897883 23248 leader_election.cc:290] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 election: Requested vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:38.897984 23243 ts_tablet_manager.cc:1403] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.000s
I20260709 12:29:38.898485 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a9b271f2894c91b543e458a845cc89" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:38.899513 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:38.900014 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a9b271f2894c91b543e458a845cc89" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:38.900419 23242 tablet_bootstrap.cc:654] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.900573 23128 raft_consensus.cc:3060] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.901525 23243 raft_consensus.cc:359] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.902361 23243 raft_consensus.cc:385] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.903185 23243 raft_consensus.cc:740] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.906227 23244 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
W20260709 12:29:38.900023 23017 leader_election.cc:343] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:38.906975 23128 raft_consensus.cc:2468] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb2b3ec3aae740189bc991bee9e52925 in term 1.
I20260709 12:29:38.907223 23244 ts_tablet_manager.cc:1403] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.004s
W20260709 12:29:38.908023 23166 leader_election.cc:343] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:38.908236 23016 leader_election.cc:304] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:38.910071 23248 raft_consensus.cc:2804] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:38.910554 23248 raft_consensus.cc:697] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 1 LEADER]: Becoming Leader. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Running, Role: LEADER
I20260709 12:29:38.911427 23248 consensus_queue.cc:237] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.912523 23127 raft_consensus.cc:2468] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 1.
I20260709 12:29:38.912159 23244 raft_consensus.cc:359] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.913372 23244 raft_consensus.cc:385] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.913815 23244 raft_consensus.cc:740] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.914485 23165 leader_election.cc:304] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:38.914834 23243 consensus_queue.cc:260] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.915802 23251 raft_consensus.cc:2804] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:38.916297 23251 raft_consensus.cc:697] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:38.917164 23251 consensus_queue.cc:237] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.919248 23243 ts_tablet_manager.cc:1434] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
W20260709 12:29:38.920367 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:38.922144 23243 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:38.920049 23244 consensus_queue.cc:260] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.926314 23244 ts_tablet_manager.cc:1434] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.006s
I20260709 12:29:38.927244 23244 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:38.932631 23243 tablet_bootstrap.cc:654] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.933043 23244 tablet_bootstrap.cc:654] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.938943 22948 catalog_manager.cc:5697] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 1 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:38.917186 23242 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:38.944200 23242 ts_tablet_manager.cc:1403] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.009s
I20260709 12:29:38.945861 23243 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:38.946612 23243 ts_tablet_manager.cc:1403] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.025s	user 0.018s	sys 0.003s
I20260709 12:29:38.944931 23244 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:38.947746 23244 ts_tablet_manager.cc:1403] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.010s
I20260709 12:29:38.949021 23242 raft_consensus.cc:359] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.950814 23243 raft_consensus.cc:359] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.952764 23242 raft_consensus.cc:385] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.952979 23243 raft_consensus.cc:385] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.953323 23242 raft_consensus.cc:740] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.953517 23243 raft_consensus.cc:740] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.951447 23244 raft_consensus.cc:359] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.954538 23244 raft_consensus.cc:385] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:38.955075 23244 raft_consensus.cc:740] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:38.956166 23244 consensus_queue.cc:260] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.954542 23242 consensus_queue.cc:260] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.958021 23243 consensus_queue.cc:260] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.960639 23243 ts_tablet_manager.cc:1434] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:29:38.961722 23243 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:38.968161 23244 ts_tablet_manager.cc:1434] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:29:38.969373 23244 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:38.969933 23249 raft_consensus.cc:493] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.971066 23243 tablet_bootstrap.cc:654] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.971167 23249 raft_consensus.cc:515] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.976946 23244 tablet_bootstrap.cc:654] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:38.970364 23251 raft_consensus.cc:493] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:38.978789 23251 raft_consensus.cc:515] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:38.982890 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2fdb33ad7b46c39013e6df43d80d28" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:38.986362 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2fdb33ad7b46c39013e6df43d80d28" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:38.987356 23128 raft_consensus.cc:2468] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 0.
W20260709 12:29:38.989435 23166 leader_election.cc:343] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:38.990725 23251 leader_election.cc:290] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:38.990523 23165 leader_election.cc:304] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:38.993368 23249 leader_election.cc:290] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:38.994742 23251 raft_consensus.cc:2804] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:38.996145 23251 raft_consensus.cc:493] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:38.996647 23251 raft_consensus.cc:3060] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:38.999686 23261 raft_consensus.cc:493] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.000388 23261 raft_consensus.cc:515] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.005551 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
W20260709 12:29:39.007485 23017 leader_election.cc:343] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.008260 23261 leader_election.cc:290] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.019487 23242 ts_tablet_manager.cc:1434] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.075s	user 0.009s	sys 0.000s
I20260709 12:29:39.028771 22949 catalog_manager.cc:5697] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "fb2b3ec3aae740189bc991bee9e52925" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.036890 23242 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:39.039732 23250 raft_consensus.cc:493] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.040452 23250 raft_consensus.cc:515] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.043344 23250 leader_election.cc:290] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:39.045547 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.048223 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
W20260709 12:29:39.050227 23165 leader_election.cc:343] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:29:39.051982 23166 leader_election.cc:343] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.052588 23166 leader_election.cc:304] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b; no voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:39.054179 23250 raft_consensus.cc:2749] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:39.072480 23249 raft_consensus.cc:493] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.073093 23249 raft_consensus.cc:515] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.075340 23249 leader_election.cc:290] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.076630 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
W20260709 12:29:39.078501 23016 leader_election.cc:343] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.078989 23016 leader_election.cc:304] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb2b3ec3aae740189bc991bee9e52925; no voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:39.079869 23248 raft_consensus.cc:2749] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:39.081700 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:39.083623 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695fef5616834fc595570c13e142e255" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:39.084447 23201 raft_consensus.cc:2468] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
W20260709 12:29:39.086220 23092 leader_election.cc:343] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.087071 23092 leader_election.cc:304] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
I20260709 12:29:39.087993 23249 raft_consensus.cc:2804] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.088846 23249 raft_consensus.cc:493] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.089178 23249 raft_consensus.cc:3060] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.096038 23248 raft_consensus.cc:493] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.096668 23248 raft_consensus.cc:515] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.099424 23248 leader_election.cc:290] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.101792 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:39.102715 23128 raft_consensus.cc:2393] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb2b3ec3aae740189bc991bee9e52925 in current term 1: Already voted for candidate 09d06605a7724754b17ada9579d17c7b in this term.
I20260709 12:29:39.104120 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:39.111572 23251 raft_consensus.cc:515] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.112119 23242 tablet_bootstrap.cc:654] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.114398 23243 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:39.114567 23251 leader_election.cc:290] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:39.115124 23243 ts_tablet_manager.cc:1403] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.154s	user 0.038s	sys 0.069s
I20260709 12:29:39.115455 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2fdb33ad7b46c39013e6df43d80d28" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:39.115648 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2fdb33ad7b46c39013e6df43d80d28" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:39.116181 23128 raft_consensus.cc:3060] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:39.117044 23166 leader_election.cc:343] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.117760 23244 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:39.118332 23244 ts_tablet_manager.cc:1403] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.149s	user 0.034s	sys 0.089s
I20260709 12:29:39.118422 23243 raft_consensus.cc:359] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.119248 23243 raft_consensus.cc:385] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.119550 23243 raft_consensus.cc:740] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.120342 23243 consensus_queue.cc:260] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.122895 23243 ts_tablet_manager.cc:1434] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:39.124048 23243 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:39.121523 23244 raft_consensus.cc:359] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.124675 23244 raft_consensus.cc:385] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.125000 23244 raft_consensus.cc:740] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.126709 23128 raft_consensus.cc:2468] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 1.
I20260709 12:29:39.127427 23244 consensus_queue.cc:260] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.128069 23165 leader_election.cc:304] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:39.128561 23248 raft_consensus.cc:493] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.129772 23248 raft_consensus.cc:515] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.131947 23243 tablet_bootstrap.cc:654] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.132812 23016 leader_election.cc:304] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb2b3ec3aae740189bc991bee9e52925; no voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:39.133577 23248 leader_election.cc:290] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.134142 23248 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.135686 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
W20260709 12:29:39.137394 23017 leader_election.cc:343] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.138785 23251 raft_consensus.cc:2804] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.139354 23251 raft_consensus.cc:697] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:39.140223 23251 consensus_queue.cc:237] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.140796 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.145685 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695fef5616834fc595570c13e142e255" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.146328 23053 raft_consensus.cc:2468] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.146416 23243 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:39.147375 23243 ts_tablet_manager.cc:1403] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.005s
W20260709 12:29:39.148525 23092 leader_election.cc:343] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.149319 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:39.151671 23244 ts_tablet_manager.cc:1434] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.003s
I20260709 12:29:39.150149 23092 leader_election.cc:304] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56; no voters: 09d06605a7724754b17ada9579d17c7b, fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:39.154477 23244 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:39.098145 23249 raft_consensus.cc:515] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.161726 23242 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:39.162210 23244 tablet_bootstrap.cc:654] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.146530 23248 raft_consensus.cc:2749] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:39.163166 23249 leader_election.cc:290] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.151947 23243 raft_consensus.cc:359] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.165040 23249 raft_consensus.cc:2749] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:39.165915 23243 raft_consensus.cc:385] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.162542 23242 ts_tablet_manager.cc:1403] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.126s	user 0.009s	sys 0.026s
I20260709 12:29:39.166710 23243 raft_consensus.cc:740] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.167878 23243 consensus_queue.cc:260] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.170321 23243 ts_tablet_manager.cc:1434] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:29:39.171351 23243 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:39.171065 23242 raft_consensus.cc:359] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.172220 23242 raft_consensus.cc:385] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.172636 23242 raft_consensus.cc:740] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.173904 22950 catalog_manager.cc:5697] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 0 to 1, leader changed from <none> to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 1 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:39.174260 23242 consensus_queue.cc:260] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
W20260709 12:29:39.177376 23016 leader_election.cc:343] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.177912 23016 leader_election.cc:304] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fb2b3ec3aae740189bc991bee9e52925; no voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:39.178742 23248 raft_consensus.cc:2749] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:39.178866 23244 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:39.178960 23243 tablet_bootstrap.cc:654] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.179539 23249 raft_consensus.cc:493] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.180044 23249 raft_consensus.cc:515] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.179538 23244 ts_tablet_manager.cc:1403] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.000s
I20260709 12:29:39.181380 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695fef5616834fc595570c13e142e255" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:39.182214 23053 raft_consensus.cc:3060] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.183092 23249 leader_election.cc:290] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.185266 23052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.186012 23052 raft_consensus.cc:2468] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.186617 23242 ts_tablet_manager.cc:1434] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.019s	user 0.010s	sys 0.000s
I20260709 12:29:39.186703 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695fef5616834fc595570c13e142e255" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:39.187479 23201 raft_consensus.cc:3060] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.186025 23244 raft_consensus.cc:359] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.188694 23244 raft_consensus.cc:385] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.189002 23244 raft_consensus.cc:740] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.189565 23243 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:39.190093 23243 ts_tablet_manager.cc:1403] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.019s	user 0.012s	sys 0.000s
I20260709 12:29:39.189885 23244 consensus_queue.cc:260] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.191027 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:39.191681 23202 raft_consensus.cc:2468] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.192263 23244 ts_tablet_manager.cc:1434] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:29:39.193311 23244 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:39.193305 23243 raft_consensus.cc:359] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.194231 23243 raft_consensus.cc:385] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.187937 23242 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:39.194828 23243 raft_consensus.cc:740] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.195724 23243 consensus_queue.cc:260] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.200539 23244 tablet_bootstrap.cc:654] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.200875 23243 ts_tablet_manager.cc:1434] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:29:39.202179 23242 tablet_bootstrap.cc:654] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.202396 23243 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:39.209146 23248 raft_consensus.cc:493] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.209743 23243 tablet_bootstrap.cc:654] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.209944 23248 raft_consensus.cc:515] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.213099 23248 leader_election.cc:290] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.213809 23244 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:39.214731 23244 ts_tablet_manager.cc:1403] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.022s	user 0.019s	sys 0.000s
I20260709 12:29:39.215320 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
W20260709 12:29:39.217317 23017 leader_election.cc:343] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.217984 23244 raft_consensus.cc:359] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.219650 23244 raft_consensus.cc:385] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.219986 23244 raft_consensus.cc:740] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.220872 23244 consensus_queue.cc:260] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.223837 23244 ts_tablet_manager.cc:1434] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:29:39.225311 23244 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:39.229511 23242 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:39.230182 23242 ts_tablet_manager.cc:1403] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.000s
I20260709 12:29:39.187379 23092 leader_election.cc:304] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:39.231895 23244 tablet_bootstrap.cc:654] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.233434 23242 raft_consensus.cc:359] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.234688 23242 raft_consensus.cc:385] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.235085 23242 raft_consensus.cc:740] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.235111 23249 raft_consensus.cc:2804] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.235864 23249 raft_consensus.cc:493] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.236361 23249 raft_consensus.cc:3060] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.236433 23242 consensus_queue.cc:260] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.239650 23242 ts_tablet_manager.cc:1434] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.003s
I20260709 12:29:39.243868 23242 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
W20260709 12:29:39.244443 23092 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.193:37909, user_credentials={real_user=slave}} blocked reactor thread for 57125.4us
I20260709 12:29:39.253127 23242 tablet_bootstrap.cc:654] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.255349 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:39.256305 23128 raft_consensus.cc:2468] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb2b3ec3aae740189bc991bee9e52925 in term 0.
I20260709 12:29:39.259961 23244 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:39.260618 23244 ts_tablet_manager.cc:1403] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.036s	user 0.017s	sys 0.016s
I20260709 12:29:39.262956 23053 raft_consensus.cc:2468] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.268358 23092 leader_election.cc:304] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:39.265347 23016 leader_election.cc:304] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:39.252650 23201 raft_consensus.cc:2468] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.270860 23279 raft_consensus.cc:2804] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.250294 23249 raft_consensus.cc:515] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.271199 23248 raft_consensus.cc:2804] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.271694 23248 raft_consensus.cc:493] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.272101 23248 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.273144 23249 leader_election.cc:290] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.264947 23244 raft_consensus.cc:359] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.269930 23242 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:39.274575 23242 ts_tablet_manager.cc:1403] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.031s	user 0.009s	sys 0.013s
I20260709 12:29:39.278584 23242 raft_consensus.cc:359] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.279687 23242 raft_consensus.cc:385] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.280067 23242 raft_consensus.cc:740] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.281114 23242 consensus_queue.cc:260] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.282471 23244 raft_consensus.cc:385] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.282963 23244 raft_consensus.cc:740] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.284044 23244 consensus_queue.cc:260] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.287194 23244 ts_tablet_manager.cc:1434] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.026s	user 0.010s	sys 0.000s
I20260709 12:29:39.289709 23242 ts_tablet_manager.cc:1434] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:29:39.290020 23249 raft_consensus.cc:493] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.290869 23242 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:39.290634 23249 raft_consensus.cc:515] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.292260 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:39.293002 23249 leader_election.cc:290] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.293277 23201 raft_consensus.cc:3060] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.293829 23244 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:39.286971 23053 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:39.297684 23242 tablet_bootstrap.cc:654] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.298148 23053 raft_consensus.cc:3060] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.304447 23243 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:39.305335 23243 ts_tablet_manager.cc:1403] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.103s	user 0.029s	sys 0.055s
I20260709 12:29:39.305953 23251 consensus_queue.cc:1048] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.310647 23052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.311371 23052 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.311707 23242 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:39.311527 23248 raft_consensus.cc:515] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.312240 23242 ts_tablet_manager.cc:1403] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.007s
I20260709 12:29:39.312409 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:39.312711 23092 leader_election.cc:304] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:39.313103 23202 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.314806 23127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:39.315398 23127 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.315560 23248 leader_election.cc:290] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 election: Requested vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.316509 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "fb2b3ec3aae740189bc991bee9e52925" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:39.317413 23243 raft_consensus.cc:359] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
W20260709 12:29:39.317988 23017 leader_election.cc:343] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:39.318248 23243 raft_consensus.cc:385] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.318614 23243 raft_consensus.cc:740] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.319356 23243 consensus_queue.cc:260] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.324707 23127 raft_consensus.cc:2468] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb2b3ec3aae740189bc991bee9e52925 in term 1.
I20260709 12:29:39.327509 23016 leader_election.cc:304] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:39.328019 23201 raft_consensus.cc:2468] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.328476 23248 raft_consensus.cc:2804] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.325656 23279 raft_consensus.cc:697] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:39.329298 23248 raft_consensus.cc:697] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 LEADER]: Becoming Leader. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Running, Role: LEADER
I20260709 12:29:39.329694 23279 consensus_queue.cc:237] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.330195 23248 consensus_queue.cc:237] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.330682 23242 raft_consensus.cc:359] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.331545 23242 raft_consensus.cc:385] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.331893 23242 raft_consensus.cc:740] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.332127 23251 consensus_queue.cc:1048] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.325217 23287 raft_consensus.cc:2804] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.332947 23242 consensus_queue.cc:260] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.333346 23287 raft_consensus.cc:493] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.333920 23287 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.334965 23243 ts_tablet_manager.cc:1434] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 12:29:39.334775 23092 leader_election.cc:304] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
I20260709 12:29:39.335824 23249 raft_consensus.cc:2804] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.336396 23053 raft_consensus.cc:2468] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.338479 23243 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:39.351219 23242 ts_tablet_manager.cc:1434] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.039s	user 0.004s	sys 0.004s
I20260709 12:29:39.354252 23242 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56: Bootstrap starting.
I20260709 12:29:39.354951 23243 tablet_bootstrap.cc:654] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.356882 23244 tablet_bootstrap.cc:654] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.374982 22948 catalog_manager.cc:5697] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 reported cstate change: term changed from 0 to 1, leader changed from <none> to fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "fb2b3ec3aae740189bc991bee9e52925" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.388870 23249 raft_consensus.cc:697] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:39.390045 23249 consensus_queue.cc:237] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.391294 23243 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:39.391916 23243 ts_tablet_manager.cc:1403] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.054s	user 0.013s	sys 0.000s
I20260709 12:29:39.394673 23243 raft_consensus.cc:359] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.395553 23243 raft_consensus.cc:385] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.396349 23243 raft_consensus.cc:740] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.397429 23243 consensus_queue.cc:260] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.400337 23243 ts_tablet_manager.cc:1434] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:39.401857 23292 raft_consensus.cc:493] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.402607 23292 raft_consensus.cc:515] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.403362 23242 tablet_bootstrap.cc:654] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.401914 23287 raft_consensus.cc:515] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.409215 23287 leader_election.cc:290] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.412031 23292 leader_election.cc:290] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.415326 23051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e744cd5af851467ab5041b62f4f5abb0" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.416168 23243 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b: Bootstrap starting.
I20260709 12:29:39.416765 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:39.417421 23201 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.422027 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e744cd5af851467ab5041b62f4f5abb0" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:39.422881 23202 raft_consensus.cc:2468] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.423586 23243 tablet_bootstrap.cc:654] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.424680 23092 leader_election.cc:304] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
I20260709 12:29:39.416105 23051 raft_consensus.cc:2468] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.425894 23052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:39.430265 23292 raft_consensus.cc:2804] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.435117 23292 raft_consensus.cc:493] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.435778 23292 raft_consensus.cc:3060] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.438956 23052 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.449673 23244 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:39.450300 23244 ts_tablet_manager.cc:1403] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.157s	user 0.009s	sys 0.006s
I20260709 12:29:39.453131 23244 raft_consensus.cc:359] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.453830 23244 raft_consensus.cc:385] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.454079 23244 raft_consensus.cc:740] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.454787 23244 consensus_queue.cc:260] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.457995 23201 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.460237 23244 ts_tablet_manager.cc:1434] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:29:39.460590 23092 leader_election.cc:304] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
I20260709 12:29:39.461390 23244 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:39.461508 23249 raft_consensus.cc:2804] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.462309 23249 raft_consensus.cc:697] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:39.463202 23249 consensus_queue.cc:237] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.469134 23244 tablet_bootstrap.cc:654] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:39.470093 23299 raft_consensus.cc:493] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.470675 23299 raft_consensus.cc:515] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.472103 23242 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56: No bootstrap required, opened a new log
I20260709 12:29:39.472728 23242 ts_tablet_manager.cc:1403] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56: Time spent bootstrapping tablet: real 0.119s	user 0.023s	sys 0.007s
I20260709 12:29:39.474011 23051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a9b271f2894c91b543e458a845cc89" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.475629 23243 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b: No bootstrap required, opened a new log
I20260709 12:29:39.476152 23243 ts_tablet_manager.cc:1403] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b: Time spent bootstrapping tablet: real 0.060s	user 0.010s	sys 0.009s
I20260709 12:29:39.475930 23248 consensus_queue.cc:1048] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.478953 23299 leader_election.cc:290] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:39.475888 23242 raft_consensus.cc:359] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.480396 23242 raft_consensus.cc:385] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.480742 23126 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a9b271f2894c91b543e458a845cc89" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:39.482254 23126 raft_consensus.cc:2393] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 09d06605a7724754b17ada9579d17c7b in current term 1: Already voted for candidate fb2b3ec3aae740189bc991bee9e52925 in this term.
I20260709 12:29:39.479806 23243 raft_consensus.cc:359] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.483757 23243 raft_consensus.cc:385] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.483050 22948 catalog_manager.cc:5697] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.484102 23243 raft_consensus.cc:740] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.480736 23242 raft_consensus.cc:740] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.485237 23165 leader_election.cc:304] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b; no voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:39.485984 23242 consensus_queue.cc:260] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.487519 23243 consensus_queue.cc:260] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.490773 23242 ts_tablet_manager.cc:1434] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:29:39.491837 23250 raft_consensus.cc:3060] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.469326 23052 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.497161 23244 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: No bootstrap required, opened a new log
I20260709 12:29:39.497738 23244 ts_tablet_manager.cc:1403] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.037s	user 0.013s	sys 0.009s
I20260709 12:29:39.500699 23244 raft_consensus.cc:359] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.501405 23244 raft_consensus.cc:385] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:39.501675 23244 raft_consensus.cc:740] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:39.502691 23292 raft_consensus.cc:515] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.503698 23250 raft_consensus.cc:2749] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:39.505640 23052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e744cd5af851467ab5041b62f4f5abb0" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:39.506114 23243 ts_tablet_manager.cc:1434] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.001s
I20260709 12:29:39.505906 23244 consensus_queue.cc:260] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.506506 23052 raft_consensus.cc:3060] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.512223 23261 consensus_queue.cc:1048] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.514354 23052 raft_consensus.cc:2468] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.519348 23287 raft_consensus.cc:493] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.519994 23287 raft_consensus.cc:515] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.523420 23287 leader_election.cc:290] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.524025 23092 leader_election.cc:304] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:39.524837 23287 raft_consensus.cc:2804] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.525537 23287 raft_consensus.cc:697] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:39.525835 23052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.526510 23052 raft_consensus.cc:2468] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.526516 23287 consensus_queue.cc:237] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.529300 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:39.530139 23201 raft_consensus.cc:2468] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.531903 23092 leader_election.cc:304] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
I20260709 12:29:39.532944 23279 raft_consensus.cc:2804] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.533299 23279 raft_consensus.cc:493] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.533632 23279 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.538231 23292 leader_election.cc:290] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.539865 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e744cd5af851467ab5041b62f4f5abb0" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:39.540542 23201 raft_consensus.cc:3060] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.551370 23201 raft_consensus.cc:2468] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.553630 23249 raft_consensus.cc:493] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:39.554430 23249 raft_consensus.cc:515] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.555624 23279 raft_consensus.cc:515] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.558614 23279 leader_election.cc:290] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.560313 23249 leader_election.cc:290] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.560907 23244 ts_tablet_manager.cc:1434] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.063s	user 0.023s	sys 0.036s
I20260709 12:29:39.558794 23052 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925" is_pre_election: true
I20260709 12:29:39.561180 23051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:39.561585 23052 raft_consensus.cc:2468] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.561851 23051 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.565521 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:39.565968 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b" is_pre_election: true
I20260709 12:29:39.566231 23201 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.566067 23092 leader_election.cc:304] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:39.570420 23202 raft_consensus.cc:2468] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 0.
I20260709 12:29:39.571477 23287 raft_consensus.cc:2804] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:39.572022 23287 raft_consensus.cc:493] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.572373 23287 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.574079 23201 raft_consensus.cc:2468] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.575646 23092 leader_election.cc:304] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
I20260709 12:29:39.576581 23249 raft_consensus.cc:2804] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.577098 23051 raft_consensus.cc:2468] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.576293 22950 catalog_manager.cc:5697] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.577644 23249 raft_consensus.cc:697] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:39.578728 23249 consensus_queue.cc:237] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.579030 22950 catalog_manager.cc:5697] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.579788 23287 raft_consensus.cc:515] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.581729 23287 leader_election.cc:290] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:39.581188 22950 catalog_manager.cc:5697] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.583328 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:39.583447 23051 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:39.584151 23201 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.584227 23051 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.589517 23201 raft_consensus.cc:2468] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.590611 23092 leader_election.cc:304] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
I20260709 12:29:39.591426 23249 raft_consensus.cc:2804] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:39.591698 23051 raft_consensus.cc:2468] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 1.
I20260709 12:29:39.591907 23249 raft_consensus.cc:697] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:39.592864 23249 consensus_queue.cc:237] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.603897 22950 catalog_manager.cc:5697] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.605252 22950 catalog_manager.cc:5697] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 0 to 1, leader changed from <none> to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:39.654258 23051 raft_consensus.cc:3060] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.660547 23250 consensus_queue.cc:1048] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.673779 23250 consensus_queue.cc:1048] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:39.680363  8743 ts_manager.cc:295] Set tserver state for fb2b3ec3aae740189bc991bee9e52925 to MAINTENANCE_MODE
I20260709 12:29:39.681376  8743 tablet_server.cc:179] TabletServer@127.8.137.193:0 shutting down...
I20260709 12:29:39.709259  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:39.710299  8743 tablet_replica.cc:333] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.711601  8743 raft_consensus.cc:2243] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:39.712060  8743 raft_consensus.cc:2272] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.714459  8743 tablet_replica.cc:333] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.715211  8743 raft_consensus.cc:2243] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:39.715637  8743 raft_consensus.cc:2272] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.717701  8743 tablet_replica.cc:333] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.718382  8743 raft_consensus.cc:2243] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:39.718818  8743 raft_consensus.cc:2272] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.720784  8743 tablet_replica.cc:333] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.721369  8743 raft_consensus.cc:2243] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:39.721887  8743 raft_consensus.cc:2272] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.723814  8743 tablet_replica.cc:333] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.724365  8743 raft_consensus.cc:2243] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:39.724797  8743 raft_consensus.cc:2272] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.727432  8743 tablet_replica.cc:333] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.728026  8743 raft_consensus.cc:2243] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:39.728432  8743 raft_consensus.cc:2272] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.730242  8743 tablet_replica.cc:333] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.730973  8743 raft_consensus.cc:2243] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:39.731739  8743 pending_rounds.cc:70] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Trying to abort 1 pending ops.
I20260709 12:29:39.731638 23201 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:39.732177  8743 pending_rounds.cc:77] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305626334546456576 op_type: NO_OP noop_request { }
I20260709 12:29:39.732602  8743 raft_consensus.cc:2889] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:39.732957  8743 raft_consensus.cc:2272] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.735816  8743 tablet_replica.cc:333] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.736634  8743 raft_consensus.cc:2243] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:39.737185  8743 raft_consensus.cc:2272] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.740010  8743 tablet_replica.cc:333] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.740710  8743 raft_consensus.cc:2243] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:39.741160  8743 raft_consensus.cc:2272] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.742995  8743 tablet_replica.cc:333] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: stopping tablet replica
I20260709 12:29:39.743499  8743 raft_consensus.cc:2243] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:39.744326  8743 raft_consensus.cc:2272] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:39.778705  8743 tablet_server.cc:196] TabletServer@127.8.137.193:0 shutdown complete.
I20260709 12:29:39.853368 23249 consensus_queue.cc:1048] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:39.865926 23092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111) [suppressed 379 similar messages]
I20260709 12:29:39.869321 23250 raft_consensus.cc:493] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:29:39.869630 23092 consensus_peers.cc:597] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:39.869943 23250 raft_consensus.cc:515] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.871793 23250 leader_election.cc:290] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:39.872982 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:39.873725 23128 raft_consensus.cc:2468] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 1.
I20260709 12:29:39.874959 23165 leader_election.cc:304] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
I20260709 12:29:39.875801 23250 raft_consensus.cc:2804] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:39.876197 23250 raft_consensus.cc:493] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:39.876528 23250 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:39.877413 23166 leader_election.cc:336] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111)
I20260709 12:29:39.882257 23250 raft_consensus.cc:515] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.884164 23250 leader_election.cc:290] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:39.885426 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:39.886126 23128 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:39.888401 23166 leader_election.cc:336] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111)
W20260709 12:29:39.892283 23092 consensus_peers.cc:597] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:39.893128 23128 raft_consensus.cc:2468] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:39.894337 23165 leader_election.cc:304] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:39.895139 23250 raft_consensus.cc:2804] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:29:39.895370 23166 consensus_peers.cc:597] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:39.895671 23250 raft_consensus.cc:697] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:39.896576 23250 consensus_queue.cc:237] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:39.903515 22950 catalog_manager.cc:5697] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 1 to 2, leader changed from fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 2 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
W20260709 12:29:39.923079 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:39.934473 23249 consensus_queue.cc:1048] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:39.949985 23287 consensus_queue.cc:1048] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:39.954277 23092 consensus_peers.cc:597] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:40.029685 23092 consensus_peers.cc:597] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:40.047113 23092 consensus_peers.cc:597] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:40.064235 23249 consensus_queue.cc:1048] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:40.080629 23092 consensus_peers.cc:597] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:40.091989 23166 consensus_peers.cc:597] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:40.135589 23249 consensus_queue.cc:1048] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:40.142741 23287 consensus_queue.cc:1048] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:40.331467 23166 consensus_peers.cc:597] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:40.369088 23128 raft_consensus.cc:1275] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:29:40.370536 23250 consensus_queue.cc:1048] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:29:40.924258 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:41.016767 23250 raft_consensus.cc:493] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader fb2b3ec3aae740189bc991bee9e52925)
I20260709 12:29:41.017854 23250 raft_consensus.cc:515] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:41.020396 23250 leader_election.cc:290] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:41.021883 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a9b271f2894c91b543e458a845cc89" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56" is_pre_election: true
I20260709 12:29:41.023134 23128 raft_consensus.cc:2468] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 1.
I20260709 12:29:41.024960 23165 leader_election.cc:304] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: 
W20260709 12:29:41.025763 23166 leader_election.cc:336] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111)
I20260709 12:29:41.025977 23250 raft_consensus.cc:2804] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:29:41.026495 23250 raft_consensus.cc:493] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting leader election (detected failure of leader fb2b3ec3aae740189bc991bee9e52925)
I20260709 12:29:41.026885 23250 raft_consensus.cc:3060] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:41.035683 23250 raft_consensus.cc:515] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:41.037431 23250 leader_election.cc:290] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:41.038641 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25a9b271f2894c91b543e458a845cc89" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:41.039309 23128 raft_consensus.cc:3060] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:29:41.042143 23166 leader_election.cc:336] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111)
I20260709 12:29:41.048648 23128 raft_consensus.cc:2468] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:41.049966 23165 leader_election.cc:304] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:41.050741 23250 raft_consensus.cc:2804] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:41.051596 23250 raft_consensus.cc:697] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:41.052795 23250 consensus_queue.cc:237] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:41.062318 22950 catalog_manager.cc:5697] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 1 to 2, leader changed from fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 2 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:41.437068 23128 raft_consensus.cc:1275] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:41.438541 23250 consensus_queue.cc:1048] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:41.447417 23166 consensus_peers.cc:597] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:41.925432 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:42.256850 23092 consensus_peers.cc:597] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.263518 23166 consensus_peers.cc:597] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.409348 23092 consensus_peers.cc:597] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.534523 23092 consensus_peers.cc:597] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.688894 23092 consensus_peers.cc:597] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.718482 23092 consensus_peers.cc:597] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.739710 23092 consensus_peers.cc:597] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.828603 23166 consensus_peers.cc:597] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:42.926442 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:42.954756 23166 consensus_peers.cc:597] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:43.927791 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:29:43.946195 23166 consensus_peers.cc:597] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:29:44.637449 23166 consensus_peers.cc:597] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:29:44.689688 23092 consensus_peers.cc:597] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.798560  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:44.806344 23337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:44.806849 23338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:44.812388  8743 server_base.cc:1061] running on GCE node
W20260709 12:29:44.812669 23341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:44.814199  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:44.814500  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:44.814774  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600184814755 us; error 0 us; skew 500 ppm
I20260709 12:29:44.815591  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:44.818516  8743 webserver.cc:533] Webserver started at http://127.8.137.193:42667/ using document root <none> and password file <none>
I20260709 12:29:44.819155  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:44.819361  8743 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:29:44.823705  8743 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.001s	sys 0.004s
I20260709 12:29:44.827980 23345 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:44.828894  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:29:44.829330  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "fb2b3ec3aae740189bc991bee9e52925"
format_stamp: "Formatted at 2026-07-09 12:29:38 on dist-test-slave-kpwv"
I20260709 12:29:44.829723  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 12:29:44.837901 23092 consensus_peers.cc:597] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:29:44.851405  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:44.852955  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:44.854624  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:44.859314 23352 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:29:44.909467 23092 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111) [suppressed 100 similar messages]
I20260709 12:29:44.918969  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:29:44.919392  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.061s	user 0.000s	sys 0.003s
I20260709 12:29:44.919734  8743 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:29:44.925645 23352 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
W20260709 12:29:44.928822 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:44.940985 23352 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:44.941803 23352 tablet_bootstrap.cc:492] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:44.942426 23352 ts_tablet_manager.cc:1403] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.017s	user 0.014s	sys 0.000s
I20260709 12:29:44.944831 23352 raft_consensus.cc:359] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:44.945422 23352 raft_consensus.cc:740] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:44.946287 23352 consensus_queue.cc:260] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:44.950202 23352 ts_tablet_manager.cc:1434] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:44.951076 23352 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:44.962678 23352 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:44.963439 23352 tablet_bootstrap.cc:492] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:44.964043 23352 ts_tablet_manager.cc:1403] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 12:29:44.966195 23352 raft_consensus.cc:359] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:44.966836 23352 raft_consensus.cc:740] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:44.967418 23352 consensus_queue.cc:260] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:44.969014 23352 ts_tablet_manager.cc:1434] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:44.969704 23352 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:44.971544  8743 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:29:44.971863  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.052s	user 0.046s	sys 0.004s
I20260709 12:29:44.983846 23352 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:29:44.984853 23352 tablet_bootstrap.cc:492] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:44.985539 23352 ts_tablet_manager.cc:1403] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.000s
I20260709 12:29:44.988095 23352 raft_consensus.cc:359] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:44.988829 23352 raft_consensus.cc:740] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:44.989554 23352 consensus_queue.cc:260] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:44.991514 23352 ts_tablet_manager.cc:1434] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:44.992489 23352 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:45.005918 23352 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:45.006742 23352 tablet_bootstrap.cc:492] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:45.007330 23352 ts_tablet_manager.cc:1403] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.015s	user 0.010s	sys 0.004s
I20260709 12:29:45.009495 23352 raft_consensus.cc:359] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.010294 23352 raft_consensus.cc:740] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.011034 23352 consensus_queue.cc:260] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.012825 23352 ts_tablet_manager.cc:1434] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:29:45.013752 23352 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:45.026933 23352 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:45.028033 23352 tablet_bootstrap.cc:492] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:45.028878 23352 ts_tablet_manager.cc:1403] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 12:29:45.032591 23352 raft_consensus.cc:359] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.033324  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:37909
I20260709 12:29:45.033422 23352 raft_consensus.cc:740] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.033483 23414 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:37909 every 8 connection(s)
I20260709 12:29:45.034495 23352 consensus_queue.cc:260] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.036778 23352 ts_tablet_manager.cc:1434] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:45.037763 23352 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:45.041663  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:45.065951 23352 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:45.067076 23352 tablet_bootstrap.cc:492] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:45.067801 23352 ts_tablet_manager.cc:1403] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.007s
I20260709 12:29:45.070993 23352 raft_consensus.cc:359] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.071738 23352 raft_consensus.cc:740] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.072541 23352 consensus_queue.cc:260] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.073169 23415 heartbeater.cc:344] Connected to a master server at 127.8.137.254:33643
I20260709 12:29:45.073690 23415 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:45.074304 23352 ts_tablet_manager.cc:1434] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:45.074723 23415 heartbeater.cc:507] Master 127.8.137.254:33643 requested a full tablet report, sending...
I20260709 12:29:45.075193 23352 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:45.075028 23128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
mode: GRACEFUL
new_leader_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:54298
I20260709 12:29:45.075781 23128 raft_consensus.cc:606] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Received request to transfer leadership to TS 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:45.079653 23128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
mode: GRACEFUL
new_leader_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:54298
I20260709 12:29:45.080331 23128 raft_consensus.cc:606] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Received request to transfer leadership to TS 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:45.081578  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:29:45.082113  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:45.082805 22950 ts_manager.cc:194] Re-registered known tserver with Master: fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909)
I20260709 12:29:45.082903  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:45.091667 23352 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:45.092756 23352 tablet_bootstrap.cc:492] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:45.093447 23352 ts_tablet_manager.cc:1403] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.018s	user 0.007s	sys 0.007s
I20260709 12:29:45.096303 23352 raft_consensus.cc:359] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.097023 23352 raft_consensus.cc:740] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.097723 23352 consensus_queue.cc:260] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.098301 22950 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51154
I20260709 12:29:45.099830 23352 ts_tablet_manager.cc:1434] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:45.100741 23352 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:45.102193 23415 heartbeater.cc:499] Master 127.8.137.254:33643 was elected leader, sending a full tablet report...
I20260709 12:29:45.106642 23249 raft_consensus.cc:993] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56: : Instructing follower 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:45.107409 23424 raft_consensus.cc:1081] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Signalling peer 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:45.109468 23202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:57962
I20260709 12:29:45.110121 23202 raft_consensus.cc:493] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:45.110507 23202 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:45.113719 23352 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:45.114692 23352 tablet_bootstrap.cc:492] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:45.115446 23352 ts_tablet_manager.cc:1403] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.015s	user 0.014s	sys 0.000s
I20260709 12:29:45.118021 23424 raft_consensus.cc:993] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56: : Instructing follower 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:45.118546 23424 raft_consensus.cc:1081] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Signalling peer 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:45.119184 23352 raft_consensus.cc:359] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.120011 23352 raft_consensus.cc:740] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.120599 23201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:57962
I20260709 12:29:45.121217 23201 raft_consensus.cc:493] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:45.120903 23352 consensus_queue.cc:260] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.121644 23201 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:45.122397 23202 raft_consensus.cc:515] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.123558 23352 ts_tablet_manager.cc:1434] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:45.124464 23352 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:45.125871 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:45.126540 23388 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:45.126997 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:45.127620 23128 raft_consensus.cc:3055] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:45.127955 23128 raft_consensus.cc:740] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:45.128845 23128 consensus_queue.cc:260] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.130127 23128 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:45.134824 23201 raft_consensus.cc:515] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.135841 23388 raft_consensus.cc:2468] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:45.137351 23166 leader_election.cc:304] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:45.137820 23201 leader_election.cc:290] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:45.138509 23336 raft_consensus.cc:2804] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:45.139731 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:45.140419 23389 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:45.141541 23127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:45.142386 23127 raft_consensus.cc:3055] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:45.142844 23127 raft_consensus.cc:740] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:45.144168 23127 consensus_queue.cc:260] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.146234 23127 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:45.146727 23202 leader_election.cc:290] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:45.149451 23128 raft_consensus.cc:2468] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:45.149730 23389 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:45.153569 23352 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:45.153736 23166 leader_election.cc:304] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:45.154776 23352 tablet_bootstrap.cc:492] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:45.154968 23427 raft_consensus.cc:2804] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:45.155593 23352 ts_tablet_manager.cc:1403] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.031s	user 0.021s	sys 0.000s
I20260709 12:29:45.156515 23427 raft_consensus.cc:697] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:45.157279 23127 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:45.150650 23336 raft_consensus.cc:697] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:45.157639 23427 consensus_queue.cc:237] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.158676 23352 raft_consensus.cc:359] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.158905 23336 consensus_queue.cc:237] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.159497 23352 raft_consensus.cc:740] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.160287 23352 consensus_queue.cc:260] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.162997 23352 ts_tablet_manager.cc:1434] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 12:29:45.164232 23352 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap starting.
I20260709 12:29:45.176700 22951 catalog_manager.cc:5697] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 1 to 2, leader changed from bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 2 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:45.179679 23352 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:29:45.180718 23352 tablet_bootstrap.cc:492] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Bootstrap complete.
I20260709 12:29:45.181437 23352 ts_tablet_manager.cc:1403] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 12:29:45.184283 23352 raft_consensus.cc:359] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.184862 23352 raft_consensus.cc:740] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb2b3ec3aae740189bc991bee9e52925, State: Initialized, Role: FOLLOWER
I20260709 12:29:45.185608 23352 consensus_queue.cc:260] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:45.187913 23352 ts_tablet_manager.cc:1434] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.004s
I20260709 12:29:45.196425 22950 catalog_manager.cc:5697] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 1 to 2, leader changed from bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 2 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:45.337182 23388 raft_consensus.cc:3060] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:45.509048 23388 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:45.514344 23388 pending_rounds.cc:85] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Aborting all ops after (but not including) 0
I20260709 12:29:45.514591 23388 pending_rounds.cc:107] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:29:45.514815 23388 raft_consensus.cc:2889] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:29:45.562487 23127 raft_consensus.cc:1275] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:45.563992 23336 consensus_queue.cc:1048] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:45.590548 23388 raft_consensus.cc:1275] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:45.594256 23428 consensus_queue.cc:1048] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:45.642333 23388 raft_consensus.cc:1275] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:45.643817 23336 consensus_queue.cc:1048] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:45.654310 23127 raft_consensus.cc:1275] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:45.658037 23449 consensus_queue.cc:1048] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
W20260709 12:29:45.930022 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:46.092824  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:46.113866 23202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
mode: GRACEFUL
new_leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:57974
I20260709 12:29:46.114517 23202 raft_consensus.cc:606] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Received request to transfer leadership to TS bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:46.117241 23127 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "695fef5616834fc595570c13e142e255"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
mode: GRACEFUL
new_leader_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:54298
I20260709 12:29:46.117816 23127 raft_consensus.cc:606] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Received request to transfer leadership to TS 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:46.120435 23202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
mode: GRACEFUL
new_leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:57974
I20260709 12:29:46.120985 23202 raft_consensus.cc:606] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Received request to transfer leadership to TS bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:46.122179  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:46.122740  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:46.123361  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:46.502358 23451 raft_consensus.cc:993] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56: : Instructing follower 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:46.502841 23456 raft_consensus.cc:1081] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Signalling peer 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:46.504372 23202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "695fef5616834fc595570c13e142e255"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:57962
I20260709 12:29:46.504913 23202 raft_consensus.cc:493] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:46.505430 23202 raft_consensus.cc:3060] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:46.511720 23202 raft_consensus.cc:515] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.513566 23202 leader_election.cc:290] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:46.514806 23127 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695fef5616834fc595570c13e142e255" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:46.514847 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "695fef5616834fc595570c13e142e255" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:46.515604 23127 raft_consensus.cc:3055] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:46.515671 23388 raft_consensus.cc:3060] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:46.516105 23127 raft_consensus.cc:740] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:46.517163 23127 consensus_queue.cc:260] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.518481 23127 raft_consensus.cc:3060] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:46.523020 23388 raft_consensus.cc:2468] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:46.524323 23166 leader_election.cc:304] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:46.525082 23449 raft_consensus.cc:2804] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:46.525477 23127 raft_consensus.cc:2468] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:46.525663 23449 raft_consensus.cc:697] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:46.526906 23449 consensus_queue.cc:237] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.535075 22950 catalog_manager.cc:5697] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 1 to 2, leader changed from bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 2 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:46.571017 23336 raft_consensus.cc:993] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b: : Instructing follower bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:46.571491 23448 raft_consensus.cc:1081] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Signalling peer bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:46.572886 23128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:54296
I20260709 12:29:46.573342 23128 raft_consensus.cc:493] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:46.573591 23128 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:46.579478 23128 raft_consensus.cc:515] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.581086 23128 leader_election.cc:290] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 3 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:46.581984 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:46.582228 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "48857b99fb4f4b489ad2a4fca48e4013" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:46.582710 23388 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:46.582818 23202 raft_consensus.cc:3055] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:46.583214 23202 raft_consensus.cc:740] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:46.584067 23202 consensus_queue.cc:260] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.585453 23202 raft_consensus.cc:3060] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:46.590579 23388 raft_consensus.cc:2468] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 3.
I20260709 12:29:46.592144 23092 leader_election.cc:304] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:46.592978 23456 raft_consensus.cc:2804] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:46.593459 23202 raft_consensus.cc:2468] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 3.
I20260709 12:29:46.593518 23456 raft_consensus.cc:697] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:46.594630 23456 consensus_queue.cc:237] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.601689 22950 catalog_manager.cc:5697] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 2 to 3, leader changed from 09d06605a7724754b17ada9579d17c7b (127.8.137.195) to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:46.705893 23336 raft_consensus.cc:993] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b: : Instructing follower bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:46.706555 23448 raft_consensus.cc:1081] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Signalling peer bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:46.708130 23128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:54296
I20260709 12:29:46.708644 23128 raft_consensus.cc:493] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:46.708986 23128 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:46.714576 23128 raft_consensus.cc:515] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.716369 23128 leader_election.cc:290] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 3 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:46.717250 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:46.717474 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:46.717988 23388 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:46.718062 23202 raft_consensus.cc:3055] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:46.718542 23202 raft_consensus.cc:740] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:46.719369 23202 consensus_queue.cc:260] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.720359 23202 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:46.723601 23388 raft_consensus.cc:2468] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 3.
I20260709 12:29:46.724741 23092 leader_election.cc:304] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:46.725579 23456 raft_consensus.cc:2804] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:46.726087 23456 raft_consensus.cc:697] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:46.726243 23202 raft_consensus.cc:2468] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 3.
I20260709 12:29:46.727166 23456 consensus_queue.cc:237] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:46.734946 22950 catalog_manager.cc:5697] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 2 to 3, leader changed from 09d06605a7724754b17ada9579d17c7b (127.8.137.195) to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:46.931102 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:46.939533 23128 raft_consensus.cc:1275] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:46.941009 23336 consensus_queue.cc:1048] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:46.950009 23388 raft_consensus.cc:1275] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:46.951735 23449 consensus_queue.cc:1048] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:47.064456 23388 raft_consensus.cc:1275] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:47.065717 23456 consensus_queue.cc:1048] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:47.075800 23202 raft_consensus.cc:1275] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:47.077626 23470 consensus_queue.cc:1048] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:47.106794 23388 raft_consensus.cc:1275] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:47.108040 23470 consensus_queue.cc:1048] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:47.118355 23202 raft_consensus.cc:1275] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:29:47.119904 23473 consensus_queue.cc:1048] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:29:47.131628  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:47.139751 23202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b4414e54d6c465084d581f259215562"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
mode: GRACEFUL
new_leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:57974
I20260709 12:29:47.140339 23202 raft_consensus.cc:606] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Received request to transfer leadership to TS bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:47.142922 23128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
mode: GRACEFUL
new_leader_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:54298
I20260709 12:29:47.143435 23128 raft_consensus.cc:606] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Received request to transfer leadership to TS 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:47.145897 23128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
mode: GRACEFUL
new_leader_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:54298
I20260709 12:29:47.146361 23128 raft_consensus.cc:606] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Received request to transfer leadership to TS 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:47.147423  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:47.147879  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:47.148399  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:47.278362 23470 raft_consensus.cc:993] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56: : Instructing follower 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:47.278973 23473 raft_consensus.cc:1081] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Signalling peer 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:47.281297 23202 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:57962
I20260709 12:29:47.281883 23202 raft_consensus.cc:493] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:47.282226 23202 raft_consensus.cc:3060] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.287539 23202 raft_consensus.cc:515] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.289463 23202 leader_election.cc:290] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:47.290246 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:47.290731 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c17bf5ce614455191209f3a51483c53" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:47.291105 23389 raft_consensus.cc:3060] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.291307 23128 raft_consensus.cc:3055] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:47.291651 23128 raft_consensus.cc:740] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:47.292411 23128 consensus_queue.cc:260] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.293653 23128 raft_consensus.cc:3060] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.298629 23389 raft_consensus.cc:2468] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:47.299911 23166 leader_election.cc:304] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:47.300318 23128 raft_consensus.cc:2468] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:47.300650 23449 raft_consensus.cc:2804] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:47.301194 23449 raft_consensus.cc:697] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:47.302124 23449 consensus_queue.cc:237] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.309804 22951 catalog_manager.cc:5697] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 1 to 2, leader changed from bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 2 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:47.478749 23448 raft_consensus.cc:993] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b: : Instructing follower bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:47.479198 23449 raft_consensus.cc:1081] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Signalling peer bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:47.480599 23128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b4414e54d6c465084d581f259215562"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:54296
I20260709 12:29:47.481067 23128 raft_consensus.cc:493] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:47.481371 23128 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.486059 23128 raft_consensus.cc:515] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.487571 23128 leader_election.cc:290] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:47.488497 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:47.488718 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:47.489187 23389 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.489249 23201 raft_consensus.cc:3055] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:47.489706 23201 raft_consensus.cc:740] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:47.490672 23201 consensus_queue.cc:260] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.492007 23201 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:47.496894 23389 raft_consensus.cc:2468] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 2.
I20260709 12:29:47.498252 23092 leader_election.cc:304] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:47.498972 23473 raft_consensus.cc:2804] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:47.499051 23201 raft_consensus.cc:2468] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 2.
I20260709 12:29:47.499418 23473 raft_consensus.cc:697] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:47.500448 23473 consensus_queue.cc:237] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.508658 22951 catalog_manager.cc:5697] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 1 to 2, leader changed from 09d06605a7724754b17ada9579d17c7b (127.8.137.195) to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:47.573089 23473 raft_consensus.cc:993] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56: : Instructing follower 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:47.573599 23470 raft_consensus.cc:1081] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Signalling peer 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:47.575214 23201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:57962
I20260709 12:29:47.576103 23201 raft_consensus.cc:493] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:47.576454 23201 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:47.582657 23201 raft_consensus.cc:515] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.584347 23201 leader_election.cc:290] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 4 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:47.585086 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:47.585351 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eb14294be772495c913f2227f7ab9f50" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:47.585722 23389 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:47.585964 23128 raft_consensus.cc:3055] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:47.586321 23128 raft_consensus.cc:740] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:47.587019 23128 consensus_queue.cc:260] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.587896 23128 raft_consensus.cc:3060] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:47.591172 23389 raft_consensus.cc:2468] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 4.
I20260709 12:29:47.592321 23166 leader_election.cc:304] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:47.592366 23128 raft_consensus.cc:2468] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 4.
I20260709 12:29:47.593060 23449 raft_consensus.cc:2804] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:47.593513 23449 raft_consensus.cc:697] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 4 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:47.594246 23449 consensus_queue.cc:237] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:47.600481 22951 catalog_manager.cc:5697] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 3 to 4, leader changed from bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 4 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:47.832428 23128 raft_consensus.cc:1275] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:47.833770 23449 consensus_queue.cc:1048] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:47.862239 23389 raft_consensus.cc:1275] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:47.863943 23448 consensus_queue.cc:1048] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:47.932209 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:47.995492 23201 raft_consensus.cc:1275] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:47.996898 23470 consensus_queue.cc:1048] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:48.006829 23389 raft_consensus.cc:1275] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:48.008760 23456 consensus_queue.cc:1048] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:48.099874 23128 raft_consensus.cc:1275] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 4 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:48.101082 23448 consensus_queue.cc:1048] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:48.108306 23389 raft_consensus.cc:1275] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 4 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:29:48.110857 23448 consensus_queue.cc:1048] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:48.154709  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:48.161842 23128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
mode: GRACEFUL
new_leader_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:54298
I20260709 12:29:48.162367 23128 raft_consensus.cc:606] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Received request to transfer leadership to TS 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:48.164911 23201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bc2fdb33ad7b46c39013e6df43d80d28"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
mode: GRACEFUL
new_leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:57974
I20260709 12:29:48.165406 23201 raft_consensus.cc:606] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Received request to transfer leadership to TS bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:48.167876 23201 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
mode: GRACEFUL
new_leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:57974
I20260709 12:29:48.168299 23201 raft_consensus.cc:606] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Received request to transfer leadership to TS bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:48.169340  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:48.169843  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:48.170420  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:48.186781 23484 raft_consensus.cc:993] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b: : Instructing follower bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:48.187335 23448 raft_consensus.cc:1081] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Signalling peer bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:48.188877 23128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bc2fdb33ad7b46c39013e6df43d80d28"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:54296
I20260709 12:29:48.189368 23128 raft_consensus.cc:493] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:48.189656 23128 raft_consensus.cc:3060] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.194674 23128 raft_consensus.cc:515] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.196138 23128 leader_election.cc:290] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:48.197124 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2fdb33ad7b46c39013e6df43d80d28" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:48.197292 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2fdb33ad7b46c39013e6df43d80d28" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:48.197710 23389 raft_consensus.cc:3060] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.197902 23201 raft_consensus.cc:3055] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:48.198319 23201 raft_consensus.cc:740] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:48.199159 23201 consensus_queue.cc:260] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.200460 23201 raft_consensus.cc:3060] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.204946 23389 raft_consensus.cc:2468] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 2.
I20260709 12:29:48.206113 23092 leader_election.cc:304] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:48.206768 23201 raft_consensus.cc:2468] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 2.
I20260709 12:29:48.206892 23470 raft_consensus.cc:2804] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:48.207552 23470 raft_consensus.cc:697] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:48.208583 23470 consensus_queue.cc:237] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.216020 22950 catalog_manager.cc:5697] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 1 to 2, leader changed from 09d06605a7724754b17ada9579d17c7b (127.8.137.195) to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:48.314155 23456 raft_consensus.cc:993] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56: : Instructing follower 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:48.314669 23470 raft_consensus.cc:1081] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Signalling peer 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:48.316061 23201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:57962
I20260709 12:29:48.316510 23201 raft_consensus.cc:493] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:48.316800 23201 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.321606 23201 raft_consensus.cc:515] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.323168 23201 leader_election.cc:290] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:48.324120 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:48.324122 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:48.325043 23128 raft_consensus.cc:3055] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:48.325134 23389 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.325444 23128 raft_consensus.cc:740] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:48.326246 23128 consensus_queue.cc:260] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.327528 23128 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:48.332566 23389 raft_consensus.cc:2468] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:48.333829 23166 leader_election.cc:304] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:48.334614 23128 raft_consensus.cc:2468] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 2.
I20260709 12:29:48.334687 23448 raft_consensus.cc:2804] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:48.335294 23448 raft_consensus.cc:697] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:48.336136 23448 consensus_queue.cc:237] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.343099 22950 catalog_manager.cc:5697] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 1 to 2, leader changed from bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 2 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:48.603425 23388 raft_consensus.cc:1275] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:48.604645 23456 consensus_queue.cc:1048] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:48.607758 23201 raft_consensus.cc:1275] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:48.609719 23456 consensus_queue.cc:1048] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:48.765620 23483 raft_consensus.cc:993] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b: : Instructing follower bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:48.766165 23448 raft_consensus.cc:1081] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Signalling peer bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:48.767670 23128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:54296
I20260709 12:29:48.768136 23128 raft_consensus.cc:493] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:48.768412 23128 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:48.772950 23128 raft_consensus.cc:515] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.774466 23128 leader_election.cc:290] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 3 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:48.775424 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:48.775415 23202 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:48.776307 23388 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:48.776364 23202 raft_consensus.cc:3055] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:48.776818 23202 raft_consensus.cc:740] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:48.777667 23202 consensus_queue.cc:260] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.779029 23202 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:48.783699 23388 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 3.
I20260709 12:29:48.785039 23092 leader_election.cc:304] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:48.785698 23202 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 3.
I20260709 12:29:48.785765 23456 raft_consensus.cc:2804] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:48.786496 23456 raft_consensus.cc:697] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:48.787352 23456 consensus_queue.cc:237] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:48.793948 22948 catalog_manager.cc:5697] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 2 to 3, leader changed from 09d06605a7724754b17ada9579d17c7b (127.8.137.195) to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:48.920588 23388 raft_consensus.cc:1275] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:48.922070 23448 consensus_queue.cc:1048] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:48.924376 23128 raft_consensus.cc:1275] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:48.926034 23448 consensus_queue.cc:1048] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:29:48.933167 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:49.176795  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:49.185308 23128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0b4414e54d6c465084d581f259215562"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
mode: GRACEFUL
new_leader_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:54298
I20260709 12:29:49.185833 23128 raft_consensus.cc:606] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 LEADER]: Received request to transfer leadership to TS 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:49.188387 23202 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
mode: GRACEFUL
new_leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:57974
I20260709 12:29:49.188968 23202 raft_consensus.cc:606] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Received request to transfer leadership to TS bfa64f51bcd64967ad3ada1069c46b56
I20260709 12:29:49.192584 23128 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
mode: GRACEFUL
new_leader_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:54298
I20260709 12:29:49.193027 23128 raft_consensus.cc:606] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Received request to transfer leadership to TS 09d06605a7724754b17ada9579d17c7b
I20260709 12:29:49.194000  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 12:29:49.194501  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:29:49.195170  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:49.247345 23202 raft_consensus.cc:1275] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:49.248697 23470 consensus_queue.cc:1048] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:49.254559 23456 raft_consensus.cc:993] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56: : Instructing follower 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:49.255424 23470 raft_consensus.cc:1081] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Signalling peer 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:49.257970 23201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:57962
I20260709 12:29:49.258533 23201 raft_consensus.cc:493] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:49.258926 23201 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:49.260257 23388 raft_consensus.cc:1275] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:49.261864 23456 consensus_queue.cc:1048] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:49.265340 23201 raft_consensus.cc:515] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.268378 23389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:49.268567 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd40c9cf4d94976b3b4d908e5e3e258" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:49.269227 23128 raft_consensus.cc:3055] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:29:49.269541 23128 raft_consensus.cc:740] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:49.270314 23128 consensus_queue.cc:260] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.271297 23128 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:49.271592 23201 leader_election.cc:290] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 4 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125)
I20260709 12:29:49.278510 23128 raft_consensus.cc:2468] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 4.
I20260709 12:29:49.279927 23165 leader_election.cc:304] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, bfa64f51bcd64967ad3ada1069c46b56; no voters: fb2b3ec3aae740189bc991bee9e52925
I20260709 12:29:49.280723 23448 raft_consensus.cc:2804] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:29:49.281203 23448 raft_consensus.cc:697] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 4 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:49.281939 23448 consensus_queue.cc:237] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.287742 22950 catalog_manager.cc:5697] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 3 to 4, leader changed from bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 4 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:49.373880 23484 raft_consensus.cc:993] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b: : Instructing follower bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:49.374411 23448 raft_consensus.cc:1081] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Signalling peer bfa64f51bcd64967ad3ada1069c46b56 to start an election
I20260709 12:29:49.375814 23128 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f"
dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
 from {username='slave'} at 127.0.0.1:54296
I20260709 12:29:49.376296 23128 raft_consensus.cc:493] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:49.376588 23128 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:49.381587 23128 raft_consensus.cc:515] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.383085 23128 leader_election.cc:290] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 3 election: Requested vote from peers fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909), 09d06605a7724754b17ada9579d17c7b (127.8.137.195:34907)
I20260709 12:29:49.384064 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:49.384250 23201 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "17f39d2dc2f34a3abd7b7a76f767c99f" candidate_uuid: "bfa64f51bcd64967ad3ada1069c46b56" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "09d06605a7724754b17ada9579d17c7b"
I20260709 12:29:49.384826 23388 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:49.384881 23201 raft_consensus.cc:3055] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:49.385397 23201 raft_consensus.cc:740] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:49.386013 23201 consensus_queue.cc:260] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.386965 23201 raft_consensus.cc:3060] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:49.390422 23388 raft_consensus.cc:2468] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 3.
I20260709 12:29:49.391947 23092 leader_election.cc:304] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bfa64f51bcd64967ad3ada1069c46b56, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:49.391973 23201 raft_consensus.cc:2468] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bfa64f51bcd64967ad3ada1069c46b56 in term 3.
I20260709 12:29:49.392802 23456 raft_consensus.cc:2804] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:49.393299 23456 raft_consensus.cc:697] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Becoming Leader. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:49.394083 23456 consensus_queue.cc:237] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.400305 22950 catalog_manager.cc:5697] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 reported cstate change: term changed from 2 to 3, leader changed from 09d06605a7724754b17ada9579d17c7b (127.8.137.195) to bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194). New cstate: current_term: 3 leader_uuid: "bfa64f51bcd64967ad3ada1069c46b56" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:49.512365 23503 raft_consensus.cc:993] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56: : Instructing follower 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:49.512964 23456 raft_consensus.cc:1081] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 LEADER]: Signalling peer 09d06605a7724754b17ada9579d17c7b to start an election
I20260709 12:29:49.514429 23201 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0b4414e54d6c465084d581f259215562"
dest_uuid: "09d06605a7724754b17ada9579d17c7b"
 from {username='slave'} at 127.0.0.1:57962
I20260709 12:29:49.514990 23201 raft_consensus.cc:493] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:49.515336 23201 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:49.520879 23201 raft_consensus.cc:515] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.522300 23201 leader_election.cc:290] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 3 election: Requested vote from peers bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125), fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909)
I20260709 12:29:49.523193 23128 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bfa64f51bcd64967ad3ada1069c46b56"
I20260709 12:29:49.523320 23388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0b4414e54d6c465084d581f259215562" candidate_uuid: "09d06605a7724754b17ada9579d17c7b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "fb2b3ec3aae740189bc991bee9e52925"
I20260709 12:29:49.523758 23128 raft_consensus.cc:3055] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:49.523901 23388 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:49.524073 23128 raft_consensus.cc:740] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bfa64f51bcd64967ad3ada1069c46b56, State: Running, Role: LEADER
I20260709 12:29:49.524852 23128 consensus_queue.cc:260] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.525753 23128 raft_consensus.cc:3060] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:49.528723 23388 raft_consensus.cc:2468] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 3.
I20260709 12:29:49.529757 23166 leader_election.cc:304] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09d06605a7724754b17ada9579d17c7b, fb2b3ec3aae740189bc991bee9e52925; no voters: 
I20260709 12:29:49.530471 23448 raft_consensus.cc:2804] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:49.530524 23128 raft_consensus.cc:2468] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09d06605a7724754b17ada9579d17c7b in term 3.
I20260709 12:29:49.531073 23448 raft_consensus.cc:697] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 3 LEADER]: Becoming Leader. State: Replica: 09d06605a7724754b17ada9579d17c7b, State: Running, Role: LEADER
I20260709 12:29:49.531926 23448 consensus_queue.cc:237] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } }
I20260709 12:29:49.539657 22950 catalog_manager.cc:5697] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b reported cstate change: term changed from 2 to 3, leader changed from bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194) to 09d06605a7724754b17ada9579d17c7b (127.8.137.195). New cstate: current_term: 3 leader_uuid: "09d06605a7724754b17ada9579d17c7b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:49.741492 23388 raft_consensus.cc:3060] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:29:49.746536 23388 raft_consensus.cc:1275] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 4 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:49.747889 23448 consensus_queue.cc:1048] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:49.757153 23128 raft_consensus.cc:1275] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 4 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:29:49.758838 23484 consensus_queue.cc:1048] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:29:49.845211 23388 raft_consensus.cc:1275] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:49.846428 23456 consensus_queue.cc:1048] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:49.854426 23201 raft_consensus.cc:1275] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Refusing update from remote peer bfa64f51bcd64967ad3ada1069c46b56: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:49.856240 23456 consensus_queue.cc:1048] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [LEADER]: Connected to new peer: Peer: permanent_uuid: "09d06605a7724754b17ada9579d17c7b" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:49.929105 23388 raft_consensus.cc:1275] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:49.930356 23507 consensus_queue.cc:1048] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb2b3ec3aae740189bc991bee9e52925" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 37909 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 12:29:49.934342 22999 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:29:49.939949 23128 raft_consensus.cc:1275] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Refusing update from remote peer 09d06605a7724754b17ada9579d17c7b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:29:49.941566 23448 consensus_queue.cc:1048] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfa64f51bcd64967ad3ada1069c46b56" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 43125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:29:50.201583  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:29:50.231099  8743 tablet_server.cc:179] TabletServer@127.8.137.193:37909 shutting down...
I20260709 12:29:50.251017  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:50.251703  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.252314  8743 raft_consensus.cc:2243] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.252856  8743 raft_consensus.cc:2272] T 695fef5616834fc595570c13e142e255 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:50.254671 23166 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
I20260709 12:29:50.255422  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.256121  8743 raft_consensus.cc:2243] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.256567  8743 raft_consensus.cc:2272] T 25a9b271f2894c91b543e458a845cc89 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:50.257702 23166 consensus_peers.cc:597] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.258932  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.259578  8743 raft_consensus.cc:2243] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.260020  8743 raft_consensus.cc:2272] T 0c17bf5ce614455191209f3a51483c53 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.262018  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.262640  8743 raft_consensus.cc:2243] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.263080  8743 raft_consensus.cc:2272] T bc2fdb33ad7b46c39013e6df43d80d28 P fb2b3ec3aae740189bc991bee9e52925 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.264992  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.265560  8743 raft_consensus.cc:2243] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.266009  8743 raft_consensus.cc:2272] T 17f39d2dc2f34a3abd7b7a76f767c99f P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.268438  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.269248  8743 raft_consensus.cc:2243] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.269711  8743 raft_consensus.cc:2272] T 0b4414e54d6c465084d581f259215562 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.272799  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.273397  8743 raft_consensus.cc:2243] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.273857  8743 raft_consensus.cc:2272] T eb14294be772495c913f2227f7ab9f50 P fb2b3ec3aae740189bc991bee9e52925 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.275779  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.276325  8743 raft_consensus.cc:2243] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.276764  8743 raft_consensus.cc:2272] T 48857b99fb4f4b489ad2a4fca48e4013 P fb2b3ec3aae740189bc991bee9e52925 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.278875  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.279640  8743 raft_consensus.cc:2243] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.280171  8743 raft_consensus.cc:2272] T e744cd5af851467ab5041b62f4f5abb0 P fb2b3ec3aae740189bc991bee9e52925 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.282389  8743 tablet_replica.cc:333] stopping tablet replica
I20260709 12:29:50.283072  8743 raft_consensus.cc:2243] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.283586  8743 raft_consensus.cc:2272] T bfd40c9cf4d94976b3b4d908e5e3e258 P fb2b3ec3aae740189bc991bee9e52925 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:29:50.304847 23166 consensus_peers.cc:597] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.315412  8743 tablet_server.cc:196] TabletServer@127.8.137.193:37909 shutdown complete.
W20260709 12:29:50.326406 23166 consensus_peers.cc:597] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.338825  8743 tablet_server.cc:179] TabletServer@127.8.137.194:0 shutting down...
W20260709 12:29:50.342347 23166 consensus_peers.cc:597] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b -> Peer fb2b3ec3aae740189bc991bee9e52925 (127.8.137.193:37909): Couldn't send request to peer fb2b3ec3aae740189bc991bee9e52925. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:37909: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:29:50.346117 23165 consensus_peers.cc:597] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b -> Peer bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125): Couldn't send request to peer bfa64f51bcd64967ad3ada1069c46b56. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.370227  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:29:50.370787 23165 consensus_peers.cc:597] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b -> Peer bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125): Couldn't send request to peer bfa64f51bcd64967ad3ada1069c46b56. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.371124  8743 tablet_replica.cc:333] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.371970  8743 raft_consensus.cc:2243] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.372716  8743 raft_consensus.cc:2272] T 48857b99fb4f4b489ad2a4fca48e4013 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.375059  8743 tablet_replica.cc:333] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.375662  8743 raft_consensus.cc:2243] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.376487  8743 raft_consensus.cc:2272] T e744cd5af851467ab5041b62f4f5abb0 P bfa64f51bcd64967ad3ada1069c46b56 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.378820  8743 tablet_replica.cc:333] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.379385  8743 raft_consensus.cc:2243] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.379814  8743 raft_consensus.cc:2272] T bfd40c9cf4d94976b3b4d908e5e3e258 P bfa64f51bcd64967ad3ada1069c46b56 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.382803  8743 tablet_replica.cc:333] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.383682  8743 raft_consensus.cc:2243] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.384089  8743 raft_consensus.cc:2272] T 0c17bf5ce614455191209f3a51483c53 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.386210  8743 tablet_replica.cc:333] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.386835  8743 raft_consensus.cc:2243] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.387215  8743 raft_consensus.cc:2272] T 0b4414e54d6c465084d581f259215562 P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.391216  8743 tablet_replica.cc:333] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.392025  8743 raft_consensus.cc:2243] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.392746  8743 raft_consensus.cc:2272] T eb14294be772495c913f2227f7ab9f50 P bfa64f51bcd64967ad3ada1069c46b56 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.395828  8743 tablet_replica.cc:333] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.396412  8743 raft_consensus.cc:2243] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.396816  8743 raft_consensus.cc:2272] T 25a9b271f2894c91b543e458a845cc89 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.399082  8743 tablet_replica.cc:333] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.399577  8743 raft_consensus.cc:2243] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.400015  8743 raft_consensus.cc:2272] T 695fef5616834fc595570c13e142e255 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.402031  8743 tablet_replica.cc:333] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.402526  8743 raft_consensus.cc:2243] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.403338  8743 raft_consensus.cc:2272] T 17f39d2dc2f34a3abd7b7a76f767c99f P bfa64f51bcd64967ad3ada1069c46b56 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.405821  8743 tablet_replica.cc:333] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56: stopping tablet replica
I20260709 12:29:50.406389  8743 raft_consensus.cc:2243] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.407212  8743 raft_consensus.cc:2272] T bc2fdb33ad7b46c39013e6df43d80d28 P bfa64f51bcd64967ad3ada1069c46b56 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.430109  8743 tablet_server.cc:196] TabletServer@127.8.137.194:0 shutdown complete.
I20260709 12:29:50.453742  8743 tablet_server.cc:179] TabletServer@127.8.137.195:0 shutting down...
I20260709 12:29:50.483361  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:50.484136  8743 tablet_replica.cc:333] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.485042  8743 raft_consensus.cc:2243] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.485589  8743 raft_consensus.cc:2272] T e744cd5af851467ab5041b62f4f5abb0 P 09d06605a7724754b17ada9579d17c7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.488523  8743 tablet_replica.cc:333] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.489429  8743 raft_consensus.cc:2243] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.489969  8743 raft_consensus.cc:2272] T 48857b99fb4f4b489ad2a4fca48e4013 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.492316  8743 tablet_replica.cc:333] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.492923  8743 raft_consensus.cc:2243] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.493763  8743 raft_consensus.cc:2272] T 0c17bf5ce614455191209f3a51483c53 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.496246  8743 tablet_replica.cc:333] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.496800  8743 raft_consensus.cc:2243] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.497578  8743 raft_consensus.cc:2272] T bfd40c9cf4d94976b3b4d908e5e3e258 P 09d06605a7724754b17ada9579d17c7b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.500950  8743 tablet_replica.cc:333] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.501554  8743 raft_consensus.cc:2243] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.501996  8743 raft_consensus.cc:2272] T 17f39d2dc2f34a3abd7b7a76f767c99f P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.504619  8743 tablet_replica.cc:333] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.505190  8743 raft_consensus.cc:2243] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:50.505622  8743 raft_consensus.cc:2272] T bc2fdb33ad7b46c39013e6df43d80d28 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.507782  8743 tablet_replica.cc:333] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.508464  8743 raft_consensus.cc:2243] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.509477  8743 raft_consensus.cc:2272] T 25a9b271f2894c91b543e458a845cc89 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.512336  8743 tablet_replica.cc:333] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.512880  8743 raft_consensus.cc:2243] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:29:50.513012 23165 consensus_peers.cc:597] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b -> Peer bfa64f51bcd64967ad3ada1069c46b56 (127.8.137.194:43125): Couldn't send request to peer bfa64f51bcd64967ad3ada1069c46b56. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:43125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:29:50.514006  8743 raft_consensus.cc:2272] T 695fef5616834fc595570c13e142e255 P 09d06605a7724754b17ada9579d17c7b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.516328  8743 tablet_replica.cc:333] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.516806  8743 raft_consensus.cc:2243] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.517554  8743 raft_consensus.cc:2272] T eb14294be772495c913f2227f7ab9f50 P 09d06605a7724754b17ada9579d17c7b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.520074  8743 tablet_replica.cc:333] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b: stopping tablet replica
I20260709 12:29:50.520582  8743 raft_consensus.cc:2243] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.521351  8743 raft_consensus.cc:2272] T 0b4414e54d6c465084d581f259215562 P 09d06605a7724754b17ada9579d17c7b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.546181  8743 tablet_server.cc:196] TabletServer@127.8.137.195:0 shutdown complete.
I20260709 12:29:50.569885  8743 master.cc:562] Master@127.8.137.254:33643 shutting down...
I20260709 12:29:50.597630  8743 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:50.598205  8743 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:50.598551  8743 tablet_replica.cc:333] T 00000000000000000000000000000000 P b38031e4847f4e9da457e2a8be0cdd0b: stopping tablet replica
I20260709 12:29:50.619071  8743 master.cc:584] Master@127.8.137.254:33643 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12915 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:29:50.668183  8743 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.137.254:36347
I20260709 12:29:50.669389  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:50.675549 23516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:50.676046 23517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:50.676661 23519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:50.677016  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:50.677938  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:50.678161  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:50.678314  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600190678297 us; error 0 us; skew 500 ppm
I20260709 12:29:50.678938  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:50.681284  8743 webserver.cc:533] Webserver started at http://127.8.137.254:44561/ using document root <none> and password file <none>
I20260709 12:29:50.681797  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:50.682009  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:50.682289  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:50.683481  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/master-0-root/instance:
uuid: "abe9516897204b0bbec75db5f5b9fcdd"
format_stamp: "Formatted at 2026-07-09 12:29:50 on dist-test-slave-kpwv"
I20260709 12:29:50.688006  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 12:29:50.691668 23524 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:50.692587  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:29:50.692847  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/master-0-root
uuid: "abe9516897204b0bbec75db5f5b9fcdd"
format_stamp: "Formatted at 2026-07-09 12:29:50 on dist-test-slave-kpwv"
I20260709 12:29:50.693097  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:50.717592  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:50.718803  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:50.757383  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.254:36347
I20260709 12:29:50.757505 23575 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.254:36347 every 8 connection(s)
I20260709 12:29:50.761454 23576 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:50.773106 23576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd: Bootstrap starting.
I20260709 12:29:50.778144 23576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:50.782581 23576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd: No bootstrap required, opened a new log
I20260709 12:29:50.784839 23576 raft_consensus.cc:359] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe9516897204b0bbec75db5f5b9fcdd" member_type: VOTER }
I20260709 12:29:50.785246 23576 raft_consensus.cc:385] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:50.785465 23576 raft_consensus.cc:740] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abe9516897204b0bbec75db5f5b9fcdd, State: Initialized, Role: FOLLOWER
I20260709 12:29:50.786078 23576 consensus_queue.cc:260] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe9516897204b0bbec75db5f5b9fcdd" member_type: VOTER }
I20260709 12:29:50.786525 23576 raft_consensus.cc:399] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:50.786830 23576 raft_consensus.cc:493] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:50.787132 23576 raft_consensus.cc:3060] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:50.791950 23576 raft_consensus.cc:515] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe9516897204b0bbec75db5f5b9fcdd" member_type: VOTER }
I20260709 12:29:50.792516 23576 leader_election.cc:304] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abe9516897204b0bbec75db5f5b9fcdd; no voters: 
I20260709 12:29:50.793783 23576 leader_election.cc:290] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:50.794421 23579 raft_consensus.cc:2804] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:50.812822 23579 raft_consensus.cc:697] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 1 LEADER]: Becoming Leader. State: Replica: abe9516897204b0bbec75db5f5b9fcdd, State: Running, Role: LEADER
I20260709 12:29:50.814373 23579 consensus_queue.cc:237] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe9516897204b0bbec75db5f5b9fcdd" member_type: VOTER }
I20260709 12:29:50.815500 23576 sys_catalog.cc:565] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:50.821588 23581 sys_catalog.cc:455] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abe9516897204b0bbec75db5f5b9fcdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe9516897204b0bbec75db5f5b9fcdd" member_type: VOTER } }
I20260709 12:29:50.822844 23581 sys_catalog.cc:458] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:50.822849 23582 sys_catalog.cc:455] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader abe9516897204b0bbec75db5f5b9fcdd. Latest consensus state: current_term: 1 leader_uuid: "abe9516897204b0bbec75db5f5b9fcdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abe9516897204b0bbec75db5f5b9fcdd" member_type: VOTER } }
I20260709 12:29:50.825240 23582 sys_catalog.cc:458] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:50.828186 23587 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:50.833516 23587 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:50.837173  8743 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:50.843572 23587 catalog_manager.cc:1383] Generated new cluster ID: eba7c38e1699474f856014245a652628
I20260709 12:29:50.843892 23587 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:50.860047 23587 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:50.861428 23587 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:50.877604 23587 catalog_manager.cc:6070] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd: Generated new TSK 0
I20260709 12:29:50.878306 23587 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:50.905114  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:50.913022 23598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:50.914469 23599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:50.916255 23601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:50.916352  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:50.917645  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:50.917985  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:50.918167  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600190918151 us; error 0 us; skew 500 ppm
I20260709 12:29:50.918843  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:50.921605  8743 webserver.cc:533] Webserver started at http://127.8.137.193:40261/ using document root <none> and password file <none>
I20260709 12:29:50.922178  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:50.922405  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:50.922780  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:50.924330  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-0-root/instance:
uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
format_stamp: "Formatted at 2026-07-09 12:29:50 on dist-test-slave-kpwv"
I20260709 12:29:50.929402  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:29:50.933476 23606 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:50.934433  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:29:50.934788  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
format_stamp: "Formatted at 2026-07-09 12:29:50 on dist-test-slave-kpwv"
I20260709 12:29:50.935053  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:50.946703  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:50.948031  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:50.949548  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:50.952241  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:50.952431  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:50.952745  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:50.952919  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:50.999257  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:44379
I20260709 12:29:50.999656 23668 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:44379 every 8 connection(s)
I20260709 12:29:51.005712  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:51.013383 23673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:51.014653 23674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:51.017724 23676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:51.018227  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:51.019718  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:51.020030  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:51.020238  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600191020217 us; error 0 us; skew 500 ppm
I20260709 12:29:51.020931  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:51.023986  8743 webserver.cc:533] Webserver started at http://127.8.137.194:37407/ using document root <none> and password file <none>
I20260709 12:29:51.024549  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:51.024613 23669 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36347
I20260709 12:29:51.024740  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:51.025336 23669 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:51.025419  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:51.026298 23669 heartbeater.cc:507] Master 127.8.137.254:36347 requested a full tablet report, sending...
I20260709 12:29:51.026975  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-1-root/instance:
uuid: "595aa3169b074e69963aa7f6d5d98a74"
format_stamp: "Formatted at 2026-07-09 12:29:51 on dist-test-slave-kpwv"
I20260709 12:29:51.028988 23541 ts_manager.cc:194] Registered new tserver with Master: 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193:44379)
I20260709 12:29:51.031749 23541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44292
I20260709 12:29:51.033286  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:29:51.037021 23681 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:51.037741  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 12:29:51.038056  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-1-root
uuid: "595aa3169b074e69963aa7f6d5d98a74"
format_stamp: "Formatted at 2026-07-09 12:29:51 on dist-test-slave-kpwv"
I20260709 12:29:51.038350  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:51.051791  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:51.053057  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:51.054562  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:51.056841  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:51.057065  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:51.057327  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:51.057508  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:51.102051  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.194:46119
I20260709 12:29:51.102138 23743 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.194:46119 every 8 connection(s)
I20260709 12:29:51.107503  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:51.119297 23748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:51.120280 23747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:51.124538 23750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:51.124976 23744 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36347
I20260709 12:29:51.125379 23744 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:51.126188 23744 heartbeater.cc:507] Master 127.8.137.254:36347 requested a full tablet report, sending...
I20260709 12:29:51.126705  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:51.127827  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:51.128072  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:51.128274  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600191128250 us; error 0 us; skew 500 ppm
I20260709 12:29:51.128334 23541 ts_manager.cc:194] Registered new tserver with Master: 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194:46119)
I20260709 12:29:51.129135  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:51.129940 23541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44300
I20260709 12:29:51.132225  8743 webserver.cc:533] Webserver started at http://127.8.137.195:33319/ using document root <none> and password file <none>
I20260709 12:29:51.132814  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:51.133033  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:51.133299  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:51.134526  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-2-root/instance:
uuid: "e8391347f1494b6bb0ded412ace196fb"
format_stamp: "Formatted at 2026-07-09 12:29:51 on dist-test-slave-kpwv"
I20260709 12:29:51.139266  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:29:51.143112 23755 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:51.144032  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:29:51.144306  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-2-root
uuid: "e8391347f1494b6bb0ded412ace196fb"
format_stamp: "Formatted at 2026-07-09 12:29:51 on dist-test-slave-kpwv"
I20260709 12:29:51.144572  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783599720900375-8743-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:51.159720  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:51.160962  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:51.162377  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:51.165004  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:51.165194  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:51.165489  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:51.165644  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:51.216061  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.195:33517
I20260709 12:29:51.216436 23817 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.195:33517 every 8 connection(s)
I20260709 12:29:51.235517 23818 heartbeater.cc:344] Connected to a master server at 127.8.137.254:36347
I20260709 12:29:51.236137 23818 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:51.237077 23818 heartbeater.cc:507] Master 127.8.137.254:36347 requested a full tablet report, sending...
I20260709 12:29:51.239253 23541 ts_manager.cc:194] Registered new tserver with Master: e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:51.240036  8743 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018985911s
I20260709 12:29:51.240873 23541 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44312
I20260709 12:29:51.269681 23541 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44318:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:51.272060 23541 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:51.322391 23783 tablet_service.cc:1511] Processing CreateTablet for tablet 1afd29529acc43eb9517981ee2e6fcb5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=10cbda4268f849768f9917e8ba857834]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:51.324043 23783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1afd29529acc43eb9517981ee2e6fcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.336313 23782 tablet_service.cc:1511] Processing CreateTablet for tablet aa6f9b3ecafc49c89b5cd3e4afe44477 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=10cbda4268f849768f9917e8ba857834]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:51.338243 23782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6f9b3ecafc49c89b5cd3e4afe44477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.339793 23709 tablet_service.cc:1511] Processing CreateTablet for tablet 1afd29529acc43eb9517981ee2e6fcb5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=10cbda4268f849768f9917e8ba857834]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:51.340255 23633 tablet_service.cc:1511] Processing CreateTablet for tablet aa6f9b3ecafc49c89b5cd3e4afe44477 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=10cbda4268f849768f9917e8ba857834]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:51.341411 23709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1afd29529acc43eb9517981ee2e6fcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.341781 23633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6f9b3ecafc49c89b5cd3e4afe44477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.347905 23708 tablet_service.cc:1511] Processing CreateTablet for tablet aa6f9b3ecafc49c89b5cd3e4afe44477 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=10cbda4268f849768f9917e8ba857834]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:51.349541 23708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6f9b3ecafc49c89b5cd3e4afe44477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.339859 23634 tablet_service.cc:1511] Processing CreateTablet for tablet 1afd29529acc43eb9517981ee2e6fcb5 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=10cbda4268f849768f9917e8ba857834]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:51.358104 23634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1afd29529acc43eb9517981ee2e6fcb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.372646 23833 tablet_bootstrap.cc:492] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb: Bootstrap starting.
I20260709 12:29:51.374333 23835 tablet_bootstrap.cc:492] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Bootstrap starting.
I20260709 12:29:51.375797 23834 tablet_bootstrap.cc:492] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74: Bootstrap starting.
I20260709 12:29:51.382167 23835 tablet_bootstrap.cc:654] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.383242 23834 tablet_bootstrap.cc:654] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.388320 23835 tablet_bootstrap.cc:492] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e: No bootstrap required, opened a new log
I20260709 12:29:51.388867 23835 ts_tablet_manager.cc:1403] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 12:29:51.388880 23833 tablet_bootstrap.cc:654] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.389328 23834 tablet_bootstrap.cc:492] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74: No bootstrap required, opened a new log
I20260709 12:29:51.389873 23834 ts_tablet_manager.cc:1403] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74: Time spent bootstrapping tablet: real 0.014s	user 0.012s	sys 0.000s
I20260709 12:29:51.392259 23835 raft_consensus.cc:359] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.393214 23835 raft_consensus.cc:385] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.393666 23835 raft_consensus.cc:740] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.393296 23834 raft_consensus.cc:359] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.394204 23834 raft_consensus.cc:385] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.394608 23834 raft_consensus.cc:740] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.394580 23835 consensus_queue.cc:260] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.396129 23833 tablet_bootstrap.cc:492] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb: No bootstrap required, opened a new log
I20260709 12:29:51.395715 23834 consensus_queue.cc:260] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.396651 23833 ts_tablet_manager.cc:1403] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb: Time spent bootstrapping tablet: real 0.024s	user 0.004s	sys 0.009s
I20260709 12:29:51.399775 23669 heartbeater.cc:499] Master 127.8.137.254:36347 was elected leader, sending a full tablet report...
I20260709 12:29:51.399494 23833 raft_consensus.cc:359] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.400321 23833 raft_consensus.cc:385] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.400627 23833 raft_consensus.cc:740] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8391347f1494b6bb0ded412ace196fb, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.400707 23835 ts_tablet_manager.cc:1434] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.006s
I20260709 12:29:51.401446 23833 consensus_queue.cc:260] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.402324 23835 tablet_bootstrap.cc:492] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Bootstrap starting.
I20260709 12:29:51.404505 23744 heartbeater.cc:499] Master 127.8.137.254:36347 was elected leader, sending a full tablet report...
I20260709 12:29:51.405668 23834 ts_tablet_manager.cc:1434] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.012s
I20260709 12:29:51.406834 23834 tablet_bootstrap.cc:492] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74: Bootstrap starting.
I20260709 12:29:51.415339 23834 tablet_bootstrap.cc:654] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.425997 23835 tablet_bootstrap.cc:654] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.427889 23834 tablet_bootstrap.cc:492] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74: No bootstrap required, opened a new log
I20260709 12:29:51.428530 23834 ts_tablet_manager.cc:1403] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.011s
I20260709 12:29:51.435985 23818 heartbeater.cc:499] Master 127.8.137.254:36347 was elected leader, sending a full tablet report...
I20260709 12:29:51.436486 23833 ts_tablet_manager.cc:1434] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb: Time spent starting tablet: real 0.040s	user 0.024s	sys 0.016s
I20260709 12:29:51.436775 23834 raft_consensus.cc:359] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.437659 23834 raft_consensus.cc:385] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.437830 23833 tablet_bootstrap.cc:492] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb: Bootstrap starting.
I20260709 12:29:51.438016 23834 raft_consensus.cc:740] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.438993 23834 consensus_queue.cc:260] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.441883 23834 ts_tablet_manager.cc:1434] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.003s
I20260709 12:29:51.445921 23833 tablet_bootstrap.cc:654] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.451817 23835 tablet_bootstrap.cc:492] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e: No bootstrap required, opened a new log
I20260709 12:29:51.452426 23835 ts_tablet_manager.cc:1403] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Time spent bootstrapping tablet: real 0.050s	user 0.019s	sys 0.007s
I20260709 12:29:51.452876 23833 tablet_bootstrap.cc:492] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb: No bootstrap required, opened a new log
I20260709 12:29:51.453449 23833 ts_tablet_manager.cc:1403] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb: Time spent bootstrapping tablet: real 0.016s	user 0.007s	sys 0.006s
I20260709 12:29:51.455597 23835 raft_consensus.cc:359] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.456413 23835 raft_consensus.cc:385] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.456856 23835 raft_consensus.cc:740] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.456439 23833 raft_consensus.cc:359] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.457320 23833 raft_consensus.cc:385] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.457638 23833 raft_consensus.cc:740] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8391347f1494b6bb0ded412ace196fb, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.457716 23835 consensus_queue.cc:260] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.458443 23833 consensus_queue.cc:260] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.460304 23835 ts_tablet_manager.cc:1434] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:51.460704 23833 ts_tablet_manager.cc:1434] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:51.485361 23840 raft_consensus.cc:493] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:51.485998 23840 raft_consensus.cc:515] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.488431 23840 leader_election.cc:290] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517), 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193:44379)
I20260709 12:29:51.497269 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1afd29529acc43eb9517981ee2e6fcb5" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8391347f1494b6bb0ded412ace196fb" is_pre_election: true
I20260709 12:29:51.498065 23793 raft_consensus.cc:2468] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 0.
I20260709 12:29:51.499096 23644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1afd29529acc43eb9517981ee2e6fcb5" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" is_pre_election: true
I20260709 12:29:51.499460 23683 leader_election.cc:304] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, e8391347f1494b6bb0ded412ace196fb; no voters: 
I20260709 12:29:51.499907 23644 raft_consensus.cc:2468] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 0.
I20260709 12:29:51.500386 23840 raft_consensus.cc:2804] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:51.500831 23840 raft_consensus.cc:493] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:51.501243 23840 raft_consensus.cc:3060] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.508116 23840 raft_consensus.cc:515] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.509795 23840 leader_election.cc:290] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 1 election: Requested vote from peers e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517), 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193:44379)
I20260709 12:29:51.510784 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1afd29529acc43eb9517981ee2e6fcb5" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8391347f1494b6bb0ded412ace196fb"
I20260709 12:29:51.510856 23644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1afd29529acc43eb9517981ee2e6fcb5" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
I20260709 12:29:51.511359 23793 raft_consensus.cc:3060] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.511518 23644 raft_consensus.cc:3060] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.512126 23839 raft_consensus.cc:493] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:51.512681 23839 raft_consensus.cc:515] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.517592 23839 leader_election.cc:290] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194:46119), e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:51.519867 23793 raft_consensus.cc:2468] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 1.
I20260709 12:29:51.521196 23683 leader_election.cc:304] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, e8391347f1494b6bb0ded412ace196fb; no voters: 
I20260709 12:29:51.522148 23840 raft_consensus.cc:2804] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:51.523509 23644 raft_consensus.cc:2468] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 1.
I20260709 12:29:51.526846 23840 raft_consensus.cc:697] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Becoming Leader. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Running, Role: LEADER
I20260709 12:29:51.527907 23840 consensus_queue.cc:237] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:51.529537 23719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6f9b3ecafc49c89b5cd3e4afe44477" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "595aa3169b074e69963aa7f6d5d98a74" is_pre_election: true
I20260709 12:29:51.530249 23719 raft_consensus.cc:2468] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 0.
I20260709 12:29:51.531229 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6f9b3ecafc49c89b5cd3e4afe44477" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8391347f1494b6bb0ded412ace196fb" is_pre_election: true
I20260709 12:29:51.531502 23607 leader_election.cc:304] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, 73721d3a5c7d4ecd8bc82b1012b83f1e; no voters: 
I20260709 12:29:51.531968 23793 raft_consensus.cc:2468] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 0.
I20260709 12:29:51.532379 23839 raft_consensus.cc:2804] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:51.532871 23839 raft_consensus.cc:493] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:51.533207 23839 raft_consensus.cc:3060] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.539453 23537 catalog_manager.cc:5697] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "595aa3169b074e69963aa7f6d5d98a74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:51.542305 23839 raft_consensus.cc:515] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.544225 23839 leader_election.cc:290] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 1 election: Requested vote from peers 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194:46119), e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:51.545379 23719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6f9b3ecafc49c89b5cd3e4afe44477" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "595aa3169b074e69963aa7f6d5d98a74"
I20260709 12:29:51.545608 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6f9b3ecafc49c89b5cd3e4afe44477" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8391347f1494b6bb0ded412ace196fb"
I20260709 12:29:51.545986 23719 raft_consensus.cc:3060] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.546231 23793 raft_consensus.cc:3060] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.555984 23793 raft_consensus.cc:2468] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 1.
I20260709 12:29:51.556281 23719 raft_consensus.cc:2468] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 1.
I20260709 12:29:51.557394 23608 leader_election.cc:304] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73721d3a5c7d4ecd8bc82b1012b83f1e, e8391347f1494b6bb0ded412ace196fb; no voters: 
I20260709 12:29:51.558179 23839 raft_consensus.cc:2804] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:51.559192 23839 raft_consensus.cc:697] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 LEADER]: Becoming Leader. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Running, Role: LEADER
I20260709 12:29:51.559911 23839 consensus_queue.cc:237] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.567305 23537 catalog_manager.cc:5697] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193). New cstate: current_term: 1 leader_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:51.612179 23537 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44326:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:51.614192 23537 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:51.639392 23634 tablet_service.cc:1511] Processing CreateTablet for tablet 6eab65d6d801452d883ae4b9efb6cd78 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=da7d069b0f214d70a483446fc7140c44]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:51.640733 23634 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eab65d6d801452d883ae4b9efb6cd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.640588 23708 tablet_service.cc:1511] Processing CreateTablet for tablet 6eab65d6d801452d883ae4b9efb6cd78 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=da7d069b0f214d70a483446fc7140c44]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:51.642055 23708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eab65d6d801452d883ae4b9efb6cd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.642241 23782 tablet_service.cc:1511] Processing CreateTablet for tablet 6eab65d6d801452d883ae4b9efb6cd78 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=da7d069b0f214d70a483446fc7140c44]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:29:51.643528 23782 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eab65d6d801452d883ae4b9efb6cd78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.645327 23633 tablet_service.cc:1511] Processing CreateTablet for tablet 0a8384c723ce440e9d7e070395edc943 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=da7d069b0f214d70a483446fc7140c44]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:51.646733 23633 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a8384c723ce440e9d7e070395edc943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.648947 23783 tablet_service.cc:1511] Processing CreateTablet for tablet 0a8384c723ce440e9d7e070395edc943 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=da7d069b0f214d70a483446fc7140c44]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:51.650246 23783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a8384c723ce440e9d7e070395edc943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.654695 23709 tablet_service.cc:1511] Processing CreateTablet for tablet 0a8384c723ce440e9d7e070395edc943 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=da7d069b0f214d70a483446fc7140c44]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:29:51.656175 23709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0a8384c723ce440e9d7e070395edc943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:51.659416 23835 tablet_bootstrap.cc:492] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Bootstrap starting.
I20260709 12:29:51.663677 23833 tablet_bootstrap.cc:492] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb: Bootstrap starting.
I20260709 12:29:51.670678 23833 tablet_bootstrap.cc:654] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.674582 23835 tablet_bootstrap.cc:654] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.681347 23834 tablet_bootstrap.cc:492] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74: Bootstrap starting.
I20260709 12:29:51.683240 23833 tablet_bootstrap.cc:492] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb: No bootstrap required, opened a new log
I20260709 12:29:51.683867 23833 ts_tablet_manager.cc:1403] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.001s
I20260709 12:29:51.687136 23833 raft_consensus.cc:359] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.687717 23833 raft_consensus.cc:385] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.687978 23833 raft_consensus.cc:740] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8391347f1494b6bb0ded412ace196fb, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.688371 23835 tablet_bootstrap.cc:492] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e: No bootstrap required, opened a new log
I20260709 12:29:51.688742 23833 consensus_queue.cc:260] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.688966 23835 ts_tablet_manager.cc:1403] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.004s
I20260709 12:29:51.689006 23834 tablet_bootstrap.cc:654] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.690989 23833 ts_tablet_manager.cc:1434] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:29:51.692109 23833 tablet_bootstrap.cc:492] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb: Bootstrap starting.
I20260709 12:29:51.694118 23835 raft_consensus.cc:359] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.694957 23835 raft_consensus.cc:385] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.695290 23835 raft_consensus.cc:740] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.696029 23835 consensus_queue.cc:260] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.698318 23833 tablet_bootstrap.cc:654] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.702070 23835 ts_tablet_manager.cc:1434] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:29:51.703282 23835 tablet_bootstrap.cc:492] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Bootstrap starting.
I20260709 12:29:51.710669 23835 tablet_bootstrap.cc:654] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.710949 23834 tablet_bootstrap.cc:492] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74: No bootstrap required, opened a new log
I20260709 12:29:51.711570 23834 ts_tablet_manager.cc:1403] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.007s
I20260709 12:29:51.711534 23833 tablet_bootstrap.cc:492] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb: No bootstrap required, opened a new log
I20260709 12:29:51.714004 23833 ts_tablet_manager.cc:1403] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.004s
I20260709 12:29:51.715116 23834 raft_consensus.cc:359] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.715981 23834 raft_consensus.cc:385] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.716326 23834 raft_consensus.cc:740] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.717283 23833 raft_consensus.cc:359] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.717317 23834 consensus_queue.cc:260] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.718207 23833 raft_consensus.cc:385] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.718524 23833 raft_consensus.cc:740] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8391347f1494b6bb0ded412ace196fb, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.719465 23833 consensus_queue.cc:260] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.721365 23835 tablet_bootstrap.cc:492] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e: No bootstrap required, opened a new log
I20260709 12:29:51.721756 23833 ts_tablet_manager.cc:1434] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.002s
I20260709 12:29:51.722018 23835 ts_tablet_manager.cc:1403] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.007s
I20260709 12:29:51.724714 23834 ts_tablet_manager.cc:1434] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:29:51.725673 23835 raft_consensus.cc:359] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.726477 23834 tablet_bootstrap.cc:492] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74: Bootstrap starting.
I20260709 12:29:51.726732 23835 raft_consensus.cc:385] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.727177 23835 raft_consensus.cc:740] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.728231 23835 consensus_queue.cc:260] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.731693 23835 ts_tablet_manager.cc:1434] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.003s
I20260709 12:29:51.734362 23834 tablet_bootstrap.cc:654] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:51.741743 23834 tablet_bootstrap.cc:492] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74: No bootstrap required, opened a new log
I20260709 12:29:51.742355 23834 ts_tablet_manager.cc:1403] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.009s
I20260709 12:29:51.745146 23834 raft_consensus.cc:359] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.745772 23834 raft_consensus.cc:385] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:51.746026 23834 raft_consensus.cc:740] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Initialized, Role: FOLLOWER
I20260709 12:29:51.746747 23834 consensus_queue.cc:260] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.748390 23834 ts_tablet_manager.cc:1434] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:29:51.765445 23839 raft_consensus.cc:493] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:51.765995 23839 raft_consensus.cc:515] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.767731 23839 leader_election.cc:290] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194:46119), e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:51.768584 23719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eab65d6d801452d883ae4b9efb6cd78" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "595aa3169b074e69963aa7f6d5d98a74" is_pre_election: true
I20260709 12:29:51.768772 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eab65d6d801452d883ae4b9efb6cd78" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8391347f1494b6bb0ded412ace196fb" is_pre_election: true
I20260709 12:29:51.769342 23719 raft_consensus.cc:2468] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 0.
I20260709 12:29:51.769349 23793 raft_consensus.cc:2468] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 0.
I20260709 12:29:51.770617 23607 leader_election.cc:304] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, 73721d3a5c7d4ecd8bc82b1012b83f1e; no voters: 
I20260709 12:29:51.771281 23839 raft_consensus.cc:2804] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:51.771588 23839 raft_consensus.cc:493] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:51.771859 23839 raft_consensus.cc:3060] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.776901 23839 raft_consensus.cc:515] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.778484 23839 leader_election.cc:290] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 1 election: Requested vote from peers 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194:46119), e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:51.779386 23719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eab65d6d801452d883ae4b9efb6cd78" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "595aa3169b074e69963aa7f6d5d98a74"
I20260709 12:29:51.779523 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eab65d6d801452d883ae4b9efb6cd78" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8391347f1494b6bb0ded412ace196fb"
I20260709 12:29:51.780090 23719 raft_consensus.cc:3060] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.780165 23793 raft_consensus.cc:3060] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.785743 23719 raft_consensus.cc:2468] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 1.
I20260709 12:29:51.785784 23793 raft_consensus.cc:2468] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 1.
I20260709 12:29:51.786890 23607 leader_election.cc:304] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, 73721d3a5c7d4ecd8bc82b1012b83f1e; no voters: 
I20260709 12:29:51.787582 23839 raft_consensus.cc:2804] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:51.788060 23839 raft_consensus.cc:697] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 LEADER]: Becoming Leader. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Running, Role: LEADER
I20260709 12:29:51.788697 23839 consensus_queue.cc:237] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.796108 23540 catalog_manager.cc:5697] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e reported cstate change: term changed from 0 to 1, leader changed from <none> to 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193). New cstate: current_term: 1 leader_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } health_report { overall_health: UNKNOWN } } }
W20260709 12:29:51.831909 23590 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:51.873359 23840 raft_consensus.cc:493] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:51.873975 23840 raft_consensus.cc:515] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.875953 23840 leader_election.cc:290] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193:44379), e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:51.876690 23644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" is_pre_election: true
I20260709 12:29:51.877295 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8391347f1494b6bb0ded412ace196fb" is_pre_election: true
I20260709 12:29:51.877425 23644 raft_consensus.cc:2468] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 0.
I20260709 12:29:51.877897 23793 raft_consensus.cc:2468] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 0.
I20260709 12:29:51.878634 23685 leader_election.cc:304] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, 73721d3a5c7d4ecd8bc82b1012b83f1e; no voters: 
I20260709 12:29:51.879447 23840 raft_consensus.cc:2804] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:51.879863 23840 raft_consensus.cc:493] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:51.880203 23840 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.885725 23840 raft_consensus.cc:515] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.887774 23840 leader_election.cc:290] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 1 election: Requested vote from peers 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193:44379), e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:51.888638 23644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
I20260709 12:29:51.888947 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8391347f1494b6bb0ded412ace196fb"
I20260709 12:29:51.889263 23644 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.889669 23793 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:51.895385 23644 raft_consensus.cc:2468] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 1.
I20260709 12:29:51.895401 23793 raft_consensus.cc:2468] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 1.
I20260709 12:29:51.896639 23685 leader_election.cc:304] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, 73721d3a5c7d4ecd8bc82b1012b83f1e; no voters: 
I20260709 12:29:51.897363 23840 raft_consensus.cc:2804] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:51.897835 23840 raft_consensus.cc:697] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Becoming Leader. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Running, Role: LEADER
I20260709 12:29:51.898644 23840 consensus_queue.cc:237] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:51.905998 23540 catalog_manager.cc:5697] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 reported cstate change: term changed from 0 to 1, leader changed from <none> to 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "595aa3169b074e69963aa7f6d5d98a74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:51.938843 23719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1afd29529acc43eb9517981ee2e6fcb5"
dest_uuid: "595aa3169b074e69963aa7f6d5d98a74"
mode: GRACEFUL
new_leader_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
 from {username='slave'} at 127.0.0.1:47800
I20260709 12:29:51.939438 23719 raft_consensus.cc:606] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Received request to transfer leadership to TS 73721d3a5c7d4ecd8bc82b1012b83f1e
I20260709 12:29:51.953294 23644 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa6f9b3ecafc49c89b5cd3e4afe44477"
dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
mode: GRACEFUL
new_leader_uuid: "595aa3169b074e69963aa7f6d5d98a74"
 from {username='slave'} at 127.0.0.1:60016
I20260709 12:29:51.953784 23644 raft_consensus.cc:606] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 LEADER]: Received request to transfer leadership to TS 595aa3169b074e69963aa7f6d5d98a74
I20260709 12:29:51.962081 23719 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943"
dest_uuid: "595aa3169b074e69963aa7f6d5d98a74"
mode: GRACEFUL
new_leader_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
 from {username='slave'} at 127.0.0.1:47800
I20260709 12:29:51.962582 23719 raft_consensus.cc:606] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Received request to transfer leadership to TS 73721d3a5c7d4ecd8bc82b1012b83f1e
I20260709 12:29:51.967726 23839 consensus_queue.cc:1048] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:51.985925 23849 consensus_queue.cc:1048] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:51.996614 23839 raft_consensus.cc:993] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e: : Instructing follower 595aa3169b074e69963aa7f6d5d98a74 to start an election
I20260709 12:29:51.997167 23849 raft_consensus.cc:1081] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 LEADER]: Signalling peer 595aa3169b074e69963aa7f6d5d98a74 to start an election
I20260709 12:29:51.999325 23719 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa6f9b3ecafc49c89b5cd3e4afe44477"
dest_uuid: "595aa3169b074e69963aa7f6d5d98a74"
 from {username='slave'} at 127.0.0.1:47784
I20260709 12:29:51.999982 23719 raft_consensus.cc:493] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:52.000373 23719 raft_consensus.cc:3060] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.003556 23840 consensus_queue.cc:1048] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:52.008174 23719 raft_consensus.cc:515] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.010727 23719 leader_election.cc:290] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 2 election: Requested vote from peers 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193:44379), e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:52.011533 23644 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6f9b3ecafc49c89b5cd3e4afe44477" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
I20260709 12:29:52.012300 23644 raft_consensus.cc:3055] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:52.012670 23644 raft_consensus.cc:740] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Running, Role: LEADER
I20260709 12:29:52.012912 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6f9b3ecafc49c89b5cd3e4afe44477" candidate_uuid: "595aa3169b074e69963aa7f6d5d98a74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8391347f1494b6bb0ded412ace196fb"
I20260709 12:29:52.013648 23793 raft_consensus.cc:3060] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.013644 23644 consensus_queue.cc:260] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.015293 23644 raft_consensus.cc:3060] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.021824 23840 consensus_queue.cc:1048] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:52.025897 23793 raft_consensus.cc:2468] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 2.
I20260709 12:29:52.027279 23683 leader_election.cc:304] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, e8391347f1494b6bb0ded412ace196fb; no voters: 
I20260709 12:29:52.028147 23840 raft_consensus.cc:2804] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:52.028692 23840 raft_consensus.cc:697] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 LEADER]: Becoming Leader. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Running, Role: LEADER
I20260709 12:29:52.029614 23840 consensus_queue.cc:237] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.032497 23644 raft_consensus.cc:2468] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595aa3169b074e69963aa7f6d5d98a74 in term 2.
I20260709 12:29:52.042361 23538 catalog_manager.cc:5697] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 reported cstate change: term changed from 1 to 2, leader changed from 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193) to 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "595aa3169b074e69963aa7f6d5d98a74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:52.048195 23871 raft_consensus.cc:993] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74: : Instructing follower 73721d3a5c7d4ecd8bc82b1012b83f1e to start an election
I20260709 12:29:52.048668 23871 raft_consensus.cc:1081] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Signalling peer 73721d3a5c7d4ecd8bc82b1012b83f1e to start an election
I20260709 12:29:52.050535 23643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1afd29529acc43eb9517981ee2e6fcb5"
dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
 from {username='slave'} at 127.0.0.1:60002
I20260709 12:29:52.051170 23643 raft_consensus.cc:493] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:52.051542 23643 raft_consensus.cc:3060] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.059176 23643 raft_consensus.cc:515] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:52.062207 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1afd29529acc43eb9517981ee2e6fcb5" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8391347f1494b6bb0ded412ace196fb"
I20260709 12:29:52.062697 23719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1afd29529acc43eb9517981ee2e6fcb5" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "595aa3169b074e69963aa7f6d5d98a74"
I20260709 12:29:52.062896 23793 raft_consensus.cc:3060] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.063400 23719 raft_consensus.cc:3055] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:52.063763 23719 raft_consensus.cc:740] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Running, Role: LEADER
I20260709 12:29:52.064667 23719 consensus_queue.cc:260] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:52.065838 23719 raft_consensus.cc:3060] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.066852 23643 leader_election.cc:290] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 2 election: Requested vote from peers e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517), 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194:46119)
I20260709 12:29:52.070724 23793 raft_consensus.cc:2468] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 2.
I20260709 12:29:52.071965 23608 leader_election.cc:304] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73721d3a5c7d4ecd8bc82b1012b83f1e, e8391347f1494b6bb0ded412ace196fb; no voters: 
I20260709 12:29:52.072820 23849 raft_consensus.cc:2804] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:52.072816 23719 raft_consensus.cc:2468] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 2.
I20260709 12:29:52.073591 23849 raft_consensus.cc:697] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 LEADER]: Becoming Leader. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Running, Role: LEADER
I20260709 12:29:52.074486 23849 consensus_queue.cc:237] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } }
I20260709 12:29:52.081362 23537 catalog_manager.cc:5697] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e reported cstate change: term changed from 1 to 2, leader changed from 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194) to 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193). New cstate: current_term: 2 leader_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:52.227785 23849 consensus_queue.cc:1048] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:52.238112 23839 consensus_queue.cc:1048] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:52.366809 23871 consensus_queue.cc:1048] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:52.377454 23840 consensus_queue.cc:1048] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:52.384743 23871 raft_consensus.cc:993] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74: : Instructing follower 73721d3a5c7d4ecd8bc82b1012b83f1e to start an election
I20260709 12:29:52.385233 23840 raft_consensus.cc:1081] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Signalling peer 73721d3a5c7d4ecd8bc82b1012b83f1e to start an election
I20260709 12:29:52.386749 23643 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943"
dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
 from {username='slave'} at 127.0.0.1:60002
I20260709 12:29:52.387452 23643 raft_consensus.cc:493] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:52.387835 23643 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.393435 23643 raft_consensus.cc:515] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.395855 23643 leader_election.cc:290] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 2 election: Requested vote from peers 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194:46119), e8391347f1494b6bb0ded412ace196fb (127.8.137.195:33517)
I20260709 12:29:52.396235 23719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "595aa3169b074e69963aa7f6d5d98a74"
I20260709 12:29:52.396406 23793 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943" candidate_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e8391347f1494b6bb0ded412ace196fb"
I20260709 12:29:52.397130 23719 raft_consensus.cc:3055] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:52.397241 23793 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.397539 23719 raft_consensus.cc:740] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 595aa3169b074e69963aa7f6d5d98a74, State: Running, Role: LEADER
I20260709 12:29:52.398463 23719 consensus_queue.cc:260] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.399544 23719 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:52.403868 23793 raft_consensus.cc:2468] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 2.
I20260709 12:29:52.405234 23608 leader_election.cc:304] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73721d3a5c7d4ecd8bc82b1012b83f1e, e8391347f1494b6bb0ded412ace196fb; no voters: 
I20260709 12:29:52.405577 23719 raft_consensus.cc:2468] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73721d3a5c7d4ecd8bc82b1012b83f1e in term 2.
I20260709 12:29:52.406085 23839 raft_consensus.cc:2804] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:52.406665 23839 raft_consensus.cc:697] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 LEADER]: Becoming Leader. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Running, Role: LEADER
I20260709 12:29:52.407512 23839 consensus_queue.cc:237] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.414084 23540 catalog_manager.cc:5697] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e reported cstate change: term changed from 1 to 2, leader changed from 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194) to 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193). New cstate: current_term: 2 leader_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:52.421265 23643 raft_consensus.cc:1275] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Refusing update from remote peer 595aa3169b074e69963aa7f6d5d98a74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:52.422778 23840 consensus_queue.cc:1048] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:52.431128 23793 raft_consensus.cc:1275] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Refusing update from remote peer 595aa3169b074e69963aa7f6d5d98a74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:52.433729 23840 consensus_queue.cc:1048] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:52.457324  8743 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:29:52.459110  8743 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:29:52.459538  8743 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:29:52.471264 23643 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943"
dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
mode: GRACEFUL
new_leader_uuid: "e8391347f1494b6bb0ded412ace196fb"
 from {username='slave'} at 127.0.0.1:60020
I20260709 12:29:52.471829 23643 raft_consensus.cc:606] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 LEADER]: Received request to transfer leadership to TS e8391347f1494b6bb0ded412ace196fb
I20260709 12:29:52.473003  8743 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:29:52.490969 23793 raft_consensus.cc:1275] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Refusing update from remote peer 73721d3a5c7d4ecd8bc82b1012b83f1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:52.492352 23839 consensus_queue.cc:1048] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:52.501941 23719 raft_consensus.cc:1275] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Refusing update from remote peer 73721d3a5c7d4ecd8bc82b1012b83f1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:52.504545 23875 consensus_queue.cc:1048] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:52.923022 23719 raft_consensus.cc:1275] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Refusing update from remote peer 73721d3a5c7d4ecd8bc82b1012b83f1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:52.924392 23881 consensus_queue.cc:1048] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:52.932407 23793 raft_consensus.cc:1275] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Refusing update from remote peer 73721d3a5c7d4ecd8bc82b1012b83f1e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:52.934123 23875 consensus_queue.cc:1048] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:52.940368 23875 raft_consensus.cc:993] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e: : Instructing follower e8391347f1494b6bb0ded412ace196fb to start an election
I20260709 12:29:52.941248 23881 raft_consensus.cc:1081] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 LEADER]: Signalling peer e8391347f1494b6bb0ded412ace196fb to start an election
I20260709 12:29:52.943248 23793 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943"
dest_uuid: "e8391347f1494b6bb0ded412ace196fb"
 from {username='slave'} at 127.0.0.1:34252
I20260709 12:29:52.943871 23793 raft_consensus.cc:493] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:52.944275 23793 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.950062 23793 raft_consensus.cc:515] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.952579 23793 leader_election.cc:290] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [CANDIDATE]: Term 3 election: Requested vote from peers 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193:44379), 595aa3169b074e69963aa7f6d5d98a74 (127.8.137.194:46119)
I20260709 12:29:52.965018 23643 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943" candidate_uuid: "e8391347f1494b6bb0ded412ace196fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e"
I20260709 12:29:52.965306 23719 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0a8384c723ce440e9d7e070395edc943" candidate_uuid: "e8391347f1494b6bb0ded412ace196fb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "595aa3169b074e69963aa7f6d5d98a74"
I20260709 12:29:52.965598 23643 raft_consensus.cc:3055] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:29:52.965966 23643 raft_consensus.cc:740] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 73721d3a5c7d4ecd8bc82b1012b83f1e, State: Running, Role: LEADER
I20260709 12:29:52.965982 23719 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.966846 23643 consensus_queue.cc:260] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.967922 23643 raft_consensus.cc:3060] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:29:52.971817 23719 raft_consensus.cc:2468] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8391347f1494b6bb0ded412ace196fb in term 3.
I20260709 12:29:52.973134 23756 leader_election.cc:304] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595aa3169b074e69963aa7f6d5d98a74, e8391347f1494b6bb0ded412ace196fb; no voters: 
I20260709 12:29:52.973913 23841 raft_consensus.cc:2804] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:29:52.973830 23643 raft_consensus.cc:2468] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8391347f1494b6bb0ded412ace196fb in term 3.
I20260709 12:29:52.975309 23841 raft_consensus.cc:697] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 3 LEADER]: Becoming Leader. State: Replica: e8391347f1494b6bb0ded412ace196fb, State: Running, Role: LEADER
I20260709 12:29:52.976161 23841 consensus_queue.cc:237] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } }
I20260709 12:29:52.982798 23540 catalog_manager.cc:5697] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb reported cstate change: term changed from 2 to 3, leader changed from 73721d3a5c7d4ecd8bc82b1012b83f1e (127.8.137.193) to e8391347f1494b6bb0ded412ace196fb (127.8.137.195). New cstate: current_term: 3 leader_uuid: "e8391347f1494b6bb0ded412ace196fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73721d3a5c7d4ecd8bc82b1012b83f1e" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 44379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "595aa3169b074e69963aa7f6d5d98a74" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8391347f1494b6bb0ded412ace196fb" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 33517 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:53.099298  8743 tablet_server.cc:179] TabletServer@127.8.137.193:0 shutting down...
I20260709 12:29:53.122466  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:53.123185  8743 tablet_replica.cc:333] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e: stopping tablet replica
I20260709 12:29:53.123751  8743 raft_consensus.cc:2243] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:53.124593  8743 raft_consensus.cc:2272] T 6eab65d6d801452d883ae4b9efb6cd78 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.126994  8743 tablet_replica.cc:333] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e: stopping tablet replica
I20260709 12:29:53.127584  8743 raft_consensus.cc:2243] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:53.128436  8743 raft_consensus.cc:2272] T 1afd29529acc43eb9517981ee2e6fcb5 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.131184  8743 tablet_replica.cc:333] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e: stopping tablet replica
I20260709 12:29:53.131762  8743 raft_consensus.cc:2243] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.132220  8743 raft_consensus.cc:2272] T 0a8384c723ce440e9d7e070395edc943 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.134802  8743 tablet_replica.cc:333] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e: stopping tablet replica
I20260709 12:29:53.135308  8743 raft_consensus.cc:2243] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.135761  8743 raft_consensus.cc:2272] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 73721d3a5c7d4ecd8bc82b1012b83f1e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.160079  8743 tablet_server.cc:196] TabletServer@127.8.137.193:0 shutdown complete.
I20260709 12:29:53.173270  8743 tablet_server.cc:179] TabletServer@127.8.137.194:0 shutting down...
I20260709 12:29:53.192178  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:53.192902  8743 tablet_replica.cc:333] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74: stopping tablet replica
I20260709 12:29:53.193526  8743 raft_consensus.cc:2243] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.194007  8743 raft_consensus.cc:2272] T 0a8384c723ce440e9d7e070395edc943 P 595aa3169b074e69963aa7f6d5d98a74 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.196301  8743 tablet_replica.cc:333] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74: stopping tablet replica
I20260709 12:29:53.196868  8743 raft_consensus.cc:2243] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:29:53.197654  8743 raft_consensus.cc:2272] T aa6f9b3ecafc49c89b5cd3e4afe44477 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.199954  8743 tablet_replica.cc:333] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74: stopping tablet replica
I20260709 12:29:53.200498  8743 raft_consensus.cc:2243] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.200932  8743 raft_consensus.cc:2272] T 6eab65d6d801452d883ae4b9efb6cd78 P 595aa3169b074e69963aa7f6d5d98a74 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.203262  8743 tablet_replica.cc:333] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74: stopping tablet replica
I20260709 12:29:53.203776  8743 raft_consensus.cc:2243] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.204234  8743 raft_consensus.cc:2272] T 1afd29529acc43eb9517981ee2e6fcb5 P 595aa3169b074e69963aa7f6d5d98a74 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.217804  8743 tablet_server.cc:196] TabletServer@127.8.137.194:0 shutdown complete.
I20260709 12:29:53.229878  8743 tablet_server.cc:179] TabletServer@127.8.137.195:0 shutting down...
I20260709 12:29:53.248669  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:29:53.249282  8743 tablet_replica.cc:333] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb: stopping tablet replica
I20260709 12:29:53.249788  8743 raft_consensus.cc:2243] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.250209  8743 raft_consensus.cc:2272] T 6eab65d6d801452d883ae4b9efb6cd78 P e8391347f1494b6bb0ded412ace196fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.252202  8743 tablet_replica.cc:333] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb: stopping tablet replica
I20260709 12:29:53.252692  8743 raft_consensus.cc:2243] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.253072  8743 raft_consensus.cc:2272] T 1afd29529acc43eb9517981ee2e6fcb5 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.255280  8743 tablet_replica.cc:333] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb: stopping tablet replica
I20260709 12:29:53.255743  8743 raft_consensus.cc:2243] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:29:53.256325  8743 pending_rounds.cc:70] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb: Trying to abort 1 pending ops.
I20260709 12:29:53.256475  8743 pending_rounds.cc:77] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb: Aborting op as it isn't in flight: id { term: 3 index: 3 } timestamp: 7305626390438129664 op_type: NO_OP noop_request { }
I20260709 12:29:53.256731  8743 raft_consensus.cc:2889] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 3 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:29:53.256960  8743 raft_consensus.cc:2272] T 0a8384c723ce440e9d7e070395edc943 P e8391347f1494b6bb0ded412ace196fb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.259166  8743 tablet_replica.cc:333] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb: stopping tablet replica
I20260709 12:29:53.259608  8743 raft_consensus.cc:2243] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:29:53.259986  8743 raft_consensus.cc:2272] T aa6f9b3ecafc49c89b5cd3e4afe44477 P e8391347f1494b6bb0ded412ace196fb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.283656  8743 tablet_server.cc:196] TabletServer@127.8.137.195:0 shutdown complete.
I20260709 12:29:53.296540  8743 master.cc:562] Master@127.8.137.254:36347 shutting down...
I20260709 12:29:53.315222  8743 raft_consensus.cc:2243] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:29:53.315738  8743 raft_consensus.cc:2272] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:29:53.316013  8743 tablet_replica.cc:333] T 00000000000000000000000000000000 P abe9516897204b0bbec75db5f5b9fcdd: stopping tablet replica
I20260709 12:29:53.336755  8743 master.cc:584] Master@127.8.137.254:36347 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2703 ms)
[----------] 6 tests from LeaderRebalancerTest (472307 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:29:53.371803  8743 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.137.254:33231
I20260709 12:29:53.372905  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:53.378113 23889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:53.378285 23890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:53.379700 23892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:53.380446  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:53.381256  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:53.381428  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:53.381619  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600193381587 us; error 0 us; skew 500 ppm
I20260709 12:29:53.382182  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:53.384434  8743 webserver.cc:533] Webserver started at http://127.8.137.254:43449/ using document root <none> and password file <none>
I20260709 12:29:53.384914  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:53.385147  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:53.385421  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:53.386484  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/master-0-root/instance:
uuid: "fc38cce3b3424ee58c6ab26ec63a51d1"
format_stamp: "Formatted at 2026-07-09 12:29:53 on dist-test-slave-kpwv"
I20260709 12:29:53.391399  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:29:53.395041 23897 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.395892  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 12:29:53.396296  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/master-0-root
uuid: "fc38cce3b3424ee58c6ab26ec63a51d1"
format_stamp: "Formatted at 2026-07-09 12:29:53 on dist-test-slave-kpwv"
I20260709 12:29:53.396601  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:53.409492  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:53.411286  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:53.456027  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.254:33231
I20260709 12:29:53.456131 23948 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.254:33231 every 8 connection(s)
I20260709 12:29:53.460229 23949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:53.471331 23949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1: Bootstrap starting.
I20260709 12:29:53.476495 23949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:53.481158 23949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1: No bootstrap required, opened a new log
I20260709 12:29:53.483852 23949 raft_consensus.cc:359] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc38cce3b3424ee58c6ab26ec63a51d1" member_type: VOTER }
I20260709 12:29:53.484411 23949 raft_consensus.cc:385] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:53.484776 23949 raft_consensus.cc:740] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc38cce3b3424ee58c6ab26ec63a51d1, State: Initialized, Role: FOLLOWER
I20260709 12:29:53.485807 23949 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc38cce3b3424ee58c6ab26ec63a51d1" member_type: VOTER }
I20260709 12:29:53.486440 23949 raft_consensus.cc:399] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:29:53.486769 23949 raft_consensus.cc:493] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:29:53.487095 23949 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:53.492791 23949 raft_consensus.cc:515] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc38cce3b3424ee58c6ab26ec63a51d1" member_type: VOTER }
I20260709 12:29:53.493430 23949 leader_election.cc:304] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc38cce3b3424ee58c6ab26ec63a51d1; no voters: 
I20260709 12:29:53.494798 23949 leader_election.cc:290] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:29:53.495139 23952 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:53.496628 23952 raft_consensus.cc:697] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 1 LEADER]: Becoming Leader. State: Replica: fc38cce3b3424ee58c6ab26ec63a51d1, State: Running, Role: LEADER
I20260709 12:29:53.497344 23952 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc38cce3b3424ee58c6ab26ec63a51d1" member_type: VOTER }
I20260709 12:29:53.498217 23949 sys_catalog.cc:565] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:29:53.500797 23953 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc38cce3b3424ee58c6ab26ec63a51d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc38cce3b3424ee58c6ab26ec63a51d1" member_type: VOTER } }
I20260709 12:29:53.500808 23954 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc38cce3b3424ee58c6ab26ec63a51d1. Latest consensus state: current_term: 1 leader_uuid: "fc38cce3b3424ee58c6ab26ec63a51d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc38cce3b3424ee58c6ab26ec63a51d1" member_type: VOTER } }
I20260709 12:29:53.501688 23953 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:53.501724 23954 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [sys.catalog]: This master's current role is: LEADER
I20260709 12:29:53.507664 23958 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:29:53.517037 23958 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:29:53.518116  8743 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:29:53.526518 23958 catalog_manager.cc:1383] Generated new cluster ID: 0b3fdadccde342ccbb910fdb800fbc47
I20260709 12:29:53.526870 23958 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:29:53.560389 23958 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:29:53.561873 23958 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:29:53.586764 23958 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1: Generated new TSK 0
I20260709 12:29:53.587445 23958 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:29:53.649618  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:53.656093 23970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:53.657188 23971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:53.660439 23973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:53.660635  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:53.661700  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:53.661907  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:53.662091  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600193662075 us; error 0 us; skew 500 ppm
I20260709 12:29:53.662770  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:53.665179  8743 webserver.cc:533] Webserver started at http://127.8.137.193:40213/ using document root <none> and password file <none>
I20260709 12:29:53.665704  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:53.665899  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:53.666200  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:53.667687  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-0-root/instance:
uuid: "f1bb62eb16d542529e317b9bbf71d0db"
format_stamp: "Formatted at 2026-07-09 12:29:53 on dist-test-slave-kpwv"
I20260709 12:29:53.672423  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:29:53.676021 23978 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.676864  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:53.677211  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "f1bb62eb16d542529e317b9bbf71d0db"
format_stamp: "Formatted at 2026-07-09 12:29:53 on dist-test-slave-kpwv"
I20260709 12:29:53.677516  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:53.702960  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:53.704155  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:53.705622  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:53.707973  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:53.708216  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.708531  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:53.708712  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.750869  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:41663
I20260709 12:29:53.750967 24040 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:41663 every 8 connection(s)
I20260709 12:29:53.756069  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:53.763140 24045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:53.763552 24046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:53.767098  8743 server_base.cc:1061] running on GCE node
W20260709 12:29:53.767912 24049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:53.768785 24041 heartbeater.cc:344] Connected to a master server at 127.8.137.254:33231
I20260709 12:29:53.768908  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:29:53.769203  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:53.769277 24041 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:53.769433  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600193769410 us; error 0 us; skew 500 ppm
I20260709 12:29:53.770200 24041 heartbeater.cc:507] Master 127.8.137.254:33231 requested a full tablet report, sending...
I20260709 12:29:53.770272  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:53.772801 23914 ts_manager.cc:194] Registered new tserver with Master: f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663)
I20260709 12:29:53.773377  8743 webserver.cc:533] Webserver started at http://127.8.137.194:36389/ using document root <none> and password file <none>
I20260709 12:29:53.773941  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:53.774152  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:53.774425  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:53.774856 23914 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60126
I20260709 12:29:53.775698  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-1-root/instance:
uuid: "48dae3b5183a4401b914bcb5dc1bf052"
format_stamp: "Formatted at 2026-07-09 12:29:53 on dist-test-slave-kpwv"
I20260709 12:29:53.780409  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:29:53.783838 24053 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.784597  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:29:53.784924  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-1-root
uuid: "48dae3b5183a4401b914bcb5dc1bf052"
format_stamp: "Formatted at 2026-07-09 12:29:53 on dist-test-slave-kpwv"
I20260709 12:29:53.785187  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:53.799386  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:53.800558  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:53.802044  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:53.804370  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:53.804620  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.804874  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:53.805047  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.847751  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.194:46737
I20260709 12:29:53.848042 24115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.194:46737 every 8 connection(s)
I20260709 12:29:53.852592  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:29:53.860409 24119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:53.860484 24120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:29:53.864902 24122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:29:53.865880  8743 server_base.cc:1061] running on GCE node
I20260709 12:29:53.866892 24116 heartbeater.cc:344] Connected to a master server at 127.8.137.254:33231
I20260709 12:29:53.867012  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:29:53.867344 24116 heartbeater.cc:461] Registering TS with master...
W20260709 12:29:53.867379  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:29:53.867785  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600193867762 us; error 0 us; skew 500 ppm
I20260709 12:29:53.868324 24116 heartbeater.cc:507] Master 127.8.137.254:33231 requested a full tablet report, sending...
I20260709 12:29:53.868423  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:29:53.870519 23914 ts_manager.cc:194] Registered new tserver with Master: 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:53.871466  8743 webserver.cc:533] Webserver started at http://127.8.137.195:41475/ using document root <none> and password file <none>
I20260709 12:29:53.872038  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:29:53.871977 23914 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60136
I20260709 12:29:53.872421  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:29:53.872769  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:29:53.874166  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-2-root/instance:
uuid: "d07747748415477ca875ddac61759b69"
format_stamp: "Formatted at 2026-07-09 12:29:53 on dist-test-slave-kpwv"
I20260709 12:29:53.879094  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:29:53.882797 24127 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.883890  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:29:53.884265  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-2-root
uuid: "d07747748415477ca875ddac61759b69"
format_stamp: "Formatted at 2026-07-09 12:29:53 on dist-test-slave-kpwv"
I20260709 12:29:53.884548  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783599720900375-8743-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:29:53.895278  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:29:53.896342  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:29:53.897595  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:29:53.899986  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:29:53.900279  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.900537  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:29:53.900717  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:29:53.940547  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.195:34527
I20260709 12:29:53.940653 24189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.195:34527 every 8 connection(s)
I20260709 12:29:53.955760 24190 heartbeater.cc:344] Connected to a master server at 127.8.137.254:33231
I20260709 12:29:53.956213 24190 heartbeater.cc:461] Registering TS with master...
I20260709 12:29:53.956907 24190 heartbeater.cc:507] Master 127.8.137.254:33231 requested a full tablet report, sending...
I20260709 12:29:53.958920 23914 ts_manager.cc:194] Registered new tserver with Master: d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:53.959046  8743 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015150426s
I20260709 12:29:53.960335 23914 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60148
I20260709 12:29:53.985879 23914 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60158:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:53.988615 23914 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:54.059592 24155 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4ea575ad024889ad3e11e44b44a807 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:54.061029 24155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4ea575ad024889ad3e11e44b44a807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.067039 24081 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4ea575ad024889ad3e11e44b44a807 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:54.070526 24080 tablet_service.cc:1511] Processing CreateTablet for tablet 3d69560971dd4d919a5c4af837200db6 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:54.072608 24154 tablet_service.cc:1511] Processing CreateTablet for tablet 3d69560971dd4d919a5c4af837200db6 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:54.074198 24079 tablet_service.cc:1511] Processing CreateTablet for tablet 89ffc6d0964a4b8f95239517fefa16bf (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:54.074785 24154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d69560971dd4d919a5c4af837200db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.075750 24081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4ea575ad024889ad3e11e44b44a807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.077611 24078 tablet_service.cc:1511] Processing CreateTablet for tablet c81e471dc3cd4be0b78ce2710eba8ddf (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:54.078526 24079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ffc6d0964a4b8f95239517fefa16bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.079907 24077 tablet_service.cc:1511] Processing CreateTablet for tablet aeb03d7b92454ec6a30cf0313afa26a3 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:54.081358 24077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb03d7b92454ec6a30cf0313afa26a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.083729 24076 tablet_service.cc:1511] Processing CreateTablet for tablet 6bcbdcd3912640e780d1d59ad1ab5994 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:54.085265 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bcbdcd3912640e780d1d59ad1ab5994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.084828 24153 tablet_service.cc:1511] Processing CreateTablet for tablet 89ffc6d0964a4b8f95239517fefa16bf (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:54.087487 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ffc6d0964a4b8f95239517fefa16bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.087743 24152 tablet_service.cc:1511] Processing CreateTablet for tablet c81e471dc3cd4be0b78ce2710eba8ddf (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:54.089128 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c81e471dc3cd4be0b78ce2710eba8ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.092964 24075 tablet_service.cc:1511] Processing CreateTablet for tablet 10c8e3c9cd4648f797f7b1e459951d8f (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:54.094456 24151 tablet_service.cc:1511] Processing CreateTablet for tablet aeb03d7b92454ec6a30cf0313afa26a3 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:54.095928 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb03d7b92454ec6a30cf0313afa26a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.095984 24074 tablet_service.cc:1511] Processing CreateTablet for tablet b893acd38c44433f970f8cf12906a2be (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:54.098851 24073 tablet_service.cc:1511] Processing CreateTablet for tablet 592beb1c07d145c3bd577dec75ba6dce (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:54.107322 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c81e471dc3cd4be0b78ce2710eba8ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.109864 24073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 592beb1c07d145c3bd577dec75ba6dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.113204 24150 tablet_service.cc:1511] Processing CreateTablet for tablet 6bcbdcd3912640e780d1d59ad1ab5994 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:54.114403 24074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b893acd38c44433f970f8cf12906a2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.115000 24150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bcbdcd3912640e780d1d59ad1ab5994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.119051 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10c8e3c9cd4648f797f7b1e459951d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.121526 24080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d69560971dd4d919a5c4af837200db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.133396 24154 tablet_service.cc:1511] Processing CreateTablet for tablet 10c8e3c9cd4648f797f7b1e459951d8f (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:54.135484 24154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10c8e3c9cd4648f797f7b1e459951d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.145486 24149 tablet_service.cc:1511] Processing CreateTablet for tablet b893acd38c44433f970f8cf12906a2be (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:54.147656 24149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b893acd38c44433f970f8cf12906a2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.153829 24148 tablet_service.cc:1511] Processing CreateTablet for tablet 592beb1c07d145c3bd577dec75ba6dce (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:54.156041 24148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 592beb1c07d145c3bd577dec75ba6dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.165340 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 8c4ea575ad024889ad3e11e44b44a807 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:54.167450 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c4ea575ad024889ad3e11e44b44a807. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.193361 24005 tablet_service.cc:1511] Processing CreateTablet for tablet 3d69560971dd4d919a5c4af837200db6 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:54.195791 24005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d69560971dd4d919a5c4af837200db6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.201303 24004 tablet_service.cc:1511] Processing CreateTablet for tablet 89ffc6d0964a4b8f95239517fefa16bf (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:54.202893 24004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89ffc6d0964a4b8f95239517fefa16bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.204512 23999 tablet_service.cc:1511] Processing CreateTablet for tablet b893acd38c44433f970f8cf12906a2be (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:54.205981 23999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b893acd38c44433f970f8cf12906a2be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.208220 24003 tablet_service.cc:1511] Processing CreateTablet for tablet c81e471dc3cd4be0b78ce2710eba8ddf (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:54.210683 24002 tablet_service.cc:1511] Processing CreateTablet for tablet aeb03d7b92454ec6a30cf0313afa26a3 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:54.213111 24001 tablet_service.cc:1511] Processing CreateTablet for tablet 6bcbdcd3912640e780d1d59ad1ab5994 (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:54.215606 24000 tablet_service.cc:1511] Processing CreateTablet for tablet 10c8e3c9cd4648f797f7b1e459951d8f (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:54.218040 23998 tablet_service.cc:1511] Processing CreateTablet for tablet 592beb1c07d145c3bd577dec75ba6dce (DEFAULT_TABLE table=test-workload [id=996a83b8835b426bb73f962ed336b14e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:54.222303 24003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c81e471dc3cd4be0b78ce2710eba8ddf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.225651 24002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb03d7b92454ec6a30cf0313afa26a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.238773 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bcbdcd3912640e780d1d59ad1ab5994. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.245972 23998 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 592beb1c07d145c3bd577dec75ba6dce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.247262 24206 tablet_bootstrap.cc:492] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.251042 24000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10c8e3c9cd4648f797f7b1e459951d8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:54.259454 24205 tablet_bootstrap.cc:492] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.261905 24206 tablet_bootstrap.cc:654] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.266451 24205 tablet_bootstrap.cc:654] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.318300 24206 tablet_bootstrap.cc:492] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:54.318861 24206 ts_tablet_manager.cc:1403] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.072s	user 0.015s	sys 0.000s
I20260709 12:29:54.321790 24206 raft_consensus.cc:359] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.322566 24206 raft_consensus.cc:385] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.323989 24206 raft_consensus.cc:740] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.326140 24208 tablet_bootstrap.cc:492] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.326556 24206 consensus_queue.cc:260] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.335614 24205 tablet_bootstrap.cc:492] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.336434 24205 ts_tablet_manager.cc:1403] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.077s	user 0.008s	sys 0.008s
I20260709 12:29:54.339145 24208 tablet_bootstrap.cc:654] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.340729 24205 raft_consensus.cc:359] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:54.341935 24205 raft_consensus.cc:385] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.342346 24205 raft_consensus.cc:740] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.345757 24208 tablet_bootstrap.cc:492] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.346241 24208 ts_tablet_manager.cc:1403] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 12:29:54.345624 24205 consensus_queue.cc:260] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:54.348419 24116 heartbeater.cc:499] Master 127.8.137.254:33231 was elected leader, sending a full tablet report...
I20260709 12:29:54.349457 24206 ts_tablet_manager.cc:1434] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.030s	user 0.019s	sys 0.007s
I20260709 12:29:54.349290 24208 raft_consensus.cc:359] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.350217 24208 raft_consensus.cc:385] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.350565 24208 raft_consensus.cc:740] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.351356 24206 tablet_bootstrap.cc:492] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.351549 24208 consensus_queue.cc:260] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.354157 24205 ts_tablet_manager.cc:1434] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.017s	user 0.013s	sys 0.003s
I20260709 12:29:54.355051 24190 heartbeater.cc:499] Master 127.8.137.254:33231 was elected leader, sending a full tablet report...
I20260709 12:29:54.355422 24205 tablet_bootstrap.cc:492] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.359725 24206 tablet_bootstrap.cc:654] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.362133 24205 tablet_bootstrap.cc:654] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.372678 24205 tablet_bootstrap.cc:492] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.372841 24206 tablet_bootstrap.cc:492] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:54.373227 24208 ts_tablet_manager.cc:1434] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.027s	user 0.009s	sys 0.007s
I20260709 12:29:54.373248 24205 ts_tablet_manager.cc:1403] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.018s	user 0.006s	sys 0.008s
I20260709 12:29:54.373565 24206 ts_tablet_manager.cc:1403] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.004s
I20260709 12:29:54.374071 24041 heartbeater.cc:499] Master 127.8.137.254:33231 was elected leader, sending a full tablet report...
I20260709 12:29:54.376858 24205 raft_consensus.cc:359] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.377678 24205 raft_consensus.cc:385] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.378036 24205 raft_consensus.cc:740] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.379035 24205 consensus_queue.cc:260] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.380383 24206 raft_consensus.cc:359] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:54.381342 24205 ts_tablet_manager.cc:1434] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:29:54.381337 24206 raft_consensus.cc:385] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.381922 24206 raft_consensus.cc:740] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.382860 24205 tablet_bootstrap.cc:492] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.382931 24206 consensus_queue.cc:260] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:54.385720 24206 ts_tablet_manager.cc:1434] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:29:54.386973 24206 tablet_bootstrap.cc:492] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.390825 24205 tablet_bootstrap.cc:654] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.394300 24206 tablet_bootstrap.cc:654] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.395692 24208 tablet_bootstrap.cc:492] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.403658 24208 tablet_bootstrap.cc:654] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.404963 24205 tablet_bootstrap.cc:492] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.405623 24205 ts_tablet_manager.cc:1403] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.023s	user 0.012s	sys 0.008s
I20260709 12:29:54.408985 24205 raft_consensus.cc:359] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.410504 24205 raft_consensus.cc:385] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.410923 24205 raft_consensus.cc:740] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.412142 24206 tablet_bootstrap.cc:492] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:54.411820 24205 consensus_queue.cc:260] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.412757 24206 ts_tablet_manager.cc:1403] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.007s
I20260709 12:29:54.414305 24205 ts_tablet_manager.cc:1434] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:29:54.416077 24205 tablet_bootstrap.cc:492] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.416551 24206 raft_consensus.cc:359] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.417357 24206 raft_consensus.cc:385] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.417665 24206 raft_consensus.cc:740] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.418679 24206 consensus_queue.cc:260] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.423499 24205 tablet_bootstrap.cc:654] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.424470 24206 ts_tablet_manager.cc:1434] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.007s
I20260709 12:29:54.425666 24206 tablet_bootstrap.cc:492] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.433717 24208 tablet_bootstrap.cc:492] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.434432 24208 ts_tablet_manager.cc:1403] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.021s
I20260709 12:29:54.435596 24206 tablet_bootstrap.cc:654] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.436802 24205 tablet_bootstrap.cc:492] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.437359 24205 ts_tablet_manager.cc:1403] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.006s
I20260709 12:29:54.437611 24208 raft_consensus.cc:359] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.438282 24208 raft_consensus.cc:385] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.438570 24208 raft_consensus.cc:740] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.439397 24208 consensus_queue.cc:260] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.441545 24208 ts_tablet_manager.cc:1434] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.003s
I20260709 12:29:54.441282 24205 raft_consensus.cc:359] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.442314 24205 raft_consensus.cc:385] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.442703 24205 raft_consensus.cc:740] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.442871 24208 tablet_bootstrap.cc:492] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.443640 24205 consensus_queue.cc:260] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.448670 24205 ts_tablet_manager.cc:1434] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:29:54.449980 24205 tablet_bootstrap.cc:492] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.450392 24208 tablet_bootstrap.cc:654] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.458817 24205 tablet_bootstrap.cc:654] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.455195 24212 raft_consensus.cc:493] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.459540 24212 raft_consensus.cc:515] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:54.462872 24208 tablet_bootstrap.cc:492] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.463594 24208 ts_tablet_manager.cc:1403] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.021s	user 0.007s	sys 0.009s
I20260709 12:29:54.465266 24206 tablet_bootstrap.cc:492] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:54.466125 24206 ts_tablet_manager.cc:1403] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.009s
I20260709 12:29:54.467115 24212 leader_election.cc:290] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663)
I20260709 12:29:54.467654 24208 raft_consensus.cc:359] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.468945 24208 raft_consensus.cc:385] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.469439 24208 raft_consensus.cc:740] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.470826 24206 raft_consensus.cc:359] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.470863 24208 consensus_queue.cc:260] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.472296 24206 raft_consensus.cc:385] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.472822 24206 raft_consensus.cc:740] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.474764 24208 ts_tablet_manager.cc:1434] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.011s	user 0.011s	sys 0.001s
I20260709 12:29:54.476518 24208 tablet_bootstrap.cc:492] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.479100 24206 consensus_queue.cc:260] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.487703 24208 tablet_bootstrap.cc:654] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.490974 24206 ts_tablet_manager.cc:1434] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.024s	user 0.011s	sys 0.003s
I20260709 12:29:54.492818 24206 tablet_bootstrap.cc:492] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.505568 24208 tablet_bootstrap.cc:492] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.506299 24208 ts_tablet_manager.cc:1403] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.030s	user 0.013s	sys 0.005s
I20260709 12:29:54.510543 24208 raft_consensus.cc:359] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.511545 24208 raft_consensus.cc:385] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.511919 24208 raft_consensus.cc:740] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.512892 24208 consensus_queue.cc:260] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
W20260709 12:29:54.514092 23962 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:54.514328 24206 tablet_bootstrap.cc:654] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.516160 24208 ts_tablet_manager.cc:1434] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 12:29:54.517665 24208 tablet_bootstrap.cc:492] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.527420 24206 tablet_bootstrap.cc:492] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:54.528000 24208 tablet_bootstrap.cc:654] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.528110 24206 ts_tablet_manager.cc:1403] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.036s	user 0.012s	sys 0.009s
I20260709 12:29:54.532429 24206 raft_consensus.cc:359] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.534006 24206 raft_consensus.cc:385] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.534495 24206 raft_consensus.cc:740] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.536453 24206 consensus_queue.cc:260] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.541378 24208 tablet_bootstrap.cc:492] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.541185 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:54.542064 24208 ts_tablet_manager.cc:1403] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.025s	user 0.008s	sys 0.013s
I20260709 12:29:54.542222 24091 raft_consensus.cc:2468] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:54.542867 24206 ts_tablet_manager.cc:1434] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 12:29:54.544193 24206 tablet_bootstrap.cc:492] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.546543 24208 raft_consensus.cc:359] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.547587 24208 raft_consensus.cc:385] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.547959 24208 raft_consensus.cc:740] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.549140 24208 consensus_queue.cc:260] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.552839 24206 tablet_bootstrap.cc:654] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.554746 24208 ts_tablet_manager.cc:1434] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.005s
I20260709 12:29:54.556089 24208 tablet_bootstrap.cc:492] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.556180 24128 leader_election.cc:304] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: 
I20260709 12:29:54.557233 24212 raft_consensus.cc:2804] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.557638 24212 raft_consensus.cc:493] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.558036 24212 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.562641 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
W20260709 12:29:54.564632 24129 leader_election.cc:343] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:54.565963 24208 tablet_bootstrap.cc:654] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.567858 24212 raft_consensus.cc:515] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:54.574553 24205 tablet_bootstrap.cc:492] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.575171 24205 ts_tablet_manager.cc:1403] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.125s	user 0.037s	sys 0.053s
I20260709 12:29:54.577855 24205 raft_consensus.cc:359] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.578579 24205 raft_consensus.cc:385] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.578925 24205 raft_consensus.cc:740] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.579213 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:54.579850 24091 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.579883 24205 consensus_queue.cc:260] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.580364 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
W20260709 12:29:54.582808 24129 leader_election.cc:343] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:54.585752 24212 leader_election.cc:290] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663)
I20260709 12:29:54.589448 24205 ts_tablet_manager.cc:1434] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.007s
I20260709 12:29:54.590874 24205 tablet_bootstrap.cc:492] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.591385 24213 raft_consensus.cc:493] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.591918 24213 raft_consensus.cc:515] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.598125 24208 tablet_bootstrap.cc:492] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.598241 24205 tablet_bootstrap.cc:654] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.598994 24208 ts_tablet_manager.cc:1403] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.043s	user 0.030s	sys 0.011s
I20260709 12:29:54.602346 24208 raft_consensus.cc:359] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.603091 24208 raft_consensus.cc:385] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.598295 24206 tablet_bootstrap.cc:492] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:54.603320 24213 leader_election.cc:290] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.603777 24206 ts_tablet_manager.cc:1403] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.060s	user 0.018s	sys 0.003s
I20260709 12:29:54.607450 24206 raft_consensus.cc:359] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.608248 24206 raft_consensus.cc:385] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.608546 24206 raft_consensus.cc:740] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.590723 24091 raft_consensus.cc:2468] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:54.609473 24206 consensus_queue.cc:260] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.611922 24128 leader_election.cc:304] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:54.614440 24212 raft_consensus.cc:2804] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:54.614979 24212 raft_consensus.cc:697] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:54.615842 24212 consensus_queue.cc:237] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:54.603396 24208 raft_consensus.cc:740] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.620322 24206 ts_tablet_manager.cc:1434] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:29:54.619352 24208 consensus_queue.cc:260] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.629676 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c81e471dc3cd4be0b78ce2710eba8ddf" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:54.630632 24091 raft_consensus.cc:2468] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:54.632131 24208 ts_tablet_manager.cc:1434] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:29:54.634002 24208 tablet_bootstrap.cc:492] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.635301 24206 tablet_bootstrap.cc:492] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.635840 23979 leader_election.cc:304] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.636962 24213 raft_consensus.cc:2804] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.637341 24213 raft_consensus.cc:493] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.637678 24213 raft_consensus.cc:3060] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.650993 24208 tablet_bootstrap.cc:654] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.654847 24206 tablet_bootstrap.cc:654] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.655525 24213 raft_consensus.cc:515] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.658659 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c81e471dc3cd4be0b78ce2710eba8ddf" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:54.659261 24091 raft_consensus.cc:3060] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.664726 24212 raft_consensus.cc:493] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.665655 24205 tablet_bootstrap.cc:492] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.665591 24212 raft_consensus.cc:515] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.667289 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c81e471dc3cd4be0b78ce2710eba8ddf" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:54.668380 24212 leader_election.cc:290] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:54.668655 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ffc6d0964a4b8f95239517fefa16bf" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:54.669237 24016 raft_consensus.cc:2468] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:54.670326 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ffc6d0964a4b8f95239517fefa16bf" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:54.670689 24129 leader_election.cc:304] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.671098 24090 raft_consensus.cc:2468] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:54.667541 24091 raft_consensus.cc:2468] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:54.666483 24205 ts_tablet_manager.cc:1403] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.076s	user 0.017s	sys 0.008s
I20260709 12:29:54.673462 23979 leader_election.cc:304] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.673696 24208 tablet_bootstrap.cc:492] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.674317 24208 ts_tablet_manager.cc:1403] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.041s	user 0.011s	sys 0.003s
W20260709 12:29:54.672228 23980 leader_election.cc:343] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d07747748415477ca875ddac61759b69 (127.8.137.195:34527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:54.675400 24238 raft_consensus.cc:2804] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:54.677598 24208 raft_consensus.cc:359] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.678344 24208 raft_consensus.cc:385] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.678668 24208 raft_consensus.cc:740] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.679446 24208 consensus_queue.cc:260] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.680073 24205 raft_consensus.cc:359] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.680948 24205 raft_consensus.cc:385] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.681315 24205 raft_consensus.cc:740] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.682216 24212 raft_consensus.cc:2804] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.682708 24212 raft_consensus.cc:493] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.683702 24206 tablet_bootstrap.cc:492] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:54.682255 24205 consensus_queue.cc:260] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.683423 24212 raft_consensus.cc:3060] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.684298 24206 ts_tablet_manager.cc:1403] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.049s	user 0.014s	sys 0.012s
I20260709 12:29:54.684885 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c81e471dc3cd4be0b78ce2710eba8ddf" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
W20260709 12:29:54.686668 23980 leader_election.cc:343] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d07747748415477ca875ddac61759b69 (127.8.137.195:34527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:54.682818 24213 leader_election.cc:290] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.687203 23913 catalog_manager.cc:5697] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:54.689760 24206 raft_consensus.cc:359] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.690618 24206 raft_consensus.cc:385] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.690836 24205 ts_tablet_manager.cc:1434] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.000s
I20260709 12:29:54.690977 24206 raft_consensus.cc:740] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.692174 24205 tablet_bootstrap.cc:492] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.694197 24212 raft_consensus.cc:515] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.695405 24208 ts_tablet_manager.cc:1434] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.006s
I20260709 12:29:54.696637 24208 tablet_bootstrap.cc:492] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.697656 24212 leader_election.cc:290] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:54.699905 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ffc6d0964a4b8f95239517fefa16bf" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:54.700542 24016 raft_consensus.cc:3060] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.700894 24205 tablet_bootstrap.cc:654] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.703222 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ffc6d0964a4b8f95239517fefa16bf" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:54.703900 24091 raft_consensus.cc:3060] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.703562 24206 consensus_queue.cc:260] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.708683 24016 raft_consensus.cc:2468] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:54.710187 24129 leader_election.cc:304] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.711529 24212 raft_consensus.cc:2804] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:54.712229 24212 raft_consensus.cc:697] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:54.712986 24091 raft_consensus.cc:2468] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:54.713119 24212 consensus_queue.cc:237] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.716894 24238 raft_consensus.cc:697] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:54.718640 24206 ts_tablet_manager.cc:1434] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.001s
I20260709 12:29:54.718556 24238 consensus_queue.cc:237] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.726321 24206 tablet_bootstrap.cc:492] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.729357 24210 raft_consensus.cc:493] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.730119 24210 raft_consensus.cc:515] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.735419 24210 leader_election.cc:290] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.735632 24208 tablet_bootstrap.cc:654] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.741528 24239 raft_consensus.cc:493] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.742271 24239 raft_consensus.cc:515] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.743726 24205 tablet_bootstrap.cc:492] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.744378 24205 ts_tablet_manager.cc:1403] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.041s
I20260709 12:29:54.745049 24206 tablet_bootstrap.cc:654] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.745860 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
W20260709 12:29:54.747375 23979 leader_election.cc:343] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:54.748238 24205 raft_consensus.cc:359] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.749161 24205 raft_consensus.cc:385] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.749940 24205 raft_consensus.cc:740] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.750521 24208 tablet_bootstrap.cc:492] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.751147 24208 ts_tablet_manager.cc:1403] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.055s	user 0.010s	sys 0.006s
I20260709 12:29:54.751390 24205 consensus_queue.cc:260] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.754362 24205 ts_tablet_manager.cc:1434] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.004s
I20260709 12:29:54.754413 24208 raft_consensus.cc:359] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.755365 24208 raft_consensus.cc:385] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.755614 24205 tablet_bootstrap.cc:492] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.755677 24208 raft_consensus.cc:740] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.761615 24208 consensus_queue.cc:260] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.763846 23914 catalog_manager.cc:5697] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 1 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:54.764981 24208 ts_tablet_manager.cc:1434] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 12:29:54.766413 24208 tablet_bootstrap.cc:492] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:54.771642 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:54.772450 24165 raft_consensus.cc:2468] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:54.773890 23980 leader_election.cc:304] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:54.774447 24208 tablet_bootstrap.cc:654] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.774920 24238 raft_consensus.cc:2804] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.775334 24238 raft_consensus.cc:493] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.775691 24238 raft_consensus.cc:3060] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.777549 24205 tablet_bootstrap.cc:654] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.779232 24212 raft_consensus.cc:493] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.779798 24212 raft_consensus.cc:515] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.783630 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69560971dd4d919a5c4af837200db6" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:54.784360 24091 raft_consensus.cc:2468] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:54.784477 23913 catalog_manager.cc:5697] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:54.785781 24128 leader_election.cc:304] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: 
I20260709 12:29:54.786728 24230 raft_consensus.cc:2804] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.787088 24230 raft_consensus.cc:493] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.781986 24212 leader_election.cc:290] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:54.787459 24230 raft_consensus.cc:3060] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.790723 24205 tablet_bootstrap.cc:492] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.791447 24205 ts_tablet_manager.cc:1403] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.009s
I20260709 12:29:54.791538 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69560971dd4d919a5c4af837200db6" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:54.792302 24016 raft_consensus.cc:2468] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:54.793068 24238 raft_consensus.cc:515] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.794688 24239 leader_election.cc:290] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.795852 24205 raft_consensus.cc:359] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.796942 24205 raft_consensus.cc:385] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.797325 24205 raft_consensus.cc:740] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.798336 24205 consensus_queue.cc:260] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.800128 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
W20260709 12:29:54.801779 23979 leader_election.cc:343] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:54.801977 24205 ts_tablet_manager.cc:1434] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:29:54.803529 24205 tablet_bootstrap.cc:492] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:54.804086 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:54.804745 24165 raft_consensus.cc:3060] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.805742 24230 raft_consensus.cc:515] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.816970 24206 tablet_bootstrap.cc:492] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:54.817615 24206 ts_tablet_manager.cc:1403] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.092s	user 0.021s	sys 0.011s
I20260709 12:29:54.818825 24230 leader_election.cc:290] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:54.812618 24205 tablet_bootstrap.cc:654] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.820856 24206 raft_consensus.cc:359] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.821404 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bcbdcd3912640e780d1d59ad1ab5994" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:54.821727 24206 raft_consensus.cc:385] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.822075 24016 raft_consensus.cc:2468] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 0.
I20260709 12:29:54.822423 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69560971dd4d919a5c4af837200db6" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:54.823043 24091 raft_consensus.cc:3060] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.823665 24055 leader_election.cc:304] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.824537 24210 raft_consensus.cc:2804] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.824890 24210 raft_consensus.cc:493] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.825201 24210 raft_consensus.cc:3060] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.822091 24206 raft_consensus.cc:740] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.830205 24208 tablet_bootstrap.cc:492] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:54.830808 24208 ts_tablet_manager.cc:1403] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.065s	user 0.027s	sys 0.030s
I20260709 12:29:54.833503 24208 raft_consensus.cc:359] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:54.830446 24206 consensus_queue.cc:260] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.834071 24208 raft_consensus.cc:385] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.821125 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bcbdcd3912640e780d1d59ad1ab5994" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:54.834532 24208 raft_consensus.cc:740] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.835709 24208 consensus_queue.cc:260] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
W20260709 12:29:54.839056 24055 leader_election.cc:343] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d07747748415477ca875ddac61759b69 (127.8.137.195:34527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:54.839143 24208 ts_tablet_manager.cc:1434] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:54.845494 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69560971dd4d919a5c4af837200db6" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:54.846454 24016 raft_consensus.cc:3060] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.858402 24091 raft_consensus.cc:2468] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:54.854089 24210 raft_consensus.cc:515] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.865726 24238 leader_election.cc:290] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.869742 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bcbdcd3912640e780d1d59ad1ab5994" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:54.870800 24015 raft_consensus.cc:3060] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.882308 24165 raft_consensus.cc:2468] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:54.884558 23980 leader_election.cc:304] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:54.890980 24128 leader_election.cc:304] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: 
I20260709 12:29:54.892431 24016 raft_consensus.cc:2468] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:54.892839 24230 raft_consensus.cc:2804] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:54.893728 24230 raft_consensus.cc:697] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:54.894958 24230 consensus_queue.cc:237] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.898954 24256 raft_consensus.cc:493] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.899842 24256 raft_consensus.cc:515] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.903237 24239 raft_consensus.cc:493] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.904168 24239 raft_consensus.cc:515] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.907577 24239 leader_election.cc:290] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.908406 24239 raft_consensus.cc:2804] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:54.909371 24239 raft_consensus.cc:697] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:54.910787 24239 consensus_queue.cc:237] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.915668 24257 raft_consensus.cc:493] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.916471 24257 raft_consensus.cc:515] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.921001 24238 raft_consensus.cc:493] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.920819 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:54.922072 24238 raft_consensus.cc:515] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.923235 24014 raft_consensus.cc:2468] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 0.
I20260709 12:29:54.920827 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:54.925438 24256 leader_election.cc:290] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.925639 24165 raft_consensus.cc:2468] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:54.927009 24055 leader_election.cc:304] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.927728 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:54.928651 24164 raft_consensus.cc:2468] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 0.
I20260709 12:29:54.930706 24238 leader_election.cc:290] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.930438 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:54.931974 24090 raft_consensus.cc:2468] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:54.932029 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:54.933008 24164 raft_consensus.cc:2468] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:54.934631 23979 leader_election.cc:304] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.934653 23980 leader_election.cc:304] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.936988 24240 raft_consensus.cc:2804] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.937439 24240 raft_consensus.cc:493] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.937813 24240 raft_consensus.cc:3060] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.935755 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:54.939131 24210 leader_election.cc:290] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.939496 24091 raft_consensus.cc:2468] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:54.940054 24210 raft_consensus.cc:2804] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.940587 24210 raft_consensus.cc:493] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.941048 24210 raft_consensus.cc:3060] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.942130 24239 raft_consensus.cc:2804] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.945300 24239 raft_consensus.cc:493] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.945858 24239 raft_consensus.cc:3060] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.929826 23914 catalog_manager.cc:5697] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:54.944796 24205 tablet_bootstrap.cc:492] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:54.947242 24205 ts_tablet_manager.cc:1403] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.144s	user 0.048s	sys 0.058s
I20260709 12:29:54.950722 24205 raft_consensus.cc:359] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.951525 24205 raft_consensus.cc:385] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:54.951884 24205 raft_consensus.cc:740] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:54.952742 24205 consensus_queue.cc:260] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.953342 24210 raft_consensus.cc:515] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.945564 23913 catalog_manager.cc:5697] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 1 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:54.956018 24205 ts_tablet_manager.cc:1434] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:29:54.958019 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:54.958371 24240 raft_consensus.cc:515] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.960059 24014 raft_consensus.cc:2393] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 48dae3b5183a4401b914bcb5dc1bf052 in current term 1: Already voted for candidate f1bb62eb16d542529e317b9bbf71d0db in this term.
I20260709 12:29:54.960444 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:54.962095 24164 raft_consensus.cc:3060] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.962657 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:54.960029 24257 leader_election.cc:290] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.963757 24091 raft_consensus.cc:2393] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f1bb62eb16d542529e317b9bbf71d0db in current term 1: Already voted for candidate 48dae3b5183a4401b914bcb5dc1bf052 in this term.
I20260709 12:29:54.966030 24240 leader_election.cc:290] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.967274 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:54.972069 24206 ts_tablet_manager.cc:1434] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.154s	user 0.057s	sys 0.060s
I20260709 12:29:54.944602 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bcbdcd3912640e780d1d59ad1ab5994" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:54.972837 24165 raft_consensus.cc:3060] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.973433 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:54.974113 24163 raft_consensus.cc:2468] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 0.
I20260709 12:29:54.974849 23980 leader_election.cc:304] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1bb62eb16d542529e317b9bbf71d0db; no voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69
I20260709 12:29:54.957044 24015 raft_consensus.cc:2468] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 1.
I20260709 12:29:54.975729 24055 leader_election.cc:304] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: 
I20260709 12:29:54.957211 24255 raft_consensus.cc:493] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:54.973452 24206 tablet_bootstrap.cc:492] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:54.976699 24255 raft_consensus.cc:515] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.977666 24055 leader_election.cc:304] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:54.971834 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:54.956398 24210 leader_election.cc:290] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.979576 24210 raft_consensus.cc:2804] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:54.978929 24164 raft_consensus.cc:2468] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 1.
I20260709 12:29:54.980037 24255 leader_election.cc:290] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.980958 24255 raft_consensus.cc:2804] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:54.981525 24165 raft_consensus.cc:2468] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 1.
I20260709 12:29:54.982753 24239 raft_consensus.cc:515] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.984234 24014 raft_consensus.cc:2393] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48dae3b5183a4401b914bcb5dc1bf052 in current term 1: Already voted for candidate f1bb62eb16d542529e317b9bbf71d0db in this term.
I20260709 12:29:54.984549 24240 raft_consensus.cc:2749] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:54.980041 24210 raft_consensus.cc:493] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:54.981526 24255 raft_consensus.cc:697] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:54.985994 24210 raft_consensus.cc:3060] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.987033 24255 consensus_queue.cc:237] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.988268 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:54.989655 24239 leader_election.cc:290] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.991878 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b893acd38c44433f970f8cf12906a2be" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:54.992580 24014 raft_consensus.cc:2468] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 0.
I20260709 12:29:54.992190 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b893acd38c44433f970f8cf12906a2be" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:54.993420 24165 raft_consensus.cc:2468] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 0.
I20260709 12:29:54.994481 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:54.995556 24164 raft_consensus.cc:3060] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:54.994864 24055 leader_election.cc:304] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:54.996132 24210 raft_consensus.cc:515] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:54.996726 24257 raft_consensus.cc:2804] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:54.994659 24206 tablet_bootstrap.cc:654] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:54.997381 24257 raft_consensus.cc:697] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:54.998828 24210 leader_election.cc:290] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:54.997386 24091 raft_consensus.cc:2393] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f1bb62eb16d542529e317b9bbf71d0db in current term 1: Already voted for candidate 48dae3b5183a4401b914bcb5dc1bf052 in this term.
I20260709 12:29:54.998428 24257 consensus_queue.cc:237] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.001331 24055 leader_election.cc:304] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:55.002306 24210 raft_consensus.cc:2804] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:55.002734 24210 raft_consensus.cc:493] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:55.003059 24210 raft_consensus.cc:3060] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:55.003568 24164 raft_consensus.cc:2468] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:55.005373 23980 leader_election.cc:304] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:55.005738 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:55.006807 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:55.007078 24165 raft_consensus.cc:2393] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 48dae3b5183a4401b914bcb5dc1bf052 in current term 1: Already voted for candidate f1bb62eb16d542529e317b9bbf71d0db in this term.
I20260709 12:29:55.007664 24014 raft_consensus.cc:2393] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 48dae3b5183a4401b914bcb5dc1bf052 in current term 1: Already voted for candidate f1bb62eb16d542529e317b9bbf71d0db in this term.
I20260709 12:29:55.008978 24239 raft_consensus.cc:2804] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:55.009682 24239 raft_consensus.cc:697] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:55.011406 24210 raft_consensus.cc:515] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.011140 24239 consensus_queue.cc:237] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.013865 24210 leader_election.cc:290] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:55.016528 24206 tablet_bootstrap.cc:492] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:55.017083 24206 ts_tablet_manager.cc:1403] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.004s
I20260709 12:29:55.009313 24055 leader_election.cc:304] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052; no voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:55.020371 24206 raft_consensus.cc:359] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.023252 24206 raft_consensus.cc:385] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:55.023659 24206 raft_consensus.cc:740] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:55.024634 24206 consensus_queue.cc:260] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.025938 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b893acd38c44433f970f8cf12906a2be" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:55.026700 24014 raft_consensus.cc:3060] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:55.026995 24206 ts_tablet_manager.cc:1434] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 12:29:55.027971 24255 raft_consensus.cc:2749] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:29:55.029592 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b893acd38c44433f970f8cf12906a2be" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:55.030297 24165 raft_consensus.cc:3060] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:55.030270 23914 catalog_manager.cc:5697] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 1 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.033715 23913 catalog_manager.cc:5697] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.035888 23913 catalog_manager.cc:5697] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.038167 24165 raft_consensus.cc:2468] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 1.
I20260709 12:29:55.039472 24055 leader_election.cc:304] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: 
I20260709 12:29:55.040309 24257 raft_consensus.cc:2804] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:55.040874 24257 raft_consensus.cc:697] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:55.041792 24257 consensus_queue.cc:237] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.042953 24014 raft_consensus.cc:2468] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 1.
I20260709 12:29:55.053555 23913 catalog_manager.cc:5697] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.090231 24014 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:55.097514 24230 consensus_queue.cc:1048] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.109910 24212 consensus_queue.cc:1048] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.148092 24165 raft_consensus.cc:3060] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:55.154491 24239 consensus_queue.cc:1048] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.162726 24257 raft_consensus.cc:493] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:55.163352 24257 raft_consensus.cc:515] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.165154 24240 consensus_queue.cc:1048] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.165717 24257 leader_election.cc:290] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:55.166985 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:55.167501 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:55.167833 24164 raft_consensus.cc:2393] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 48dae3b5183a4401b914bcb5dc1bf052 in current term 1: Already voted for candidate f1bb62eb16d542529e317b9bbf71d0db in this term.
I20260709 12:29:55.169857 24055 leader_election.cc:304] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052; no voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:55.171129 24257 raft_consensus.cc:3060] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:55.179570 24257 raft_consensus.cc:2749] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:55.202476 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807"
dest_uuid: "d07747748415477ca875ddac61759b69"
mode: GRACEFUL
new_leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
 from {username='slave'} at 127.0.0.1:35694
I20260709 12:29:55.203222 24165 raft_consensus.cc:606] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Received request to transfer leadership to TS f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:55.203277 24230 consensus_queue.cc:1048] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.209699 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3d69560971dd4d919a5c4af837200db6"
dest_uuid: "d07747748415477ca875ddac61759b69"
mode: GRACEFUL
new_leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
 from {username='slave'} at 127.0.0.1:35694
I20260709 12:29:55.210341 24165 raft_consensus.cc:606] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Received request to transfer leadership to TS f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:55.218321 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89ffc6d0964a4b8f95239517fefa16bf"
dest_uuid: "d07747748415477ca875ddac61759b69"
mode: GRACEFUL
new_leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
 from {username='slave'} at 127.0.0.1:35694
I20260709 12:29:55.218971 24165 raft_consensus.cc:606] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Received request to transfer leadership to TS f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:55.220659 24230 consensus_queue.cc:1048] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.225239 24230 raft_consensus.cc:993] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69: : Instructing follower f1bb62eb16d542529e317b9bbf71d0db to start an election
I20260709 12:29:55.225768 24230 raft_consensus.cc:1081] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Signalling peer f1bb62eb16d542529e317b9bbf71d0db to start an election
I20260709 12:29:55.227646 24014 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89ffc6d0964a4b8f95239517fefa16bf"
dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
 from {username='slave'} at 127.0.0.1:57896
I20260709 12:29:55.228224 24014 raft_consensus.cc:493] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:55.228613 24014 raft_consensus.cc:3060] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.236042 24014 raft_consensus.cc:515] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.238816 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ffc6d0964a4b8f95239517fefa16bf" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:55.239071 24014 leader_election.cc:290] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 2 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:55.239432 24091 raft_consensus.cc:3060] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.239782 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89ffc6d0964a4b8f95239517fefa16bf" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:55.240388 24165 raft_consensus.cc:3055] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:55.240715 24165 raft_consensus.cc:740] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:55.241533 24165 consensus_queue.cc:260] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.242887 24165 raft_consensus.cc:3060] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.245134 24014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3"
dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
mode: GRACEFUL
new_leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:57910
I20260709 12:29:55.245698 24014 raft_consensus.cc:606] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Received request to transfer leadership to TS 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:55.247426 24091 raft_consensus.cc:2468] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 2.
I20260709 12:29:55.248744 23979 leader_election.cc:304] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:55.249562 24240 raft_consensus.cc:2804] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:55.249560 24165 raft_consensus.cc:2468] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 2.
I20260709 12:29:55.250301 24240 raft_consensus.cc:697] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:55.251165 24240 consensus_queue.cc:237] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.254900 24014 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f"
dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
mode: GRACEFUL
new_leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:57910
I20260709 12:29:55.255842 24014 raft_consensus.cc:606] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Received request to transfer leadership to TS 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:55.258406 23913 catalog_manager.cc:5697] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 1 to 2, leader changed from d07747748415477ca875ddac61759b69 (127.8.137.195) to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 2 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.279191 24091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce"
dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
mode: GRACEFUL
new_leader_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:35898
I20260709 12:29:55.279969 24091 raft_consensus.cc:606] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Received request to transfer leadership to TS d07747748415477ca875ddac61759b69
I20260709 12:29:55.399248 24240 consensus_queue.cc:1048] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.405683 24213 raft_consensus.cc:993] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db: : Instructing follower 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:55.406235 24239 raft_consensus.cc:1081] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Signalling peer 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:55.413650 24090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3"
dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:35896
I20260709 12:29:55.414284 24090 raft_consensus.cc:493] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:55.414721 24090 raft_consensus.cc:3060] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.418715 24257 consensus_queue.cc:1048] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:55.419209 24239 consensus_queue.cc:1048] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.425391 24275 consensus_queue.cc:1048] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.425810 24090 raft_consensus.cc:515] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.428795 24090 leader_election.cc:290] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:55.429548 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:55.430287 24014 raft_consensus.cc:3055] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:55.430702 24014 raft_consensus.cc:740] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:55.430692 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb03d7b92454ec6a30cf0313afa26a3" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:55.433382 24014 consensus_queue.cc:260] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.434738 24014 raft_consensus.cc:3060] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.445041 24239 consensus_queue.cc:1048] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:55.445621 24014 raft_consensus.cc:2468] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 2.
I20260709 12:29:55.447000 24055 leader_election.cc:304] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: d07747748415477ca875ddac61759b69
I20260709 12:29:55.447155 24210 consensus_queue.cc:1048] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.448887 24257 raft_consensus.cc:2804] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:55.449451 24257 raft_consensus.cc:697] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:55.451053 24275 consensus_queue.cc:1048] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.450271 24257 consensus_queue.cc:237] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.469054 23914 catalog_manager.cc:5697] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 1 to 2, leader changed from f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193) to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.474610 24213 raft_consensus.cc:993] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db: : Instructing follower 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:55.475297 24240 raft_consensus.cc:1081] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Signalling peer 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:55.477193 24090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f"
dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:35896
I20260709 12:29:55.481629 24090 raft_consensus.cc:493] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:55.482061 24090 raft_consensus.cc:3060] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.482124 24230 raft_consensus.cc:993] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69: : Instructing follower f1bb62eb16d542529e317b9bbf71d0db to start an election
I20260709 12:29:55.484819 24230 raft_consensus.cc:1081] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Signalling peer f1bb62eb16d542529e317b9bbf71d0db to start an election
I20260709 12:29:55.487023 24016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3d69560971dd4d919a5c4af837200db6"
dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
 from {username='slave'} at 127.0.0.1:57896
I20260709 12:29:55.487630 24016 raft_consensus.cc:493] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:55.488004 24016 raft_consensus.cc:3060] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.491771 24239 consensus_queue.cc:1048] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:55.492892 24090 raft_consensus.cc:515] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.498658 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:55.499217 24016 raft_consensus.cc:515] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.500896 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10c8e3c9cd4648f797f7b1e459951d8f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:55.501710 24016 leader_election.cc:290] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 2 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:55.502713 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69560971dd4d919a5c4af837200db6" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:55.503315 24091 raft_consensus.cc:3060] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.503525 24090 leader_election.cc:290] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:55.499300 24015 raft_consensus.cc:3055] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:55.504711 24015 raft_consensus.cc:740] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:55.505576 24015 consensus_queue.cc:260] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
W20260709 12:29:55.506662 24239 consensus_queue.cc:1175] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d07747748415477ca875ddac61759b69" responder_term: 1 status { last_received { term: 1 index: 1 } last_committed_idx: 1 last_received_current_leader { term: 1 index: 1 } }
I20260709 12:29:55.506873 24015 raft_consensus.cc:3060] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.508006 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3d69560971dd4d919a5c4af837200db6" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:55.508553 24165 raft_consensus.cc:3055] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:55.508867 24165 raft_consensus.cc:740] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:55.509688 24165 consensus_queue.cc:260] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.511005 24165 raft_consensus.cc:3060] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.513893 24015 raft_consensus.cc:2468] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 2.
I20260709 12:29:55.515105 24091 raft_consensus.cc:2468] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 2.
I20260709 12:29:55.516501 23979 leader_election.cc:304] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:55.516597 24055 leader_election.cc:304] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: d07747748415477ca875ddac61759b69
I20260709 12:29:55.517426 24257 raft_consensus.cc:2804] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:55.517499 24239 raft_consensus.cc:2804] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:55.517987 24257 raft_consensus.cc:697] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:55.518484 24239 raft_consensus.cc:697] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:55.518923 24257 consensus_queue.cc:237] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.519419 24239 consensus_queue.cc:237] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.526850 24165 raft_consensus.cc:2468] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 2.
I20260709 12:29:55.531312 23913 catalog_manager.cc:5697] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 1 to 2, leader changed from f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193) to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.536760 23911 catalog_manager.cc:5697] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 1 to 2, leader changed from d07747748415477ca875ddac61759b69 (127.8.137.195) to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 2 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:55.557827 24257 consensus_queue.cc:1048] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.564603 24210 raft_consensus.cc:993] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052: : Instructing follower d07747748415477ca875ddac61759b69 to start an election
I20260709 12:29:55.565112 24210 raft_consensus.cc:1081] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Signalling peer d07747748415477ca875ddac61759b69 to start an election
I20260709 12:29:55.569767 24165 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce"
dest_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:35686
I20260709 12:29:55.570384 24165 raft_consensus.cc:493] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:55.570827 24165 raft_consensus.cc:3060] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.571626 24256 consensus_queue.cc:1048] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.580621 24165 raft_consensus.cc:515] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.582935 24165 leader_election.cc:290] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:55.583185 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:55.583523 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "592beb1c07d145c3bd577dec75ba6dce" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:55.583837 24015 raft_consensus.cc:3060] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.584241 24091 raft_consensus.cc:3055] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:55.584795 24091 raft_consensus.cc:740] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:55.585564 24091 consensus_queue.cc:260] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.586923 24091 raft_consensus.cc:3060] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.587373 24256 consensus_queue.cc:1048] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:55.591743 24015 raft_consensus.cc:2468] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 2.
I20260709 12:29:55.592664 24091 raft_consensus.cc:2468] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 2.
I20260709 12:29:55.593312 24129 leader_election.cc:304] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:55.594434 24275 raft_consensus.cc:2804] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:55.594983 24275 raft_consensus.cc:697] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 2 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:55.595827 24275 consensus_queue.cc:237] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:55.602586 24257 consensus_queue.cc:1048] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:55.605448 23911 catalog_manager.cc:5697] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 1 to 2, leader changed from 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194) to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:55.674991 24267 raft_consensus.cc:993] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69: : Instructing follower f1bb62eb16d542529e317b9bbf71d0db to start an election
I20260709 12:29:55.675472 24267 raft_consensus.cc:1081] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Signalling peer f1bb62eb16d542529e317b9bbf71d0db to start an election
I20260709 12:29:55.677040 24016 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807"
dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
 from {username='slave'} at 127.0.0.1:57896
I20260709 12:29:55.677512 24016 raft_consensus.cc:493] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:55.677789 24016 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.683202 24016 raft_consensus.cc:515] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:55.685312 24016 leader_election.cc:290] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 2 election: Requested vote from peers d07747748415477ca875ddac61759b69 (127.8.137.195:34527), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:55.686350 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:55.686682 24091 raft_consensus.cc:1275] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Refusing update from remote peer f1bb62eb16d542529e317b9bbf71d0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:55.686476 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:55.687147 24090 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.687388 24165 raft_consensus.cc:3055] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:55.687804 24165 raft_consensus.cc:740] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:55.688199 24239 consensus_queue.cc:1048] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:55.688705 24165 consensus_queue.cc:260] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:55.690336 24165 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.698503 24165 raft_consensus.cc:2468] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 2.
I20260709 12:29:55.700562 23980 leader_election.cc:304] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:55.701495 24239 raft_consensus.cc:2804] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:55.702771 24165 raft_consensus.cc:1275] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Refusing update from remote peer f1bb62eb16d542529e317b9bbf71d0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:55.703086 24090 raft_consensus.cc:2468] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 2.
I20260709 12:29:55.704785 24238 consensus_queue.cc:1048] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:55.705915 24239 raft_consensus.cc:697] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:55.706955 24239 consensus_queue.cc:237] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:29:55.716477 23911 catalog_manager.cc:5697] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 1 to 2, leader changed from d07747748415477ca875ddac61759b69 (127.8.137.195) to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 2 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:55.886655 24016 raft_consensus.cc:1275] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:55.888113 24210 consensus_queue.cc:1048] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:55.896664 24165 raft_consensus.cc:3060] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.904373 24165 raft_consensus.cc:1275] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:55.905712 24210 consensus_queue.cc:1048] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:55.979278 24016 raft_consensus.cc:1275] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:55.980873 24210 consensus_queue.cc:1048] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:55.990765 24165 raft_consensus.cc:3060] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:55.998188 24165 raft_consensus.cc:1275] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:55.999585 24255 consensus_queue.cc:1048] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:56.015822 24090 raft_consensus.cc:1275] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:56.016937 24267 consensus_queue.cc:1048] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:56.025035 24016 raft_consensus.cc:1275] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:56.026629 24230 consensus_queue.cc:1048] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:56.043376 24090 raft_consensus.cc:1275] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Refusing update from remote peer f1bb62eb16d542529e317b9bbf71d0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:56.044546 24239 consensus_queue.cc:1048] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:56.052140 24165 raft_consensus.cc:1275] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Refusing update from remote peer f1bb62eb16d542529e317b9bbf71d0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:56.054356 24213 consensus_queue.cc:1048] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:56.110946 24165 raft_consensus.cc:1275] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Refusing update from remote peer f1bb62eb16d542529e317b9bbf71d0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:56.112323 24213 consensus_queue.cc:1048] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:56.123021 24090 raft_consensus.cc:1275] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Refusing update from remote peer f1bb62eb16d542529e317b9bbf71d0db: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:29:56.124962 24213 consensus_queue.cc:1048] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:29:58.309957 23911 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51448:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:29:58.313215 23911 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:29:58.361969 24001 tablet_service.cc:1511] Processing CreateTablet for tablet fbac3c7c05844f52bd9002dfa8bbc412 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:58.362946 24075 tablet_service.cc:1511] Processing CreateTablet for tablet fbac3c7c05844f52bd9002dfa8bbc412 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:58.363364 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbac3c7c05844f52bd9002dfa8bbc412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.364248 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbac3c7c05844f52bd9002dfa8bbc412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.364223 24150 tablet_service.cc:1511] Processing CreateTablet for tablet fbac3c7c05844f52bd9002dfa8bbc412 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:29:58.365525 24150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbac3c7c05844f52bd9002dfa8bbc412. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.366547 24077 tablet_service.cc:1511] Processing CreateTablet for tablet 528b90894334494c99bb554676db6492 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:58.367856 24077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 528b90894334494c99bb554676db6492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.377085 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 528b90894334494c99bb554676db6492 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:58.378571 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 528b90894334494c99bb554676db6492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.390100 24309 tablet_bootstrap.cc:492] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:58.389974 24152 tablet_service.cc:1511] Processing CreateTablet for tablet 528b90894334494c99bb554676db6492 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:29:58.392172 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 528b90894334494c99bb554676db6492. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.399338 24001 tablet_service.cc:1511] Processing CreateTablet for tablet 077f4a9eb0e24431a0ade022c0a16a46 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:58.401075 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 077f4a9eb0e24431a0ade022c0a16a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.407328 24308 tablet_bootstrap.cc:492] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:58.414845 24308 tablet_bootstrap.cc:654] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.416994 24075 tablet_service.cc:1511] Processing CreateTablet for tablet 077f4a9eb0e24431a0ade022c0a16a46 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:58.419145 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 077f4a9eb0e24431a0ade022c0a16a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.464088 24309 tablet_bootstrap.cc:654] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.463723 24077 tablet_service.cc:1511] Processing CreateTablet for tablet c8be97b034694a4690c8802b447f4e21 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:58.465317 24077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8be97b034694a4690c8802b447f4e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.468274 24310 tablet_bootstrap.cc:492] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:58.468734 24148 tablet_service.cc:1511] Processing CreateTablet for tablet 077f4a9eb0e24431a0ade022c0a16a46 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:29:58.470156 24148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 077f4a9eb0e24431a0ade022c0a16a46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.468734 24006 tablet_service.cc:1511] Processing CreateTablet for tablet c8be97b034694a4690c8802b447f4e21 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:58.474030 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8be97b034694a4690c8802b447f4e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.481799 24310 tablet_bootstrap.cc:654] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.482455 24152 tablet_service.cc:1511] Processing CreateTablet for tablet c8be97b034694a4690c8802b447f4e21 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:29:58.484301 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8be97b034694a4690c8802b447f4e21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.497712 24308 tablet_bootstrap.cc:492] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:58.499876 24308 ts_tablet_manager.cc:1403] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.093s	user 0.024s	sys 0.040s
I20260709 12:29:58.501971 24309 tablet_bootstrap.cc:492] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:58.502563 24309 ts_tablet_manager.cc:1403] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.113s	user 0.018s	sys 0.011s
I20260709 12:29:58.509716 24150 tablet_service.cc:1511] Processing CreateTablet for tablet 299a83e96d9c4834999f3e4f608a8f3f (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:58.511375 24001 tablet_service.cc:1511] Processing CreateTablet for tablet 299a83e96d9c4834999f3e4f608a8f3f (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:58.512910 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 299a83e96d9c4834999f3e4f608a8f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.512566 23999 tablet_service.cc:1511] Processing CreateTablet for tablet bfd1439fade84cda8117c094209726da (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:58.514304 23999 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd1439fade84cda8117c094209726da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.505833 24309 raft_consensus.cc:359] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.514966 24309 raft_consensus.cc:385] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.514782 24308 raft_consensus.cc:359] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.515264 24075 tablet_service.cc:1511] Processing CreateTablet for tablet 299a83e96d9c4834999f3e4f608a8f3f (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:29:58.516784 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 299a83e96d9c4834999f3e4f608a8f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.515285 24309 raft_consensus.cc:740] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.518620 24309 consensus_queue.cc:260] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.520035 24150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 299a83e96d9c4834999f3e4f608a8f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.515673 24308 raft_consensus.cc:385] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.524490 24308 raft_consensus.cc:740] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.530154 24308 consensus_queue.cc:260] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.542155 24078 tablet_service.cc:1511] Processing CreateTablet for tablet bfd1439fade84cda8117c094209726da (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:58.543262 24310 tablet_bootstrap.cc:492] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:58.543588 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd1439fade84cda8117c094209726da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.543816 24310 ts_tablet_manager.cc:1403] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.076s	user 0.011s	sys 0.008s
I20260709 12:29:58.561334 24154 tablet_service.cc:1511] Processing CreateTablet for tablet bfd1439fade84cda8117c094209726da (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:29:58.544687 24308 ts_tablet_manager.cc:1434] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
I20260709 12:29:58.562968 24154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfd1439fade84cda8117c094209726da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.563591 24308 tablet_bootstrap.cc:492] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
W20260709 12:29:58.566298 23962 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:29:58.540091 24309 ts_tablet_manager.cc:1434] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.001s
I20260709 12:29:58.573328 24073 tablet_service.cc:1511] Processing CreateTablet for tablet a017bacb4b9c420c834372f8af70b858 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:58.574739 24073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a017bacb4b9c420c834372f8af70b858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.576188 24006 tablet_service.cc:1511] Processing CreateTablet for tablet a017bacb4b9c420c834372f8af70b858 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:58.578804 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a017bacb4b9c420c834372f8af70b858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.573668 24308 tablet_bootstrap.cc:654] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.584911 24149 tablet_service.cc:1511] Processing CreateTablet for tablet a017bacb4b9c420c834372f8af70b858 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:29:58.585682 24005 tablet_service.cc:1511] Processing CreateTablet for tablet e63a629f649d4454b7469534a32760a0 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:58.587185 24149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a017bacb4b9c420c834372f8af70b858. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.587486 24005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63a629f649d4454b7469534a32760a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.606639 24309 tablet_bootstrap.cc:492] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:58.608027 24297 raft_consensus.cc:493] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:58.608618 24297 raft_consensus.cc:515] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.562054 24310 raft_consensus.cc:359] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.610579 24310 raft_consensus.cc:385] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.611035 24310 raft_consensus.cc:740] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.612092 24310 consensus_queue.cc:260] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.613515 24297 leader_election.cc:290] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.613374 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbac3c7c05844f52bd9002dfa8bbc412" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:58.614261 24090 raft_consensus.cc:2468] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:58.616122 24308 tablet_bootstrap.cc:492] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:58.616683 24308 ts_tablet_manager.cc:1403] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.008s
I20260709 12:29:58.616531 23979 leader_election.cc:304] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:58.613418 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbac3c7c05844f52bd9002dfa8bbc412" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:58.617379 24297 raft_consensus.cc:2804] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:58.617705 24164 raft_consensus.cc:2468] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:58.618784 24075 tablet_service.cc:1511] Processing CreateTablet for tablet e63a629f649d4454b7469534a32760a0 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:58.620126 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63a629f649d4454b7469534a32760a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.617790 24297 raft_consensus.cc:493] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:58.621986 24297 raft_consensus.cc:3060] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.622681 24153 tablet_service.cc:1511] Processing CreateTablet for tablet e63a629f649d4454b7469534a32760a0 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:29:58.623037 24151 tablet_service.cc:1511] Processing CreateTablet for tablet 1f54ac6f71234841a0ef335e4b4fca18 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:58.624155 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e63a629f649d4454b7469534a32760a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.624869 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f54ac6f71234841a0ef335e4b4fca18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.627665 24001 tablet_service.cc:1511] Processing CreateTablet for tablet 1f54ac6f71234841a0ef335e4b4fca18 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:58.628957 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f54ac6f71234841a0ef335e4b4fca18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.632728 24308 raft_consensus.cc:359] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.633592 24308 raft_consensus.cc:385] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.633883 24308 raft_consensus.cc:740] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.621459 24078 tablet_service.cc:1511] Processing CreateTablet for tablet 1f54ac6f71234841a0ef335e4b4fca18 (DEFAULT_TABLE table=soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:29:58.643548 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1f54ac6f71234841a0ef335e4b4fca18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:29:58.644508 24310 ts_tablet_manager.cc:1434] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.085s	user 0.005s	sys 0.002s
I20260709 12:29:58.647357 24310 tablet_bootstrap.cc:492] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:58.654654 24297 raft_consensus.cc:515] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.657903 24308 consensus_queue.cc:260] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.659005 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbac3c7c05844f52bd9002dfa8bbc412" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:58.659718 24090 raft_consensus.cc:3060] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.660386 24308 ts_tablet_manager.cc:1434] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.043s	user 0.008s	sys 0.000s
I20260709 12:29:58.661376 24297 leader_election.cc:290] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.669952 24308 tablet_bootstrap.cc:492] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:58.680781 24310 tablet_bootstrap.cc:654] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.694000 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbac3c7c05844f52bd9002dfa8bbc412" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:58.694774 24164 raft_consensus.cc:3060] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.697667 24308 tablet_bootstrap.cc:654] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.703474 24210 raft_consensus.cc:493] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:58.704236 24210 raft_consensus.cc:515] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.708083 24210 leader_election.cc:290] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.711301 24308 tablet_bootstrap.cc:492] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:58.711997 24308 ts_tablet_manager.cc:1403] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.009s
I20260709 12:29:58.712284 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "528b90894334494c99bb554676db6492" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:58.713012 24309 tablet_bootstrap.cc:654] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.715508 24308 raft_consensus.cc:359] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.716359 24308 raft_consensus.cc:385] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.716639 24308 raft_consensus.cc:740] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
W20260709 12:29:58.716488 24055 leader_election.cc:343] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:58.717278 24090 raft_consensus.cc:2468] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:58.717804 24308 consensus_queue.cc:260] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.718082 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "528b90894334494c99bb554676db6492" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:58.718899 23979 leader_election.cc:304] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:58.720240 24213 raft_consensus.cc:2804] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:58.720510 24308 ts_tablet_manager.cc:1434] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:29:58.721063 24213 raft_consensus.cc:697] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:58.721702 24308 tablet_bootstrap.cc:492] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:58.721985 24213 consensus_queue.cc:237] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.729779 24308 tablet_bootstrap.cc:654] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
W20260709 12:29:58.732308 24055 leader_election.cc:343] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d07747748415477ca875ddac61759b69 (127.8.137.195:34527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:58.738008 24055 leader_election.cc:304] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052; no voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:58.739814 24210 raft_consensus.cc:2749] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:58.745988 24164 raft_consensus.cc:2468] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:58.755972 23911 catalog_manager.cc:5697] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 1 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:58.757386 24310 tablet_bootstrap.cc:492] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:58.758221 24310 ts_tablet_manager.cc:1403] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.111s	user 0.023s	sys 0.041s
I20260709 12:29:58.766413 24310 raft_consensus.cc:359] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.767380 24310 raft_consensus.cc:385] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.767721 24310 raft_consensus.cc:740] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.768558 24310 consensus_queue.cc:260] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.769557 24308 tablet_bootstrap.cc:492] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:58.770011 24309 tablet_bootstrap.cc:492] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:58.770139 24308 ts_tablet_manager.cc:1403] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.049s	user 0.030s	sys 0.015s
I20260709 12:29:58.770612 24309 ts_tablet_manager.cc:1403] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.171s	user 0.010s	sys 0.009s
I20260709 12:29:58.771009 24310 ts_tablet_manager.cc:1434] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.012s	user 0.002s	sys 0.006s
I20260709 12:29:58.772194 24310 tablet_bootstrap.cc:492] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:58.773625 24308 raft_consensus.cc:359] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.773876 24309 raft_consensus.cc:359] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.774449 24308 raft_consensus.cc:385] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.774709 24309 raft_consensus.cc:385] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.774827 24308 raft_consensus.cc:740] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.775141 24309 raft_consensus.cc:740] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.775902 24308 consensus_queue.cc:260] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.776105 24309 consensus_queue.cc:260] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.780460 24310 tablet_bootstrap.cc:654] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.781129 24308 ts_tablet_manager.cc:1434] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:29:58.782269 24308 tablet_bootstrap.cc:492] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:58.792629 24309 ts_tablet_manager.cc:1434] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:29:58.793069 24310 tablet_bootstrap.cc:492] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:58.793697 24310 ts_tablet_manager.cc:1403] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 12:29:58.793857 24309 tablet_bootstrap.cc:492] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:58.792631 24308 tablet_bootstrap.cc:654] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.797961 24310 raft_consensus.cc:359] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.798949 24310 raft_consensus.cc:385] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.799238 24310 raft_consensus.cc:740] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.800177 24310 consensus_queue.cc:260] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.802743 24309 tablet_bootstrap.cc:654] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.802737 24310 ts_tablet_manager.cc:1434] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:29:58.810029 24310 tablet_bootstrap.cc:492] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:58.816617 24309 tablet_bootstrap.cc:492] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:58.817476 24309 ts_tablet_manager.cc:1403] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.005s
I20260709 12:29:58.819938 24310 tablet_bootstrap.cc:654] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.821239 24309 raft_consensus.cc:359] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.822245 24309 raft_consensus.cc:385] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.821768 24308 tablet_bootstrap.cc:492] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:58.822677 24309 raft_consensus.cc:740] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.822957 24308 ts_tablet_manager.cc:1403] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.005s
I20260709 12:29:58.823673 24309 consensus_queue.cc:260] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.826689 24309 ts_tablet_manager.cc:1434] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:29:58.826316 24308 raft_consensus.cc:359] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.827155 24308 raft_consensus.cc:385] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.827495 24308 raft_consensus.cc:740] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.827809 24309 tablet_bootstrap.cc:492] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:58.828276 24308 consensus_queue.cc:260] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.829348 24310 tablet_bootstrap.cc:492] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:58.829936 24310 ts_tablet_manager.cc:1403] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.008s
I20260709 12:29:58.830755 24308 ts_tablet_manager.cc:1434] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:29:58.831830 24308 tablet_bootstrap.cc:492] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:58.833463 24310 raft_consensus.cc:359] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.834367 24310 raft_consensus.cc:385] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.834709 24310 raft_consensus.cc:740] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.835705 24310 consensus_queue.cc:260] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.836274 24309 tablet_bootstrap.cc:654] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.838441 24310 ts_tablet_manager.cc:1434] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:58.839774 24310 tablet_bootstrap.cc:492] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:58.842221 24308 tablet_bootstrap.cc:654] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.842931 24213 raft_consensus.cc:493] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:58.843467 24213 raft_consensus.cc:515] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.845741 24213 leader_election.cc:290] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.847473 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077f4a9eb0e24431a0ade022c0a16a46" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:58.848310 24090 raft_consensus.cc:2468] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:58.852681 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077f4a9eb0e24431a0ade022c0a16a46" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:58.853732 24164 raft_consensus.cc:2468] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:58.855521 24310 tablet_bootstrap.cc:654] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.855624 23979 leader_election.cc:304] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:58.857523 24297 raft_consensus.cc:493] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:58.858199 24297 raft_consensus.cc:515] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.862875 24309 tablet_bootstrap.cc:492] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:58.863739 24309 ts_tablet_manager.cc:1403] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.036s	user 0.019s	sys 0.015s
I20260709 12:29:58.863623 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "528b90894334494c99bb554676db6492" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:58.864852 24163 raft_consensus.cc:2468] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:58.866106 24297 leader_election.cc:290] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.871558 24297 raft_consensus.cc:2804] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:58.872115 24297 raft_consensus.cc:493] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:58.872566 24297 raft_consensus.cc:3060] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.875470 23980 leader_election.cc:304] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:58.876446 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "528b90894334494c99bb554676db6492" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:58.877030 24213 raft_consensus.cc:2804] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:58.877482 24090 raft_consensus.cc:2468] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:58.877693 24213 raft_consensus.cc:493] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:58.881067 24213 raft_consensus.cc:3060] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.868520 24309 raft_consensus.cc:359] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.883381 24309 raft_consensus.cc:385] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.883947 24309 raft_consensus.cc:740] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.885157 24309 consensus_queue.cc:260] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.894743 24310 tablet_bootstrap.cc:492] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:58.895622 24310 ts_tablet_manager.cc:1403] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.056s	user 0.014s	sys 0.035s
I20260709 12:29:58.897068 24308 tablet_bootstrap.cc:492] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:58.899021 24308 ts_tablet_manager.cc:1403] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.067s	user 0.028s	sys 0.027s
I20260709 12:29:58.900070 24310 raft_consensus.cc:359] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.901176 24310 raft_consensus.cc:385] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.901577 24310 raft_consensus.cc:740] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.897984 24213 raft_consensus.cc:515] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.902552 24310 consensus_queue.cc:260] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.903237 24297 raft_consensus.cc:515] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.905572 24213 leader_election.cc:290] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.905987 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "528b90894334494c99bb554676db6492" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:58.906872 24164 raft_consensus.cc:3060] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.902422 24308 raft_consensus.cc:359] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.907917 24308 raft_consensus.cc:385] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.908277 24308 raft_consensus.cc:740] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.908428 24309 ts_tablet_manager.cc:1434] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.044s	user 0.017s	sys 0.010s
I20260709 12:29:58.909680 24309 tablet_bootstrap.cc:492] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:58.908510 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "528b90894334494c99bb554676db6492" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:58.910655 24090 raft_consensus.cc:3060] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.909313 24308 consensus_queue.cc:260] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.913369 24308 ts_tablet_manager.cc:1434] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 12:29:58.914414 24308 tablet_bootstrap.cc:492] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:58.918380 24090 raft_consensus.cc:2468] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:58.919353 24164 raft_consensus.cc:2468] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:58.920828 24310 ts_tablet_manager.cc:1434] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.001s
I20260709 12:29:58.921310 23980 leader_election.cc:304] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:58.922132 24310 tablet_bootstrap.cc:492] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:58.927281 24309 tablet_bootstrap.cc:654] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.930130 24310 tablet_bootstrap.cc:654] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.930249 24308 tablet_bootstrap.cc:654] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:58.935607 24328 raft_consensus.cc:2804] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:58.936349 24328 raft_consensus.cc:697] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:58.938436 24328 consensus_queue.cc:237] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.944700 24213 raft_consensus.cc:493] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:58.945303 24213 raft_consensus.cc:515] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.949018 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be97b034694a4690c8802b447f4e21" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:58.950124 24213 leader_election.cc:290] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.950788 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be97b034694a4690c8802b447f4e21" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:58.951462 24164 raft_consensus.cc:2468] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:58.949764 24090 raft_consensus.cc:2468] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
I20260709 12:29:58.952740 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077f4a9eb0e24431a0ade022c0a16a46" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:58.953378 24091 raft_consensus.cc:3060] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.960238 23979 leader_election.cc:304] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:58.961090 24091 raft_consensus.cc:2468] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:58.962307 23979 leader_election.cc:304] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:58.963162 24213 raft_consensus.cc:2804] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:58.963546 24213 raft_consensus.cc:493] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:58.963904 24213 raft_consensus.cc:3060] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.965972 23913 catalog_manager.cc:5697] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 1 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:58.971660 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "077f4a9eb0e24431a0ade022c0a16a46" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:58.972566 24164 raft_consensus.cc:3060] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:58.980758 24317 raft_consensus.cc:493] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:58.981529 24317 raft_consensus.cc:515] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.982095 24333 raft_consensus.cc:2804] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:58.983783 24317 leader_election.cc:290] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.985745 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:58.985983 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
W20260709 12:29:58.987474 24055 leader_election.cc:343] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d07747748415477ca875ddac61759b69 (127.8.137.195:34527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:58.988297 24297 leader_election.cc:290] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:58.989018 24309 tablet_bootstrap.cc:492] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:58.989580 24309 ts_tablet_manager.cc:1403] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.080s	user 0.020s	sys 0.016s
I20260709 12:29:58.990782 24164 raft_consensus.cc:2468] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:58.992947 24309 raft_consensus.cc:359] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:58.993803 24309 raft_consensus.cc:385] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:58.994138 24309 raft_consensus.cc:740] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:58.994947 24309 consensus_queue.cc:260] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
W20260709 12:29:58.995324 24055 leader_election.cc:343] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:58.996105 24055 leader_election.cc:304] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052; no voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:58.997220 24317 raft_consensus.cc:2749] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:58.997512 24309 ts_tablet_manager.cc:1434] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:58.998571 24309 tablet_bootstrap.cc:492] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:59.000808 24213 raft_consensus.cc:515] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.003734 24213 leader_election.cc:290] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.004120 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be97b034694a4690c8802b447f4e21" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:59.004804 24091 raft_consensus.cc:3060] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.004570 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8be97b034694a4690c8802b447f4e21" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:59.005527 24163 raft_consensus.cc:3060] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.007718 24309 tablet_bootstrap.cc:654] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.016732 24163 raft_consensus.cc:2468] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:59.018476 24308 tablet_bootstrap.cc:492] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:59.018370 23980 leader_election.cc:304] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:59.019621 24308 ts_tablet_manager.cc:1403] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.105s	user 0.020s	sys 0.018s
I20260709 12:29:59.020272 24309 tablet_bootstrap.cc:492] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:59.020882 24309 ts_tablet_manager.cc:1403] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.023s	user 0.006s	sys 0.011s
I20260709 12:29:59.020164 24213 raft_consensus.cc:2804] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.021467 24213 raft_consensus.cc:697] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:59.024417 24310 tablet_bootstrap.cc:492] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:59.022660 24213 consensus_queue.cc:237] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.025130 24310 ts_tablet_manager.cc:1403] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.103s	user 0.051s	sys 0.047s
I20260709 12:29:59.024698 24309 raft_consensus.cc:359] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.026922 24309 raft_consensus.cc:385] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.027228 24309 raft_consensus.cc:740] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.023280 24295 raft_consensus.cc:493] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.027987 24309 consensus_queue.cc:260] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.028179 24295 raft_consensus.cc:515] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.028739 24310 raft_consensus.cc:359] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.021890 24333 raft_consensus.cc:697] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:59.029764 24310 raft_consensus.cc:385] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.030205 24310 raft_consensus.cc:740] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.030448 24309 ts_tablet_manager.cc:1434] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:29:59.031064 24310 consensus_queue.cc:260] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.032490 24309 tablet_bootstrap.cc:492] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:59.032500 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd1439fade84cda8117c094209726da" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
W20260709 12:29:59.034976 24129 leader_election.cc:343] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:59.035936 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd1439fade84cda8117c094209726da" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:59.036724 24090 raft_consensus.cc:2468] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:59.038230 24128 leader_election.cc:304] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:59.030639 24333 consensus_queue.cc:237] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.030382 24295 leader_election.cc:290] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:59.040130 24309 tablet_bootstrap.cc:654] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.043067 24308 raft_consensus.cc:359] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.046037 24308 raft_consensus.cc:385] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.046324 24295 raft_consensus.cc:2804] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.047070 24295 raft_consensus.cc:493] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.047631 24295 raft_consensus.cc:3060] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.046476 24308 raft_consensus.cc:740] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.049307 24308 consensus_queue.cc:260] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.054064 24309 tablet_bootstrap.cc:492] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:59.054677 24309 ts_tablet_manager.cc:1403] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.003s
I20260709 12:29:59.057863 24309 raft_consensus.cc:359] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.058663 24309 raft_consensus.cc:385] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.059006 24309 raft_consensus.cc:740] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.059971 24309 consensus_queue.cc:260] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.061144 24328 raft_consensus.cc:493] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.061628 24328 raft_consensus.cc:515] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.062913 24309 ts_tablet_manager.cc:1434] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:29:59.064066 24309 tablet_bootstrap.cc:492] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:59.065165 24328 leader_election.cc:290] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.066030 24308 ts_tablet_manager.cc:1434] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.002s
I20260709 12:29:59.067502 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:59.068120 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:59.068142 24090 raft_consensus.cc:2468] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 0.
W20260709 12:29:59.069581 23980 leader_election.cc:343] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d07747748415477ca875ddac61759b69 (127.8.137.195:34527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:59.070796 23979 leader_election.cc:304] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: d07747748415477ca875ddac61759b69
I20260709 12:29:59.072841 24297 raft_consensus.cc:2804] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.073298 24297 raft_consensus.cc:493] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.073697 24297 raft_consensus.cc:3060] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.074911 24309 tablet_bootstrap.cc:654] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.077224 24308 tablet_bootstrap.cc:492] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:59.021479 24091 raft_consensus.cc:2468] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
W20260709 12:29:59.101936 24128 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.8.137.194:46737, user_credentials={real_user=slave}} blocked reactor thread for 63791.7us
I20260709 12:29:59.111776 24338 raft_consensus.cc:493] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.109680 24297 raft_consensus.cc:515] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.117354 24310 ts_tablet_manager.cc:1434] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.092s	user 0.040s	sys 0.040s
I20260709 12:29:59.110132 24334 raft_consensus.cc:493] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.118108 24334 raft_consensus.cc:515] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.118688 24310 tablet_bootstrap.cc:492] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:59.119917 24297 leader_election.cc:290] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.120288 24334 leader_election.cc:290] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.120023 23911 catalog_manager.cc:5697] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 1 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.121783 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f54ac6f71234841a0ef335e4b4fca18" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:59.121917 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:59.122941 24091 raft_consensus.cc:3060] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:29:59.123585 23979 leader_election.cc:343] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:59.117394 24338 raft_consensus.cc:515] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.126190 24310 tablet_bootstrap.cc:654] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.126979 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:59.129007 24338 leader_election.cc:290] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:59.113412 24295 raft_consensus.cc:515] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.131879 24295 leader_election.cc:290] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:59.131628 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63a629f649d4454b7469534a32760a0" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:59.132761 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f54ac6f71234841a0ef335e4b4fca18" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
W20260709 12:29:59.134466 23980 leader_election.cc:343] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d07747748415477ca875ddac61759b69 (127.8.137.195:34527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:59.136158 24089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd1439fade84cda8117c094209726da" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:59.136895 24089 raft_consensus.cc:3060] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.134513 24309 tablet_bootstrap.cc:492] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
W20260709 12:29:59.137171 23980 leader_election.cc:343] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d07747748415477ca875ddac61759b69 (127.8.137.195:34527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:59.137697 24309 ts_tablet_manager.cc:1403] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.036s
I20260709 12:29:59.137908 23980 leader_election.cc:304] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1bb62eb16d542529e317b9bbf71d0db; no voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69
I20260709 12:29:59.139057 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd1439fade84cda8117c094209726da" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:59.140687 24328 raft_consensus.cc:2749] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:59.141513 24308 tablet_bootstrap.cc:654] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.142854 24016 raft_consensus.cc:2468] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:59.140718 24309 raft_consensus.cc:359] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.144788 24309 raft_consensus.cc:385] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.145139 24309 raft_consensus.cc:740] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
W20260709 12:29:59.148149 24129 leader_election.cc:343] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:59.149231 24129 leader_election.cc:304] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:59.145975 24309 consensus_queue.cc:260] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.152256 24309 ts_tablet_manager.cc:1434] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.003s
I20260709 12:29:59.134718 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63a629f649d4454b7469534a32760a0" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:59.154022 24310 tablet_bootstrap.cc:492] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:59.154296 24295 raft_consensus.cc:2804] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.154836 24295 raft_consensus.cc:493] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.154933 24310 ts_tablet_manager.cc:1403] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.036s	user 0.009s	sys 0.010s
W20260709 12:29:59.154117 24128 leader_election.cc:343] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:59.155227 24295 raft_consensus.cc:3060] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.158792 24091 raft_consensus.cc:2468] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1bb62eb16d542529e317b9bbf71d0db in term 1.
I20260709 12:29:59.160421 24309 tablet_bootstrap.cc:492] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db: Bootstrap starting.
I20260709 12:29:59.161240 23979 leader_election.cc:304] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: d07747748415477ca875ddac61759b69
I20260709 12:29:59.162526 24334 raft_consensus.cc:2804] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.163072 24334 raft_consensus.cc:697] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Leader. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:59.164461 24334 consensus_queue.cc:237] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.167757 24309 tablet_bootstrap.cc:654] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.167949 23911 catalog_manager.cc:5697] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 1 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.171612 24310 raft_consensus.cc:359] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.172688 24310 raft_consensus.cc:385] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.173336 24310 raft_consensus.cc:740] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.173548 24308 tablet_bootstrap.cc:492] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:59.174113 24308 ts_tablet_manager.cc:1403] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.098s	user 0.021s	sys 0.020s
I20260709 12:29:59.174141 24310 consensus_queue.cc:260] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.176754 24310 ts_tablet_manager.cc:1434] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 12:29:59.177793 24310 tablet_bootstrap.cc:492] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:59.179831 24308 raft_consensus.cc:359] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.180500 24308 raft_consensus.cc:385] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.180783 24308 raft_consensus.cc:740] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.181412 24308 consensus_queue.cc:260] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.182953 24308 ts_tablet_manager.cc:1434] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:29:59.183851 24308 tablet_bootstrap.cc:492] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052: Bootstrap starting.
I20260709 12:29:59.192493 24328 consensus_queue.cc:1048] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.193800 24297 consensus_queue.cc:1048] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.200940 24309 tablet_bootstrap.cc:492] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db: No bootstrap required, opened a new log
I20260709 12:29:59.201521 24309 ts_tablet_manager.cc:1403] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db: Time spent bootstrapping tablet: real 0.041s	user 0.009s	sys 0.012s
I20260709 12:29:59.203634 24310 tablet_bootstrap.cc:654] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.204546 24309 raft_consensus.cc:359] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.204895 24089 raft_consensus.cc:2468] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:59.205318 24309 raft_consensus.cc:385] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.205629 24309 raft_consensus.cc:740] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.206642 24128 leader_election.cc:304] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:59.207546 24338 raft_consensus.cc:2804] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.208033 24338 raft_consensus.cc:697] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:59.208912 24338 consensus_queue.cc:237] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.206324 24309 consensus_queue.cc:260] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.213757 24309 ts_tablet_manager.cc:1434] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:29:59.215901 24308 tablet_bootstrap.cc:654] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.227202 24295 raft_consensus.cc:515] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.231521 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63a629f649d4454b7469534a32760a0" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:59.232410 24016 raft_consensus.cc:3060] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.232221 24089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63a629f649d4454b7469534a32760a0" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
W20260709 12:29:59.234503 24128 leader_election.cc:343] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:29:59.235812 24295 leader_election.cc:290] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:59.240712 24310 tablet_bootstrap.cc:492] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:59.241703 24310 ts_tablet_manager.cc:1403] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.064s	user 0.013s	sys 0.002s
I20260709 12:29:59.241755 23914 catalog_manager.cc:5697] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db reported cstate change: term changed from 0 to 1, leader changed from <none> to f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193). New cstate: current_term: 1 leader_uuid: "f1bb62eb16d542529e317b9bbf71d0db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.247186 24310 raft_consensus.cc:359] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.248764 24310 raft_consensus.cc:385] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.252489 24310 raft_consensus.cc:740] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.253767 24310 consensus_queue.cc:260] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.257297 24310 ts_tablet_manager.cc:1434] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.002s
I20260709 12:29:59.258761 24310 tablet_bootstrap.cc:492] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69: Bootstrap starting.
I20260709 12:29:59.259941 24295 raft_consensus.cc:493] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.260712 24295 raft_consensus.cc:515] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.266466 24089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f54ac6f71234841a0ef335e4b4fca18" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:59.267544 24089 raft_consensus.cc:2468] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:59.273042 24128 leader_election.cc:304] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: 
I20260709 12:29:59.274704 24337 raft_consensus.cc:2804] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.275234 24337 raft_consensus.cc:493] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.275653 24337 raft_consensus.cc:3060] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.279186 24317 raft_consensus.cc:493] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.280000 24317 raft_consensus.cc:515] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.284574 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a017bacb4b9c420c834372f8af70b858" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:59.285571 24015 raft_consensus.cc:2468] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 0.
I20260709 12:29:59.289165 24055 leader_election.cc:304] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:59.293565 24210 raft_consensus.cc:2804] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:29:59.294117 24210 raft_consensus.cc:493] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:29:59.294569 24210 raft_consensus.cc:3060] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.312405 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f54ac6f71234841a0ef335e4b4fca18" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:59.313251 24337 raft_consensus.cc:515] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.313596 24015 raft_consensus.cc:2468] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 0.
I20260709 12:29:59.315544 24337 leader_election.cc:290] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:59.315815 24317 leader_election.cc:290] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.313251 24210 raft_consensus.cc:515] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.269986 24295 leader_election.cc:290] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:59.321377 24089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f54ac6f71234841a0ef335e4b4fca18" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:29:59.322201 24089 raft_consensus.cc:3060] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.324986 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1f54ac6f71234841a0ef335e4b4fca18" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:59.325706 24015 raft_consensus.cc:3060] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.327566 24016 raft_consensus.cc:2468] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:59.330134 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a017bacb4b9c420c834372f8af70b858" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:59.331092 24164 raft_consensus.cc:2468] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 0.
I20260709 12:29:59.335930 24129 leader_election.cc:304] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:59.341039 23914 catalog_manager.cc:5697] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:59.342077 24015 raft_consensus.cc:2468] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:59.341806 24295 raft_consensus.cc:2804] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.345805 24295 raft_consensus.cc:697] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:59.346712 24295 consensus_queue.cc:237] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.345837 24129 leader_election.cc:304] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:59.348088 24337 raft_consensus.cc:2804] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.348587 24337 raft_consensus.cc:697] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:59.349360 24310 tablet_bootstrap.cc:654] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69: Neither blocks nor log segments found. Creating new log.
I20260709 12:29:59.349408 24337 consensus_queue.cc:237] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.350409 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a017bacb4b9c420c834372f8af70b858" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:59.351046 24015 raft_consensus.cc:3060] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.351462 24308 tablet_bootstrap.cc:492] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052: No bootstrap required, opened a new log
I20260709 12:29:59.351977 24308 ts_tablet_manager.cc:1403] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent bootstrapping tablet: real 0.168s	user 0.053s	sys 0.078s
I20260709 12:29:59.352129 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a017bacb4b9c420c834372f8af70b858" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:59.352761 24164 raft_consensus.cc:3060] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.354820 24308 raft_consensus.cc:359] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.355533 24308 raft_consensus.cc:385] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.355816 24308 raft_consensus.cc:740] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.356509 24308 consensus_queue.cc:260] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.358717 24308 ts_tablet_manager.cc:1434] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:29:59.359377 24210 leader_election.cc:290] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.362738 24089 raft_consensus.cc:2468] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 1.
I20260709 12:29:59.363819 24164 raft_consensus.cc:2468] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 1.
I20260709 12:29:59.365923 24055 leader_election.cc:304] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: 
I20260709 12:29:59.367316 24210 raft_consensus.cc:2804] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:29:59.367882 24210 raft_consensus.cc:697] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:59.368743 24210 consensus_queue.cc:237] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.375882 24310 tablet_bootstrap.cc:492] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69: No bootstrap required, opened a new log
I20260709 12:29:59.376538 24310 ts_tablet_manager.cc:1403] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69: Time spent bootstrapping tablet: real 0.118s	user 0.013s	sys 0.005s
I20260709 12:29:59.374467 24015 raft_consensus.cc:2468] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 1.
I20260709 12:29:59.380019 24310 raft_consensus.cc:359] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.380738 24310 raft_consensus.cc:385] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:29:59.381006 24310 raft_consensus.cc:740] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Initialized, Role: FOLLOWER
I20260709 12:29:59.381811 24310 consensus_queue.cc:260] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.383620 24310 ts_tablet_manager.cc:1434] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:29:59.385129 23911 catalog_manager.cc:5697] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:59.386751 23911 catalog_manager.cc:5697] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 0 to 1, leader changed from <none> to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:29:59.387732 24297 consensus_queue.cc:1048] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.396842 23914 catalog_manager.cc:5697] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 1 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.401525 24334 consensus_queue.cc:1048] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.433796 24334 consensus_queue.cc:1048] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.434119 24213 raft_consensus.cc:493] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.434697 24213 raft_consensus.cc:515] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.436831 24213 leader_election.cc:290] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.437600 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd1439fade84cda8117c094209726da" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:59.438308 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd1439fade84cda8117c094209726da" candidate_uuid: "f1bb62eb16d542529e317b9bbf71d0db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d07747748415477ca875ddac61759b69" is_pre_election: true
I20260709 12:29:59.438601 24090 raft_consensus.cc:2393] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1bb62eb16d542529e317b9bbf71d0db in current term 1: Already voted for candidate d07747748415477ca875ddac61759b69 in this term.
I20260709 12:29:59.440310 23979 leader_election.cc:304] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1bb62eb16d542529e317b9bbf71d0db; no voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69
I20260709 12:29:59.441694 24213 raft_consensus.cc:3060] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.450551 24213 raft_consensus.cc:2749] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:59.451660 24334 consensus_queue.cc:1048] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.458246 24213 consensus_queue.cc:1048] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.472302 24333 consensus_queue.cc:1048] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.550359 24295 raft_consensus.cc:493] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:29:59.551051 24295 raft_consensus.cc:515] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.553153 24295 leader_election.cc:290] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:29:59.553956 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db" is_pre_election: true
I20260709 12:29:59.554204 24089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052" is_pre_election: true
I20260709 12:29:59.554992 24089 raft_consensus.cc:2393] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d07747748415477ca875ddac61759b69 in current term 1: Already voted for candidate f1bb62eb16d542529e317b9bbf71d0db in this term.
I20260709 12:29:59.556335 24129 leader_election.cc:304] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d07747748415477ca875ddac61759b69; no voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:59.557111 24295 raft_consensus.cc:3060] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.562947 24295 raft_consensus.cc:2749] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:29:59.604686 24015 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f"
dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
mode: GRACEFUL
new_leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:57910
I20260709 12:29:59.605376 24015 raft_consensus.cc:606] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Received request to transfer leadership to TS 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:59.610158 24163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bfd1439fade84cda8117c094209726da"
dest_uuid: "d07747748415477ca875ddac61759b69"
mode: GRACEFUL
new_leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:35694
I20260709 12:29:59.610782 24163 raft_consensus.cc:606] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Received request to transfer leadership to TS 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:59.611438 24333 consensus_queue.cc:1048] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.622231 24333 consensus_queue.cc:1048] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.623669 24163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e63a629f649d4454b7469534a32760a0"
dest_uuid: "d07747748415477ca875ddac61759b69"
mode: GRACEFUL
new_leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:35694
I20260709 12:29:59.624383 24163 raft_consensus.cc:606] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Received request to transfer leadership to TS 48dae3b5183a4401b914bcb5dc1bf052
I20260709 12:29:59.630672 24213 raft_consensus.cc:993] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db: : Instructing follower 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:59.631161 24333 raft_consensus.cc:1081] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Signalling peer 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:59.632808 24089 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f"
dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:35896
I20260709 12:29:59.633332 24089 raft_consensus.cc:493] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:59.633736 24089 raft_consensus.cc:3060] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.640908 24089 raft_consensus.cc:515] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.642850 24089 leader_election.cc:290] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.643592 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:59.644088 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "299a83e96d9c4834999f3e4f608a8f3f" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:59.644333 24015 raft_consensus.cc:3055] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:59.644768 24163 raft_consensus.cc:3060] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.644786 24015 raft_consensus.cc:740] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:29:59.645828 24015 consensus_queue.cc:260] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.646904 24015 raft_consensus.cc:3060] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.652145 24163 raft_consensus.cc:2468] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 2.
I20260709 12:29:59.653347 24055 leader_election.cc:304] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: 
I20260709 12:29:59.653704 24015 raft_consensus.cc:2468] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 2.
I20260709 12:29:59.654270 24210 raft_consensus.cc:2804] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:59.654775 24210 raft_consensus.cc:697] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:59.655501 24210 consensus_queue.cc:237] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.662395 23911 catalog_manager.cc:5697] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 1 to 2, leader changed from f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193) to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.726270 24295 consensus_queue.cc:1048] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.733193 24337 raft_consensus.cc:993] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: : Instructing follower 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:59.733681 24337 raft_consensus.cc:1081] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Signalling peer 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:59.735939 24090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bfd1439fade84cda8117c094209726da"
dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:35882
I20260709 12:29:59.737015 24090 raft_consensus.cc:493] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:59.737425 24090 raft_consensus.cc:3060] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.740379 24295 consensus_queue.cc:1048] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.744902 24090 raft_consensus.cc:515] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.747596 24090 leader_election.cc:290] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.750087 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd1439fade84cda8117c094209726da" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:59.750154 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfd1439fade84cda8117c094209726da" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:59.750994 24163 raft_consensus.cc:3055] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:59.751323 24163 raft_consensus.cc:740] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:59.752287 24295 consensus_queue.cc:1048] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.752861 24163 consensus_queue.cc:260] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.753897 24163 raft_consensus.cc:3060] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.761688 24163 raft_consensus.cc:2468] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 2.
I20260709 12:29:59.762989 24055 leader_election.cc:304] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, d07747748415477ca875ddac61759b69; no voters: f1bb62eb16d542529e317b9bbf71d0db
I20260709 12:29:59.764673 24090 raft_consensus.cc:3060] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:29:59.764863 24210 raft_consensus.cc:2804] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:59.768154 24210 raft_consensus.cc:697] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:59.769007 24210 consensus_queue.cc:237] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.773701 24295 consensus_queue.cc:1048] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.777927 23911 catalog_manager.cc:5697] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 1 to 2, leader changed from d07747748415477ca875ddac61759b69 (127.8.137.195) to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.780975 24337 raft_consensus.cc:993] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69: : Instructing follower 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:59.781488 24295 raft_consensus.cc:1081] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Signalling peer 48dae3b5183a4401b914bcb5dc1bf052 to start an election
I20260709 12:29:59.782997 24090 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e63a629f649d4454b7469534a32760a0"
dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
 from {username='slave'} at 127.0.0.1:35882
I20260709 12:29:59.783514 24090 raft_consensus.cc:493] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:29:59.783880 24090 raft_consensus.cc:3060] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.790349 24090 raft_consensus.cc:515] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.792452 24090 leader_election.cc:290] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), d07747748415477ca875ddac61759b69 (127.8.137.195:34527)
I20260709 12:29:59.793102 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63a629f649d4454b7469534a32760a0" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:29:59.793464 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e63a629f649d4454b7469534a32760a0" candidate_uuid: "48dae3b5183a4401b914bcb5dc1bf052" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d07747748415477ca875ddac61759b69"
I20260709 12:29:59.793754 24014 raft_consensus.cc:3060] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.794127 24163 raft_consensus.cc:3055] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:29:59.794466 24163 raft_consensus.cc:740] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:29:59.795286 24163 consensus_queue.cc:260] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.796588 24163 raft_consensus.cc:3060] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:29:59.800060 24014 raft_consensus.cc:2468] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 2.
I20260709 12:29:59.801188 24055 leader_election.cc:304] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48dae3b5183a4401b914bcb5dc1bf052, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:29:59.801908 24210 raft_consensus.cc:2804] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:29:59.802306 24210 raft_consensus.cc:697] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Becoming Leader. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:29:59.802477 24163 raft_consensus.cc:2468] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48dae3b5183a4401b914bcb5dc1bf052 in term 2.
I20260709 12:29:59.803066 24210 consensus_queue.cc:237] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:29:59.810263 23911 catalog_manager.cc:5697] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 reported cstate change: term changed from 1 to 2, leader changed from d07747748415477ca875ddac61759b69 (127.8.137.195) to 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194). New cstate: current_term: 2 leader_uuid: "48dae3b5183a4401b914bcb5dc1bf052" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: UNKNOWN } } }
I20260709 12:29:59.834342 24295 consensus_queue.cc:1048] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.834748 24210 consensus_queue.cc:1048] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:29:59.848160 24295 consensus_queue.cc:1048] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:29:59.856570 24210 consensus_queue.cc:1048] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:00.118211 24014 raft_consensus.cc:1275] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:00.119486 24210 consensus_queue.cc:1048] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:00.127663 24164 raft_consensus.cc:1275] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:00.129845 24374 consensus_queue.cc:1048] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:00.220659 24014 raft_consensus.cc:3060] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:00.227945 24014 raft_consensus.cc:1275] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:00.229427 24210 consensus_queue.cc:1048] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:00.238137 24164 raft_consensus.cc:1275] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:00.240504 24374 consensus_queue.cc:1048] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:00.313015 24164 raft_consensus.cc:1275] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:00.314539 24374 consensus_queue.cc:1048] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:00.325418 24014 raft_consensus.cc:1275] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer 48dae3b5183a4401b914bcb5dc1bf052: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:00.326787 24210 consensus_queue.cc:1048] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:02.637966 23913 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51448:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:30:02.663918 23913 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:51448:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:30:02.814724  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:30:02.828385 23980 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:34527, user_credentials={real_user=slave}} blocked reactor thread for 60395.7us
I20260709 12:30:02.873701 24411 tablet.cc:1724] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:02.941905 23979 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:46737, user_credentials={real_user=slave}} blocked reactor thread for 57738.1us
I20260709 12:30:02.985601 24417 tablet.cc:1724] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:02.994949 24191 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:299a83e96d9c4834999f3e4f608a8f3f)
W20260709 12:30:03.004701 24191 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:299a83e96d9c4834999f3e4f608a8f3f)
W20260709 12:30:03.007085 24191 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:299a83e96d9c4834999f3e4f608a8f3f)
I20260709 12:30:03.054740 24436 tablet.cc:1724] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:03.124449 24431 log.cc:927] Time spent T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:30:03.139945 24442 tablet.cc:1724] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:03.173959 24443 log.cc:927] Time spent T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052: Append to log took a long time: real 0.100s	user 0.002s	sys 0.000s
I20260709 12:30:03.172683 24411 tablet.cc:1724] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:03.209954 24437 log.cc:927] Time spent T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: Append to log took a long time: real 0.176s	user 0.006s	sys 0.000s
I20260709 12:30:03.220203 24449 tablet.cc:1724] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.240763 24451 tablet.cc:1724] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.246891 24453 tablet.cc:1724] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.253700 24454 tablet.cc:1724] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.257787 24446 tablet.cc:1724] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.258069 24445 tablet.cc:1724] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.272351 24436 tablet.cc:1724] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.269632 24452 tablet.cc:1724] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.280406 24456 tablet.cc:1724] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.286804 24091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a017bacb4b9c420c834372f8af70b858"
dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
mode: GRACEFUL
new_leader_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:33450
I20260709 12:30:03.283638 24411 tablet.cc:1724] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.291841 24091 raft_consensus.cc:606] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Received request to transfer leadership to TS d07747748415477ca875ddac61759b69
I20260709 12:30:03.303586 24452 tablet.cc:1724] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.304816 24411 tablet.cc:1724] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.329160 24456 tablet.cc:1724] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.333281 24446 tablet.cc:1724] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.334052 24442 tablet.cc:1724] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.354472 24454 tablet.cc:1724] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.343389 24417 tablet.cc:1724] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.347613 24449 tablet.cc:1724] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.330499 24453 tablet.cc:1724] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.359319 24016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fbac3c7c05844f52bd9002dfa8bbc412"
dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
mode: GRACEFUL
new_leader_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:50362
I20260709 12:30:03.364333 24016 raft_consensus.cc:606] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Received request to transfer leadership to TS d07747748415477ca875ddac61759b69
I20260709 12:30:03.366153  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:30:03.367177  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:03.374270 24456 tablet.cc:1724] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.387015 24016 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807"
dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
mode: GRACEFUL
new_leader_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:50362
I20260709 12:30:03.387698 24016 raft_consensus.cc:606] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Received request to transfer leadership to TS d07747748415477ca875ddac61759b69
I20260709 12:30:03.389016 24445 tablet.cc:1724] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.389081 24454 tablet.cc:1724] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:03.403939 24418 raft_consensus.cc:993] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db: : Instructing follower d07747748415477ca875ddac61759b69 to start an election
I20260709 12:30:03.404927 24423 raft_consensus.cc:1081] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Signalling peer d07747748415477ca875ddac61759b69 to start an election
I20260709 12:30:03.407259 24091 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b893acd38c44433f970f8cf12906a2be"
dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
mode: GRACEFUL
new_leader_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:33450
I20260709 12:30:03.407717 24162 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807"
dest_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:35684
I20260709 12:30:03.408278 24091 raft_consensus.cc:606] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Received request to transfer leadership to TS d07747748415477ca875ddac61759b69
I20260709 12:30:03.408746 24162 raft_consensus.cc:493] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.409477 24162 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:03.418311  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:03.419380  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:03.425227 24162 raft_consensus.cc:515] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:30:03.429282 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:30:03.429649 24162 leader_election.cc:290] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 3 election: Requested vote from peers 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737), f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663)
I20260709 12:30:03.431131 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c4ea575ad024889ad3e11e44b44a807" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:30:03.431875 24016 raft_consensus.cc:3055] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:03.432240 24016 raft_consensus.cc:740] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:30:03.433336 24016 consensus_queue.cc:260] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:30:03.430109 24091 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:03.434975 24016 raft_consensus.cc:3060] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:03.443509 24016 raft_consensus.cc:2468] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 3.
I20260709 12:30:03.443881 23912 catalog_manager.cc:6242] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1: soft_deleted_table [id=5816930921e24de1b3b6defcd8807fb8] alter complete (version 1)
I20260709 12:30:03.445173 24129 leader_election.cc:304] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:30:03.445281 24091 raft_consensus.cc:2468] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 3.
I20260709 12:30:03.446573 24433 raft_consensus.cc:2804] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:03.447085 24433 raft_consensus.cc:697] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 3 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:30:03.448606 24433 consensus_queue.cc:237] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } }
I20260709 12:30:03.462478 23914 catalog_manager.cc:5697] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 2 to 3, leader changed from f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193) to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:03.502300 24420 raft_consensus.cc:993] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052: : Instructing follower d07747748415477ca875ddac61759b69 to start an election
I20260709 12:30:03.503000 24424 raft_consensus.cc:1081] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Signalling peer d07747748415477ca875ddac61759b69 to start an election
I20260709 12:30:03.505041 24159 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b893acd38c44433f970f8cf12906a2be"
dest_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:35686
I20260709 12:30:03.505661 24159 raft_consensus.cc:493] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.506094 24159 raft_consensus.cc:3060] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.512696 24159 raft_consensus.cc:515] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.514935 24159 leader_election.cc:290] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:30:03.515820 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b893acd38c44433f970f8cf12906a2be" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:30:03.516094 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b893acd38c44433f970f8cf12906a2be" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:30:03.516542 24016 raft_consensus.cc:3060] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.516757 24091 raft_consensus.cc:3055] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.517189 24091 raft_consensus.cc:740] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:30:03.518245 24091 consensus_queue.cc:260] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.519618 24091 raft_consensus.cc:3060] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.523007 24016 raft_consensus.cc:2468] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 2.
I20260709 12:30:03.524260 24129 leader_election.cc:304] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:30:03.525007 24433 raft_consensus.cc:2804] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.525533 24433 raft_consensus.cc:697] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 2 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:30:03.526891 24091 raft_consensus.cc:2468] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 2.
I20260709 12:30:03.526564 24433 consensus_queue.cc:237] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.535374 23914 catalog_manager.cc:5697] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 1 to 2, leader changed from 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194) to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:03.624825 24424 raft_consensus.cc:993] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052: : Instructing follower d07747748415477ca875ddac61759b69 to start an election
I20260709 12:30:03.625422 24420 raft_consensus.cc:1081] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Signalling peer d07747748415477ca875ddac61759b69 to start an election
I20260709 12:30:03.627262 24159 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a017bacb4b9c420c834372f8af70b858"
dest_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:35686
I20260709 12:30:03.627799 24159 raft_consensus.cc:493] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.628130 24159 raft_consensus.cc:3060] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.633925 24159 raft_consensus.cc:515] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.635746 24159 leader_election.cc:290] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:30:03.636615 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a017bacb4b9c420c834372f8af70b858" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:30:03.636809 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a017bacb4b9c420c834372f8af70b858" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:30:03.637404 24016 raft_consensus.cc:3060] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.637485 24091 raft_consensus.cc:3055] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.637930 24091 raft_consensus.cc:740] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 48dae3b5183a4401b914bcb5dc1bf052, State: Running, Role: LEADER
I20260709 12:30:03.638697 24091 consensus_queue.cc:260] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.639958 24091 raft_consensus.cc:3060] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.643106 24016 raft_consensus.cc:2468] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 2.
I20260709 12:30:03.644418 24129 leader_election.cc:304] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:30:03.645555 24433 raft_consensus.cc:2804] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.646587 24091 raft_consensus.cc:2468] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 2.
I20260709 12:30:03.646921 24433 raft_consensus.cc:697] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 2 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:30:03.648698 24433 consensus_queue.cc:237] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.656385 23914 catalog_manager.cc:5697] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 1 to 2, leader changed from 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194) to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:03.674015 24423 raft_consensus.cc:993] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db: : Instructing follower d07747748415477ca875ddac61759b69 to start an election
I20260709 12:30:03.674666 24419 raft_consensus.cc:1081] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Signalling peer d07747748415477ca875ddac61759b69 to start an election
I20260709 12:30:03.676551 24159 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fbac3c7c05844f52bd9002dfa8bbc412"
dest_uuid: "d07747748415477ca875ddac61759b69"
 from {username='slave'} at 127.0.0.1:35684
I20260709 12:30:03.677318 24159 raft_consensus.cc:493] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:03.677743 24159 raft_consensus.cc:3060] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.683871 24159 raft_consensus.cc:515] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.685643 24159 leader_election.cc:290] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 2 election: Requested vote from peers f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663), 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737)
I20260709 12:30:03.686573 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbac3c7c05844f52bd9002dfa8bbc412" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f1bb62eb16d542529e317b9bbf71d0db"
I20260709 12:30:03.686969 24086 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbac3c7c05844f52bd9002dfa8bbc412" candidate_uuid: "d07747748415477ca875ddac61759b69" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "48dae3b5183a4401b914bcb5dc1bf052"
I20260709 12:30:03.687371 24016 raft_consensus.cc:3055] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:03.687633 24086 raft_consensus.cc:3060] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.687772 24016 raft_consensus.cc:740] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1bb62eb16d542529e317b9bbf71d0db, State: Running, Role: LEADER
I20260709 12:30:03.688669 24016 consensus_queue.cc:260] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.689733 24016 raft_consensus.cc:3060] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:03.695883 24086 raft_consensus.cc:2468] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 2.
I20260709 12:30:03.695983 24016 raft_consensus.cc:2468] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d07747748415477ca875ddac61759b69 in term 2.
I20260709 12:30:03.697113 24129 leader_election.cc:304] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d07747748415477ca875ddac61759b69, f1bb62eb16d542529e317b9bbf71d0db; no voters: 
I20260709 12:30:03.697806 24433 raft_consensus.cc:2804] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:03.698314 24433 raft_consensus.cc:697] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 2 LEADER]: Becoming Leader. State: Replica: d07747748415477ca875ddac61759b69, State: Running, Role: LEADER
I20260709 12:30:03.699180 24433 consensus_queue.cc:237] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } }
I20260709 12:30:03.705927 23914 catalog_manager.cc:5697] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 reported cstate change: term changed from 1 to 2, leader changed from f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193) to d07747748415477ca875ddac61759b69 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "d07747748415477ca875ddac61759b69" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d07747748415477ca875ddac61759b69" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 34527 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:03.925191 24016 raft_consensus.cc:1275] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.926286 24433 consensus_queue.cc:1048] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:03.942760 24086 raft_consensus.cc:1275] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:03.944453 24433 consensus_queue.cc:1048] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:03.991297 24086 raft_consensus.cc:1275] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 3 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:03.992738 24433 consensus_queue.cc:1048] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:04.002897 24016 raft_consensus.cc:1275] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 3 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:04.006868 24433 consensus_queue.cc:1048] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:04.085497 24086 raft_consensus.cc:1275] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:30:04.087009 24464 consensus_queue.cc:1048] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:04.094892 24016 raft_consensus.cc:1275] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:30:04.099157 24470 consensus_queue.cc:1048] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:30:04.118323 24015 raft_consensus.cc:1275] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:30:04.119601 24470 consensus_queue.cc:1048] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1bb62eb16d542529e317b9bbf71d0db" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 41663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:04.128019 24086 raft_consensus.cc:1275] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Refusing update from remote peer d07747748415477ca875ddac61759b69: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:30:04.130190 24470 consensus_queue.cc:1048] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [LEADER]: Connected to new peer: Peer: permanent_uuid: "48dae3b5183a4401b914bcb5dc1bf052" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 46737 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:04.421774  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:04.426916  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:04.427443  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:04.433264  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:04.433620  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:05.434739  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:05.440440  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:05.440862  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:05.445850  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:05.446271  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:06.447410  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:06.452440  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:06.452947  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:06.458515  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:06.459049  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:07.460134  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:07.465927  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:07.466549  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:07.471544  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:07.472059  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:08.473043  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:08.478132  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:08.478653  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:08.484061  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:08.484582  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:09.485682  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:09.491073  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:09.491516  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:09.496289  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:09.496680  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:10.497573  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:10.503137  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:10.503621  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:10.508876  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:10.509385  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:11.510681  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:11.519461  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:11.519968  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:11.525861  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:11.526319  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:12.527617  8743 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:30:12.532560  8743 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:30:12.533038  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:12.538275  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:12.538851  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:13.603876  8743 tablet_server.cc:179] TabletServer@127.8.137.193:0 shutting down...
W20260709 12:30:13.608201 24129 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 6 similar messages]
W20260709 12:30:13.612573 24129 consensus_peers.cc:597] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.649835  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:13.650875  8743 tablet_replica.cc:333] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.652376  8743 raft_consensus.cc:2243] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.653499  8743 raft_consensus.cc:2272] T 1f54ac6f71234841a0ef335e4b4fca18 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:13.656222 24055 consensus_peers.cc:597] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.657461  8743 tablet_replica.cc:333] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.658177  8743 raft_consensus.cc:2243] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.658838  8743 raft_consensus.cc:2272] T e63a629f649d4454b7469534a32760a0 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.662662  8743 tablet_replica.cc:333] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.663403  8743 raft_consensus.cc:2243] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.663954  8743 raft_consensus.cc:2272] T 299a83e96d9c4834999f3e4f608a8f3f P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.667814  8743 tablet_replica.cc:333] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.668853  8743 raft_consensus.cc:2243] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.669672  8743 raft_consensus.cc:2272] T a017bacb4b9c420c834372f8af70b858 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.672808  8743 tablet_replica.cc:333] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.673431  8743 raft_consensus.cc:2243] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.674360  8743 raft_consensus.cc:2272] T 528b90894334494c99bb554676db6492 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.677088  8743 tablet_replica.cc:333] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.677760  8743 raft_consensus.cc:2243] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.678443  8743 raft_consensus.cc:2272] T bfd1439fade84cda8117c094209726da P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.681249  8743 tablet_replica.cc:333] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.681921  8743 raft_consensus.cc:2243] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.682436  8743 raft_consensus.cc:2272] T 8c4ea575ad024889ad3e11e44b44a807 P f1bb62eb16d542529e317b9bbf71d0db [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.685338  8743 tablet_replica.cc:333] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.686030  8743 raft_consensus.cc:2243] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.687264  8743 raft_consensus.cc:2272] T 3d69560971dd4d919a5c4af837200db6 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.689790  8743 tablet_replica.cc:333] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.690376  8743 raft_consensus.cc:2243] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.691643  8743 raft_consensus.cc:2272] T c81e471dc3cd4be0b78ce2710eba8ddf P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:13.694633 24129 consensus_peers.cc:597] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.695133  8743 tablet_replica.cc:333] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.695741  8743 raft_consensus.cc:2243] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.696257  8743 raft_consensus.cc:2272] T fbac3c7c05844f52bd9002dfa8bbc412 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.699263  8743 tablet_replica.cc:333] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.699962  8743 raft_consensus.cc:2243] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.700512  8743 raft_consensus.cc:2272] T 6bcbdcd3912640e780d1d59ad1ab5994 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.704516  8743 tablet_replica.cc:333] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
W20260709 12:30:13.704564 24129 consensus_peers.cc:597] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.705464  8743 raft_consensus.cc:2243] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.706843  8743 raft_consensus.cc:2272] T c8be97b034694a4690c8802b447f4e21 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:13.707199 24129 consensus_peers.cc:597] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.710258  8743 tablet_replica.cc:333] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.711314  8743 raft_consensus.cc:2243] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.712021  8743 raft_consensus.cc:2272] T aeb03d7b92454ec6a30cf0313afa26a3 P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.715252  8743 tablet_replica.cc:333] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.715896  8743 raft_consensus.cc:2243] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.716567  8743 raft_consensus.cc:2272] T 10c8e3c9cd4648f797f7b1e459951d8f P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.720220  8743 tablet_replica.cc:333] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
W20260709 12:30:13.720211 24055 consensus_peers.cc:597] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.721186  8743 raft_consensus.cc:2243] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.721911  8743 raft_consensus.cc:2272] T 592beb1c07d145c3bd577dec75ba6dce P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.724653  8743 tablet_replica.cc:333] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.725250  8743 raft_consensus.cc:2243] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.726145  8743 raft_consensus.cc:2272] T 077f4a9eb0e24431a0ade022c0a16a46 P f1bb62eb16d542529e317b9bbf71d0db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.728857  8743 tablet_replica.cc:333] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.729439  8743 raft_consensus.cc:2243] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.729980  8743 raft_consensus.cc:2272] T b893acd38c44433f970f8cf12906a2be P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.733028  8743 tablet_replica.cc:333] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db: stopping tablet replica
I20260709 12:30:13.733767  8743 raft_consensus.cc:2243] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.735251  8743 raft_consensus.cc:2272] T 89ffc6d0964a4b8f95239517fefa16bf P f1bb62eb16d542529e317b9bbf71d0db [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:13.762138 24055 consensus_peers.cc:597] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.791008  8743 tablet_server.cc:196] TabletServer@127.8.137.193:0 shutdown complete.
W20260709 12:30:13.819166 24055 consensus_peers.cc:597] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:41663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:13.848661 24129 consensus_peers.cc:597] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:41663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.853312  8743 tablet_server.cc:179] TabletServer@127.8.137.194:0 shutting down...
I20260709 12:30:13.898051  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:13.899101  8743 tablet_replica.cc:333] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.900345  8743 raft_consensus.cc:2243] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.901499  8743 raft_consensus.cc:2272] T e63a629f649d4454b7469534a32760a0 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.905350  8743 tablet_replica.cc:333] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.906075  8743 raft_consensus.cc:2243] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.906710  8743 raft_consensus.cc:2272] T 1f54ac6f71234841a0ef335e4b4fca18 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.909231  8743 tablet_replica.cc:333] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.909826  8743 raft_consensus.cc:2243] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.911093  8743 raft_consensus.cc:2272] T 299a83e96d9c4834999f3e4f608a8f3f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.914768  8743 tablet_replica.cc:333] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.915472  8743 raft_consensus.cc:2243] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:13.915992 24055 consensus_peers.cc:597] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:41663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.916251  8743 raft_consensus.cc:2272] T c8be97b034694a4690c8802b447f4e21 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.919237  8743 tablet_replica.cc:333] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.919806  8743 raft_consensus.cc:2243] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.920768  8743 raft_consensus.cc:2272] T 10c8e3c9cd4648f797f7b1e459951d8f P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.923537  8743 tablet_replica.cc:333] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.924350  8743 raft_consensus.cc:2243] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.925666  8743 raft_consensus.cc:2272] T aeb03d7b92454ec6a30cf0313afa26a3 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:13.927770 24128 consensus_peers.cc:597] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 -> Peer 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737): Couldn't send request to peer 48dae3b5183a4401b914bcb5dc1bf052. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:13.928857  8743 tablet_replica.cc:333] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.929582  8743 raft_consensus.cc:2243] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.930123  8743 raft_consensus.cc:2272] T 592beb1c07d145c3bd577dec75ba6dce P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.932569  8743 tablet_replica.cc:333] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.933156  8743 raft_consensus.cc:2243] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.933599  8743 raft_consensus.cc:2272] T c81e471dc3cd4be0b78ce2710eba8ddf P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.936060  8743 tablet_replica.cc:333] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.936610  8743 raft_consensus.cc:2243] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.937095  8743 raft_consensus.cc:2272] T 3d69560971dd4d919a5c4af837200db6 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.939543  8743 tablet_replica.cc:333] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.940099  8743 raft_consensus.cc:2243] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.940564  8743 raft_consensus.cc:2272] T 89ffc6d0964a4b8f95239517fefa16bf P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.943081  8743 tablet_replica.cc:333] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.943632  8743 raft_consensus.cc:2243] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.944099  8743 raft_consensus.cc:2272] T 077f4a9eb0e24431a0ade022c0a16a46 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.946333  8743 tablet_replica.cc:333] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.946954  8743 raft_consensus.cc:2243] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.947391  8743 raft_consensus.cc:2272] T b893acd38c44433f970f8cf12906a2be P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.949716  8743 tablet_replica.cc:333] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.950196  8743 raft_consensus.cc:2243] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.950749  8743 raft_consensus.cc:2272] T fbac3c7c05844f52bd9002dfa8bbc412 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.952968  8743 tablet_replica.cc:333] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.953460  8743 raft_consensus.cc:2243] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.954294  8743 raft_consensus.cc:2272] T 6bcbdcd3912640e780d1d59ad1ab5994 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.956643  8743 tablet_replica.cc:333] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.957260  8743 raft_consensus.cc:2243] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.957799  8743 raft_consensus.cc:2272] T a017bacb4b9c420c834372f8af70b858 P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.960239  8743 tablet_replica.cc:333] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.960852  8743 raft_consensus.cc:2243] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.961405  8743 raft_consensus.cc:2272] T 528b90894334494c99bb554676db6492 P 48dae3b5183a4401b914bcb5dc1bf052 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.963848  8743 tablet_replica.cc:333] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.964473  8743 raft_consensus.cc:2243] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:13.965415  8743 raft_consensus.cc:2272] T bfd1439fade84cda8117c094209726da P 48dae3b5183a4401b914bcb5dc1bf052 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:13.967646  8743 tablet_replica.cc:333] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052: stopping tablet replica
I20260709 12:30:13.968173  8743 raft_consensus.cc:2243] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:13.968721  8743 raft_consensus.cc:2272] T 8c4ea575ad024889ad3e11e44b44a807 P 48dae3b5183a4401b914bcb5dc1bf052 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:13.994028 24128 consensus_peers.cc:597] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 -> Peer 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737): Couldn't send request to peer 48dae3b5183a4401b914bcb5dc1bf052. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:14.017707  8743 tablet_server.cc:196] TabletServer@127.8.137.194:0 shutdown complete.
I20260709 12:30:14.065653  8743 tablet_server.cc:179] TabletServer@127.8.137.195:0 shutting down...
W20260709 12:30:14.088398 24129 consensus_peers.cc:597] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 -> Peer f1bb62eb16d542529e317b9bbf71d0db (127.8.137.193:41663): Couldn't send request to peer f1bb62eb16d542529e317b9bbf71d0db. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:41663: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:14.114995  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:14.115955  8743 tablet_replica.cc:333] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.117221  8743 raft_consensus.cc:2243] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:14.118568  8743 raft_consensus.cc:2272] T 1f54ac6f71234841a0ef335e4b4fca18 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:14.119835 24128 consensus_peers.cc:597] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 -> Peer 48dae3b5183a4401b914bcb5dc1bf052 (127.8.137.194:46737): Couldn't send request to peer 48dae3b5183a4401b914bcb5dc1bf052. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:46737: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:14.121639  8743 tablet_replica.cc:333] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.122385  8743 raft_consensus.cc:2243] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.123028  8743 raft_consensus.cc:2272] T e63a629f649d4454b7469534a32760a0 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.125393  8743 tablet_replica.cc:333] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.126021  8743 raft_consensus.cc:2243] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.126538  8743 raft_consensus.cc:2272] T 299a83e96d9c4834999f3e4f608a8f3f P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.128998  8743 tablet_replica.cc:333] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.129647  8743 raft_consensus.cc:2243] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.130133  8743 raft_consensus.cc:2272] T c81e471dc3cd4be0b78ce2710eba8ddf P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.132473  8743 tablet_replica.cc:333] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.133085  8743 raft_consensus.cc:2243] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:14.133958  8743 raft_consensus.cc:2272] T 592beb1c07d145c3bd577dec75ba6dce P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.136233  8743 tablet_replica.cc:333] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.136852  8743 raft_consensus.cc:2243] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.137370  8743 raft_consensus.cc:2272] T 077f4a9eb0e24431a0ade022c0a16a46 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.139796  8743 tablet_replica.cc:333] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.140429  8743 raft_consensus.cc:2243] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:14.141305  8743 raft_consensus.cc:2272] T b893acd38c44433f970f8cf12906a2be P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.143656  8743 tablet_replica.cc:333] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.144269  8743 raft_consensus.cc:2243] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.144826  8743 raft_consensus.cc:2272] T 89ffc6d0964a4b8f95239517fefa16bf P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.147161  8743 tablet_replica.cc:333] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.147743  8743 raft_consensus.cc:2243] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.148330  8743 raft_consensus.cc:2272] T c8be97b034694a4690c8802b447f4e21 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.151463  8743 tablet_replica.cc:333] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.152511  8743 raft_consensus.cc:2243] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.152968  8743 raft_consensus.cc:2272] T 10c8e3c9cd4648f797f7b1e459951d8f P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.155148  8743 tablet_replica.cc:333] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.155710  8743 raft_consensus.cc:2243] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.156318  8743 raft_consensus.cc:2272] T aeb03d7b92454ec6a30cf0313afa26a3 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.159576  8743 tablet_replica.cc:333] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.160377  8743 raft_consensus.cc:2243] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.160971  8743 raft_consensus.cc:2272] T 3d69560971dd4d919a5c4af837200db6 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.164304  8743 tablet_replica.cc:333] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.165180  8743 raft_consensus.cc:2243] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:14.166205  8743 raft_consensus.cc:2272] T fbac3c7c05844f52bd9002dfa8bbc412 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.168516  8743 tablet_replica.cc:333] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.169083  8743 raft_consensus.cc:2243] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.169518  8743 raft_consensus.cc:2272] T 6bcbdcd3912640e780d1d59ad1ab5994 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.171795  8743 tablet_replica.cc:333] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.172350  8743 raft_consensus.cc:2243] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:14.173244  8743 raft_consensus.cc:2272] T a017bacb4b9c420c834372f8af70b858 P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.175587  8743 tablet_replica.cc:333] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.176141  8743 raft_consensus.cc:2243] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.176692  8743 raft_consensus.cc:2272] T 528b90894334494c99bb554676db6492 P d07747748415477ca875ddac61759b69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.178994  8743 tablet_replica.cc:333] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.179527  8743 raft_consensus.cc:2243] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:14.180023  8743 raft_consensus.cc:2272] T bfd1439fade84cda8117c094209726da P d07747748415477ca875ddac61759b69 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.182148  8743 tablet_replica.cc:333] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69: stopping tablet replica
I20260709 12:30:14.182729  8743 raft_consensus.cc:2243] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:14.183625  8743 raft_consensus.cc:2272] T 8c4ea575ad024889ad3e11e44b44a807 P d07747748415477ca875ddac61759b69 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.227967  8743 tablet_server.cc:196] TabletServer@127.8.137.195:0 shutdown complete.
I20260709 12:30:14.268738  8743 master.cc:562] Master@127.8.137.254:33231 shutting down...
I20260709 12:30:14.297152  8743 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:14.297910  8743 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:14.298327  8743 tablet_replica.cc:333] T 00000000000000000000000000000000 P fc38cce3b3424ee58c6ab26ec63a51d1: stopping tablet replica
I20260709 12:30:14.322538  8743 master.cc:584] Master@127.8.137.254:33231 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21011 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:30:14.383500  8743 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.8.137.254:42857
I20260709 12:30:14.384697  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:14.391731 24501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:14.391981 24502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:14.392925 24504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:14.393292  8743 server_base.cc:1061] running on GCE node
I20260709 12:30:14.394373  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:14.394559  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:14.395107  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600214395065 us; error 0 us; skew 500 ppm
I20260709 12:30:14.395674  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:14.398334  8743 webserver.cc:533] Webserver started at http://127.8.137.254:42675/ using document root <none> and password file <none>
I20260709 12:30:14.398886  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:14.399075  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:14.399386  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:14.400612  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/master-0-root/instance:
uuid: "98d9a8c9d30646e0b6b93bbd28f195a3"
format_stamp: "Formatted at 2026-07-09 12:30:14 on dist-test-slave-kpwv"
I20260709 12:30:14.405351  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:30:14.409407 24509 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.410494  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:30:14.410885  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/master-0-root
uuid: "98d9a8c9d30646e0b6b93bbd28f195a3"
format_stamp: "Formatted at 2026-07-09 12:30:14 on dist-test-slave-kpwv"
I20260709 12:30:14.411160  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:14.434043  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:14.435406  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:14.485664  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.254:42857
I20260709 12:30:14.485684 24560 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.254:42857 every 8 connection(s)
I20260709 12:30:14.490726 24561 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:14.503968 24561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3: Bootstrap starting.
I20260709 12:30:14.509078 24561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:14.513617 24561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3: No bootstrap required, opened a new log
I20260709 12:30:14.516040 24561 raft_consensus.cc:359] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9a8c9d30646e0b6b93bbd28f195a3" member_type: VOTER }
I20260709 12:30:14.516499 24561 raft_consensus.cc:385] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:14.516680 24561 raft_consensus.cc:740] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98d9a8c9d30646e0b6b93bbd28f195a3, State: Initialized, Role: FOLLOWER
I20260709 12:30:14.517273 24561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9a8c9d30646e0b6b93bbd28f195a3" member_type: VOTER }
I20260709 12:30:14.517715 24561 raft_consensus.cc:399] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:30:14.517925 24561 raft_consensus.cc:493] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:30:14.518147 24561 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:14.523157 24561 raft_consensus.cc:515] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9a8c9d30646e0b6b93bbd28f195a3" member_type: VOTER }
I20260709 12:30:14.523671 24561 leader_election.cc:304] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98d9a8c9d30646e0b6b93bbd28f195a3; no voters: 
I20260709 12:30:14.524767 24561 leader_election.cc:290] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:30:14.525207 24564 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:14.526744 24564 raft_consensus.cc:697] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 1 LEADER]: Becoming Leader. State: Replica: 98d9a8c9d30646e0b6b93bbd28f195a3, State: Running, Role: LEADER
I20260709 12:30:14.527449 24564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9a8c9d30646e0b6b93bbd28f195a3" member_type: VOTER }
I20260709 12:30:14.527878 24561 sys_catalog.cc:565] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:30:14.532596 24566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98d9a8c9d30646e0b6b93bbd28f195a3. Latest consensus state: current_term: 1 leader_uuid: "98d9a8c9d30646e0b6b93bbd28f195a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9a8c9d30646e0b6b93bbd28f195a3" member_type: VOTER } }
I20260709 12:30:14.532977 24565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98d9a8c9d30646e0b6b93bbd28f195a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98d9a8c9d30646e0b6b93bbd28f195a3" member_type: VOTER } }
I20260709 12:30:14.533290 24566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:14.534366 24565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [sys.catalog]: This master's current role is: LEADER
I20260709 12:30:14.536159 24571 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:30:14.541198 24571 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:30:14.544759  8743 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:30:14.552668 24571 catalog_manager.cc:1383] Generated new cluster ID: 5f909d0d66ae49b19f8c891bb398c017
I20260709 12:30:14.553043 24571 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:30:14.579490 24571 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:30:14.580940 24571 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:30:14.604939 24571 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3: Generated new TSK 0
I20260709 12:30:14.605623 24571 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:30:14.611614  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:14.617928 24582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:14.619200 24583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:14.621366 24585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:14.621433  8743 server_base.cc:1061] running on GCE node
I20260709 12:30:14.622565  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:14.622846  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:14.623040  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600214623023 us; error 0 us; skew 500 ppm
I20260709 12:30:14.623588  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:14.626200  8743 webserver.cc:533] Webserver started at http://127.8.137.193:39203/ using document root <none> and password file <none>
I20260709 12:30:14.626820  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:14.627022  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:14.627311  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:14.628463  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-0-root/instance:
uuid: "6ce569893fd14fb995f3c987e622fc10"
format_stamp: "Formatted at 2026-07-09 12:30:14 on dist-test-slave-kpwv"
I20260709 12:30:14.633921  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.001s
I20260709 12:30:14.639333 24590 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.640336  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.002s
I20260709 12:30:14.640657  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-0-root
uuid: "6ce569893fd14fb995f3c987e622fc10"
format_stamp: "Formatted at 2026-07-09 12:30:14 on dist-test-slave-kpwv"
I20260709 12:30:14.640947  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:14.656104  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:14.657332  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:14.658929  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:14.661301  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:14.661481  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.661787  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:14.661984  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.710702  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.193:43703
I20260709 12:30:14.710881 24652 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.193:43703 every 8 connection(s)
I20260709 12:30:14.716634  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:14.723788 24657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:14.725307 24658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:14.730155 24660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:14.730532  8743 server_base.cc:1061] running on GCE node
I20260709 12:30:14.731412 24653 heartbeater.cc:344] Connected to a master server at 127.8.137.254:42857
I20260709 12:30:14.731907 24653 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:14.732295  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:14.732612  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:14.732822  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600214732801 us; error 0 us; skew 500 ppm
I20260709 12:30:14.732843 24653 heartbeater.cc:507] Master 127.8.137.254:42857 requested a full tablet report, sending...
I20260709 12:30:14.733789  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:14.735723 24526 ts_manager.cc:194] Registered new tserver with Master: 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703)
I20260709 12:30:14.736831  8743 webserver.cc:533] Webserver started at http://127.8.137.194:43351/ using document root <none> and password file <none>
I20260709 12:30:14.737627  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:14.737869  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:14.738199  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:14.738459 24526 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36330
I20260709 12:30:14.739926  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-1-root/instance:
uuid: "4b08b50e5e4146fa845229e427293f3d"
format_stamp: "Formatted at 2026-07-09 12:30:14 on dist-test-slave-kpwv"
I20260709 12:30:14.745287  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.001s
I20260709 12:30:14.749140 24665 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.750077  8743 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:30:14.750448  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-1-root
uuid: "4b08b50e5e4146fa845229e427293f3d"
format_stamp: "Formatted at 2026-07-09 12:30:14 on dist-test-slave-kpwv"
I20260709 12:30:14.750818  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:14.775305  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:14.776630  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:14.778281  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:14.780882  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:14.781153  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.781461  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:14.781684  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.826689  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.194:41381
I20260709 12:30:14.826820 24727 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.194:41381 every 8 connection(s)
I20260709 12:30:14.831845  8743 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:30:14.839529 24731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:30:14.840516 24732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:14.843113  8743 server_base.cc:1061] running on GCE node
W20260709 12:30:14.844506 24734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:30:14.845835  8743 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:30:14.846163  8743 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:30:14.846374  8743 hybrid_clock.cc:648] HybridClock initialized: now 1783600214846336 us; error 0 us; skew 500 ppm
I20260709 12:30:14.847076  8743 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:30:14.847069 24728 heartbeater.cc:344] Connected to a master server at 127.8.137.254:42857
I20260709 12:30:14.847806 24728 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:14.848938 24728 heartbeater.cc:507] Master 127.8.137.254:42857 requested a full tablet report, sending...
I20260709 12:30:14.850520  8743 webserver.cc:533] Webserver started at http://127.8.137.195:41527/ using document root <none> and password file <none>
I20260709 12:30:14.851186  8743 fs_manager.cc:362] Metadata directory not provided
I20260709 12:30:14.851398  8743 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:30:14.851527 24526 ts_manager.cc:194] Registered new tserver with Master: 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:14.851681  8743 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:30:14.853138  8743 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-2-root/instance:
uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
format_stamp: "Formatted at 2026-07-09 12:30:14 on dist-test-slave-kpwv"
I20260709 12:30:14.853216 24526 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36336
I20260709 12:30:14.858392  8743 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 12:30:14.862305 24739 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.863185  8743 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:30:14.863521  8743 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-2-root
uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
format_stamp: "Formatted at 2026-07-09 12:30:14 on dist-test-slave-kpwv"
I20260709 12:30:14.863796  8743 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskcjUeod/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783599720900375-8743-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:30:14.881253  8743 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:30:14.883162  8743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:30:14.884967  8743 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:30:14.888052  8743 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:30:14.888394  8743 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.888607  8743 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:30:14.888744  8743 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:30:14.932126  8743 rpc_server.cc:307] RPC server started. Bound to: 127.8.137.195:42195
I20260709 12:30:14.932233 24801 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.8.137.195:42195 every 8 connection(s)
I20260709 12:30:14.946524 24802 heartbeater.cc:344] Connected to a master server at 127.8.137.254:42857
I20260709 12:30:14.947081 24802 heartbeater.cc:461] Registering TS with master...
I20260709 12:30:14.947825 24802 heartbeater.cc:507] Master 127.8.137.254:42857 requested a full tablet report, sending...
I20260709 12:30:14.949796 24526 ts_manager.cc:194] Registered new tserver with Master: c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:14.950379  8743 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01486464s
I20260709 12:30:14.951634 24526 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36350
I20260709 12:30:14.978407 24526 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36354:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:14.981304 24526 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:15.067396 24767 tablet_service.cc:1511] Processing CreateTablet for tablet 60c01c5a4ca34a3baf3de9616c7efce3 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:15.067818 24618 tablet_service.cc:1511] Processing CreateTablet for tablet 60c01c5a4ca34a3baf3de9616c7efce3 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:15.068907 24767 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c01c5a4ca34a3baf3de9616c7efce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.069268 24618 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c01c5a4ca34a3baf3de9616c7efce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.071736 24617 tablet_service.cc:1511] Processing CreateTablet for tablet 07feb8bf7d60420f910f92bb22070acd (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:15.073290 24617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07feb8bf7d60420f910f92bb22070acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.074397 24766 tablet_service.cc:1511] Processing CreateTablet for tablet 07feb8bf7d60420f910f92bb22070acd (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:15.075006 24765 tablet_service.cc:1511] Processing CreateTablet for tablet 42a5285e1d844f10a11559914dce63c5 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:15.075945 24766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07feb8bf7d60420f910f92bb22070acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.075891 24616 tablet_service.cc:1511] Processing CreateTablet for tablet 42a5285e1d844f10a11559914dce63c5 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:15.077559 24764 tablet_service.cc:1511] Processing CreateTablet for tablet 6d64ccaae0eb400f8010fb8849239fc3 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:15.078459 24615 tablet_service.cc:1511] Processing CreateTablet for tablet 6d64ccaae0eb400f8010fb8849239fc3 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:15.079159 24764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d64ccaae0eb400f8010fb8849239fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.080307 24693 tablet_service.cc:1511] Processing CreateTablet for tablet 60c01c5a4ca34a3baf3de9616c7efce3 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:15.081753 24693 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 60c01c5a4ca34a3baf3de9616c7efce3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.082194 24763 tablet_service.cc:1511] Processing CreateTablet for tablet 2c4639d5f9864cfe9bf2132351763b98 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:15.083745 24763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c4639d5f9864cfe9bf2132351763b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.083382 24613 tablet_service.cc:1511] Processing CreateTablet for tablet e5ab1d11815d4a859c2141da9167ed32 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:15.085814 24612 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3aa7ecc7294d738b3e788d66914ea8 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:15.086263 24614 tablet_service.cc:1511] Processing CreateTablet for tablet 2c4639d5f9864cfe9bf2132351763b98 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:15.087852 24762 tablet_service.cc:1511] Processing CreateTablet for tablet e5ab1d11815d4a859c2141da9167ed32 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:15.088666 24611 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd6dc8d1cc34003aef4b0eb4823ca1c (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:15.089329 24762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab1d11815d4a859c2141da9167ed32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.090466 24611 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd6dc8d1cc34003aef4b0eb4823ca1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.090080 24692 tablet_service.cc:1511] Processing CreateTablet for tablet 07feb8bf7d60420f910f92bb22070acd (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:15.091957 24613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab1d11815d4a859c2141da9167ed32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.091562 24761 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3aa7ecc7294d738b3e788d66914ea8 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:15.093387 24615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d64ccaae0eb400f8010fb8849239fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.091977 24692 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07feb8bf7d60420f910f92bb22070acd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.094570 24616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a5285e1d844f10a11559914dce63c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.094553 24691 tablet_service.cc:1511] Processing CreateTablet for tablet 42a5285e1d844f10a11559914dce63c5 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:15.096211 24612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3aa7ecc7294d738b3e788d66914ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.096397 24691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a5285e1d844f10a11559914dce63c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.099061 24614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c4639d5f9864cfe9bf2132351763b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.092787 24765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42a5285e1d844f10a11559914dce63c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.107028 24761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3aa7ecc7294d738b3e788d66914ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.113164 24690 tablet_service.cc:1511] Processing CreateTablet for tablet 6d64ccaae0eb400f8010fb8849239fc3 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:15.114709 24690 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d64ccaae0eb400f8010fb8849239fc3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.119556 24689 tablet_service.cc:1511] Processing CreateTablet for tablet 2c4639d5f9864cfe9bf2132351763b98 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:15.121218 24689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2c4639d5f9864cfe9bf2132351763b98. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.123827 24688 tablet_service.cc:1511] Processing CreateTablet for tablet e5ab1d11815d4a859c2141da9167ed32 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:15.125337 24688 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e5ab1d11815d4a859c2141da9167ed32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.126806 24686 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd6dc8d1cc34003aef4b0eb4823ca1c (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:15.128188 24687 tablet_service.cc:1511] Processing CreateTablet for tablet 3f3aa7ecc7294d738b3e788d66914ea8 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:15.130414 24760 tablet_service.cc:1511] Processing CreateTablet for tablet 0cd6dc8d1cc34003aef4b0eb4823ca1c (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:15.131887 24760 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd6dc8d1cc34003aef4b0eb4823ca1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.128247 24686 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cd6dc8d1cc34003aef4b0eb4823ca1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.152691 24687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f3aa7ecc7294d738b3e788d66914ea8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.168730 24610 tablet_service.cc:1511] Processing CreateTablet for tablet 96761ddfa0044fea9016033ae2131861 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:15.170828 24759 tablet_service.cc:1511] Processing CreateTablet for tablet 96761ddfa0044fea9016033ae2131861 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:15.168730 24685 tablet_service.cc:1511] Processing CreateTablet for tablet 96761ddfa0044fea9016033ae2131861 (DEFAULT_TABLE table=test-workload [id=0298b9b64c4343b39640fe03baf0f18e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:15.172576 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96761ddfa0044fea9016033ae2131861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.173738 24685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96761ddfa0044fea9016033ae2131861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.170419 24610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96761ddfa0044fea9016033ae2131861. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:15.222370 24819 tablet_bootstrap.cc:492] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.252298 24817 tablet_bootstrap.cc:492] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.263557 24817 tablet_bootstrap.cc:654] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.265913 24819 tablet_bootstrap.cc:654] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.278716 24818 tablet_bootstrap.cc:492] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.299214 24818 tablet_bootstrap.cc:654] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.314960 24817 tablet_bootstrap.cc:492] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.321342 24819 tablet_bootstrap.cc:492] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.322053 24819 ts_tablet_manager.cc:1403] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.100s	user 0.010s	sys 0.023s
I20260709 12:30:15.322855 24817 ts_tablet_manager.cc:1403] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.071s	user 0.007s	sys 0.016s
I20260709 12:30:15.323536 24818 tablet_bootstrap.cc:492] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.324098 24818 ts_tablet_manager.cc:1403] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.009s
I20260709 12:30:15.325038 24819 raft_consensus.cc:359] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.325692 24819 raft_consensus.cc:385] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.325960 24819 raft_consensus.cc:740] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.326449 24817 raft_consensus.cc:359] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.326699 24819 consensus_queue.cc:260] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.327363 24817 raft_consensus.cc:385] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.327194 24818 raft_consensus.cc:359] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.327740 24817 raft_consensus.cc:740] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.328006 24818 raft_consensus.cc:385] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.328334 24818 raft_consensus.cc:740] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.328696 24817 consensus_queue.cc:260] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.329120 24818 consensus_queue.cc:260] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.336933 24802 heartbeater.cc:499] Master 127.8.137.254:42857 was elected leader, sending a full tablet report...
I20260709 12:30:15.337019 24819 ts_tablet_manager.cc:1434] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.008s
I20260709 12:30:15.337024 24817 ts_tablet_manager.cc:1434] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 12:30:15.337932 24818 ts_tablet_manager.cc:1434] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.014s	user 0.012s	sys 0.002s
I20260709 12:30:15.338205 24728 heartbeater.cc:499] Master 127.8.137.254:42857 was elected leader, sending a full tablet report...
I20260709 12:30:15.338475 24819 tablet_bootstrap.cc:492] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.339174 24818 tablet_bootstrap.cc:492] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.341838 24653 heartbeater.cc:499] Master 127.8.137.254:42857 was elected leader, sending a full tablet report...
I20260709 12:30:15.343014 24817 tablet_bootstrap.cc:492] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.347512 24818 tablet_bootstrap.cc:654] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.348135 24819 tablet_bootstrap.cc:654] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.355161 24817 tablet_bootstrap.cc:654] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.359478 24818 tablet_bootstrap.cc:492] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.360069 24818 ts_tablet_manager.cc:1403] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.010s
I20260709 12:30:15.362394 24819 tablet_bootstrap.cc:492] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.363231 24819 ts_tablet_manager.cc:1403] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.025s	user 0.010s	sys 0.003s
I20260709 12:30:15.363257 24818 raft_consensus.cc:359] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.364284 24818 raft_consensus.cc:385] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.364589 24818 raft_consensus.cc:740] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.365500 24818 consensus_queue.cc:260] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.368309 24818 ts_tablet_manager.cc:1434] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:30:15.369375 24818 tablet_bootstrap.cc:492] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.370671 24817 tablet_bootstrap.cc:492] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.371870 24817 ts_tablet_manager.cc:1403] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.009s
I20260709 12:30:15.368335 24819 raft_consensus.cc:359] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.374845 24819 raft_consensus.cc:385] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.375236 24819 raft_consensus.cc:740] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.375582 24817 raft_consensus.cc:359] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.376488 24817 raft_consensus.cc:385] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.376148 24819 consensus_queue.cc:260] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.376837 24817 raft_consensus.cc:740] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.377470 24818 tablet_bootstrap.cc:654] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.377954 24817 consensus_queue.cc:260] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.378744 24819 ts_tablet_manager.cc:1434] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.001s
I20260709 12:30:15.380021 24819 tablet_bootstrap.cc:492] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.380795 24817 ts_tablet_manager.cc:1434] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:30:15.381939 24817 tablet_bootstrap.cc:492] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.388877 24817 tablet_bootstrap.cc:654] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.389714 24819 tablet_bootstrap.cc:654] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.394558 24818 tablet_bootstrap.cc:492] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.395152 24818 ts_tablet_manager.cc:1403] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.026s	user 0.002s	sys 0.023s
I20260709 12:30:15.398569 24818 raft_consensus.cc:359] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.399394 24818 raft_consensus.cc:385] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.399749 24818 raft_consensus.cc:740] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.400652 24818 consensus_queue.cc:260] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.405360 24818 ts_tablet_manager.cc:1434] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.007s
I20260709 12:30:15.406481 24818 tablet_bootstrap.cc:492] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.414278 24819 tablet_bootstrap.cc:492] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.414183 24824 raft_consensus.cc:493] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.415020 24817 tablet_bootstrap.cc:492] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.414981 24824 raft_consensus.cc:515] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.415637 24817 ts_tablet_manager.cc:1403] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.034s	user 0.012s	sys 0.015s
I20260709 12:30:15.416759 24818 tablet_bootstrap.cc:654] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.419114 24817 raft_consensus.cc:359] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.419909 24817 raft_consensus.cc:385] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.420204 24817 raft_consensus.cc:740] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.421276 24817 consensus_queue.cc:260] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.420131 24824 leader_election.cc:290] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.425472 24819 ts_tablet_manager.cc:1403] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.014s
I20260709 12:30:15.431348 24825 raft_consensus.cc:493] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.429726 24819 raft_consensus.cc:359] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.432039 24819 raft_consensus.cc:385] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.431968 24825 raft_consensus.cc:515] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.432564 24819 raft_consensus.cc:740] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.433935 24819 consensus_queue.cc:260] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
W20260709 12:30:15.436575 24803 tablet.cc:2406] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:30:15.436596 24825 leader_election.cc:290] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.437790 24818 tablet_bootstrap.cc:492] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.438050 24819 ts_tablet_manager.cc:1434] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.002s
I20260709 12:30:15.438707 24818 ts_tablet_manager.cc:1403] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.005s
I20260709 12:30:15.439666 24819 tablet_bootstrap.cc:492] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.450856 24819 tablet_bootstrap.cc:654] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.452593 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:15.453469 24628 raft_consensus.cc:2468] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:15.442402 24818 raft_consensus.cc:359] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.455068 24667 leader_election.cc:304] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:15.456012 24824 raft_consensus.cc:2804] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.456506 24824 raft_consensus.cc:493] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.456979 24824 raft_consensus.cc:3060] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.458889 24817 ts_tablet_manager.cc:1434] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.043s	user 0.015s	sys 0.027s
I20260709 12:30:15.460110 24817 tablet_bootstrap.cc:492] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.460681 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
W20260709 12:30:15.462478 24667 leader_election.cc:343] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.454524 24818 raft_consensus.cc:385] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.463735 24818 raft_consensus.cc:740] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.467250 24817 tablet_bootstrap.cc:654] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.467708 24824 raft_consensus.cc:515] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.468091 24818 consensus_queue.cc:260] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.472005 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:15.472630 24628 raft_consensus.cc:3060] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.474300 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:15.474447 24824 leader_election.cc:290] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
W20260709 12:30:15.475881 24667 leader_election.cc:343] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.478168 24818 ts_tablet_manager.cc:1434] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 12:30:15.479910 24818 tablet_bootstrap.cc:492] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.486680 24818 tablet_bootstrap.cc:654] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.492748 24817 tablet_bootstrap.cc:492] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.493337 24817 ts_tablet_manager.cc:1403] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.009s
I20260709 12:30:15.492751 24819 tablet_bootstrap.cc:492] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.495270 24819 ts_tablet_manager.cc:1403] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.012s
I20260709 12:30:15.496577 24817 raft_consensus.cc:359] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.497423 24817 raft_consensus.cc:385] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.497732 24817 raft_consensus.cc:740] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.498523 24817 consensus_queue.cc:260] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.499894 24703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:15.500978 24817 ts_tablet_manager.cc:1434] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:30:15.501160 24703 raft_consensus.cc:2393] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6ce569893fd14fb995f3c987e622fc10 in current term 1: Already voted for candidate 4b08b50e5e4146fa845229e427293f3d in this term.
I20260709 12:30:15.501324 24819 raft_consensus.cc:359] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.502135 24817 tablet_bootstrap.cc:492] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.502171 24819 raft_consensus.cc:385] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.502740 24819 raft_consensus.cc:740] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.503674 24819 consensus_queue.cc:260] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.505061 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
W20260709 12:30:15.507575 24592 leader_election.cc:343] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.508342 24592 leader_election.cc:304] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10; no voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:15.509863 24817 tablet_bootstrap.cc:654] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.515126 24819 ts_tablet_manager.cc:1434] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.001s
I20260709 12:30:15.516944 24628 raft_consensus.cc:2468] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:15.520792 24819 tablet_bootstrap.cc:492] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.521637 24825 raft_consensus.cc:2749] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.524600 24667 leader_election.cc:304] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:15.527040 24824 raft_consensus.cc:2804] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.528486 24824 raft_consensus.cc:697] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:15.530424 24817 tablet_bootstrap.cc:492] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.529860 24824 consensus_queue.cc:237] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.531114 24817 ts_tablet_manager.cc:1403] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.012s
I20260709 12:30:15.532349 24819 tablet_bootstrap.cc:654] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.534578 24817 raft_consensus.cc:359] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.535480 24817 raft_consensus.cc:385] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.535921 24817 raft_consensus.cc:740] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.537314 24817 consensus_queue.cc:260] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
W20260709 12:30:15.539265 24574 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:15.540292 24817 ts_tablet_manager.cc:1434] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:15.541949 24817 tablet_bootstrap.cc:492] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.552158 24818 tablet_bootstrap.cc:492] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.552809 24818 ts_tablet_manager.cc:1403] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.073s	user 0.020s	sys 0.044s
I20260709 12:30:15.553125 24817 tablet_bootstrap.cc:654] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.553988 24819 tablet_bootstrap.cc:492] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.554570 24819 ts_tablet_manager.cc:1403] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.034s	user 0.014s	sys 0.009s
I20260709 12:30:15.556340 24818 raft_consensus.cc:359] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.557194 24818 raft_consensus.cc:385] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.557495 24818 raft_consensus.cc:740] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.557797 24819 raft_consensus.cc:359] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.559280 24819 raft_consensus.cc:385] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.558516 24818 consensus_queue.cc:260] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.565565 24818 ts_tablet_manager.cc:1434] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 12:30:15.560114 24819 raft_consensus.cc:740] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.567615 24818 tablet_bootstrap.cc:492] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.567401 24819 consensus_queue.cc:260] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.572919 24524 catalog_manager.cc:5697] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 1 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.575537 24819 ts_tablet_manager.cc:1434] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.002s
I20260709 12:30:15.575838 24818 tablet_bootstrap.cc:654] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.576759 24819 tablet_bootstrap.cc:492] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.579752 24817 tablet_bootstrap.cc:492] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.580389 24817 ts_tablet_manager.cc:1403] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.039s	user 0.012s	sys 0.011s
I20260709 12:30:15.583560 24825 raft_consensus.cc:493] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.584172 24825 raft_consensus.cc:515] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.584084 24817 raft_consensus.cc:359] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.585439 24817 raft_consensus.cc:385] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.585551 24819 tablet_bootstrap.cc:654] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.585877 24817 raft_consensus.cc:740] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.586864 24825 leader_election.cc:290] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.586912 24817 consensus_queue.cc:260] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.590219 24817 ts_tablet_manager.cc:1434] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.002s
I20260709 12:30:15.589726 24703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c01c5a4ca34a3baf3de9616c7efce3" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:15.590973 24703 raft_consensus.cc:2468] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:15.591622 24817 tablet_bootstrap.cc:492] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.592429 24592 leader_election.cc:304] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:15.592856 24825 raft_consensus.cc:493] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.589709 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c01c5a4ca34a3baf3de9616c7efce3" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:15.593382 24825 raft_consensus.cc:515] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.594049 24819 tablet_bootstrap.cc:492] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.593967 24777 raft_consensus.cc:2468] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:15.594673 24819 ts_tablet_manager.cc:1403] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.018s	user 0.014s	sys 0.002s
I20260709 12:30:15.595592 24825 leader_election.cc:290] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.596174 24825 raft_consensus.cc:2804] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.596525 24825 raft_consensus.cc:493] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.596861 24825 raft_consensus.cc:3060] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.597971 24819 raft_consensus.cc:359] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.598796 24819 raft_consensus.cc:385] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.599118 24819 raft_consensus.cc:740] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.600205 24819 consensus_queue.cc:260] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.601929 24817 tablet_bootstrap.cc:654] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.603371 24818 tablet_bootstrap.cc:492] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.603438 24819 ts_tablet_manager.cc:1434] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:30:15.604130 24818 ts_tablet_manager.cc:1403] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.037s	user 0.017s	sys 0.000s
I20260709 12:30:15.604785 24819 tablet_bootstrap.cc:492] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.611095 24825 raft_consensus.cc:515] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.611657 24703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07feb8bf7d60420f910f92bb22070acd" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:15.612125 24819 tablet_bootstrap.cc:654] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.612682 24703 raft_consensus.cc:2468] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:15.613706 24825 leader_election.cc:290] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Requested vote from peers c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.614346 24818 raft_consensus.cc:359] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.615161 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07feb8bf7d60420f910f92bb22070acd" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:15.615643 24817 tablet_bootstrap.cc:492] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.615968 24777 raft_consensus.cc:2468] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:15.616291 24817 ts_tablet_manager.cc:1403] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.009s
I20260709 12:30:15.617146 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c01c5a4ca34a3baf3de9616c7efce3" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:15.615156 24818 raft_consensus.cc:385] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.617839 24776 raft_consensus.cc:3060] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.617926 24818 raft_consensus.cc:740] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.619191 24818 consensus_queue.cc:260] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.620254 24817 raft_consensus.cc:359] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.621112 24817 raft_consensus.cc:385] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.621474 24817 raft_consensus.cc:740] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.622395 24817 consensus_queue.cc:260] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.625051 24817 ts_tablet_manager.cc:1434] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:30:15.625084 24819 tablet_bootstrap.cc:492] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.622241 24818 ts_tablet_manager.cc:1434] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:30:15.625990 24819 ts_tablet_manager.cc:1403] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.006s
I20260709 12:30:15.627070 24818 tablet_bootstrap.cc:492] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.627219 24776 raft_consensus.cc:2468] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:15.628394 24817 tablet_bootstrap.cc:492] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.629200 24819 raft_consensus.cc:359] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.629491 24703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "60c01c5a4ca34a3baf3de9616c7efce3" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:15.630208 24703 raft_consensus.cc:3060] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.630336 24592 leader_election.cc:304] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:15.631287 24825 raft_consensus.cc:2804] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.631659 24825 raft_consensus.cc:493] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.632027 24825 raft_consensus.cc:3060] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.634729 24818 tablet_bootstrap.cc:654] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.636358 24817 tablet_bootstrap.cc:654] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.638084 24819 raft_consensus.cc:385] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.638558 24819 raft_consensus.cc:740] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.639654 24592 leader_election.cc:304] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.639814 24819 consensus_queue.cc:260] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.641240 24849 raft_consensus.cc:2804] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.643323 24703 raft_consensus.cc:2468] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:15.646389 24819 ts_tablet_manager.cc:1434] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 12:30:15.646715 24818 tablet_bootstrap.cc:492] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.647434 24818 ts_tablet_manager.cc:1403] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 12:30:15.645712 24825 raft_consensus.cc:515] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.647593 24819 tablet_bootstrap.cc:492] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.649835 24825 leader_election.cc:290] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.649940 24823 raft_consensus.cc:493] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.650527 24823 raft_consensus.cc:515] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.650658 24818 raft_consensus.cc:359] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.651379 24818 raft_consensus.cc:385] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.646697 24849 raft_consensus.cc:697] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:15.651722 24817 tablet_bootstrap.cc:492] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.652520 24817 ts_tablet_manager.cc:1403] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260709 12:30:15.652590 24849 consensus_queue.cc:237] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } }
I20260709 12:30:15.653257 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07feb8bf7d60420f910f92bb22070acd" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:15.653949 24776 raft_consensus.cc:3060] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.655275 24823 leader_election.cc:290] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.655923 24817 raft_consensus.cc:359] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.656786 24817 raft_consensus.cc:385] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.657209 24817 raft_consensus.cc:740] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.651700 24818 raft_consensus.cc:740] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.658105 24817 consensus_queue.cc:260] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.651634 24703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07feb8bf7d60420f910f92bb22070acd" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:15.658305 24818 consensus_queue.cc:260] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.663075 24703 raft_consensus.cc:3060] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.663599 24817 ts_tablet_manager.cc:1434] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:30:15.664697 24817 tablet_bootstrap.cc:492] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:15.667716 24818 ts_tablet_manager.cc:1434] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.003s
I20260709 12:30:15.669088 24818 tablet_bootstrap.cc:492] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.671872 24776 raft_consensus.cc:2468] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:15.674350 24823 raft_consensus.cc:493] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.675298 24823 raft_consensus.cc:515] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.681231 24823 leader_election.cc:290] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.682682 24703 raft_consensus.cc:2468] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:15.681427 24818 tablet_bootstrap.cc:654] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.685241 24592 leader_election.cc:304] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:15.687662 24817 tablet_bootstrap.cc:654] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.690356 24819 tablet_bootstrap.cc:654] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.694188 24849 raft_consensus.cc:2804] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.696333 24824 raft_consensus.cc:493] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.704221 24849 raft_consensus.cc:697] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:15.704066 24824 raft_consensus.cc:515] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.706068 24849 consensus_queue.cc:237] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.707167 24824 leader_election.cc:290] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.709275 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5285e1d844f10a11559914dce63c5" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:15.710405 24776 raft_consensus.cc:2468] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:15.709376 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5285e1d844f10a11559914dce63c5" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:15.711836 24667 leader_election.cc:304] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.711858 24628 raft_consensus.cc:2468] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:15.712142 24525 catalog_manager.cc:5697] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.714427 24824 raft_consensus.cc:2804] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.714900 24824 raft_consensus.cc:493] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.715262 24824 raft_consensus.cc:3060] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.716109 24627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c4639d5f9864cfe9bf2132351763b98" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:15.716892 24627 raft_consensus.cc:2468] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 0.
I20260709 12:30:15.720257 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:15.721505 24628 raft_consensus.cc:2393] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in current term 1: Already voted for candidate 4b08b50e5e4146fa845229e427293f3d in this term.
I20260709 12:30:15.726017 24823 raft_consensus.cc:493] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.726708 24823 raft_consensus.cc:515] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.728962 24823 leader_election.cc:290] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.719661 24741 leader_election.cc:304] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.730962 24817 tablet_bootstrap.cc:492] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:15.730966 24819 tablet_bootstrap.cc:492] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.731827 24819 ts_tablet_manager.cc:1403] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.084s	user 0.015s	sys 0.010s
I20260709 12:30:15.733330 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:15.734102 24628 raft_consensus.cc:2468] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 0.
I20260709 12:30:15.735026 24819 raft_consensus.cc:359] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.731640 24824 raft_consensus.cc:515] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.735785 24823 raft_consensus.cc:2804] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.735846 24819 raft_consensus.cc:385] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.737432 24823 raft_consensus.cc:493] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.738121 24823 raft_consensus.cc:3060] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.736675 24741 leader_election.cc:304] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.731693 24817 ts_tablet_manager.cc:1403] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.067s	user 0.012s	sys 0.001s
I20260709 12:30:15.741174 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5285e1d844f10a11559914dce63c5" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:15.739418 24824 leader_election.cc:290] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.741920 24628 raft_consensus.cc:3060] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.742363 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5285e1d844f10a11559914dce63c5" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:15.743113 24776 raft_consensus.cc:3060] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.744421 24817 raft_consensus.cc:359] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.745293 24817 raft_consensus.cc:385] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.745929 24817 raft_consensus.cc:740] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.747556 24817 consensus_queue.cc:260] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.748790 24823 raft_consensus.cc:515] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.750653 24628 raft_consensus.cc:2468] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:15.737466 24819 raft_consensus.cc:740] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.752606 24819 consensus_queue.cc:260] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.753197 24667 leader_election.cc:304] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:15.754459 24835 raft_consensus.cc:2804] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.756592 24824 raft_consensus.cc:493] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.757220 24824 raft_consensus.cc:515] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.756879 24523 catalog_manager.cc:5697] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.760461 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:15.761194 24628 raft_consensus.cc:2468] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:15.762431 24667 leader_election.cc:304] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:15.765123 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:15.765858 24777 raft_consensus.cc:2468] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:15.769572 24824 leader_election.cc:290] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.770489 24824 raft_consensus.cc:2804] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.771230 24824 raft_consensus.cc:493] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.772004 24824 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.773933 24776 raft_consensus.cc:2468] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:15.773030 24817 ts_tablet_manager.cc:1434] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.001s
I20260709 12:30:15.777396 24863 raft_consensus.cc:2804] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.777917 24863 raft_consensus.cc:493] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.778275 24863 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.779429 24703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:15.780460 24835 raft_consensus.cc:697] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:15.780989 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c4639d5f9864cfe9bf2132351763b98" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:15.781709 24702 raft_consensus.cc:2468] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 0.
I20260709 12:30:15.781476 24835 consensus_queue.cc:237] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.783412 24701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:15.785358 24741 leader_election.cc:304] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10
I20260709 12:30:15.785694 24819 ts_tablet_manager.cc:1434] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.053s	user 0.017s	sys 0.020s
I20260709 12:30:15.790066 24864 raft_consensus.cc:493] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.790791 24864 raft_consensus.cc:515] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.793962 24819 tablet_bootstrap.cc:492] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:15.794235 24864 leader_election.cc:290] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.795193 24864 raft_consensus.cc:3060] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.796866 24823 leader_election.cc:290] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.803679 24863 raft_consensus.cc:515] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.806798 24863 leader_election.cc:290] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.810884 24819 tablet_bootstrap.cc:654] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.811739 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6dc8d1cc34003aef4b0eb4823ca1c" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:15.812363 24627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c4639d5f9864cfe9bf2132351763b98" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:15.813094 24627 raft_consensus.cc:3060] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.810972 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6dc8d1cc34003aef4b0eb4823ca1c" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:15.813962 24628 raft_consensus.cc:2468] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 0.
I20260709 12:30:15.812505 24702 raft_consensus.cc:2468] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 0.
I20260709 12:30:15.823406 24818 tablet_bootstrap.cc:492] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.824028 24818 ts_tablet_manager.cc:1403] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.155s	user 0.019s	sys 0.015s
I20260709 12:30:15.824551 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c4639d5f9864cfe9bf2132351763b98" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:15.825227 24702 raft_consensus.cc:3060] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.826979 24818 raft_consensus.cc:359] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.827716 24818 raft_consensus.cc:385] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.827988 24818 raft_consensus.cc:740] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.828809 24818 consensus_queue.cc:260] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.828864 24526 catalog_manager.cc:5697] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 1 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.831110 24627 raft_consensus.cc:2468] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 1.
I20260709 12:30:15.833333 24818 ts_tablet_manager.cc:1434] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:30:15.834378 24818 tablet_bootstrap.cc:492] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:15.837158 24819 tablet_bootstrap.cc:492] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:15.837651 24819 ts_tablet_manager.cc:1403] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.005s
I20260709 12:30:15.840546 24819 raft_consensus.cc:359] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.841259 24819 raft_consensus.cc:385] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.841562 24819 raft_consensus.cc:740] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.842257 24819 consensus_queue.cc:260] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.842653 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:15.843245 24628 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.844530 24819 ts_tablet_manager.cc:1434] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:30:15.844852 24702 raft_consensus.cc:2468] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 1.
I20260709 12:30:15.846297 24741 leader_election.cc:304] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.847033 24703 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:15.847266 24741 leader_election.cc:304] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.847746 24818 tablet_bootstrap.cc:654] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:15.849093 24867 raft_consensus.cc:2804] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.849537 24867 raft_consensus.cc:493] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.849869 24867 raft_consensus.cc:3060] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.851452 24866 raft_consensus.cc:2804] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.852056 24866 raft_consensus.cc:697] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Becoming Leader. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:15.841383 24824 raft_consensus.cc:515] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.854707 24701 raft_consensus.cc:2393] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in current term 1: Already voted for candidate 4b08b50e5e4146fa845229e427293f3d in this term.
I20260709 12:30:15.857151 24627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:15.858898 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:15.859529 24824 leader_election.cc:290] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.852926 24866 consensus_queue.cc:237] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.859843 24776 raft_consensus.cc:2393] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b08b50e5e4146fa845229e427293f3d in current term 1: Already voted for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in this term.
I20260709 12:30:15.861174 24667 leader_election.cc:304] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d; no voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:15.863242 24835 raft_consensus.cc:2749] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.863921 24835 raft_consensus.cc:493] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.864380 24835 raft_consensus.cc:515] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.866564 24835 leader_election.cc:290] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.867003 24864 raft_consensus.cc:2749] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:15.867764 24628 raft_consensus.cc:2468] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 1.
I20260709 12:30:15.868952 24741 leader_election.cc:304] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:15.869329 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3aa7ecc7294d738b3e788d66914ea8" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:15.870076 24776 raft_consensus.cc:2468] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:15.870241 24864 raft_consensus.cc:2804] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.870535 24867 raft_consensus.cc:515] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.870812 24864 raft_consensus.cc:697] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Becoming Leader. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:15.873651 24867 leader_election.cc:290] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:15.871366 24667 leader_election.cc:304] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.873028 24864 consensus_queue.cc:237] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.875629 24835 raft_consensus.cc:2804] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.875633 24701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6dc8d1cc34003aef4b0eb4823ca1c" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:15.877148 24835 raft_consensus.cc:493] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.877537 24701 raft_consensus.cc:3060] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.877576 24835 raft_consensus.cc:3060] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.870898 24627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3aa7ecc7294d738b3e788d66914ea8" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
W20260709 12:30:15.879762 24667 leader_election.cc:343] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:15.882826 24526 catalog_manager.cc:5697] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:15.886363 24701 raft_consensus.cc:2468] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 1.
I20260709 12:30:15.887817 24741 leader_election.cc:304] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.888501 24818 tablet_bootstrap.cc:492] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:15.888803 24866 raft_consensus.cc:2804] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.889294 24818 ts_tablet_manager.cc:1403] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.012s
I20260709 12:30:15.889391 24866 raft_consensus.cc:697] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Becoming Leader. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:15.890466 24866 consensus_queue.cc:237] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.898677 24818 raft_consensus.cc:359] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.899223 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6dc8d1cc34003aef4b0eb4823ca1c" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:15.898355 24835 raft_consensus.cc:515] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.900400 24628 raft_consensus.cc:3060] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.899704 24818 raft_consensus.cc:385] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:15.903859 24818 raft_consensus.cc:740] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:15.904054 24627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3aa7ecc7294d738b3e788d66914ea8" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:15.906925 24818 consensus_queue.cc:260] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.907792 24627 raft_consensus.cc:3060] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.909063 24818 ts_tablet_manager.cc:1434] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.001s
I20260709 12:30:15.910356 24835 leader_election.cc:290] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.912189 24628 raft_consensus.cc:2468] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 1.
I20260709 12:30:15.912680 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f3aa7ecc7294d738b3e788d66914ea8" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:15.913408 24776 raft_consensus.cc:3060] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.917048 24526 catalog_manager.cc:5697] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:15.918551 24526 catalog_manager.cc:5697] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:15.919574 24776 raft_consensus.cc:2468] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:15.920938 24667 leader_election.cc:304] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:15.921770 24835 raft_consensus.cc:2804] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.923079 24627 raft_consensus.cc:2468] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:15.923100 24835 raft_consensus.cc:697] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:15.924243 24835 consensus_queue.cc:237] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.933914 24526 catalog_manager.cc:5697] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 1 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:15.968412 24849 raft_consensus.cc:493] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:15.969071 24849 raft_consensus.cc:515] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.970816 24849 leader_election.cc:290] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.971807 24701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96761ddfa0044fea9016033ae2131861" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:15.972061 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96761ddfa0044fea9016033ae2131861" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:15.972438 24701 raft_consensus.cc:2468] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:15.972796 24776 raft_consensus.cc:2468] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:15.973479 24592 leader_election.cc:304] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:15.974251 24849 raft_consensus.cc:2804] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:15.974591 24849 raft_consensus.cc:493] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:15.974973 24849 raft_consensus.cc:3060] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.978364 24835 consensus_queue.cc:1048] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:15.982312 24849 raft_consensus.cc:515] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:15.984997 24849 leader_election.cc:290] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:15.985782 24701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96761ddfa0044fea9016033ae2131861" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:15.986430 24701 raft_consensus.cc:3060] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.986701 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96761ddfa0044fea9016033ae2131861" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:15.987378 24776 raft_consensus.cc:3060] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:15.994566 24701 raft_consensus.cc:2468] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:15.997026 24592 leader_election.cc:304] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:15.997782 24776 raft_consensus.cc:2468] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:15.998775 24849 raft_consensus.cc:2804] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:15.999310 24849 raft_consensus.cc:697] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:16.000185 24849 consensus_queue.cc:237] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.000409 24835 consensus_queue.cc:1048] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.009526 24523 catalog_manager.cc:5697] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.116552 24825 consensus_queue.cc:1048] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.119216 24825 consensus_queue.cc:1048] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.132339 24854 consensus_queue.cc:1048] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.147430 24854 consensus_queue.cc:1048] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.180159 24702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42a5285e1d844f10a11559914dce63c5"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
mode: GRACEFUL
new_leader_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:59770
I20260709 12:30:16.181049 24702 raft_consensus.cc:606] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Received request to transfer leadership to TS 6ce569893fd14fb995f3c987e622fc10
I20260709 12:30:16.186864 24702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
mode: GRACEFUL
new_leader_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:59770
I20260709 12:30:16.187435 24702 raft_consensus.cc:606] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Received request to transfer leadership to TS 6ce569893fd14fb995f3c987e622fc10
I20260709 12:30:16.201596 24777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2c4639d5f9864cfe9bf2132351763b98"
dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
mode: GRACEFUL
new_leader_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:35932
I20260709 12:30:16.202160 24777 raft_consensus.cc:606] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Received request to transfer leadership to TS 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:16.207687 24777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32"
dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
mode: GRACEFUL
new_leader_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:35932
I20260709 12:30:16.208244 24777 raft_consensus.cc:606] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Received request to transfer leadership to TS 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:16.217581 24777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0cd6dc8d1cc34003aef4b0eb4823ca1c"
dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
mode: GRACEFUL
new_leader_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:35932
I20260709 12:30:16.218178 24777 raft_consensus.cc:606] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Received request to transfer leadership to TS 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:16.227547 24865 consensus_queue.cc:1048] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.235647 24835 raft_consensus.cc:993] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d: : Instructing follower 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:16.236222 24835 raft_consensus.cc:1081] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Signalling peer 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:16.236901 24627 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "96761ddfa0044fea9016033ae2131861"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
mode: GRACEFUL
new_leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
 from {username='slave'} at 127.0.0.1:52258
I20260709 12:30:16.237669 24627 raft_consensus.cc:606] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Received request to transfer leadership to TS c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:16.238579 24626 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42a5285e1d844f10a11559914dce63c5"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:52238
I20260709 12:30:16.239319 24626 raft_consensus.cc:493] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:16.239691 24626 raft_consensus.cc:3060] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.246862 24890 consensus_queue.cc:1048] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.249012 24626 raft_consensus.cc:515] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.251240 24626 leader_election.cc:290] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:16.253688 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5285e1d844f10a11559914dce63c5" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:16.254377 24702 raft_consensus.cc:3055] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:16.254052 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42a5285e1d844f10a11559914dce63c5" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:16.254768 24702 raft_consensus.cc:740] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:16.256485 24702 consensus_queue.cc:260] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.259933 24702 raft_consensus.cc:3060] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.266081 24702 raft_consensus.cc:2468] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:16.267134 24592 leader_election.cc:304] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:16.267882 24825 raft_consensus.cc:2804] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:16.268778 24825 raft_consensus.cc:697] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:16.269882 24825 consensus_queue.cc:237] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.272740 24866 consensus_queue.cc:1048] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.278622 24523 catalog_manager.cc:5697] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 1 to 2, leader changed from 4b08b50e5e4146fa845229e427293f3d (127.8.137.194) to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.285599 24867 consensus_queue.cc:1048] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.287143 24867 consensus_queue.cc:1048] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.289723 24866 consensus_queue.cc:1048] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.304265 24864 consensus_queue.cc:1048] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.305575 24867 raft_consensus.cc:993] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3: : Instructing follower 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:16.308694 24823 raft_consensus.cc:993] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3: : Instructing follower 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:16.309249 24823 raft_consensus.cc:1081] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Signalling peer 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:16.306406 24867 raft_consensus.cc:1081] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Signalling peer 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:16.314924 24702 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2c4639d5f9864cfe9bf2132351763b98"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:59764
I20260709 12:30:16.316007 24702 raft_consensus.cc:493] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:16.316920 24702 raft_consensus.cc:3060] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.314414 24703 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0cd6dc8d1cc34003aef4b0eb4823ca1c"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:59764
I20260709 12:30:16.318315 24703 raft_consensus.cc:493] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:16.318919 24703 raft_consensus.cc:3060] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.323422 24890 consensus_queue.cc:1048] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.328148 24866 consensus_queue.cc:1048] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.343150 24703 raft_consensus.cc:515] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.344923 24701 raft_consensus.cc:1240] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Rejecting Update request from peer c9ee0c5aee904bc6a5597cc141dcd3e3 for earlier term 1. Current term is 2. Ops: []
I20260709 12:30:16.345616 24703 leader_election.cc:290] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:16.347107 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6dc8d1cc34003aef4b0eb4823ca1c" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:16.347386 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cd6dc8d1cc34003aef4b0eb4823ca1c" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:16.347759 24626 raft_consensus.cc:3060] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.348114 24776 raft_consensus.cc:3055] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:16.348485 24776 raft_consensus.cc:740] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:16.349712 24776 consensus_queue.cc:260] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.351696 24776 raft_consensus.cc:3060] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.354954 24890 consensus_queue.cc:1048] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.360473 24626 raft_consensus.cc:2468] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:16.361093 24702 raft_consensus.cc:515] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.361856 24667 leader_election.cc:304] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:16.363224 24776 raft_consensus.cc:2468] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:16.364573 24824 raft_consensus.cc:2804] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:16.365160 24824 raft_consensus.cc:697] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:16.366086 24824 consensus_queue.cc:237] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.369633 24702 leader_election.cc:290] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:16.371789 24866 raft_consensus.cc:993] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3: : Instructing follower 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:16.372265 24866 raft_consensus.cc:1081] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Signalling peer 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:16.374269 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c4639d5f9864cfe9bf2132351763b98" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:16.374511 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2c4639d5f9864cfe9bf2132351763b98" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:16.375042 24626 raft_consensus.cc:3060] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.375093 24776 raft_consensus.cc:3055] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:16.378515 24776 raft_consensus.cc:740] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:16.378841 24700 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:59764
I20260709 12:30:16.379473 24700 raft_consensus.cc:493] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:16.379926 24776 consensus_queue.cc:260] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.381337 24776 raft_consensus.cc:3060] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.379961 24700 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.385579 24626 raft_consensus.cc:2468] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:16.386886 24667 leader_election.cc:304] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:16.387676 24890 raft_consensus.cc:2804] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:16.388129 24890 raft_consensus.cc:697] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:16.388876 24890 consensus_queue.cc:237] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.389642 24776 raft_consensus.cc:2468] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:16.390631 24700 raft_consensus.cc:515] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.392993 24700 leader_election.cc:290] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:16.393674 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:16.394212 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:16.394338 24626 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.395021 24776 raft_consensus.cc:3055] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:16.395370 24776 raft_consensus.cc:740] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:16.396209 24776 consensus_queue.cc:260] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.397526 24776 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.397071 24524 catalog_manager.cc:5697] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 1 to 2, leader changed from c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195) to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 2 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.402354 24626 raft_consensus.cc:2468] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:16.403735 24667 leader_election.cc:304] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:16.404588 24890 raft_consensus.cc:2804] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:16.405102 24890 raft_consensus.cc:697] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:16.405388 24776 raft_consensus.cc:2468] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:16.405975 24890 consensus_queue.cc:237] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.414580 24524 catalog_manager.cc:5697] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 1 to 2, leader changed from c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195) to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 2 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.416102 24524 catalog_manager.cc:5697] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 1 to 2, leader changed from c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195) to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 2 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.441921 24825 consensus_queue.cc:1048] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:16.449363 24849 raft_consensus.cc:993] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10: : Instructing follower c9ee0c5aee904bc6a5597cc141dcd3e3 to start an election
I20260709 12:30:16.449937 24849 raft_consensus.cc:1081] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Signalling peer c9ee0c5aee904bc6a5597cc141dcd3e3 to start an election
I20260709 12:30:16.451807 24777 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "96761ddfa0044fea9016033ae2131861"
dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
 from {username='slave'} at 127.0.0.1:35924
I20260709 12:30:16.453217 24777 raft_consensus.cc:493] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:16.453662 24777 raft_consensus.cc:3060] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.457454 24884 consensus_queue.cc:1048] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:16.462035 24777 raft_consensus.cc:515] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.464455 24777 leader_election.cc:290] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 2 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:16.465934 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96761ddfa0044fea9016033ae2131861" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:16.469413 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96761ddfa0044fea9016033ae2131861" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:16.470042 24626 raft_consensus.cc:3055] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:16.470342 24626 raft_consensus.cc:740] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:16.471032 24626 consensus_queue.cc:260] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.472047 24626 raft_consensus.cc:3060] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.477937 24626 raft_consensus.cc:2468] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 2.
I20260709 12:30:16.479136 24741 leader_election.cc:304] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:16.479995 24864 raft_consensus.cc:2804] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:16.480487 24864 raft_consensus.cc:697] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 LEADER]: Becoming Leader. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:16.481308 24864 consensus_queue.cc:237] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.488721 24524 catalog_manager.cc:5697] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 reported cstate change: term changed from 1 to 2, leader changed from 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193) to c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:16.593823 24890 raft_consensus.cc:993] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d: : Instructing follower 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:16.594341 24865 raft_consensus.cc:1081] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Signalling peer 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:16.596153 24626 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:52238
I20260709 12:30:16.596668 24626 raft_consensus.cc:493] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:16.596963 24626 raft_consensus.cc:3060] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.602535 24626 raft_consensus.cc:515] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.604233 24626 leader_election.cc:290] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:16.605660 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:16.605611 24700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6d64ccaae0eb400f8010fb8849239fc3" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:16.606513 24777 raft_consensus.cc:3060] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.606566 24700 raft_consensus.cc:3055] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:16.607025 24700 raft_consensus.cc:740] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:16.607888 24700 consensus_queue.cc:260] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.609222 24700 raft_consensus.cc:3060] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.614383 24777 raft_consensus.cc:2468] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:16.615780 24592 leader_election.cc:304] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:16.616628 24849 raft_consensus.cc:2804] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:16.616775 24700 raft_consensus.cc:2468] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:16.617079 24849 raft_consensus.cc:697] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:16.618088 24849 consensus_queue.cc:237] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:16.625164 24524 catalog_manager.cc:5697] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 1 to 2, leader changed from 4b08b50e5e4146fa845229e427293f3d (127.8.137.194) to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:16.729868 24700 raft_consensus.cc:1275] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.731153 24849 consensus_queue.cc:1048] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:16.738358 24777 raft_consensus.cc:3060] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.750031 24777 raft_consensus.cc:1275] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.751456 24849 consensus_queue.cc:1048] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:16.781556 24776 raft_consensus.cc:1275] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.782958 24865 consensus_queue.cc:1048] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:16.791159 24626 raft_consensus.cc:1275] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.793182 24835 consensus_queue.cc:1048] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:16.795630 24626 raft_consensus.cc:1275] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.797181 24835 consensus_queue.cc:1048] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:16.807052 24776 raft_consensus.cc:1275] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.808697 24824 consensus_queue.cc:1048] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:16.889425 24700 raft_consensus.cc:3060] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:16.895184 24700 raft_consensus.cc:1275] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Refusing update from remote peer c9ee0c5aee904bc6a5597cc141dcd3e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.896697 24864 consensus_queue.cc:1048] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:16.906251 24626 raft_consensus.cc:1275] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Refusing update from remote peer c9ee0c5aee904bc6a5597cc141dcd3e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.908034 24902 consensus_queue.cc:1048] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:16.927626 24626 raft_consensus.cc:1275] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.927764 24776 raft_consensus.cc:1275] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:16.929081 24890 consensus_queue.cc:1048] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:16.929919 24824 consensus_queue.cc:1048] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:17.110777 24700 raft_consensus.cc:1275] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:17.112142 24884 consensus_queue.cc:1048] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:17.122025 24776 raft_consensus.cc:1275] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:17.124274 24825 consensus_queue.cc:1048] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:19.272704 24524 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46206:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:30:19.276894 24524 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:30:19.335997 24612 tablet_service.cc:1511] Processing CreateTablet for tablet ad96890857ec428694da7080c2fddf26 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:19.337368 24612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad96890857ec428694da7080c2fddf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.337080 24687 tablet_service.cc:1511] Processing CreateTablet for tablet ad96890857ec428694da7080c2fddf26 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:19.338449 24687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad96890857ec428694da7080c2fddf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.338809 24764 tablet_service.cc:1511] Processing CreateTablet for tablet ad96890857ec428694da7080c2fddf26 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:30:19.340132 24764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad96890857ec428694da7080c2fddf26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.341504 24689 tablet_service.cc:1511] Processing CreateTablet for tablet a12efdb159cf47dcb9d429a97eeef237 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:19.342914 24689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12efdb159cf47dcb9d429a97eeef237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.343931 24759 tablet_service.cc:1511] Processing CreateTablet for tablet a12efdb159cf47dcb9d429a97eeef237 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:19.344149 24610 tablet_service.cc:1511] Processing CreateTablet for tablet a12efdb159cf47dcb9d429a97eeef237 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:30:19.345289 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12efdb159cf47dcb9d429a97eeef237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.345505 24610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a12efdb159cf47dcb9d429a97eeef237. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.365546 24617 tablet_service.cc:1511] Processing CreateTablet for tablet ab45a25466a1484081f5e7e94ff26561 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:19.367132 24617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab45a25466a1484081f5e7e94ff26561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.367439 24685 tablet_service.cc:1511] Processing CreateTablet for tablet ab45a25466a1484081f5e7e94ff26561 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:19.368767 24685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab45a25466a1484081f5e7e94ff26561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.368901 24764 tablet_service.cc:1511] Processing CreateTablet for tablet ab45a25466a1484081f5e7e94ff26561 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:30:19.370206 24764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab45a25466a1484081f5e7e94ff26561. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.383587 24613 tablet_service.cc:1511] Processing CreateTablet for tablet caf686d556df490cbb73587efca0b6dd (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:19.385107 24613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf686d556df490cbb73587efca0b6dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.386920 24691 tablet_service.cc:1511] Processing CreateTablet for tablet caf686d556df490cbb73587efca0b6dd (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:19.388899 24691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf686d556df490cbb73587efca0b6dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.402437 24919 tablet_bootstrap.cc:492] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:19.411351 24764 tablet_service.cc:1511] Processing CreateTablet for tablet caf686d556df490cbb73587efca0b6dd (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:30:19.416272 24920 tablet_bootstrap.cc:492] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:19.416519 24764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf686d556df490cbb73587efca0b6dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.426116 24919 tablet_bootstrap.cc:654] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.432243 24920 tablet_bootstrap.cc:654] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.431849 24612 tablet_service.cc:1511] Processing CreateTablet for tablet 1bfa91f364ec497198b5528195934b51 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:19.434741 24685 tablet_service.cc:1511] Processing CreateTablet for tablet 1bfa91f364ec497198b5528195934b51 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:19.438791 24921 tablet_bootstrap.cc:492] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.440656 24759 tablet_service.cc:1511] Processing CreateTablet for tablet 1bfa91f364ec497198b5528195934b51 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:30:19.442283 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bfa91f364ec497198b5528195934b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.445645 24617 tablet_service.cc:1511] Processing CreateTablet for tablet d169eba347c2424a9bb86068108e7f62 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:19.446254 24613 tablet_service.cc:1511] Processing CreateTablet for tablet 1a8fed2a979948a3bc7bd4585b8421d4 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:19.446569 24761 tablet_service.cc:1511] Processing CreateTablet for tablet 1a8fed2a979948a3bc7bd4585b8421d4 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:19.447923 24761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a8fed2a979948a3bc7bd4585b8421d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.448262 24687 tablet_service.cc:1511] Processing CreateTablet for tablet 1a8fed2a979948a3bc7bd4585b8421d4 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:30:19.451565 24685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bfa91f364ec497198b5528195934b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.449240 24612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1bfa91f364ec497198b5528195934b51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.451807 24763 tablet_service.cc:1511] Processing CreateTablet for tablet d169eba347c2424a9bb86068108e7f62 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:19.449461 24689 tablet_service.cc:1511] Processing CreateTablet for tablet d169eba347c2424a9bb86068108e7f62 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:30:19.452939 24617 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d169eba347c2424a9bb86068108e7f62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.453332 24763 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d169eba347c2424a9bb86068108e7f62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.463017 24687 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a8fed2a979948a3bc7bd4585b8421d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.463889 24689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d169eba347c2424a9bb86068108e7f62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.468876 24921 tablet_bootstrap.cc:654] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.471140 24610 tablet_service.cc:1511] Processing CreateTablet for tablet f5073cee60714ee9b1d6e5eb034177f7 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:19.472671 24610 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5073cee60714ee9b1d6e5eb034177f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.474972 24613 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1a8fed2a979948a3bc7bd4585b8421d4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.492540 24919 tablet_bootstrap.cc:492] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:19.493438 24919 ts_tablet_manager.cc:1403] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.091s	user 0.009s	sys 0.007s
I20260709 12:30:19.497469 24919 raft_consensus.cc:359] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.498523 24919 raft_consensus.cc:385] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.503486 24919 raft_consensus.cc:740] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.497632 24691 tablet_service.cc:1511] Processing CreateTablet for tablet f5073cee60714ee9b1d6e5eb034177f7 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:19.504335 24919 consensus_queue.cc:260] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.505059 24691 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5073cee60714ee9b1d6e5eb034177f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.505074 24920 tablet_bootstrap.cc:492] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:19.511710 24920 ts_tablet_manager.cc:1403] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.096s	user 0.006s	sys 0.007s
I20260709 12:30:19.513446 24921 tablet_bootstrap.cc:492] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:19.514349 24921 ts_tablet_manager.cc:1403] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.076s	user 0.014s	sys 0.002s
I20260709 12:30:19.517637 24921 raft_consensus.cc:359] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.506434 24919 ts_tablet_manager.cc:1434] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:30:19.519835 24919 tablet_bootstrap.cc:492] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:19.518399 24921 raft_consensus.cc:385] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.521765 24921 raft_consensus.cc:740] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.522567 24921 consensus_queue.cc:260] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.525113 24921 ts_tablet_manager.cc:1434] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.010s	user 0.003s	sys 0.002s
I20260709 12:30:19.526149 24921 tablet_bootstrap.cc:492] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.527153 24920 raft_consensus.cc:359] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.527983 24920 raft_consensus.cc:385] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.528270 24920 raft_consensus.cc:740] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.529027 24920 consensus_queue.cc:260] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.534143 24919 tablet_bootstrap.cc:654] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.528905 24759 tablet_service.cc:1511] Processing CreateTablet for tablet f5073cee60714ee9b1d6e5eb034177f7 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:30:19.538203 24759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f5073cee60714ee9b1d6e5eb034177f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.539570 24920 ts_tablet_manager.cc:1434] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.028s	user 0.000s	sys 0.007s
I20260709 12:30:19.540696 24920 tablet_bootstrap.cc:492] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:19.546530 24689 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4fa56ac4f74ff4911afffedc164b88 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:19.554803 24689 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4fa56ac4f74ff4911afffedc164b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.556506 24920 tablet_bootstrap.cc:654] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.559957 24764 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4fa56ac4f74ff4911afffedc164b88 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:30:19.561842 24764 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4fa56ac4f74ff4911afffedc164b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.572501 24921 tablet_bootstrap.cc:654] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.573990 24612 tablet_service.cc:1511] Processing CreateTablet for tablet 0f4fa56ac4f74ff4911afffedc164b88 (DEFAULT_TABLE table=soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
W20260709 12:30:19.579257 24574 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:30:19.584898 24612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f4fa56ac4f74ff4911afffedc164b88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:30:19.584951 24849 raft_consensus.cc:493] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.586931 24849 raft_consensus.cc:515] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.590075 24700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12efdb159cf47dcb9d429a97eeef237" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
W20260709 12:30:19.591846 24592 leader_election.cc:343] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.594158 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12efdb159cf47dcb9d429a97eeef237" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:19.594895 24777 raft_consensus.cc:2468] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:19.596478 24592 leader_election.cc:304] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:19.597456 24921 tablet_bootstrap.cc:492] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:19.597714 24849 leader_election.cc:290] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:19.598079 24921 ts_tablet_manager.cc:1403] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.072s	user 0.017s	sys 0.004s
I20260709 12:30:19.598439 24849 raft_consensus.cc:2804] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.598904 24849 raft_consensus.cc:493] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.599596 24849 raft_consensus.cc:3060] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.604393 24921 raft_consensus.cc:359] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.605278 24921 raft_consensus.cc:385] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.605576 24921 raft_consensus.cc:740] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.611466 24921 consensus_queue.cc:260] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.614390 24921 ts_tablet_manager.cc:1434] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:30:19.615424 24921 tablet_bootstrap.cc:492] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.630803 24921 tablet_bootstrap.cc:654] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.636673 24919 tablet_bootstrap.cc:492] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:19.637251 24919 ts_tablet_manager.cc:1403] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.118s	user 0.009s	sys 0.028s
I20260709 12:30:19.640477 24919 raft_consensus.cc:359] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.641287 24919 raft_consensus.cc:385] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.641601 24919 raft_consensus.cc:740] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.642580 24919 consensus_queue.cc:260] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.645658 24919 ts_tablet_manager.cc:1434] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:30:19.649235 24921 tablet_bootstrap.cc:492] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:19.649935 24921 ts_tablet_manager.cc:1403] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.004s
I20260709 12:30:19.651217 24919 tablet_bootstrap.cc:492] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:19.653616 24921 raft_consensus.cc:359] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.654465 24921 raft_consensus.cc:385] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.654815 24921 raft_consensus.cc:740] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.655582 24921 consensus_queue.cc:260] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.658087 24921 ts_tablet_manager.cc:1434] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:30:19.659205 24921 tablet_bootstrap.cc:492] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.663007 24919 tablet_bootstrap.cc:654] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.667385 24921 tablet_bootstrap.cc:654] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.680063 24921 tablet_bootstrap.cc:492] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:19.680655 24921 ts_tablet_manager.cc:1403] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.008s
I20260709 12:30:19.683943 24921 raft_consensus.cc:359] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.684696 24921 raft_consensus.cc:385] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.685086 24921 raft_consensus.cc:740] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.685901 24921 consensus_queue.cc:260] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.687074 24849 raft_consensus.cc:515] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.688500 24921 ts_tablet_manager.cc:1434] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:30:19.691784 24920 tablet_bootstrap.cc:492] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:19.693832 24920 ts_tablet_manager.cc:1403] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.153s	user 0.048s	sys 0.085s
I20260709 12:30:19.694536 24849 leader_election.cc:290] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:19.695762 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12efdb159cf47dcb9d429a97eeef237" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:19.696424 24777 raft_consensus.cc:3060] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.698850 24920 raft_consensus.cc:359] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.699867 24700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a12efdb159cf47dcb9d429a97eeef237" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:19.700160 24920 raft_consensus.cc:385] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.700623 24700 raft_consensus.cc:3060] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.700835 24920 raft_consensus.cc:740] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.702148 24920 consensus_queue.cc:260] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.704532 24777 raft_consensus.cc:2468] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:19.706061 24592 leader_election.cc:304] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:19.706004 24920 ts_tablet_manager.cc:1434] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.012s	user 0.010s	sys 0.000s
I20260709 12:30:19.707708 24928 raft_consensus.cc:2804] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.708182 24928 raft_consensus.cc:697] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:19.708508 24920 tablet_bootstrap.cc:492] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:19.709048 24928 consensus_queue.cc:237] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.712769 24700 raft_consensus.cc:2468] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:19.718323 24920 tablet_bootstrap.cc:654] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.732993 24920 tablet_bootstrap.cc:492] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:19.733723 24920 ts_tablet_manager.cc:1403] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.007s
I20260709 12:30:19.738101 24920 raft_consensus.cc:359] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.739316 24920 raft_consensus.cc:385] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.739732 24920 raft_consensus.cc:740] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.740693 24920 consensus_queue.cc:260] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.744036 24920 ts_tablet_manager.cc:1434] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.008s
I20260709 12:30:19.745591 24920 tablet_bootstrap.cc:492] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:19.755757 24921 tablet_bootstrap.cc:492] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.756152 24920 tablet_bootstrap.cc:654] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.763473 24921 tablet_bootstrap.cc:654] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.786123 24920 tablet_bootstrap.cc:492] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:19.786805 24920 ts_tablet_manager.cc:1403] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.042s	user 0.004s	sys 0.023s
I20260709 12:30:19.789077 24920 raft_consensus.cc:359] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.789686 24920 raft_consensus.cc:385] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.789954 24920 raft_consensus.cc:740] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
W20260709 12:30:19.783630 24592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41381, user_credentials={real_user=slave}} blocked reactor thread for 57945.2us
I20260709 12:30:19.790536 24920 consensus_queue.cc:260] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.798846 24920 ts_tablet_manager.cc:1434] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:30:19.800158 24920 tablet_bootstrap.cc:492] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:19.800370 24919 tablet_bootstrap.cc:492] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:19.800987 24919 ts_tablet_manager.cc:1403] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.150s	user 0.040s	sys 0.086s
I20260709 12:30:19.804177 24919 raft_consensus.cc:359] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.804930 24919 raft_consensus.cc:385] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.805233 24919 raft_consensus.cc:740] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.805168 24526 catalog_manager.cc:5697] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:19.806200 24919 consensus_queue.cc:260] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.807791 24920 tablet_bootstrap.cc:654] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.808655 24921 tablet_bootstrap.cc:492] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:19.808689 24919 ts_tablet_manager.cc:1434] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:30:19.809434 24921 ts_tablet_manager.cc:1403] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.054s	user 0.009s	sys 0.008s
I20260709 12:30:19.810034 24919 tablet_bootstrap.cc:492] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:19.816371 24919 tablet_bootstrap.cc:654] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.817790 24921 raft_consensus.cc:359] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.818964 24921 raft_consensus.cc:385] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.819382 24921 raft_consensus.cc:740] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.820567 24921 consensus_queue.cc:260] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.824220 24921 ts_tablet_manager.cc:1434] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.004s
I20260709 12:30:19.825363 24921 tablet_bootstrap.cc:492] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.835357 24921 tablet_bootstrap.cc:654] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.844297 24920 tablet_bootstrap.cc:492] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:19.845041 24920 ts_tablet_manager.cc:1403] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.045s	user 0.010s	sys 0.011s
I20260709 12:30:19.848366 24920 raft_consensus.cc:359] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.849160 24920 raft_consensus.cc:385] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.849473 24920 raft_consensus.cc:740] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.850286 24920 consensus_queue.cc:260] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.852077 24921 tablet_bootstrap.cc:492] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:19.852866 24921 ts_tablet_manager.cc:1403] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.010s
I20260709 12:30:19.852909 24920 ts_tablet_manager.cc:1434] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.006s
I20260709 12:30:19.854070 24920 tablet_bootstrap.cc:492] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:19.856253 24921 raft_consensus.cc:359] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.857012 24921 raft_consensus.cc:385] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.857326 24921 raft_consensus.cc:740] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.858145 24921 consensus_queue.cc:260] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.861034 24920 tablet_bootstrap.cc:654] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.861057 24921 ts_tablet_manager.cc:1434] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:30:19.862514 24921 tablet_bootstrap.cc:492] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.871241 24921 tablet_bootstrap.cc:654] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.873337 24920 tablet_bootstrap.cc:492] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:19.874059 24920 ts_tablet_manager.cc:1403] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.005s
I20260709 12:30:19.874488 24903 raft_consensus.cc:493] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.875275 24903 raft_consensus.cc:515] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.878289 24903 leader_election.cc:290] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:19.878108 24920 raft_consensus.cc:359] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.879618 24920 raft_consensus.cc:385] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.879954 24920 raft_consensus.cc:740] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.880309 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad96890857ec428694da7080c2fddf26" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:19.880973 24777 raft_consensus.cc:2468] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:19.880731 24920 consensus_queue.cc:260] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.879070 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad96890857ec428694da7080c2fddf26" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:19.882741 24626 raft_consensus.cc:2468] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:19.883365 24920 ts_tablet_manager.cc:1434] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.003s
I20260709 12:30:19.884155 24667 leader_election.cc:304] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:19.884526 24920 tablet_bootstrap.cc:492] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:19.885682 24903 raft_consensus.cc:2804] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.886053 24903 raft_consensus.cc:493] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.886492 24903 raft_consensus.cc:3060] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.892041 24921 tablet_bootstrap.cc:492] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:19.892654 24921 ts_tablet_manager.cc:1403] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.030s	user 0.014s	sys 0.005s
I20260709 12:30:19.895714 24921 raft_consensus.cc:359] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.896535 24921 raft_consensus.cc:385] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.896842 24921 raft_consensus.cc:740] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.897606 24921 consensus_queue.cc:260] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.899243 24903 raft_consensus.cc:515] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.902340 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad96890857ec428694da7080c2fddf26" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:19.902765 24919 tablet_bootstrap.cc:492] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:19.902966 24626 raft_consensus.cc:3060] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.903338 24919 ts_tablet_manager.cc:1403] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.094s	user 0.038s	sys 0.045s
I20260709 12:30:19.904081 24920 tablet_bootstrap.cc:654] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.906157 24919 raft_consensus.cc:359] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.907076 24919 raft_consensus.cc:385] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.907449 24919 raft_consensus.cc:740] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.908285 24919 consensus_queue.cc:260] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.910496 24919 ts_tablet_manager.cc:1434] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:30:19.911614 24919 tablet_bootstrap.cc:492] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:19.920070 24919 tablet_bootstrap.cc:654] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.920288 24921 ts_tablet_manager.cc:1434] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 12:30:19.925098 24921 tablet_bootstrap.cc:492] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.925787 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad96890857ec428694da7080c2fddf26" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:19.926494 24777 raft_consensus.cc:3060] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.927879 24931 raft_consensus.cc:493] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:19.928437 24931 raft_consensus.cc:515] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.931787 24931 leader_election.cc:290] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:19.932287 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf686d556df490cbb73587efca0b6dd" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:19.932711 24921 tablet_bootstrap.cc:654] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.933050 24628 raft_consensus.cc:2468] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:19.935405 24667 leader_election.cc:304] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:19.936643 24931 raft_consensus.cc:2804] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:19.937167 24931 raft_consensus.cc:493] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:19.937613 24931 raft_consensus.cc:3060] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.935315 24919 tablet_bootstrap.cc:492] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:19.935367 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf686d556df490cbb73587efca0b6dd" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:19.943193 24919 ts_tablet_manager.cc:1403] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.032s	user 0.008s	sys 0.006s
W20260709 12:30:19.944298 24667 leader_election.cc:343] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:19.947273 24903 leader_election.cc:290] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:19.916254 24626 raft_consensus.cc:2468] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:19.949409 24667 leader_election.cc:304] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:19.949290 24931 raft_consensus.cc:515] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.950810 24903 raft_consensus.cc:2804] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.951650 24903 raft_consensus.cc:697] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:19.950644 24919 raft_consensus.cc:359] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.952491 24919 raft_consensus.cc:385] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.952803 24919 raft_consensus.cc:740] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.953627 24919 consensus_queue.cc:260] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.955039 24777 raft_consensus.cc:2468] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:19.955402 24903 consensus_queue.cc:237] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.956749 24931 leader_election.cc:290] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:19.957410 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf686d556df490cbb73587efca0b6dd" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:19.958838 24776 raft_consensus.cc:3060] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.960286 24919 ts_tablet_manager.cc:1434] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:30:19.961432 24919 tablet_bootstrap.cc:492] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:19.955108 24921 tablet_bootstrap.cc:492] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:19.963379 24921 ts_tablet_manager.cc:1403] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.004s
I20260709 12:30:19.965726 24920 tablet_bootstrap.cc:492] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:19.966372 24920 ts_tablet_manager.cc:1403] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.082s	user 0.030s	sys 0.016s
I20260709 12:30:19.966480 24921 raft_consensus.cc:359] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.967252 24921 raft_consensus.cc:385] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.957625 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf686d556df490cbb73587efca0b6dd" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:19.967423 24776 raft_consensus.cc:2468] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:19.967857 24626 raft_consensus.cc:3060] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:19.969542 24920 raft_consensus.cc:359] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.970338 24920 raft_consensus.cc:385] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:19.970664 24920 raft_consensus.cc:740] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.971457 24920 consensus_queue.cc:260] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.973968 24920 ts_tablet_manager.cc:1434] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:30:19.974995 24919 tablet_bootstrap.cc:654] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:19.967518 24921 raft_consensus.cc:740] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:19.975567 24667 leader_election.cc:304] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:19.976521 24921 consensus_queue.cc:260] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:19.977625 24945 raft_consensus.cc:2804] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:19.981755 24626 raft_consensus.cc:2468] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:19.983111 24921 ts_tablet_manager.cc:1434] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:30:19.983834 24920 tablet_bootstrap.cc:492] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:19.992684 24921 tablet_bootstrap.cc:492] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d: Bootstrap starting.
I20260709 12:30:19.999888 24921 tablet_bootstrap.cc:654] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.002408 24920 tablet_bootstrap.cc:654] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.004703 24525 catalog_manager.cc:5697] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 1 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.004071 24919 tablet_bootstrap.cc:492] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:20.013119 24933 raft_consensus.cc:493] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.013839 24933 raft_consensus.cc:515] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.018781 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:20.021453 24921 tablet_bootstrap.cc:492] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d: No bootstrap required, opened a new log
I20260709 12:30:20.022034 24921 ts_tablet_manager.cc:1403] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.001s
I20260709 12:30:20.022831 24700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:20.023890 24700 raft_consensus.cc:2468] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:20.025154 24921 raft_consensus.cc:359] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.026078 24921 raft_consensus.cc:385] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.026397 24921 raft_consensus.cc:740] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.026968 24592 leader_election.cc:304] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.027948 24903 raft_consensus.cc:493] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:30:20.028052 24592 leader_election.cc:343] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.028110 24936 raft_consensus.cc:2804] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.028584 24903 raft_consensus.cc:515] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.028959 24936 raft_consensus.cc:493] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.012691 24919 ts_tablet_manager.cc:1403] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.051s	user 0.023s	sys 0.008s
I20260709 12:30:20.031395 24903 leader_election.cc:290] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.029567 24936 raft_consensus.cc:3060] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.033561 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d169eba347c2424a9bb86068108e7f62" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
W20260709 12:30:20.035228 24667 leader_election.cc:343] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.029063 24933 leader_election.cc:290] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.034910 24919 raft_consensus.cc:359] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.036331 24919 raft_consensus.cc:385] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.036638 24919 raft_consensus.cc:740] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.037318 24933 raft_consensus.cc:493] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.027402 24921 consensus_queue.cc:260] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.037891 24933 raft_consensus.cc:515] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.039525 24908 raft_consensus.cc:493] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.037228 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d169eba347c2424a9bb86068108e7f62" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:20.039950 24933 leader_election.cc:290] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.040462 24626 raft_consensus.cc:2468] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:20.040264 24908 raft_consensus.cc:515] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.042049 24667 leader_election.cc:304] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:20.040020 24921 ts_tablet_manager.cc:1434] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 12:30:20.043437 24908 leader_election.cc:290] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:20.043599 24903 raft_consensus.cc:2804] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.041966 24919 consensus_queue.cc:260] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.044214 24903 raft_consensus.cc:493] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.044773 24903 raft_consensus.cc:3060] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.046164 24919 ts_tablet_manager.cc:1434] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:30:20.046283 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bfa91f364ec497198b5528195934b51" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:20.047056 24702 raft_consensus.cc:2468] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:20.047387 24919 tablet_bootstrap.cc:492] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:20.049870 24592 leader_election.cc:304] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.051182 24928 raft_consensus.cc:2804] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.051539 24928 raft_consensus.cc:493] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.051854 24928 raft_consensus.cc:3060] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.054087 24903 raft_consensus.cc:515] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.056087 24919 tablet_bootstrap.cc:654] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.056674 24903 leader_election.cc:290] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.059810 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d169eba347c2424a9bb86068108e7f62" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:20.060510 24626 raft_consensus.cc:3060] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.061674 24936 raft_consensus.cc:515] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.043690 24945 raft_consensus.cc:697] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:20.067510 24920 tablet_bootstrap.cc:492] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:20.068146 24920 ts_tablet_manager.cc:1403] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.085s	user 0.007s	sys 0.010s
I20260709 12:30:20.068181 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d169eba347c2424a9bb86068108e7f62" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.070358 24903 raft_consensus.cc:493] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.071326 24903 raft_consensus.cc:515] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
W20260709 12:30:20.073211 24667 leader_election.cc:343] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.066735 24945 consensus_queue.cc:237] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.075546 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:20.076253 24628 raft_consensus.cc:2468] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 0.
I20260709 12:30:20.078121 24626 raft_consensus.cc:2468] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:20.079840 24903 leader_election.cc:290] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.079861 24700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:20.081004 24700 raft_consensus.cc:2468] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 0.
I20260709 12:30:20.082638 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:20.083238 24776 raft_consensus.cc:2468] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:20.083896 24936 leader_election.cc:290] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.084227 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bfa91f364ec497198b5528195934b51" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:20.084540 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:20.084900 24776 raft_consensus.cc:2468] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:20.085160 24628 raft_consensus.cc:2468] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:20.087700 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:20.088320 24702 raft_consensus.cc:3060] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.088405 24741 leader_election.cc:304] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:20.090277 24908 raft_consensus.cc:2804] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.090677 24908 raft_consensus.cc:493] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.091015 24908 raft_consensus.cc:3060] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.096534 24667 leader_election.cc:304] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.106560 24903 raft_consensus.cc:2804] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.107124 24903 raft_consensus.cc:493] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.107580 24903 raft_consensus.cc:3060] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.102865 24908 raft_consensus.cc:515] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.104605 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
W20260709 12:30:20.110641 24592 leader_election.cc:343] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:30:20.111282 24908 leader_election.cc:290] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:20.081044 24928 raft_consensus.cc:515] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.112567 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:20.113194 24628 raft_consensus.cc:3060] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.114499 24928 leader_election.cc:290] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.119496 24920 raft_consensus.cc:359] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.120230 24920 raft_consensus.cc:385] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.105603 24702 raft_consensus.cc:2468] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:20.121299 24667 leader_election.cc:304] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:20.120507 24920 raft_consensus.cc:740] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.123663 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:20.123471 24920 consensus_queue.cc:260] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.124454 24700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bfa91f364ec497198b5528195934b51" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:20.127074 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bfa91f364ec497198b5528195934b51" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.127398 24700 raft_consensus.cc:3060] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.127712 24776 raft_consensus.cc:3060] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.129570 24919 tablet_bootstrap.cc:492] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:20.125289 24592 leader_election.cc:304] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:20.125504 24945 raft_consensus.cc:2804] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.130147 24919 ts_tablet_manager.cc:1403] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.083s	user 0.035s	sys 0.043s
I20260709 12:30:20.130906 24903 raft_consensus.cc:515] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.133772 24903 leader_election.cc:290] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.130458 24945 raft_consensus.cc:697] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:20.134269 24950 raft_consensus.cc:493] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.131654 24936 raft_consensus.cc:2804] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.135078 24936 raft_consensus.cc:697] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.134812 24950 raft_consensus.cc:515] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.134864 24945 consensus_queue.cc:237] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.135974 24936 consensus_queue.cc:237] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.137053 24950 leader_election.cc:290] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.137048 24700 raft_consensus.cc:2468] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:20.138885 24627 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a8fed2a979948a3bc7bd4585b8421d4" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:20.139578 24627 raft_consensus.cc:2468] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:20.139725 24775 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a8fed2a979948a3bc7bd4585b8421d4" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:20.140482 24775 raft_consensus.cc:2468] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 0.
I20260709 12:30:20.138445 24525 catalog_manager.cc:5697] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 1 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.140990 24920 ts_tablet_manager.cc:1434] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.072s	user 0.007s	sys 0.000s
I20260709 12:30:20.133807 24919 raft_consensus.cc:359] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.141539 24919 raft_consensus.cc:385] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.141938 24919 raft_consensus.cc:740] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.142087 24920 tablet_bootstrap.cc:492] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10: Bootstrap starting.
I20260709 12:30:20.142664 24592 leader_election.cc:304] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.142959 24919 consensus_queue.cc:260] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.143661 24849 raft_consensus.cc:2804] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.144176 24849 raft_consensus.cc:697] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.145025 24849 consensus_queue.cc:237] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.146059 24919 ts_tablet_manager.cc:1434] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.002s
I20260709 12:30:20.147114 24919 tablet_bootstrap.cc:492] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:20.148233 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:20.141995 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.153411 24777 raft_consensus.cc:2393] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 4b08b50e5e4146fa845229e427293f3d in current term 1: Already voted for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in this term.
I20260709 12:30:20.154165 24919 tablet_bootstrap.cc:654] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.142726 24702 raft_consensus.cc:2393] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in current term 1: Already voted for candidate 4b08b50e5e4146fa845229e427293f3d in this term.
I20260709 12:30:20.161157 24667 leader_election.cc:304] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:20.163178 24667 leader_election.cc:304] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d; no voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:20.164803 24903 raft_consensus.cc:2804] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.165504 24950 raft_consensus.cc:2749] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.166049 24776 raft_consensus.cc:2468] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:20.163477 24920 tablet_bootstrap.cc:654] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.165478 24903 raft_consensus.cc:493] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.167590 24903 raft_consensus.cc:3060] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.126263 24628 raft_consensus.cc:2468] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 1.
I20260709 12:30:20.173327 24741 leader_election.cc:304] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:20.174670 24919 tablet_bootstrap.cc:492] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:20.175247 24919 ts_tablet_manager.cc:1403] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.028s	user 0.021s	sys 0.004s
I20260709 12:30:20.181988 24920 tablet_bootstrap.cc:492] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10: No bootstrap required, opened a new log
I20260709 12:30:20.182564 24920 ts_tablet_manager.cc:1403] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.001s
I20260709 12:30:20.183357 24908 raft_consensus.cc:2804] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.183905 24908 raft_consensus.cc:697] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Becoming Leader. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:20.184749 24908 consensus_queue.cc:237] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.185434 24920 raft_consensus.cc:359] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.185135 24903 raft_consensus.cc:515] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.186246 24920 raft_consensus.cc:385] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.187832 24919 raft_consensus.cc:359] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.189962 24920 raft_consensus.cc:740] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.190109 24919 raft_consensus.cc:385] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.190807 24919 raft_consensus.cc:740] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.191643 24919 consensus_queue.cc:260] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.192764 24920 consensus_queue.cc:260] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.193949 24919 ts_tablet_manager.cc:1434] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:30:20.198490 24920 ts_tablet_manager.cc:1434] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 12:30:20.198555 24903 leader_election.cc:290] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.199882 24919 tablet_bootstrap.cc:492] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3: Bootstrap starting.
I20260709 12:30:20.204794 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a8fed2a979948a3bc7bd4585b8421d4" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:20.205694 24628 raft_consensus.cc:3060] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.200687 24525 catalog_manager.cc:5697] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.210441 24919 tablet_bootstrap.cc:654] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3: Neither blocks nor log segments found. Creating new log.
I20260709 12:30:20.218499 24628 raft_consensus.cc:2468] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:20.235699 24524 catalog_manager.cc:5697] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 1 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.236689 24523 catalog_manager.cc:5697] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195). New cstate: current_term: 1 leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:20.237984 24667 leader_election.cc:304] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.231966 24919 tablet_bootstrap.cc:492] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3: No bootstrap required, opened a new log
I20260709 12:30:20.240355 24945 raft_consensus.cc:2804] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.240525 24919 ts_tablet_manager.cc:1403] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent bootstrapping tablet: real 0.046s	user 0.015s	sys 0.013s
I20260709 12:30:20.241219 24945 raft_consensus.cc:697] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:20.242138 24945 consensus_queue.cc:237] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.244321 24919 raft_consensus.cc:359] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.245286 24919 raft_consensus.cc:385] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:30:20.245622 24919 raft_consensus.cc:740] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Initialized, Role: FOLLOWER
I20260709 12:30:20.246348 24919 consensus_queue.cc:260] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.233946 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1a8fed2a979948a3bc7bd4585b8421d4" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.247660 24776 raft_consensus.cc:3060] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.248637 24919 ts_tablet_manager.cc:1434] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:30:20.261119 24776 raft_consensus.cc:2468] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 1.
I20260709 12:30:20.262854 24936 raft_consensus.cc:493] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.263424 24936 raft_consensus.cc:515] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.265722 24936 leader_election.cc:290] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.265136 24523 catalog_manager.cc:5697] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.267683 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4fa56ac4f74ff4911afffedc164b88" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:20.268400 24702 raft_consensus.cc:2468] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:20.269454 24592 leader_election.cc:304] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.270283 24936 raft_consensus.cc:2804] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:30:20.270761 24936 raft_consensus.cc:493] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:30:20.271163 24936 raft_consensus.cc:3060] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.275532 24526 catalog_manager.cc:5697] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 1 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.276842 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4fa56ac4f74ff4911afffedc164b88" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" is_pre_election: true
I20260709 12:30:20.277281 24849 consensus_queue.cc:1048] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.277607 24776 raft_consensus.cc:2468] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 0.
I20260709 12:30:20.279127 24936 raft_consensus.cc:515] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.281301 24936 leader_election.cc:290] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.286079 24700 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4fa56ac4f74ff4911afffedc164b88" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:20.286862 24700 raft_consensus.cc:3060] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.294842 24849 consensus_queue.cc:1048] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.296824 24700 raft_consensus.cc:2468] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:20.298197 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4fa56ac4f74ff4911afffedc164b88" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.298853 24776 raft_consensus.cc:3060] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.302901 24592 leader_election.cc:304] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.305224 24849 raft_consensus.cc:2804] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:30:20.306030 24849 raft_consensus.cc:697] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.305807 24776 raft_consensus.cc:2468] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 1.
I20260709 12:30:20.307137 24849 consensus_queue.cc:237] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.320993 24526 catalog_manager.cc:5697] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 1 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.386200 24945 consensus_queue.cc:1048] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.398475 24931 consensus_queue.cc:1048] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.454653 24702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ad96890857ec428694da7080c2fddf26"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
mode: GRACEFUL
new_leader_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:59770
I20260709 12:30:20.455148 24702 raft_consensus.cc:606] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Received request to transfer leadership to TS 6ce569893fd14fb995f3c987e622fc10
I20260709 12:30:20.455686 24908 raft_consensus.cc:493] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.456219 24908 raft_consensus.cc:515] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.458077 24908 leader_election.cc:290] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:20.458971 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d169eba347c2424a9bb86068108e7f62" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:20.459897 24628 raft_consensus.cc:2393] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in current term 1: Already voted for candidate 4b08b50e5e4146fa845229e427293f3d in this term.
I20260709 12:30:20.459513 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d169eba347c2424a9bb86068108e7f62" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:20.461874 24908 raft_consensus.cc:493] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:30:20.461825 24741 leader_election.cc:304] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10
I20260709 12:30:20.462495 24908 raft_consensus.cc:515] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.463145 24953 raft_consensus.cc:3060] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.464711 24908 leader_election.cc:290] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:20.465337 24777 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561"
dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
mode: GRACEFUL
new_leader_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:35932
I20260709 12:30:20.466029 24777 raft_consensus.cc:606] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Received request to transfer leadership to TS 6ce569893fd14fb995f3c987e622fc10
I20260709 12:30:20.466259 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ce569893fd14fb995f3c987e622fc10" is_pre_election: true
I20260709 12:30:20.466694 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b08b50e5e4146fa845229e427293f3d" is_pre_election: true
I20260709 12:30:20.467542 24702 raft_consensus.cc:2393] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in current term 1: Already voted for candidate 6ce569893fd14fb995f3c987e622fc10 in this term.
I20260709 12:30:20.468550 24741 leader_election.cc:304] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10
I20260709 12:30:20.469689 24908 raft_consensus.cc:3060] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:30:20.471318 24953 raft_consensus.cc:2749] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.473042 24702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "caf686d556df490cbb73587efca0b6dd"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
mode: GRACEFUL
new_leader_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:59770
I20260709 12:30:20.473623 24702 raft_consensus.cc:606] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Received request to transfer leadership to TS 6ce569893fd14fb995f3c987e622fc10
I20260709 12:30:20.476109 24908 raft_consensus.cc:2749] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:30:20.478785 24628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1bfa91f364ec497198b5528195934b51"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
mode: GRACEFUL
new_leader_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:52258
I20260709 12:30:20.479377 24628 raft_consensus.cc:606] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Received request to transfer leadership to TS 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:20.490811 24628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
mode: GRACEFUL
new_leader_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:52258
I20260709 12:30:20.491362 24628 raft_consensus.cc:606] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Received request to transfer leadership to TS 4b08b50e5e4146fa845229e427293f3d
I20260709 12:30:20.496966 24628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f4fa56ac4f74ff4911afffedc164b88"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
mode: GRACEFUL
new_leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
 from {username='slave'} at 127.0.0.1:52258
I20260709 12:30:20.497607 24628 raft_consensus.cc:606] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Received request to transfer leadership to TS c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:20.529603 24931 consensus_queue.cc:1048] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.540834 24950 consensus_queue.cc:1048] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.548892 24849 consensus_queue.cc:1048] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.554811 24931 raft_consensus.cc:993] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d: : Instructing follower 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:20.555341 24931 raft_consensus.cc:1081] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Signalling peer 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:20.557266 24628 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "caf686d556df490cbb73587efca0b6dd"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:52238
I20260709 12:30:20.557837 24628 raft_consensus.cc:493] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:20.558238 24628 raft_consensus.cc:3060] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.563787 24936 raft_consensus.cc:993] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: : Instructing follower 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:20.564352 24928 raft_consensus.cc:1081] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Signalling peer 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:20.564611 24628 raft_consensus.cc:515] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.566874 24628 leader_election.cc:290] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.568629 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf686d556df490cbb73587efca0b6dd" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.569197 24776 raft_consensus.cc:3060] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.569669 24701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf686d556df490cbb73587efca0b6dd" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:20.570334 24701 raft_consensus.cc:3055] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:20.570684 24701 raft_consensus.cc:740] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:20.571502 24701 consensus_queue.cc:260] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.572790 24701 raft_consensus.cc:3060] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.573441 24700 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1bfa91f364ec497198b5528195934b51"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:59754
I20260709 12:30:20.574072 24700 raft_consensus.cc:493] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:20.574445 24700 raft_consensus.cc:3060] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.577519 24933 consensus_queue.cc:1048] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.580719 24701 raft_consensus.cc:2468] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:20.583583 24936 consensus_queue.cc:1048] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.584089 24700 raft_consensus.cc:515] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.585719 24776 raft_consensus.cc:2468] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:20.586047 24592 leader_election.cc:304] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.586490 24700 leader_election.cc:290] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.587195 24936 raft_consensus.cc:2804] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:20.587816 24936 raft_consensus.cc:697] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.588793 24936 consensus_queue.cc:237] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.590566 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bfa91f364ec497198b5528195934b51" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:20.591277 24628 raft_consensus.cc:3055] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:20.591609 24628 raft_consensus.cc:740] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.592449 24628 consensus_queue.cc:260] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.593952 24628 raft_consensus.cc:3060] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.603323 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1bfa91f364ec497198b5528195934b51" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.603384 24628 raft_consensus.cc:2468] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:20.604142 24777 raft_consensus.cc:3060] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.611119 24525 catalog_manager.cc:5697] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 1 to 2, leader changed from 4b08b50e5e4146fa845229e427293f3d (127.8.137.194) to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.612118 24777 raft_consensus.cc:2468] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:20.616443 24667 leader_election.cc:304] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:20.618711 24931 raft_consensus.cc:2804] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:20.619222 24931 raft_consensus.cc:697] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:20.620077 24931 consensus_queue.cc:237] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.627903 24928 raft_consensus.cc:993] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10: : Instructing follower 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:20.628371 24928 raft_consensus.cc:1081] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Signalling peer 4b08b50e5e4146fa845229e427293f3d to start an election
I20260709 12:30:20.630133 24701 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
 from {username='slave'} at 127.0.0.1:59754
I20260709 12:30:20.630765 24701 raft_consensus.cc:493] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:20.631150 24701 raft_consensus.cc:3060] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.634174 24931 consensus_queue.cc:1048] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.636673 24928 consensus_queue.cc:1048] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.641417 24701 raft_consensus.cc:515] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.643144 24700 raft_consensus.cc:1240] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Rejecting Update request from peer 6ce569893fd14fb995f3c987e622fc10 for earlier term 1. Current term is 2. Ops: []
I20260709 12:30:20.645330 24526 catalog_manager.cc:5697] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 1 to 2, leader changed from 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193) to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 2 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.646062 24928 consensus_queue.cc:1059] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.646929 24928 raft_consensus.cc:3055] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:20.647260 24928 raft_consensus.cc:740] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.647938 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:20.648077 24928 consensus_queue.cc:260] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.649296 24928 raft_consensus.cc:3060] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.649775 24701 leader_election.cc:290] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.651127 24776 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f5073cee60714ee9b1d6e5eb034177f7" candidate_uuid: "4b08b50e5e4146fa845229e427293f3d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.651727 24776 raft_consensus.cc:3060] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.659869 24776 raft_consensus.cc:2468] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:20.664832 24667 leader_election.cc:304] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:20.666522 24950 raft_consensus.cc:2804] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:20.667078 24950 raft_consensus.cc:697] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Becoming Leader. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:20.667954 24950 consensus_queue.cc:237] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.663261 24931 consensus_queue.cc:1048] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:30:20.677819 24628 raft_consensus.cc:2468] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b08b50e5e4146fa845229e427293f3d in term 2.
I20260709 12:30:20.685432 24526 catalog_manager.cc:5697] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d reported cstate change: term changed from 1 to 2, leader changed from 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193) to 4b08b50e5e4146fa845229e427293f3d (127.8.137.194). New cstate: current_term: 2 leader_uuid: "4b08b50e5e4146fa845229e427293f3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.696288 24908 consensus_queue.cc:1048] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.708336 24908 consensus_queue.cc:1048] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.719321 24953 raft_consensus.cc:993] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: : Instructing follower 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:20.719797 24908 raft_consensus.cc:1081] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Signalling peer 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:20.721849 24626 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:52252
I20260709 12:30:20.722555 24626 raft_consensus.cc:493] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:20.722975 24626 raft_consensus.cc:3060] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.728451 24626 raft_consensus.cc:515] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.730185 24626 leader_election.cc:290] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.731122 24701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:20.731539 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab45a25466a1484081f5e7e94ff26561" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.731866 24701 raft_consensus.cc:3060] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.732147 24777 raft_consensus.cc:3055] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:20.732508 24777 raft_consensus.cc:740] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:20.733430 24777 consensus_queue.cc:260] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.734843 24777 raft_consensus.cc:3060] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.740140 24701 raft_consensus.cc:2468] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:20.741830 24592 leader_election.cc:304] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, 6ce569893fd14fb995f3c987e622fc10; no voters: 
I20260709 12:30:20.742722 24928 raft_consensus.cc:2804] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:20.742826 24777 raft_consensus.cc:2468] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:20.743315 24928 raft_consensus.cc:697] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.744270 24928 consensus_queue.cc:237] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.751722 24523 catalog_manager.cc:5697] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 1 to 2, leader changed from c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195) to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:20.768347 24928 consensus_queue.cc:1048] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.780125 24849 consensus_queue.cc:1048] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.787385 24931 consensus_queue.cc:1048] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:30:20.790874 24849 raft_consensus.cc:993] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10: : Instructing follower c9ee0c5aee904bc6a5597cc141dcd3e3 to start an election
I20260709 12:30:20.791471 24849 raft_consensus.cc:1081] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Signalling peer c9ee0c5aee904bc6a5597cc141dcd3e3 to start an election
I20260709 12:30:20.793710 24776 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f4fa56ac4f74ff4911afffedc164b88"
dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
 from {username='slave'} at 127.0.0.1:35924
I20260709 12:30:20.794296 24776 raft_consensus.cc:493] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:20.794672 24776 raft_consensus.cc:3060] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.805485 24950 consensus_queue.cc:1048] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:30:20.807878 24776 raft_consensus.cc:515] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.810019 24776 leader_election.cc:290] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 2 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:20.812866 24701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4fa56ac4f74ff4911afffedc164b88" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:20.813657 24701 raft_consensus.cc:3060] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.815066 24626 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f4fa56ac4f74ff4911afffedc164b88" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:20.815655 24626 raft_consensus.cc:3055] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:20.815903 24626 raft_consensus.cc:740] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.816614 24626 consensus_queue.cc:260] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.817765 24626 raft_consensus.cc:3060] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.823567 24626 raft_consensus.cc:2468] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 2.
I20260709 12:30:20.824120 24701 raft_consensus.cc:2468] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 2.
I20260709 12:30:20.825562 24741 leader_election.cc:304] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:20.826534 24908 raft_consensus.cc:2804] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:20.827095 24908 raft_consensus.cc:697] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 LEADER]: Becoming Leader. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:20.827996 24908 consensus_queue.cc:237] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.836164 24525 catalog_manager.cc:5697] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 reported cstate change: term changed from 1 to 2, leader changed from 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193) to c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:20.882519 24945 raft_consensus.cc:993] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d: : Instructing follower 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:20.883049 24903 raft_consensus.cc:1081] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Signalling peer 6ce569893fd14fb995f3c987e622fc10 to start an election
I20260709 12:30:20.884742 24626 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ad96890857ec428694da7080c2fddf26"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
 from {username='slave'} at 127.0.0.1:52238
I20260709 12:30:20.885232 24626 raft_consensus.cc:493] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:20.885524 24626 raft_consensus.cc:3060] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.890719 24626 raft_consensus.cc:515] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.892342 24626 leader_election.cc:290] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Requested vote from peers 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381), c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195:42195)
I20260709 12:30:20.893234 24701 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad96890857ec428694da7080c2fddf26" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:20.893639 24777 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ad96890857ec428694da7080c2fddf26" candidate_uuid: "6ce569893fd14fb995f3c987e622fc10" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
I20260709 12:30:20.893929 24701 raft_consensus.cc:3055] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:20.894325 24777 raft_consensus.cc:3060] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.894357 24701 raft_consensus.cc:740] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:20.895346 24701 consensus_queue.cc:260] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.896382 24701 raft_consensus.cc:3060] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:20.900390 24777 raft_consensus.cc:2468] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:20.901678 24592 leader_election.cc:304] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:20.901827 24701 raft_consensus.cc:2468] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ce569893fd14fb995f3c987e622fc10 in term 2.
I20260709 12:30:20.902529 24936 raft_consensus.cc:2804] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:20.903059 24936 raft_consensus.cc:697] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Becoming Leader. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:20.903829 24936 consensus_queue.cc:237] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:20.911870 24525 catalog_manager.cc:5697] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 reported cstate change: term changed from 1 to 2, leader changed from 4b08b50e5e4146fa845229e427293f3d (127.8.137.194) to 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193). New cstate: current_term: 2 leader_uuid: "6ce569893fd14fb995f3c987e622fc10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: UNKNOWN } } }
I20260709 12:30:21.005451 24776 raft_consensus.cc:1275] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.007622 24936 consensus_queue.cc:1048] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:21.022079 24701 raft_consensus.cc:1275] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.025436 24936 consensus_queue.cc:1048] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:21.055581 24626 raft_consensus.cc:1275] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.056926 24903 consensus_queue.cc:1048] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:21.068015 24777 raft_consensus.cc:1275] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.069368 24903 consensus_queue.cc:1048] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:21.142323 24626 raft_consensus.cc:1275] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.143921 24903 consensus_queue.cc:1048] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:21.157433 24777 raft_consensus.cc:1275] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 4b08b50e5e4146fa845229e427293f3d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.159456 24903 consensus_queue.cc:1048] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:21.236886 24701 raft_consensus.cc:1275] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.238229 24928 consensus_queue.cc:1048] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:21.244467 24700 raft_consensus.cc:1275] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Refusing update from remote peer c9ee0c5aee904bc6a5597cc141dcd3e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.246379 24953 consensus_queue.cc:1048] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:21.250448 24776 raft_consensus.cc:1275] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.253527 24928 consensus_queue.cc:1048] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:21.260797 24626 raft_consensus.cc:1275] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Refusing update from remote peer c9ee0c5aee904bc6a5597cc141dcd3e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.263106 24908 consensus_queue.cc:1048] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:21.461112 24700 raft_consensus.cc:1275] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.462368 24928 consensus_queue.cc:1048] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:21.477166 24776 raft_consensus.cc:1275] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Refusing update from remote peer 6ce569893fd14fb995f3c987e622fc10: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:21.478739 24988 consensus_queue.cc:1048] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:30:23.501981 24525 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46206:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:30:23.526870 24525 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:46206:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:30:23.641127  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:23.771477 25023 tablet.cc:1724] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:23.780081 24592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.195:42195, user_credentials={real_user=slave}} blocked reactor thread for 50424.2us
I20260709 12:30:23.832791 25036 tablet.cc:1724] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:23.884027 24592 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.194:41381, user_credentials={real_user=slave}} blocked reactor thread for 72079.2us
W20260709 12:30:23.872061 25012 log.cc:927] Time spent T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3: Append to log took a long time: real 0.137s	user 0.003s	sys 0.002s
W20260709 12:30:23.915093 25035 log.cc:927] Time spent T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:30:23.885129 25048 tablet.cc:1724] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:23.933406 25050 tablet.cc:1724] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:23.938689 25051 tablet.cc:1724] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:23.902099 25045 tablet.cc:1724] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:23.957074 25047 tablet.cc:1724] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:23.987175 25019 log.cc:927] Time spent T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: Append to log took a long time: real 0.250s	user 0.004s	sys 0.000s
W20260709 12:30:24.011374 25052 log.cc:927] Time spent T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3: Append to log took a long time: real 0.077s	user 0.000s	sys 0.002s
W20260709 12:30:24.015200 25046 log.cc:927] Time spent T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3: Append to log took a long time: real 0.084s	user 0.002s	sys 0.000s
W20260709 12:30:24.039381 25026 log.cc:927] Time spent T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3: Append to log took a long time: real 0.288s	user 0.004s	sys 0.000s
I20260709 12:30:24.064549 25053 tablet.cc:1724] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.068234 25056 tablet.cc:1724] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.077189 25057 tablet.cc:1724] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:30:24.081161 24667 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.8.137.193:43703, user_credentials={real_user=slave}} blocked reactor thread for 89402.7us
I20260709 12:30:24.087086 25036 tablet.cc:1724] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.083004 25045 tablet.cc:1724] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.095098 25023 tablet.cc:1724] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.108806 25058 tablet.cc:1724] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.110050 25060 tablet.cc:1724] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.115247 25050 tablet.cc:1724] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.119208 25045 tablet.cc:1724] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.121151 25051 tablet.cc:1724] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.123945 24628 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "07feb8bf7d60420f910f92bb22070acd"
dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
mode: GRACEFUL
new_leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
 from {username='slave'} at 127.0.0.1:40554
I20260709 12:30:24.106061 25048 tablet.cc:1724] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.124897 24628 raft_consensus.cc:606] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Received request to transfer leadership to TS c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:24.131747 25045 tablet.cc:1724] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.132499 25036 tablet.cc:1724] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.147835 25047 tablet.cc:1724] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.163442 25053 tablet.cc:1724] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.169098 25050 tablet.cc:1724] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.159472 25060 tablet.cc:1724] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.190462 25057 tablet.cc:1724] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.214915 25047 tablet.cc:1724] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:30:24.224815 24702 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32"
dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
mode: GRACEFUL
new_leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
 from {username='slave'} at 127.0.0.1:47494
I20260709 12:30:24.225597 24702 raft_consensus.cc:606] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Received request to transfer leadership to TS c9ee0c5aee904bc6a5597cc141dcd3e3
I20260709 12:30:24.227442  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:30:24.228125  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:24.232949 24526 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3: soft_deleted_table [id=c1f74abf6abd4071b072d2bfe5bdf4bc] alter complete (version 1)
I20260709 12:30:24.336485 25061 raft_consensus.cc:993] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d: : Instructing follower c9ee0c5aee904bc6a5597cc141dcd3e3 to start an election
I20260709 12:30:24.337181 25024 raft_consensus.cc:1081] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Signalling peer c9ee0c5aee904bc6a5597cc141dcd3e3 to start an election
I20260709 12:30:24.338865 24770 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32"
dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
 from {username='slave'} at 127.0.0.1:35914
I20260709 12:30:24.339390 24770 raft_consensus.cc:493] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:24.339720 24770 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:24.346163 24770 raft_consensus.cc:515] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:24.348059 24770 leader_election.cc:290] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 3 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:24.348987 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:24.349109 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e5ab1d11815d4a859c2141da9167ed32" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:24.349856 24628 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:24.349931 24702 raft_consensus.cc:3055] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:30:24.350298 24702 raft_consensus.cc:740] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4b08b50e5e4146fa845229e427293f3d, State: Running, Role: LEADER
I20260709 12:30:24.351117 24702 consensus_queue.cc:260] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:24.352183 24702 raft_consensus.cc:3060] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:30:24.355779 24628 raft_consensus.cc:2468] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 3.
I20260709 12:30:24.357095 24741 leader_election.cc:304] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ce569893fd14fb995f3c987e622fc10, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:24.357908 25043 raft_consensus.cc:2804] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:30:24.358435 25043 raft_consensus.cc:697] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 3 LEADER]: Becoming Leader. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:24.358856 24702 raft_consensus.cc:2468] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 3.
I20260709 12:30:24.359575 25043 consensus_queue.cc:237] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:24.369534 24526 catalog_manager.cc:5697] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 reported cstate change: term changed from 2 to 3, leader changed from 4b08b50e5e4146fa845229e427293f3d (127.8.137.194) to c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195). New cstate: current_term: 3 leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:24.396847 24936 raft_consensus.cc:993] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10: : Instructing follower c9ee0c5aee904bc6a5597cc141dcd3e3 to start an election
I20260709 12:30:24.397400 25042 raft_consensus.cc:1081] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Signalling peer c9ee0c5aee904bc6a5597cc141dcd3e3 to start an election
I20260709 12:30:24.398936 24770 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "07feb8bf7d60420f910f92bb22070acd"
dest_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3"
 from {username='slave'} at 127.0.0.1:35924
I20260709 12:30:24.399564 24770 raft_consensus.cc:493] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:30:24.399915 24770 raft_consensus.cc:3060] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:24.405638 24770 raft_consensus.cc:515] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:24.407428 24770 leader_election.cc:290] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 2 election: Requested vote from peers 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703), 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381)
I20260709 12:30:24.408528 24702 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07feb8bf7d60420f910f92bb22070acd" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4b08b50e5e4146fa845229e427293f3d"
I20260709 12:30:24.408483 24628 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "07feb8bf7d60420f910f92bb22070acd" candidate_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6ce569893fd14fb995f3c987e622fc10"
I20260709 12:30:24.409333 24702 raft_consensus.cc:3060] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:24.409423 24628 raft_consensus.cc:3055] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:30:24.409847 24628 raft_consensus.cc:740] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6ce569893fd14fb995f3c987e622fc10, State: Running, Role: LEADER
I20260709 12:30:24.410804 24628 consensus_queue.cc:260] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:24.412204 24628 raft_consensus.cc:3060] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:30:24.417102 24702 raft_consensus.cc:2468] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 2.
I20260709 12:30:24.418615 24741 leader_election.cc:304] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b08b50e5e4146fa845229e427293f3d, c9ee0c5aee904bc6a5597cc141dcd3e3; no voters: 
I20260709 12:30:24.419130 24628 raft_consensus.cc:2468] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9ee0c5aee904bc6a5597cc141dcd3e3 in term 2.
I20260709 12:30:24.419569 25043 raft_consensus.cc:2804] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:30:24.420076 25043 raft_consensus.cc:697] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 LEADER]: Becoming Leader. State: Replica: c9ee0c5aee904bc6a5597cc141dcd3e3, State: Running, Role: LEADER
I20260709 12:30:24.420984 25043 consensus_queue.cc:237] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } }
I20260709 12:30:24.428215 24526 catalog_manager.cc:5697] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 reported cstate change: term changed from 1 to 2, leader changed from 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193) to c9ee0c5aee904bc6a5597cc141dcd3e3 (127.8.137.195). New cstate: current_term: 2 leader_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c9ee0c5aee904bc6a5597cc141dcd3e3" member_type: VOTER last_known_addr { host: "127.8.137.195" port: 42195 } health_report { overall_health: HEALTHY } } }
I20260709 12:30:24.746356 24628 raft_consensus.cc:1275] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 3 FOLLOWER]: Refusing update from remote peer c9ee0c5aee904bc6a5597cc141dcd3e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:24.747646 25043 consensus_queue.cc:1048] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:24.761135 24702 raft_consensus.cc:1275] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 3 FOLLOWER]: Refusing update from remote peer c9ee0c5aee904bc6a5597cc141dcd3e3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:30:24.762717 25043 consensus_queue.cc:1048] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:30:24.903920 24702 raft_consensus.cc:1275] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Refusing update from remote peer c9ee0c5aee904bc6a5597cc141dcd3e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:24.905261 25043 consensus_queue.cc:1048] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b08b50e5e4146fa845229e427293f3d" member_type: VOTER last_known_addr { host: "127.8.137.194" port: 41381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:24.915735 24628 raft_consensus.cc:1275] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Refusing update from remote peer c9ee0c5aee904bc6a5597cc141dcd3e3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:30:24.917479 25075 consensus_queue.cc:1048] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ce569893fd14fb995f3c987e622fc10" member_type: VOTER last_known_addr { host: "127.8.137.193" port: 43703 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:30:25.229259  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:25.235271  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:25.235718  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:26.236636  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:26.242650  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:26.243166  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:27.244139  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:27.249748  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:27.250229  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:28.251369  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:28.257670  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:28.258157  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:29.259271  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:29.264329  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:29.264856  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:30.266032  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:30.271442  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:30.271979  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:31.273582  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:31.281483  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:31.282248  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:32.283828  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:32.288983  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:32.289394  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:33.290763  8743 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:30:33.295665  8743 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:30:33.296088  8743 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:30:34.362658  8743 tablet_server.cc:179] TabletServer@127.8.137.193:0 shutting down...
W20260709 12:30:34.392783 24667 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 14 similar messages]
W20260709 12:30:34.398069 24667 consensus_peers.cc:597] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:34.414273 24667 consensus_peers.cc:597] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.415791  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:34.416859  8743 tablet_replica.cc:333] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.419323  8743 raft_consensus.cc:2243] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.420482  8743 raft_consensus.cc:2272] T 1a8fed2a979948a3bc7bd4585b8421d4 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:34.421907 24667 consensus_peers.cc:597] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.426877  8743 tablet_replica.cc:333] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.428112  8743 raft_consensus.cc:2243] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:30:34.428599 24741 consensus_peers.cc:597] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.429009  8743 raft_consensus.cc:2272] T 1bfa91f364ec497198b5528195934b51 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.432065  8743 tablet_replica.cc:333] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.432775  8743 raft_consensus.cc:2243] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.433349  8743 raft_consensus.cc:2272] T f5073cee60714ee9b1d6e5eb034177f7 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.436180  8743 tablet_replica.cc:333] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.436880  8743 raft_consensus.cc:2243] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.438319  8743 raft_consensus.cc:2272] T caf686d556df490cbb73587efca0b6dd P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.441694  8743 tablet_replica.cc:333] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.442405  8743 raft_consensus.cc:2243] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.443085  8743 raft_consensus.cc:2272] T 3f3aa7ecc7294d738b3e788d66914ea8 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.445921  8743 tablet_replica.cc:333] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.446633  8743 raft_consensus.cc:2243] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.447234  8743 raft_consensus.cc:2272] T e5ab1d11815d4a859c2141da9167ed32 P 6ce569893fd14fb995f3c987e622fc10 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.450093  8743 tablet_replica.cc:333] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.450830  8743 raft_consensus.cc:2243] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.451370  8743 raft_consensus.cc:2272] T 07feb8bf7d60420f910f92bb22070acd P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.455315  8743 tablet_replica.cc:333] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.456331  8743 raft_consensus.cc:2243] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.457055  8743 raft_consensus.cc:2272] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.460909  8743 tablet_replica.cc:333] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.461926  8743 raft_consensus.cc:2243] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.463053  8743 raft_consensus.cc:2272] T ab45a25466a1484081f5e7e94ff26561 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.465761  8743 tablet_replica.cc:333] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.466468  8743 raft_consensus.cc:2243] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.467097  8743 raft_consensus.cc:2272] T 96761ddfa0044fea9016033ae2131861 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.469933  8743 tablet_replica.cc:333] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.470616  8743 raft_consensus.cc:2243] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.471233  8743 raft_consensus.cc:2272] T 0f4fa56ac4f74ff4911afffedc164b88 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.474128  8743 tablet_replica.cc:333] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.474795  8743 raft_consensus.cc:2243] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.475468  8743 raft_consensus.cc:2272] T d169eba347c2424a9bb86068108e7f62 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.478104  8743 tablet_replica.cc:333] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.478787  8743 raft_consensus.cc:2243] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.480005  8743 raft_consensus.cc:2272] T 6d64ccaae0eb400f8010fb8849239fc3 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.483016  8743 tablet_replica.cc:333] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.483530  8743 raft_consensus.cc:2243] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.484385  8743 raft_consensus.cc:2272] T 42a5285e1d844f10a11559914dce63c5 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.486919  8743 tablet_replica.cc:333] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.487421  8743 raft_consensus.cc:2243] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.487929  8743 raft_consensus.cc:2272] T 2c4639d5f9864cfe9bf2132351763b98 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.490399  8743 tablet_replica.cc:333] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.490940  8743 raft_consensus.cc:2243] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.491788  8743 raft_consensus.cc:2272] T 60c01c5a4ca34a3baf3de9616c7efce3 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.494575  8743 tablet_replica.cc:333] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.495261  8743 raft_consensus.cc:2243] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.496577  8743 raft_consensus.cc:2272] T ad96890857ec428694da7080c2fddf26 P 6ce569893fd14fb995f3c987e622fc10 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:34.497350 24741 consensus_peers.cc:597] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.500106  8743 tablet_replica.cc:333] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10: stopping tablet replica
I20260709 12:30:34.500793  8743 raft_consensus.cc:2243] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.501732  8743 raft_consensus.cc:2272] T a12efdb159cf47dcb9d429a97eeef237 P 6ce569893fd14fb995f3c987e622fc10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.554016  8743 tablet_server.cc:196] TabletServer@127.8.137.193:0 shutdown complete.
W20260709 12:30:34.578307 24667 consensus_peers.cc:597] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:43703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.607197  8743 tablet_server.cc:179] TabletServer@127.8.137.194:0 shutting down...
W20260709 12:30:34.612327 24741 consensus_peers.cc:597] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 -> Peer 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381): Couldn't send request to peer 4b08b50e5e4146fa845229e427293f3d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:30:34.642472 24667 consensus_peers.cc:597] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:43703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.643779  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:34.644594  8743 tablet_replica.cc:333] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.645551  8743 raft_consensus.cc:2243] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.646716  8743 raft_consensus.cc:2272] T 1bfa91f364ec497198b5528195934b51 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.649325  8743 tablet_replica.cc:333] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.650032  8743 raft_consensus.cc:2243] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.651173  8743 raft_consensus.cc:2272] T 1a8fed2a979948a3bc7bd4585b8421d4 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.654212  8743 tablet_replica.cc:333] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.654961  8743 raft_consensus.cc:2243] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.655997  8743 raft_consensus.cc:2272] T f5073cee60714ee9b1d6e5eb034177f7 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.658710  8743 tablet_replica.cc:333] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.659348  8743 raft_consensus.cc:2243] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.659940  8743 raft_consensus.cc:2272] T caf686d556df490cbb73587efca0b6dd P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.662381  8743 tablet_replica.cc:333] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.663087  8743 raft_consensus.cc:2243] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.664240  8743 raft_consensus.cc:2272] T 3f3aa7ecc7294d738b3e788d66914ea8 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.667066  8743 tablet_replica.cc:333] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.667703  8743 raft_consensus.cc:2243] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.668192  8743 raft_consensus.cc:2272] T e5ab1d11815d4a859c2141da9167ed32 P 4b08b50e5e4146fa845229e427293f3d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.670523  8743 tablet_replica.cc:333] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.671213  8743 raft_consensus.cc:2243] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.671729  8743 raft_consensus.cc:2272] T 07feb8bf7d60420f910f92bb22070acd P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.673979  8743 tablet_replica.cc:333] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.674646  8743 raft_consensus.cc:2243] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.675573  8743 raft_consensus.cc:2272] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.677837  8743 tablet_replica.cc:333] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.678408  8743 raft_consensus.cc:2243] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.679383  8743 raft_consensus.cc:2272] T 2c4639d5f9864cfe9bf2132351763b98 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.681885  8743 tablet_replica.cc:333] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.682512  8743 raft_consensus.cc:2243] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.683082  8743 raft_consensus.cc:2272] T ab45a25466a1484081f5e7e94ff26561 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.685413  8743 tablet_replica.cc:333] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.686055  8743 raft_consensus.cc:2243] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.686626  8743 raft_consensus.cc:2272] T 96761ddfa0044fea9016033ae2131861 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.689286  8743 tablet_replica.cc:333] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.689939  8743 raft_consensus.cc:2243] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.690537  8743 raft_consensus.cc:2272] T 0f4fa56ac4f74ff4911afffedc164b88 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.695106  8743 tablet_replica.cc:333] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.695781  8743 raft_consensus.cc:2243] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.696887  8743 raft_consensus.cc:2272] T d169eba347c2424a9bb86068108e7f62 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.699679  8743 tablet_replica.cc:333] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.700282  8743 raft_consensus.cc:2243] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.700793  8743 raft_consensus.cc:2272] T 6d64ccaae0eb400f8010fb8849239fc3 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.703168  8743 tablet_replica.cc:333] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.703733  8743 raft_consensus.cc:2243] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.704268  8743 raft_consensus.cc:2272] T 42a5285e1d844f10a11559914dce63c5 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.706553  8743 tablet_replica.cc:333] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.707172  8743 raft_consensus.cc:2243] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.707686  8743 raft_consensus.cc:2272] T 60c01c5a4ca34a3baf3de9616c7efce3 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.709952  8743 tablet_replica.cc:333] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.710449  8743 raft_consensus.cc:2243] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.711047  8743 raft_consensus.cc:2272] T a12efdb159cf47dcb9d429a97eeef237 P 4b08b50e5e4146fa845229e427293f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.713541  8743 tablet_replica.cc:333] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d: stopping tablet replica
I20260709 12:30:34.714092  8743 raft_consensus.cc:2243] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.714725  8743 raft_consensus.cc:2272] T ad96890857ec428694da7080c2fddf26 P 4b08b50e5e4146fa845229e427293f3d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:30:34.736945 24741 consensus_peers.cc:597] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:43703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.762862  8743 tablet_server.cc:196] TabletServer@127.8.137.194:0 shutdown complete.
W20260709 12:30:34.806840 24741 consensus_peers.cc:597] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 -> Peer 6ce569893fd14fb995f3c987e622fc10 (127.8.137.193:43703): Couldn't send request to peer 6ce569893fd14fb995f3c987e622fc10. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.193:43703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.807855  8743 tablet_server.cc:179] TabletServer@127.8.137.195:0 shutting down...
W20260709 12:30:34.811478 24741 consensus_peers.cc:597] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 -> Peer 4b08b50e5e4146fa845229e427293f3d (127.8.137.194:41381): Couldn't send request to peer 4b08b50e5e4146fa845229e427293f3d. Status: Network error: Client connection negotiation failed: client connection to 127.8.137.194:41381: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:30:34.841687  8743 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:30:34.842552  8743 tablet_replica.cc:333] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.843318  8743 raft_consensus.cc:2243] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.843910  8743 raft_consensus.cc:2272] T 1bfa91f364ec497198b5528195934b51 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.846585  8743 tablet_replica.cc:333] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.847347  8743 raft_consensus.cc:2243] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.847910  8743 raft_consensus.cc:2272] T f5073cee60714ee9b1d6e5eb034177f7 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.850121  8743 tablet_replica.cc:333] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.850772  8743 raft_consensus.cc:2243] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.851284  8743 raft_consensus.cc:2272] T caf686d556df490cbb73587efca0b6dd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.853662  8743 tablet_replica.cc:333] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.854403  8743 raft_consensus.cc:2243] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.854959  8743 raft_consensus.cc:2272] T 1a8fed2a979948a3bc7bd4585b8421d4 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.857136  8743 tablet_replica.cc:333] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.857692  8743 raft_consensus.cc:2243] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.858182  8743 raft_consensus.cc:2272] T 3f3aa7ecc7294d738b3e788d66914ea8 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.860479  8743 tablet_replica.cc:333] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.861109  8743 raft_consensus.cc:2243] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.861624  8743 raft_consensus.cc:2272] T ad96890857ec428694da7080c2fddf26 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.864018  8743 tablet_replica.cc:333] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.864583  8743 raft_consensus.cc:2243] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.865103  8743 raft_consensus.cc:2272] T a12efdb159cf47dcb9d429a97eeef237 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.867509  8743 tablet_replica.cc:333] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.868073  8743 raft_consensus.cc:2243] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.869011  8743 raft_consensus.cc:2272] T 0f4fa56ac4f74ff4911afffedc164b88 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.871353  8743 tablet_replica.cc:333] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.871977  8743 raft_consensus.cc:2243] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.872573  8743 raft_consensus.cc:2272] T d169eba347c2424a9bb86068108e7f62 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.874812  8743 tablet_replica.cc:333] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.875371  8743 raft_consensus.cc:2243] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.875859  8743 raft_consensus.cc:2272] T 42a5285e1d844f10a11559914dce63c5 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.878122  8743 tablet_replica.cc:333] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.878731  8743 raft_consensus.cc:2243] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.879277  8743 raft_consensus.cc:2272] T 6d64ccaae0eb400f8010fb8849239fc3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.881548  8743 tablet_replica.cc:333] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.882107  8743 raft_consensus.cc:2243] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.883514  8743 raft_consensus.cc:2272] T e5ab1d11815d4a859c2141da9167ed32 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.885828  8743 tablet_replica.cc:333] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.886404  8743 raft_consensus.cc:2243] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.887312  8743 raft_consensus.cc:2272] T 07feb8bf7d60420f910f92bb22070acd P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.889645  8743 tablet_replica.cc:333] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.890627  8743 raft_consensus.cc:2243] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.891292  8743 raft_consensus.cc:2272] T 0cd6dc8d1cc34003aef4b0eb4823ca1c P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.894217  8743 tablet_replica.cc:333] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.894855  8743 raft_consensus.cc:2243] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.895448  8743 raft_consensus.cc:2272] T ab45a25466a1484081f5e7e94ff26561 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.897791  8743 tablet_replica.cc:333] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.898341  8743 raft_consensus.cc:2243] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:30:34.899271  8743 raft_consensus.cc:2272] T 96761ddfa0044fea9016033ae2131861 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.901544  8743 tablet_replica.cc:333] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.902082  8743 raft_consensus.cc:2243] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.902577  8743 raft_consensus.cc:2272] T 2c4639d5f9864cfe9bf2132351763b98 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.904706  8743 tablet_replica.cc:333] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3: stopping tablet replica
I20260709 12:30:34.905226  8743 raft_consensus.cc:2243] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:30:34.905711  8743 raft_consensus.cc:2272] T 60c01c5a4ca34a3baf3de9616c7efce3 P c9ee0c5aee904bc6a5597cc141dcd3e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:34.948793  8743 tablet_server.cc:196] TabletServer@127.8.137.195:0 shutdown complete.
I20260709 12:30:34.990006  8743 master.cc:562] Master@127.8.137.254:42857 shutting down...
I20260709 12:30:35.013418  8743 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:30:35.014117  8743 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:30:35.014477  8743 tablet_replica.cc:333] T 00000000000000000000000000000000 P 98d9a8c9d30646e0b6b93bbd28f195a3: stopping tablet replica
I20260709 12:30:35.036088  8743 master.cc:584] Master@127.8.137.254:42857 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (20710 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (41722 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (514030 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 12:30:35.097385  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2231 messages since previous log ~32 seconds ago
I20260709 12:30:35.097597  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2231 messages since previous log ~32 seconds ago
I20260709 12:30:35.097738  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2228 messages since previous log ~32 seconds ago
I20260709 12:30:35.097903  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~50 seconds ago
I20260709 12:30:35.098058  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~50 seconds ago
I20260709 12:30:35.098209  8743 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 1 messages since previous log ~243 seconds ago
I20260709 12:30:35.098346  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 1 messages since previous log ~243 seconds ago
I20260709 12:30:35.098501  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 4 messages since previous log ~163 seconds ago
I20260709 12:30:35.098794  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 11 messages since previous log ~0 seconds ago
I20260709 12:30:35.099114  8743 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~19 seconds ago
